ICRR 12886 Report Number : ICRR12886 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 05/07/2008 PROJ ID : P082865 Appraisal Actual Project Name : Co: Programmatic US$M ): Project Costs (US$M): 200.0 200.0 Labor Reform And Social Structural Adjustment Loan Country : Colombia Loan /Credit (US$M): Loan/ US$M ): 200.0 200.0 Sector Board : SP US$M): Cofinancing (US$M ): Sector (s): Health (40%) Primary education (30%) Other social services (20%) Secondary education (10%) Theme (s): Improving labor markets (23% - P) Education for all (22% - P) Other social protection and risk management (22% - P) Social safety nets (22% - P) Education for the knowledge economy (11% - S) L/C Number : L7259 Board Approval Date : 11/04/2004 Partners involved : Closing Date : 03/31/2005 03/31/2005 Evaluator : Panel Reviewer : Group Manager : Group : Rene I. Vandendries Fareed M. A. Hassan James Sackey IEGCR 2. Project Objectives and Components: a. Objectives: This ICR Review examines the second operation of a series of three programmatic loans to Colombia for a Labor Reform and Social Program. The program development objectives (PDOs) proposed to: (i) strengthen Colombia's social protection system and improve the delivery of social services; (2) raise human capital formation; and (3) improve employability. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project components (sub-objectives) covered: (i) social protection, i.e. institution of an effective authority in charge of developing a comprehensive social risk management system; (2) early childhood, i.e. greater access to and better targeting of feeding programs for poor children; (3) health, i.e. expansion of health insurance for the poor; (4) increased immunization rates; (5) labor and training, i.e. reforms to support employment and expand social protection; (6) education, i.e. reforms to increase coverage as well as quality; and, (7) transparency, i.e. greater control of social programs through periodic evaluation and participatory oversight by citizens. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The program of Bank support for Colombia's Labor Reform and Social Program was at first supported by a sequence of two one-tranche programmatic SALs, both for $ 200 million. This second operation was approved on November 4, 2004 and disbursed at effectiveness on November 10, 2004. In view of the rapid progress made in the reforms and Colombia's request for continued policy support, a third one -tranche operation, also for $200 million, was approved in 2006. There was no cofinancing. 3. Relevance of Objectives & Design: The objectives of this operation and of the overall program were highly relevant to the situation in Colombia in the early 2000s. They addressed highly important social sector issues . Following the crisis of 1999/2000 the unemployment rate was very high, at 17 percent in 2002. Poverty was severe and the country's education levels continued to lag behind comparator countries . The existing system of social protection programs was uncoordinated, inefficient and ineffective . The operation and the program were well -designed and implemented because of close working relations between the Bank and a committed Government and implementing agencies . 4. Achievement of Objectives (Efficacy): Social Protection . The key achievement was the creation, in February 2003, of the Ministry of Social Protection (MSP) by the new Government which had come to power in August 2002. This was before appraisal of the first operation in the program and was done by the merging of the Ministries of Labor and Health . The new Ministry became the institutional backbone of Colombia's social protection programs, housing in one ministry all public sector programs responsible for social risk management . By the end of this second operation of the program the ministry was functioning effectively Early Childhood Development . Better targeting and access of poor children to the feeding programs of the Colombia Institute for Family Well-being (ICBF) was a major theme of the program. The number of children reached, all poor according to the norms that had been set, grew from 78,000 in 2002 (the base year) to 517,000 at completion of the loan. This second operation also helped initiate evaluation of the two largest ICBF programs. Health and Health Insurance . Health insurance coverage for the poor through the Subsidized Health Insurance Regime was 10.7 million in the base year, 2002. Through a reduction in evasion of taxes that fund the system, simplification of the funding process, and increased efficiency in the health sector, health insurance coverage for the poor reached 13.5 million at completion of the loan, exceeding the program target . Immunization Coverage . In 2002, average immunization rates of key vaccines were 80 percent, a decrease from the mid-90s. The target immunization rate of at least 90 percent, so as to ensure protection against epidemics, was maintained . Labor and Training . A new Labor Reform Law (Law 789 of 2002), approved prior to the initiation of the program, aimed to support employment creation and expand social protection . The law also established norms to create a National Skills Formation System (SNFT) and expand the apprenticeship system . The loan supported all of these objectives, including an increase in the efficiency of the public training institute (SENA). While national employment rose and the unemployment rate fell, it is impossible to estimate how much of this was a result of the labor reform. On the other hand, the number of private training courses approved by SENA grew from none in the base year, to 138 at the completion of the loan. The pool of apprentices grew from 33,000 annually in the base year, to 74,000. The number of graduates from SENA increased from 900,000 in the base year, to 2.9 million at the completion of the loan. Education . The focus in education was on increasing coverage and improving quality, the latter through both student and teacher testing . Through changes in school funding mechanisms and in the student -to-teacher ratio, basic and secondary enrollment increased from 7.8 million students in the base year to 8.5 million at completion of the loan, achieving the program target . Only about a third of municipalities tested students in the base year, slightly more than 90 percent did at the completion of the loan . Teacher testing did not get off the ground as planned under this operation . Transparency . The program included the development of tools to improve oversight of government action in the social sectors. Progress has been made. While the first operation had helped strengthen impact evaluation work, the second operation focused more on dissemination of findings to the public . 5. Efficiency (not applicable to DPLs): Economic rates of return are not relevant for programmatic development policy loans . Bank budgetary lending and supervision costs were relatively modest at about US $310,000, for US$200 million in lending. Part of the explanation lies in the very low supervision costs (about US$23,000) as a result of the close spacing of the operations, with supervision taking place during preparation of the follow -up loan. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Good progress was made in implementing the reform program, although in some areas program targets were not fully met. There is now an effective and efficient Ministry of Social Protection, a much improved human capital base (through Early Childhood Development, improved health and education ), a better trained labor force, and increasing transparency and oversight of social services . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risks of not sustaining the achievements of this operation are low because, with strong government commitment, the reform program is progressing well . MSP has become gradually more efficient, the human capital development programs are well established, training is being intensified, and development of the monitoring and evaluation system is progressing . a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: The Bank collaborated closely with client ministries and institutions and supported their longer term objectives as well as needed intermediate steps . The operation was designed to ensure appropriate monitoring and hence allow for corrective action, when necessary . Supervision was intense through regular discussions between the Bank and Government on the Bank's overall social sector assistance to Colombia . The Bank also provided quality technical assistance, drawing on a large supply of sector work, and coordinated closely with IDB on its program of social sector operations . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Probably the most significant achievement of the Borrower was the creation of MSP in early 2003, shortly after a new social sector reform -conscious Government had come to power . Through the MSP the Government was successful in achieving coordination among the relevant ministries and institutions responsible for social sector development programs in Colombia . A lack of coordination had plagued the sector for many years. There was a clear vision and strategy for a labor and social reform agenda . This was the core of the reform program and major contributor to its success . a. Government Performance :Highly Satisfactory b. Implementing Agency Performance :Highly Satisfactory c. Overall Borrower Performance :Highly Satisfactory 10. M&E Design, Implementation, & Utilization: The M&E was well designed, given that it focused on tracking outcome and output indicators which could be monitored because of generally readily available information . Implementation was smooth because the program was closely aligned with the National Development Plan . The monitoring system allowed for timely adjustments in budgetary decisions in function of the results being achieved . a. M&E Quality Rating : High 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Highly Satisfactory See Section 9 Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR contains several useful lessons, whose essence may be summarized as follows . Commitment by client institutions is a necessary condition for success and enhances the value added of Bank support. Close cooperation between the borrower and the Bank facilitates both the Bank's understanding of the client's needs and the transfer of Bank knowledge and experience . A good M&E system is a precondition for results -based management. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clearly written, comprehensive and exhaustive . It focuses appropriately on the program as a whole, with occasional references to the contribution of individual operations . The ICR is internally consistent with one small exception: the outcome of loan # 1 is rated "satisfactory" in the summary table, but highly satisfactory in para 3.3. An ICR for a series of programmatic loans could be further improved by including a short summary table showing the contribution of the separate loans, in addition to the current treatment of these contributions throughout the tables and text . a.Quality of ICR Rating : Satisfactory