ICRR 11846 Report Number : ICRR11846 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 08/25/2004 PROJ ID : P046112 Appraisal Actual Project Name : Co Pasto Education Project Costs 12 10.9 US$M ) (US$M) Country : Colombia Loan/ US$M ) 7.2 Loan /Credit (US$M) 7.1 Sector (s): Board: ED - Secondary Cofinancing education (40%), Primary US$M ) (US$M) education (40%), Central government administration (10%), Tertiary education (5%), Sub-national government administration (5%) L/C Number : L4242; LP299 Board Approval 98 FY ) (FY) Partners involved : Closing Date 06/30/2002 08/31/2003 Prepared by : Reviewed by : Group Manager : Group : Helen Abadzi Jorge Garcia-Garcia Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The development objectives of the Colombia Pasto Education Project were to : (a) improve the capacity of schools, education communities, and the municipality to provide their respective education services effectively and in a participatory way; (b) contribute to the municipality's goal of improving student learning, school access, and retention rates. b. Components There were three components: (a) The Basic Unattended Educational Needs component, which provided resources (educational materials, teacher training, civil works, classroom furniture and equipment) for public school grades Kinder to grade 9, and performs urgent infrastructure rehabilitation (building new classrooms) for US$6.7 million; (b) Institutional Strengthening component, which provided resources (technical assistance, training, and systems development ) to strengthen the Municipal Secretariat of Education to fully assume decentralized management as well as strengthen the capacity of the various actors co-managing school networks (for US$2.1 million); and (c) the 12 School Networks Component, with each network comprising about 15 public schools and created on the basis of geographic contiguity and socioeconomic homogeneity criteria to develop and implement demand -driven education sub-projects through school and community capacity -building (for US$3.2 million). c. Comments on Project Cost, Financing and Dates This was a loan given directly to a municipality in Colombia but guaranteed by the central government . It was a pilot project aimed to provide lessons and valuable experiences for the design of a national strategy to support regionalized investment programs. Some subcomponents were revised in the course of implementation . The project closed after extensions of 14 months, and US$0.1 million were cancelled. 3. Achievement of Relevant Objectives: The project was implemented during a period of high violence in neighboring municipalities that resulted in a stream of refugees and inundation of the schools with students who were average and with limited education. Nevertheless, objectives were substantially achieved . Achievements included: (a) Improve the capacity of the municipality to provide education services - achieved.. Public schools received basic educational materials and support, such as 1727 classrooms and 1232 pedagogical toolkits; 136 schools were rehabilitated and many others received smaller interventions, such as fences and 5000 student desks. Local communities provided materials and equipment . School improvement plans were developed for school networks and 30 were financed as subprojects. The legal framework of networks was strengthened, and financing formulas were developed and applied . Teacher payroll was delegated to municipal authorities in 1999, and educational resources were transferred to the municipality in 2000. The municipality acquired capacities in budgeting, planning, and information technology . (b) Contribute to the municipality's goal of improving student learning, school access, and retention rates - partly achieved. Public school enrollments increased by an additional 7284 students, while private school enrollments increased by 4393 students. However, refugee students often tended to be overage, so net enrollment rates actually decreased by 13 percentage points. Similarly, dropout and repetition rates remained slightly higher than expected (5.2% vs 3.9% dropout and 4.2% vs. 2.8% for repetition). A functional monitoring system was put in place early on to assess the impact of the project with both pedagogical and social indicators . The evaluation study concluded that project implementation was satisfactory because investment were efficient and sustainable, and the community perceived the project positively. 4. Significant Outcomes/Impacts: - Overall, the project benefited 92,028 students, and training was provided to 90% of the teachers (2646 persons). Gross enrollment rate targets were met ( 37% for preschool, 113% for primary, and 89% for secondary). -The municipality was significantly strengthened in its efforts to plan and manage schools and to decentralize its administrative task via school networks in accordance with the competencies established by the General Law on Education (Law 115 of 1994). Despite the municipality's lack of experience in managing a Bank project, the administrators were able to overcome major obstacles and negotiate with regional and national authorities on transferring responsibility of the management of education services . An example of successful subprojects was the establishment of youth orchestras in the poorest school networks. - The ICR reports substantial improvements in test scores between 98/99 and 2003; eg. 16.6% vs. 41.6% for grade 3 math, but large reductions as well, such as 41.7% vs. 23% for grade 3 language. These are unusually large fluctuations that may reflect administration of different tests or inclusion of private schools in the testing of 2003, and cannot be accepted at face value. 5. Significant Shortcomings (including non-compliance with safeguard policies): During some periods of the project, implementation was slow . Teachers' trade union pressure against municipal-level employment and school networks created some obstacles, and thus the role of the networks was diminished. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. NOTE: 7. Lessons of Broad Applicability: - It is possible for well-organized and motivated municipalities to implement World Bank projects on their own with little support from a central government. To succeed, implementation might take into consideration the implications of the decentralization model for the various national and local actors, including the respective spheres of authority and responsibility . - Participation of local stakeholders, including communities and universities, remains a key to the success of regionalized projects. Stakeholder participation may protect the project from political and economic instabilities and reduce opposition from specific groups . - It is important to have a well-designed and fully operational monitoring and evaluation system in place at the early stages of the project cycle. Locally conducted impact evaluation studies have the additional benefit of ownership that may facilitate decisionmaking on the basis of feedback from the study . 8. Assessment Recommended? Yes No Why? This project has some innovative features, such as development of school networks within a municipal setting. It would be useful to do the field review in conjunction with other projects . 9. Comments on Quality of ICR: Overall, the ICR is rated satisfactory . However, it is extremely long, over three times the guidelines' size . The achievement test scores on table 2 are inexplicably variable, and it is unclear what level D means . Also, the ICR does not include clear enrollment and dropout targets in Annex 1 from the beginning of the project, just estimates from the last PSR.