ICRR 12763 Report Number : ICRR12763 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/10/2007 PROJ ID : P079260 Appraisal Actual Project Name : Education US$M ): Project Costs (US$M): 5.5 5.5 Participation Improvement Project Country : Kosovo Loan/ US$M): Loan /Credit (US$M): 4.5 4.5 Sector Board : ED Cofinancing (US$M): US$M ): 0.9 0.9 Sector (s): Primary education (38%) Secondary education (38%) Sub-national government administration (19%) Central government administration (5%) Theme (s): Education for all (25% - P) Vulnerability assessment and monitoring (25% - P) Participation and civic engagement (24% - P) Gender (13% - S) Municipal governance and institution building (13% - S) L/C Number : CH036 Board Approval Date : 02/14/2005 Partners involved : Canada Closing Date : 12/31/2006 12/31/2006 Evaluator : Panel Reviewer : Group Manager : Group : Helen Abadzi Howard Nial White Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The project was to improve educational attainment at primary and secondary education levels and to enhance access of vulnerable groups to education . The outcome indicators (revised for greater specificity during mid -term review) were: 25% reduction of the dropout rate among the ethnic minorities in 75% of the targeted schools; (ii) 25% improvement in the survival rate in targeted primary and secondary schools among the ethnic minorities between the grades (where feasible until Grade 9) in 75% of the targeted schools by the beginning of the next academic year following the completion of the school based planning and grant activities; and (iii) 30% improvement of girls' survival rate between grades in secondary education . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (a): School Development Grants (appraisal and actual cost US$ 3.79 million; each US$10,000-15,000) to approximately half of the 550 primary schools and 110 secondary schools in Kosovo ’s 30 municipalities in order to help increase enrollment, attendance, and retention in primary and secondary education; finance training, community outreach activities, educational materials and school supplies, and carry out small civil works aimed at improving security and the educational environment . (b) Improving the Education Management Information System (appraisal and actual cost US$ 0.30 million) to strengthen and ensure the sustainability of the existing educational management information system in order to monitor enrollment, attendance, repetition, dropout and completion rates . (c) Project Management (appraisal and actual cost US$0.51 million) to implement, monitor, and evaluate the project . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: This was an IDA grant for post-conflict reconstruction. The borrower's counterpart was only US$ 100,000. The project closed as scheduled and according to the ICR disbursed almost 100% of the funds. The exact amount canceled is not shown in the ICR . Canadian CIDA financing was partly used for grants to 100 schools. 3. Relevance of Objectives & Design: The project appears relevant because Kosovo had lower enrollment and completion rates than neighboring countries, especially at secondary level . Although school facilities were subject to emergency reconstruction immediately after the war, many buildings continued to be inadequate . The development objective remained relevant after project completion because secondary school enrollments did not increase to the extent expected among the ethnic minorities and girls. However, there were several constraints on school attainment, and it is not clear that the project addressed the most pressing of these, especially as the funds allocated were too low to improve access - and the school grants were smaller than originally planned, limiting the extent of possible improvements . 4. Achievement of Objectives (Efficacy): Improving educational attainment at primary and secondary education levels and enhancing access of vulnerable groups to education (modest). Though most activities were carried out as planned, the objectives were not attained to the extent expected . Also improvements cannot be clearly attributed to project activities . About 300,000 students benefited from the project; 514 schools received grants based on their school development plans (up from 300) and another 90 schools obtained a second award based on earlier successful implementation of plans. The total number of grants awarded by schools in this project was 604 schools (534 primary and 70 secondary schools). About 56% of the grants involved infrastructure, 30% computers and musical instruments, 14% textbooks and clothes, and only 2.6% involved teacher training; An educational management information system to collect on school indicators was piloted in 30 schools. Its design was finalized, and the installation in municipalities were completed . It was in the process of installation for all schools after project completion . Since there were no census data however, gross and net enrollment rates could not be calculated; Improvements by ethnic minorities were noted in the survival rates of grades (e.g. from 111-154% in grades 1, 3, and 5) but deterioration in others, e.g up to -20% in grades 2, or 4. These mixed results might be due partially to the high mobility of students/families in the post-conflict environment and the reliability of data . The dropout rate among ethnic minorities (Roma and others) was reduced by 43% from 2002/03 (15%) to 2004/05 (8%), exceeding the target of 25% reductions; and The girls’ survival rate between grades in secondary education in participating schools was expected to increase by 30%, but instead was reduced from 92% in 2002/03 to 2003/04 to 82% in 2002/04 to 2003/05. Migration due to the conflict may have been one reason . 5. Efficiency (not applicable to DPLs): The ICR does not offer evidence, but it states that small grants to schools were a relatively efficient means of improving school infrastructure and environment . Fences were built to protect students, and some transportation arrangements were made. A 5% contribution by the communities was required, and communities generally raised the funds. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: School development plans were successful, and the number developed exceeded the original target by more than 70%. According to external evaluation, the implementation resulted in various achievements including increased public awareness and community participation in education reform . According to a beneficiary assessment, participants and beneficiaries believed that the project made an important contribution to the education system . Most of the stakeholders also believed that enrolment, attendance and completion rates increased whereas drop -out rate decreased in the schools that received grants . However, ethnic minorities and girls did not benefit from the project to the extent expected. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Although the political status of Kosovo is not likely to affect the institutional development that has resulted from the school grants, uncertainties are likely to affect population movements and the willingness to send girls to schools . The financing of the educational system will continue to remain heavily dependent on donor financing . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: The Bank helped design a simple and focused project when the government ’s institutional capacity was still weak. It was supervised every 2-4 months, and problems that arose were quickly resolved . However, targets were initially ambitious (and adjusted during the mid-term review), and weaknesses in the management information system made it difficult to monitor and assess progress in some key areas . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: At project effectiveness, the provisional government was still developing . Organizational conflicts arose between different departments, and capacity to carry out civil works was limited . Thus, implementation initially stagnated due to a lack of leadership and clear division of responsibilities . After the midterm review, the government demonstrated stronger leadership and ownership . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: A monitoring and evaluation design was put in place at appraisal that initially was vague . Monitoring indicators were clarified at midterm review. However, some were not realistic, such as the calculation of gross enrollment data without a census ( although a tentative rate could have been calculated after the 2000 Kosovo Living Standards Measurement Survey.) The country had limited implementation capacity and the management information system could only be partly implemented. One problem is that attendance and enrollment were not carefully monitored at the school level, but no specific training was given for that purpose . The ICR states that the monitoring and evaluation system was based on a more qualitative approach, but details are not given . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): There were no financial management issues . Management was handled satisfactorily and the financial monitoring reports were prepared and submitted on time; and The project design called for community members to participate in school management activities and to contribute financially. Although financial contributions exceeded expectations, community members had limited interest in school management. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Post-conflict reconstruction can take place satisfactorily if organizational conflicts are minimized and roles clarified from the start. A simple and focused project design is appropriate for a post -conflict context, given the frequently weak government capacity; and Community participation is instrumental in rebuilding the education system in a post -conflict environment. However, communities that are not in extreme poverty may find it easier to participate financially than to devote time and help manage schools . When there is no tradition in involving communities in school management, it may be hard to convince parents to spend time in school -related activities. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR describes the project events satisfactorily . However, it does not clarify the meaning of dropout and survival targets. Does "25 percent reduction in 75% of schools" mean an average 25% reduction across the best performing 3/4 of schools, or a minimum of 25% in at least 3/4 of schools? It is unclear. Although targets were exceeded in this project, in other cases such an ambiguity would be important . a.Quality of ICR Rating : Satisfactory