ICRR 12835 Report Number : ICRR12835 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/10/2008 PROJ ID : P043255 Appraisal Actual Project Name : RY Basic Education US$M ): Project Costs (US$M): 62.6 57.3 Expansion Project Country : Yemen Loan /Credit (US$M): Loan/ US$M ): 56.0 65.6 Sector Board : ED US$M): Cofinancing (US$M ): Sector (s): Primary education (76%) Tertiary education (10%) Central government administration (10%) Other social services (4%) Theme (s): Education for all (25% - P) Gender (25% - P) Rural services and infrastructure (24% - P) Social analysis and monitoring (13% - S) Participation and civic engagement (13% - S) L/C Number : C3422 Board Approval Date : 10/17/2000 Partners involved : Closing Date : 06/30/2006 06/30/2007 Evaluator : Panel Reviewer : Group Manager : Group : Helen Abadzi Kris Hallberg Soniya Carvalho IEGSG 2. Project Objectives and Components: a. Objectives: The development objective of the Basic Education Expansion Project (BEEP) was to increase the grade 1-6 enrollment of rural children, especially girls, in basic education of adequate quality . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (a) Expansion of Access (US$38.90m at appraisal, US$48.3m actual) for school construction, extension, education and rehabilitation and school maintenance . (b) Quality Improvement (US$9.7m at appraisal, US$11.20m actual) for improving teacher skills and deployment, supplying furniture and teaching /learning materials to schools (c) Capacity Building (US$7.10m at appraisal, US$7.8m actual) for community participation, planning, monitoring and evaluation in the Ministry of Education as well as project management . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: After extension of one year, the project was closed on June 30, 2007. The exchange rate between SDRs and US dollars increased the loan funds available . As a result, reallocation among various categories changed repeatedly, and 125 additional schools were built. Ultimately, all funds were disbursed (ICR p. 3). 3. Relevance of Objectives & Design: The overall relevance of Project objectives is substantial, both at the time of preparation and at completion . As stated in the 2006 Country Assistance Strategy (CAS), about 42% of the population of 19.2 million still live below the national poverty line, most of them (83%) in rural areas, and Yemen ranked 151st out of 177 countries on the 2005 Human Development Index. Despite significant improvements in basic education indicators, many areas of concern remain without access to basic education for girls, and 70% of women are illiterate. Efforts to simplify design have compromised some aspects of the project . Risks related to the implementation of the maintenance grant mechanisms were underestimated, and there were no real outcome indicators of quality . 4. Achievement of Objectives (Efficacy): Increasing the grade 1-6 enrollment of rural children, especially girls, in basic education of adequate quality (substantial). - Expansion of Access : 2,961 additional classrooms have been constructed and made available to school children in the grades 1-6. The large part of these classrooms have been provided through the construction of 497 new schools with 3 and 6 classrooms. All new constructed schools were adequately equipped . The number of enrolled children in the surveyed schools has increased by 44,949 or about 80.7%. The number of boys enrolled in basic education has increased by 26,376 (70.8%) and number of girls enrolled in basic education has increased by 18,799 (99.8%). - Girls' share in total enrolment of school children has been increased by more than the targeted figure of 60%. The increase realized for girls share by the end of the project as average for all five governorates was 70%. - New teaching/learning methodologies were introduced with the revised education curricula for grades 1 – 6, and extensive in-service teachers training has taken place . Teachers were deployed to rural areas, and efforts were made to link remuneration with work performance . (However, there are no indicators measuring these achievements). - Community participation was activated on different activities such as identifying type of school required, selecting the school site, supervising the construction process . About 196 parental councils were established that contributed to schools in cash and in kind . 5. Efficiency (not applicable to DPLs): -Overall efficiency is modest. Civil works, which accounted for 70% of project expenditures) were constructed with modest efficiency. Reviews of earlier projects and a 2004 IEG project performance assessment review had raised the issues of high unit costs and mismanagement potential . The project under review started with a unit cost of US$138 in 2001 and increased it to US$154 in 2002. Subsequently it adopted cost -effective school designs and decentralized bidding procedures which lowered unit costs per square meter (US$151 in 2003). By comparison, unit costs of the Social Fund for Development were US$ 158 in 2003, but those of the Department of Public Works (PWP) were slightly lower, US$147. The average unit cost of classroom construction was US$ 13,700, which is relatively high by international standards . In comments on the draft ICR Review, the Region noted that the slightly higher unit cost of BEEP projects compared to PWP costs are explained by the higher percentage of BEEP projects located in rural areas where construction is more expensive due to difficult access, and the fact that PWP schools have fewer facilities than PWP schools. - The number of teachers who benefited from in -service training surpassed targets, though the unit cost per trainee was 20% lower than expected. However, the ICR does not give the reasons for reduced expenditures . There are few other indicators to suggest the efficiency of delivering instruction to students (e.g. the percentage of officially available time students were engaged in instruction ). ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project contributed significantly to the provision of appropriate access and improving the learning environments for grades 1-6 students especially in the remote rural areas through building new schools, extending and rehabilitating existing schools . - The overall enrollment of school children has increased by more than the targeted figure 30%; the increase by the end of the project on average for the five supported governorates was 48%. The gross enrollment rate of girls in grades 1-6 has not only reached the target (54%) but exceeded the target in all five targeted governorates; field survey has indicated that the average increase is 78%, based on population projections . The gender gap was reduced in the five governorates as Gender Parity Index improved globally from 0.39 in 1999 to 0.72 in 2006. - The project directly addressed the CAS objective of creation of an early stream of social benefits for the poor by increasing enrollment in basic education (grades 1-6) in 70 underserved rural districts. According to the beneficiary survey, 35 to 50% of the families in the communities covered by the project were considered as poor . -As a result of the project, community participation improved in school identification, construction, maintenance and management. Around 70% of schools surveyed have established Father or Mother Councils, and 83% of the councils were actually in operation at the time of an evaluation survey . The beneficiary communities became involved in the school construction and rehabilitation activities . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome is rated moderate . The government is committed to increased school access and quality for all. But project outcomes are at risk from limited maintenance of civil works, difficulties in assigning teachers to rural areas, and uncertainty regarding the continuing operation of parents' councils . Thus it is possible that the newly established schools will not operate in full capacity in the future . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Overall, Bank performance was satisfactory . During appraisal, lessons from earlier operations were taken into account, and the project components were kept simple . However, the monitoring and evaluation was rudimentary, and quality indicators were not included (although available). Supervision essentially was continuous. The task manager was based in the Country Office; hence was available to provide just -in–time advice to the Borrower. Residence in the country office has also helped to strengthen policy and technical collaboration with other donors . However, the Bank did not sufficiently focus on the quality of education provided in the schools and the basic skills of students . The ICR says very little regarding the extent to which students acquired basic skills. at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Highly Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Borrower performance is rated satisfactory . Overall, the government maintained a high financial commitment to education. Over the past decade, the overall allocation to the education sector has oscillated between 5 and 7% of GDP (5.4% in 2005). The abolition of school fees for girls (grades 1-6) and boys (grades 1-3) confirmed its commitment to achieving the MDGs of universal primary enrolment and of gender parity . During implementation, the Ministry of Education directly implemented the project under the overall guidance and oversight of a Steering Committee, while a credit administration unit played only a nominal role . The Steering Committee approved annual plans and studies in a timely manner . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: - The project had only a rudimentary monitoring plan . It supported annual educational surveys and a comprehensive school survey that were used to estimate outcome performance indicators . An evaluation of the teacher training activities was conducted, but the ICR does not mention its outcomes . The project included no monitoring indicators of capacity improvement . The ICR does not offer indicators of actual attendance, completion, or transition rates. -The ICR states that indicators for the quality component were not developed because the government had no capacity. However, completion and transition rates might have been used as proxy quality indicators . In addition, baseline data from an existing UNESCO study could have been updated, with a focus on governorates covered under the project, in order to assess the project's impact on learning outcomes (UNESCO/ERDC 2000, "Reasons for the Students' Level in the Primary School in Reading and Writing from the Perceptions of Teachers and Supervisors ", Sana'a, Yemen). - The ICR cites no evidence that monitoring and evaluation findings were used in the decision processes of this project or for next projects. The limited amount of monitoring information may have impeded such decisions . In comments on the draft ICR Review, the Region noted that Annual Education Surveys were used during supervision to assess progress toward the achievement of PDOs . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): No special issues listed in this section . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: - In countries or areas where implementation capacity is limited, a single development objective and a simple project design can achieve substantial changes in development trends . Under such circumstances, it may be wise to limit the project’s scope to a small number of provinces . -Community involvement in the definition of educational needs is important in ensuring the enrollment of poor students. However, clear research evidence is needed to understand better how (or if) parental involvement improves learning outcomes. - Decentralization of task managers to country offices facilitates rapid response to project -related problems as well coordination with donors. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: Overall, ICR quality is satisfactory but with some shortcomings Indicators of educational quality and learning outcomes are not provided. Detailed data are given about certain aspects of community participation, but little evidence is provided to establish a link between these activities and improved instructional delivery . a.Quality of ICR Rating : Satisfactory