IEG Report Number: ICRR14707 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 03/31/2015 Country: Guinea Project ID: P050046 Appraisal Actual Project Name: Education For All Project Costs (US$M): 420.1 420.1 Project L/C Number: Loan/Credit (US$M): 70.0 92.2 Sector Board: Education Cofinancing (US$M): 350.1 n/a Cofinanciers: GTZ, AfDB, EC, AFD, Board Approval Date : 07/24/2001 USAID, Islamic Dev Closing Date: 06/30/2005 12/31/2013 Bank, JICA, KfW, OPEC Fund, UNICEF, UNESCO Sector(s): Primary education (40%); Tertiary education (26%); General education sector (15%); Sub-national government administration (13%); Vocational training (6%) Theme(s): Rural services and infrastructure (23%); HIV/AIDS (22%); Gender (22%); Education for all (22%); Decentralization (11%) Prepared by: Reviewed by: ICR Review Group: Coordinator: Judith Hahn Gaubatz Judyth L. Twigg Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: The project underwent multiple restructurings over the course of the project period, including revisions to the project objectives and/or key associated outcome targets. The main thrust of the project (overarching goals and project design) remained essentially the same throughout the restructurings; however, for the sake of clarity, this review separates the project into four distinct periods: Project Period 1 (2001-2005) According to the Project Appraisal Document (PAD, page 3), the project objective was as follows:  To establish a sustainable framework for good quality universal basic education . The Development Credit Agreement (DCA, Schedule 2) states the objective as follows:  To assist the Borrower in its efforts to implement the first phase of its Program , through the establishment of a sustainable framework for reaching high quality universal primary education in its territory . Note: The DCA statement of objectives refers to the Borrower's three-phase education reform plan ("Program"), which was to be supported by the Bank through a three-phase Adaptable Program Loan (APL). In 2008, at the time of Additional Financing, a decision was made to discontinue the support for the APL. Future assistance would be provided through a stand-alone investment operation and donor harmonization efforts would continue through a sector wide approach. For the purposes of this review, the project objective is considered as three separate, specific objectives:  (i) To establish a sustainable framework for education  (ii) To reach universal primary /basic education  (iii) To reach high quality of primary /basic education Project Period 2 (2005-2008) In 2005, key associated outcome targets were revised for specific objectives (ii) and (iii). A Restructuring/Project Paper was not required at the time, but a memorandum identifying the revisions was cleared by the Country Director (dated 6/29/2005). The project objective was not changed. Project Period 3 (2008-2011) In 2008, the project objective was formally revised as follows (Financing Agreement, 2008, page 5):  To assist the Recipient in its efforts to improve conditions for teaching and learning in its territory , through expanded access to education , enhanced quality of learning , and strengthened capacity for decentralized management of the sector . For the purposes of this review, the project objective is again considered as three separate, specific objectives:  (i) To expand access to education  (ii) To enhance quality of learning  (iii) To strengthen capacity for decentralized management of the sector Project Period 4 (2011-2013) In 2011, the objective was formally revised again (Restructuring Paper, 2011, page 5) such that specific objective (iii) was as follows:  (iii) To strengthen human resource management *** Each of the Project Periods 1-4 are assessed separately, and a combined weighted outcome rating is determined according to the amount of the Credit that disbursed during each of the four periods. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval: 06/29/2005 c. Components: The appraised and actual project costs reported below reflect IDA financing only. Actual costs for the total project were not reported in the ICR, due to difficulties in tracking donor contributions. Project Period 1 (2001-2005) Actual: US$29.6 million (32.1% of total IDA financing) 1. Improving Access to Education (Appraisal: US$ 19.7 million; Actual: US$ 28.1 million): This component aimed to ensure that all children (born in 1997 and later) complete primary schooling, with a focus on under-served regions and multi-grade programs. Activities included: construction of 1500 primary school classrooms; and renovation of teacher training institutions. 2. Improving the Quality of Schooling (Appraisal: US$ 41.2 million; Actual: US$ 43.3 million): This component aimed to ensure that all children receive a "relevant and meaningful" education, with a focus on establishing a framework for more effective support of classroom personnel. Activities for the early childhood level included pilot testing of early childhood development programs. Activities for the primary level included school improvement grants for primary schools; teacher training curriculum reform; general education curriculum reform; provision of textbooks; provision of health education, clean water supplies and low-cost health and nutrition services in schools; and school mapping to support the adoption of multi-grade teaching. Activities for the secondary level included pilot testing of school improvement grants for secondary schools; pilot testing of science and math teaching materials; and pilot testing of an adult basic literacy approach. Activities for vocational and professional training included pilot testing of a dual-vocational training approach; and pre- and in-service training for primary and secondary school teachers. Activities for higher education included construction of a central university library; review and reform of all university programs of study; provision of research and innovation grants; and upgrading qualifications of university faculty. 3. Strengthening Capacity for Decentralized Management (Appraisal: US$ 9.1 million; Actual: US$ 11.1 million): This component aimed to ensure that sufficient local resources and capacity are available to carry out the sector reform efforts. Support for the central and national institutions would aim to shift their roles to setting policies and establishing effective monitoring systems, while support for local institutions would aim to develop their new implementation responsibilities. Activities included development of education information systems; development of financial management systems; and construction of a central ministry administrative building (for the Ministries of Education, Higher Education, and Vocational/Technical Education). Project Period 2 (2005-2008) Actual: US$ 50.1 million (54.3% of total IDA financing) According to the ICR (page 10 and Annex 10), the following revisions were made to the project design:  Component 1: Construction of latrines and wells in primary schools was dropped.  Component 3: The activities to construct a central university library and a central ministry administrative building were dropped, as they were considered less important to the achievement of the project objectives. Project resources were re-directed towards the renovation and upgrading of seven existing faculty libraries, and increased outputs in textbook provision, secondary school and teacher training institution construction, teacher training, higher education curriculum reform, and school health.  Key associated outcome targets: Targets were revised for the following indicators: percent of girls enrolled in primary school; completion rate for primary school; primary level textbook:pupil ratio; and average class size in primary schools. Project Period 3 (2008-2011) Actual: US$ 7.5 million (8.1% of total IDA financing) According to the Project Paper (2008), Additional Financing (AF) of US$ 10.0 million was approved primarily as an expedient way to support the government through a fiscal crisis. As project implementation progress was considered satisfactory at the time, this project was selected from the country portfolio to provide supplemental financing to the government during the ongoing food and oil crisis.  Project objective: Although the project's goal remained essentially the same, the statement of objectives was clarified as follows: To assist the Recipient in its efforts to improve conditions for teaching and learning in its territory , through expanded access to education , enhanced quality of learning , and strengthened capacity for decentralized management of the sector . The specific focus on primary/basic education was dropped, in order to increase the relevance of project activities in non-primary education.  Component 3 (AF Appraisal: US$10.0 million; AF Actual: US$ 9.6 million): The additional activities in this component aimed to sustain project achievements in improving quality of teaching personnel. Activities included the provision of teacher remuneration and incentives for the months of November and December 2008.  Key associated outcome targets: Targets were again revised for the following indicators: percent of girls enrolled in primary school; completion rate for primary school; and average class size in primary schools.  Implementing Agency: The primary implementing agency role was changed from the three education ministries (Education, Higher Education, and Vocational/Technical Education) to the Ministry of Economy and Finance, given the main project activity to fund salaries. Project Period 4 (2011-2013) Actual: US$ 5.1 million (5.5% of total IDA financing) Shortly after approval of the Additional Financing, a coup d'etat took place in the country. During this period of unrest, the government fell into arrears and almost all donor activity (including the Bank) was suspended for two years. Bank engagement resumed in 2011, and the newly elected government requested that the remaining AF funds (US$ 5.5 million) be used for improving human resource management in the education sector.  Project objective: The project objective was revised such that specific objective (iii) was as follows: "to strengthen human resource management."  Component 3: The payments for teacher salaries and incentives for the month of November 2008 had been completed, but these payments were dropped for the second month. The new project activities included support for enforcement of legal texts; review of recruitment and deployment policies; and implementation of a data information system and merit/performance-based incentive system.  Implementing Agency: The primary implementing agency role was returned to the three education Ministries. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project cost:  The appraised and actual (as reported by the project team) project cost was US$ 420.1 million. Financing:  The project was to be financed in part by an IDA Credit of US$ 70.0 million. The actual Credit disbursed was US$ 82.6 million. The difference was due to exchange rate fluctuations.  The project was also be financed through significant funding from donors (US$ 350.1 million). The appraised co-financing amounts included the following: African Development Bank (US$ 0.64 million); European Commission (US$ 10.4 million); Agence Francaise de Developpement (US$ 8.1 million); Government of Germany (US$ 1.6 million); USAID (US$ 21.1 million); HIPC Fund (US$ 34.75 million); Islamic Development Bank (US$ 3.4 million); JICA (US$ 4.9 million); KfW (US$ 5.9 million); OPEC Fund (US$ 1.7 million). The ICR does not report on the actual contributions from donors, due to difficulties in tracking through the extended project period.  Additional Financing (AF) was approved in 2008 in the form of a US$ 10.0 million IDA grant. The actual AF grant disbursed was US$ 9.6 million. Borrower contribution :  The planned Borrower contribution was US$ 232.1 million. The ICR does not report whether this contribution was provided in full, although it does refer to persistent delays in providing counterpart funding. According to the project team, adequate counterpart funding was provided through 2008, after which the Additional Financing was approved and no further counterpart funding was required. Dates:  June 2005: The project's closing date was extended from December 2005 to December 2007. The closing date was later extended again from December 2007 to December 2008 (no date provided for approval of project extension).  October 2008: Additional Financing (AF) was approved in the amount of US$ 10.0 million. Although the project's goal remained essentially the same, the statement of objectives was clarified in a formal revision. The closing date was extended from December 2008 to December 2009, in tandem with the closing date of the Additional Financing grant.  December 2008 - November 2009: A coup d'etat took place in the country which led to the suspension of project disbursements and, eventually, all Bank operations.  December 2009: The project closed according to schedule. (After the Bank re-engaged, the project closing date was retroactively extended).  September 2011: The Bank opened dialogue with the newly elected government and resumed operations after the government cleared its arrears. One of the three project objectives was revised to sharpen focus on human resource management issues. The project closing date (including for the Additional Financing) was retroactively extended to June 2013. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Project Period 1 and 2: High At the time of project appraisal, the country was facing critical challenges in the education sector. Access to primary education was low, with a gross primary school enrollment rate of 61% (2000). There were also significant disparities in access among boys and girls, and among rural and urban areas. Educational outcomes were also poor, with less than half of primary level students achieving minimal competency levels in French and math. Direct and indirect costs for schooling served as a significant barrier to access for poor households. Sectoral inefficiences due to weak management and lack of financing were also prominent issues. In response to these challenges, the government prepared an "Education For All" sector reform program (2001-2013), of which the main objective was to ensure equitable access to a high quality education system that is relevant for their communities and for their economic well-being. The Bank's Country Assistance Strategy (1997-2000) at the time of appraisal identified improved service delivery for poverty reduction, including in basic education, as a priority. The current Country Partnership Strategy (2014-2017) continues to prioritize access to basic services. Project Period 3: High As noted in the Project Paper (2008, page 1), the project was restructured in part to "better align the project development objective with project activities, including those under the Additional Financing." The original project objective was more explicitly stated, including removing the reference to primary education and therefore broadening the relevant scope of activities beyond primary education. The re-stated objectives remained highly relevant to country and Bank strategies at the time of restructuring and completion. Project Period 4: High The final project restructuring revised the third objective to focus specifically on human resource management in the education sector. This reflected the newly elected government's adjusted priorities to address inefficiencies in pubic spending alongside reforming personnel management in education. b. Relevance of Design: Project Period 1: Modest The majority of project interventions were likely to contribute to the intended outcomes, including construction of classrooms, teacher training and provision of textbooks. The project design identifies a number of key policy reforms to establish a sustainable policy framework for the education sector, including the abolition of school fees, and decentralization of school management and teacher training. New initiatives to improve quality of education were also to be pilot tested, including school improvement grants, literacy programs, and early childhood development centers. However, there was not a systematic plan to evaluate the pilots to inform decisions for scaling up. Another shortcoming in the project design is the broad range of project activities (encompassing early childhood, higher education, vocational education, and adult literacy) that were not directly relevant to the objectives as articulated in project documents, which focus on primary/basic education. Project Period 2: Substantial According to the project documents (Country Director Memorandum, 2005, page 2), the project restructuring was undertaken in part to reduce the scope of project activities by eliminating "non-essential activities which are not closely linked to development objectives or less relevant in the present country context." Project resources were therefore shifted to activities that were more directly relevant to improving primary education quality. Project Period 3: Substantial The activities funded by the Additional Financing grant included teacher remuneration and incentives during the months of November and December 2008. These project interventions were intended to sustain project progress in improving access and quality of education, alongside efforts by the government to strengthen decentralized management of the education sector. Project Period 4: Substantial The revised project activities included more effective management of teacher recruitment and deployment, and implementation of information systems. These activities were likely to contribute to achievement of the revised objective to strengthen human resource management. 4. Achievement of Objectives (Efficacy): To establish a sustainable framework for education Project Period 1 and 2: Modest Outputs  Technical assistance to central ministries in the areas of teacher personnel management, planning and budgeting, policy monitoring, and financial management.  Conducting of national assessments to evaluate learning outcomes in primary education, including a 2005 longitudinal study, school evaluation for grades 2, 4 and 6, and three assessments of grade 4 learning outcomes. Outcomes  The policy of charging school fees for primary/secondary schools was abolished.  Recruitment of teachers was decentralized to the sub-regional level (even though final authorization remained the responsibility of the central ministries).  Pre-service teacher training was decentralized to regional teacher training institutions.  Initiatives on school improvement grant programs and reading programs were pilot tested. However, the following policy reforms were not adopted/enforced:  School-level autonomy in textbook selection.  Adoption of other national languages (in addition to French) as languages of instruction.  Lengthening of the school year. Also, a significant lack of public financing, including for non-salary expenditures, hampered efforts to decentralize budget planning and execution to local levels. To achieve universal primary education / To expand access to primary education There was progress made towards achieving universal primary education over the project period; however, there were shortfalls in achieving some targets. Project Period 1: Modest Project Period 2 (revised targets): Modest Project Period 3 and 4 (revised targets): Substantial Outputs  Construction of 1,120 primary school classrooms. This fell short of the target of 1,500.  Recruitment and training of 8,652 primary school teachers. This surpassed the target of 6,000. The ICR (page 42) reports that an independent evaluation showed that the level of proficiency of the project-trained teachers was "superior" to the level of teachers trained before project intervention. Outcomes  Initial enrollment/intake rate for Grade 1 increased from 70% in 2008 to 85% in 2013. This fell short of the Project Period 1 and 2 target of 90%, but achieved the Project Period 3 and 4 target of 83%.  Gross enrollment rate for girls increased from 51% in 2000/01 to 75% in 2013. This achieved the Project Period 1 target of 59%, the Project Period 2 target of 75%, and the Project Period 3 and 4 target of 72%. To achieve high quality primary education / To enhance the quality of learning There was evidence of improved learning outcomes and improved aspects of the learning environment.. Project Period 1: Substantial Project Period 2 (revised targets): Substantial Project Period 3 and 4 (revised targets): Substantial Outputs Primary schools  Training of 8,652 primary school teachers. This surpassed the target of 6,000. The ICR (page 42) reports findings of an independent evaluation in which the level of proficiency of the project-trained teachers was "superior" to the level of teachers trained before project intervention.  Provision of school improvement grants to 3,600 primary schools. According to the project team, these grants were largely used for provision of school supplies that were formerly funded through school fees (now abolished).  Provision of 8.7 million textbooks. There was no target specified. However, the ICR (page 42) reports that a 2006 textbook audit found several weaknesses in the textbook distribution system that resulted in "considerable loss." A plan to address the system weaknesses was prepared and the project continued to finance textbooks through 2008.  Payment of teacher salaries and incentives (in November 2008).  Provision of micro-nutrient supplements and de-worming medicines for school health programs.  Conducting of pilot reading program in primary schools. Professional development  Construction/renovation of five teacher training institutions.  Rehabilitation of seven university libraries.  Development of multi-grade teaching strategy and pedagogical materials.  Review of 50 university programs as part of higher education curriculum reform efforts.  Training of 400 secondary school teachers in science modules.  Provision of 114 research and innovation grants to university staff. Other  Construction of 70 early childhood education (ECD) community centers. This achieved the target of 70. The project also supported the implementation of a national ECD strategy and training of ECD teachers.  Establishment of adult basic literacy programs, implemented by 43 non-governmental organizations. These programs served approximately 35,000 beneficiaries. Outcomes Learning outcomes  Grade 4 standardized test scores for French increased from 38% in 2000/01 to 41.2% in 2013. This fell slightly short of the target of 42% (for all Project Periods). Similarly, 47.6% of students in Grade 4 were able to read in French at a satisfactory level, assessed according to international standards. This fell short of the target of 50%.  There were no data on math scores, as this indicator was dropped in 2005. Learning environment  The primary school completion rate increased from 36% in 2005 to 59% in 2013. This achieved the Project Period 2 target of 58%, but fell short of the Project Period 3 and 4 target of 63%.  The textbook:student ratio at primary schools increased from 1:1 in 2000/01 to 3:1 in 2013. This achieved the Project Period 1 target of 1.5:1 and Project Period 2-4 target of 3:1.  The average class size in primary schools decreased from 55 in 2005 to 46 in 2013. This achieved the Project Period 1 target of 48, and the Project 2-4 target of 50. To strengthen capacity for decentralized management Project Period 3: Modest Outputs  Technical assistance for decentralization of planning, budgeting and data collection processes.  Establishment of planning and statistical unit in each education ministry. Each unit was able to prepare annual statistics books by the end of the project period.  Introduction of medium-term expenditure framework, but with limited progress in its implementation. Outcomes  38 prefectures prepared and implemented decentralized development plans, surpassing the target of 10. Although budgeting and resource management are increasingly being transferred to the decentralized units, the ICR (page 21) notes that "a lack of public financing hampered efforts to decentralize the budget execution which undermined the actual devolution of planning and budgeting functions to the local level.”  According to the ICR (page 27), "capacity building of [decentralized] personnel resulted in more timely availability of data as well as greater use of the data for local planning and management." There is no specific evidence cited. To strengthen human resource management Project Period 4: Substantial Outputs  Development of a teacher transfer and appointment scale.  Conducting of three audits of contract teachers, in order to remove “ghost” teachers from the payroll.  Conducting of assessment on the teacher merit and performance system.  Initial development of a regional personnel database. The database was reviewed and finalized, but not yet implemented.  Study tours for human resources staff. Outcomes  Recruitment of teachers was decentralized to the sub-regional level (even though final authorization was still granted at the central Ministry level).  100% of teachers were deployed using the new personnel movement approach, surpassing the target of 95%. 5. Efficiency: Efficiency is rated Modest. The PAD (Annex 4) provides an extensive economic analysis; however, most of the information pertains to an analysis of the Sector Expenditure Plan (2001-2013) of the entire education sector rather than the project specifically. The ICR provides the following data on the efficiency of project resources: For the construction of primary school classrooms, the cost per classroom built was estimated at US$ 11,300. Although this cost is 26 percent higher than the appraised estimate of US$ 9,000, it is consistent with, or even more cost-effective than, costs for another country in the same time period (Ghana: US$ 16,651). For provision of primary school textbooks, the average cost per textbook was estimated at US$ 1.4. This was consistent with the range of unit costs for other countries in the same time period (Cote d'Ivoire, Burundi, Niger: US$ 1.1 to US$ 1.4). However, the ICR (page 42) reports that a 2006 textbook audit found several weaknesses in the textbook distribution system that resulted in "considerable loss." However, the ICR does not provide the total costs for these two activities, in proportion to total project costs. Also, according to the project team, enhanced donor coordination was a positive aspect of project implementation. While this could have contributed to efficient use of project resources, there is not enough information provided in the ICR for clear validation. In addition, there were significant implementation delays, including an extended period of suspended Bank activity, throughout the project period that likely diminished efficiency. a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Project Period 1 (2001-2005): Moderately Unsatisfactory Relevance of the project objectives is rated High, while relevance of the project design is rated Modest due to a significant number of activities that addressed non-basic education (i.e. early childhood education, higher education, vocational and technical education) although the project objectives centered on primary education. Achievement of the objective to establish a sustainable framework for education is rated Modest due to shortcomings in adopting critical policy reforms. Achievement of the objective to reach universal primary education is rated Modest due to some progress in improving Grade 1 enrollment rates but shortfalls in achieving targets. Achievement of the objective to reach high quality basic education is rated Substantial due to evidence of improved learning outcomes as well as improved aspects of the learning environment. Efficiency is rated Modest. Project Period 2 (2005-2008): Moderately Unsatisfactory Ratings are same as above for Project Period 1 except for the following: Relevance of the project design is rated Substantial due to a reduced scope of activities to focus more on achieving outcomes in primary education. Project Period 3 (2008-2011): Moderately Satisfactory Relevance of the project objectives is rated High, while relevance of the project design is rated Substantial. Achievement of the objective to expand access to education is rated Substantial due to achievements in increasing Grade 1 enrollment rates, including reaching targets. Achievement of the objective to enhance quality of learning is rated Substantial due to evidence of improved learning outcomes, as well as improved aspects of the learning environment. Achievement of the objective to strengthen capacity for decentralized management of the sector is rated Modest due to lack of sufficient evidence. Efficiency is rated Modest. Project Period 4 (2011-2013): Moderately Satisfactory Ratings are same as above for Project Period 3 except for the following: Achievement of the objective to strengthen human resource management is rated Substantial due to effective implementation of activities to support this objective. According to OPCS/IEG harmonized guidelines, the final outcome rating of a restructured project is determined by weighting the outcome for each project period according to the proportion of the Credit/Grant that disbursed during that period. Project Period 1 is rated Moderately Unsatisfactory (32.1% disbursed); Project Period 2 is rated Moderately Unsatisfactory (54.3%); Project Period 3 is rated Moderately Satisfactory (8.1%); and Project Period 4 is rated Moderately Satisfactory (5.5%). The weighted final outcome rating is Moderately Unsatisfactory. a. Outcome Rating: Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The current government is likely to sustain its commitment to the policy reforms supported by the project. However, the ability of the government to finance recurrent costs for the education sector, including those financed through this project, is uncertain, as the share of public funding for education is likely to decrease. The recent Ebola outbreak is also expected to negatively impact the country's economy and growth. Some of the activities supported by the project - school grants, teacher training, provision of textbooks - will continue to be supported by a Global Program on Education (GPE) project, although the Bank itself is not planning a follow-up basic education project. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The original project design was complex, addressing multiple levels of education from early childhood to higher education, with activities financed by multiple major donors and implemented by various institutions. Although the design was intentionally aimed at strengthening coordination among donors, as well as capacity of the government to lead and manage these efforts, even a minimal level of project implementation capacity did not exist among the three education ministries. The ICR (page 11) notes lessons from the previous Bank-financed education project (Sector Adjustment Program of Education - PASE 2) that were not sufficiently incorporated into the project design, including weaknesses in contract management (for school construction) and weaknesses in the textbook distribution system. These continued to pose challenges throughout the project period. The ICR (page 12) also notes that "a number of preparatory activities that were required for smooth project implementation from the beginning (development of operational manuals, procurement plans, and reporting formats) had not been completed by project effectiveness." The risk assessment underestimated the overall risk rating as “Modest,” with various government capacity issues rated only Modest. The M&E design was overall satisfactory, with relevant and measurable indicators capturing both outputs and outcomes. Quality-at-Entry Rating: Moderately Unsatisfactory b. Quality of supervision: The first two years of project implementation saw only 13 percent of the Credit disbursed; the ICR (page 12) notes that this was due to the lack of readiness, frequent turnover in task team leadership (five different TTLs in three years), and high staff turnover in counterpart staff. The Mid-Term Review (October 2004) offered the opportunity to restructure the project by reducing the scope of activities, and the pace of implementation improved considerably, alongside increased Bank staff continuity. The disbursement rate increased, and project funds were committed to key activities including construction of classrooms, provision of textbooks, rehabilitation of teacher training institutions, and school improvement grants. The ICR (page 31) notes a 2006 Quality of Supervision Assessment finding that there were discrepancies between the Implementation Supervision Report ratings (which were mostly Satisfactory) and the text descriptions (which highlighted numerous challenges). In 2008, the Bank team responded to the need of the government to address a significant budget gap. The Bank team worked in an expedient manner to prepare Additional Financing, while also restructuring the project to "better align the project development objective with project activities... and to refine certain performance indicators to adjust some of the targets given more recent population estimates" (Project Paper, 2008, page 1). The ICR (page 32) also reports that the Bank team provided "more intense and targeted supervision missions which aimed to identify and address any bottlenecks which were impeding timely implementation," particularly with regards to fiduciary issues. The M&E framework was constantly revisited throughout the project period, although some key indicators and targets were not formally revised (or dropped) until late into the project period. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The ICR (page 38) notes that the early project period was marked by "limited ownership by the Government," while subsequent years of the project period were marked by various macroeconomic and fiscal management crises (leading to cessation of HIPC interim debt relief provisions and reductions in donor aid). The ICR (page 13) reports that these conditions affected the project in the following ways: education budgets fell consistently below targets and counterpart funding was delayed (which led to implementation delays); and suspension of Bank disbursements threatened the sustainability of project achievements to date. The most recently elected government in 2011 has provided a more favorable environment for reform efforts and project implementation support. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: Performance of the three main education ministries – Education, Higher Education, and Vocational and Technical Education - was overall marked by inadequate capacity. There was an initial period of high staff turnover, after which the core staff in the project coordinating unit remained sufficiently stable to facilitate capacity building and project implementation efforts. There were significant implementation delays, in large part due to procurement and contract management problems in the key activities of school construction and textbook provision. However, through intensive Bank support, these challenges were addressed such that a number of critical project activities were implemented as planned. The original project design envisioned significant donor activity in a cohesive, coordinated sector project; according to the project team, coordination with donors did materialize effectively and was a positive aspect of the project's implementation. There were no problems reported in safeguards compliance. Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The key indicators were relevant and measurable, capturing both outputs and outcomes; however, the indicators were mostly focused on the access and quality objectives, and limited for the sectoral management objectives. The M&E arrangements relied upon existing data collection systems, including the education management information system. Realistic baseline and target figures were identified and constantly revisited throughout the project period. Although project design included a number of important innovations to be tested as pilots, it is not clear whether these pilots were to be systematically assessed as part of the project design, in order to inform decisions about scaling up. b. M&E Implementation: The M&E framework was revised during the multiple project restructurings. Key indicators were clarified in their wording, targets made more realistic, and new indicators added to better capture project outcomes, Baseline and target figures were modified when updated data became available. However, as noted in the ICR (page 14), it was only at the time of the 2011 restructuring that a number of less relevant indicators were formally dropped. The ICR notes that several evaluative activities were conducted, including a textbook audit, a teacher audit, and assessments of the school grants, reading, and adult literacy programs. c. M&E Utilization: As reported in the ICR (pages 14-15), the M&E activities "did not fully incorporate/utilize evaluations of key programs and pilots to improve project performance. More specifically, while the evaluations undertaken were able to identify issues and corrective measures during implementation, these assessments were not designed to measure the impact or cost-effectiveness of the interventions on access and learning outcomes to inform decisions related to scaling-up the interventions or to inform shifts in strategy.” M&E Quality Rating: Modest 11. Other Issues a. Safeguards: The project was classified as an Environmental Category "B" project due to construction of schools and latrines, and provision of water supplies. The safeguard policy on Environmental Assessment (OP/BP 4.01) was triggered, and an Environmental and Social Impact Assessment was carried out. A safeguards audit was conducted in January 2014, which concluded that "the size and nature of infrastructure works carried out under the Project did not have any significant negative biophysical or socioeconomic impacts and did not result in the relocation of populations, or loss of livelihood or access to economic assets of populations" (ICR, page 17). b. Fiduciary Compliance: Financial management: The ICR (page 15) reports that there were no major problems reported in accounting systems use, preparation of financial statements, adherence to auditing requirements, or qualifications of accounting personnel. One rating of Unsatisfactory (2003-04) due to a change in project management staff without prior consultation was subsequently resolved. Audit reports were submitted on time and with no qualifications, with one exception in 2009: an undocumented expenditure of US$ 215,000 was noted, and although the Ministry of Economy and Finance was delayed in responding to this audit finding, it later refunded the amount in full. Procurement: Procurement performance was less than satisfactory during the initial project period due to weak procurement capacity, frequent staff turnover, and lack of readiness of procurement processes (planning, document preparation, evaluation of offers, and contract management systems). The ICR (page 16) notes that multiple changes in Bank task team leadership and insufficient technical support in the field office further contributed to the following: procurement delays, higher prices than otherwise would have been obtained, and use of inappropriate processes and methods triggering cancellation and re-launching of large procurements. However, due to intensive Bank support beginning in late 2004, procurement performance improved significantly. The Bank team provided increased procurement support and training, and closely monitored the procurement schedule, which "significantly boosted capacity, implementation, and the quality and reliability of procurement processes" (ICR, page 16). Although there continued to be weaknesses in timeliness and contract management, all contracts adhered to Bank procedures and requirements. c. Unintended Impacts (positive or negative): None reported. d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Significant Significant Outcome: Bank Performance: Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lessons drawn from the ICR (page 34-35), adapted by IEG:  Evaluative activities, including student assessments and pilot test assessments, have less impact when they are not strategically used to inform policy making and project implementation support. In the case of this project, some of these activities were conducted, but there was no systematic plan established at the outset to use the findings to inform decision-making.  During periods of intense political transitions and crisis, maintaining a minimal level of communication with the government can facilitate swift re-engagement following the crisis and bring important dividends. In the case of this project, the Bank team demonstrated a high level of commitment to supporting the education sector by maintaining interactions with the government and key stakeholders even during the period of official disengagement. Lessons drawn by IEG:  In a fragile environment, simplifying the project design and ensuring project readiness are critical factors for satisfactory quality at entry. In the case of this project, the multiple decentralized agencies and institutions involved made for an overly complex project, and the lack of prepared documents (i.e. operations manual) at project launch contributed to a slow start.  Project restructuring would need to be undertaken as soon as it is feasible to improve likelihood of success. In the case of this project, much needed restructuring(s) did not take place until late into project implementation. Although the project performed on a more satisfactory level in the latter part of the project period, the majority of Credit/Grant had already disbursed and therefore the outcome rating remained in the unsatisfactory range. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is commendable in articulating the shifting context and long implementation experience of this project. The ICR is overall consistent with guidelines and provides a detailed discussion of all outputs and results in Annex 2; however, the section on project achievements (Section 3.2) should discuss outcomes and results achieved for each project objective over the entire project period, not the outcomes and results achieved only within the time frame of each project restructuring. a.Quality of ICR Rating : Satisfactory