Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004615 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF-18789 ON A GRANT IN THE AMOUNT OF US$32 MILLION TO THE LEBANESE REPUBLIC FOR AN EMERGENCY EDUCATION SYSTEM STABILIZATION PROJECT MAY 30, 2019 Education Global Practice Middle East And North Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective Oct 9, 2018) Currency Unit = LBP Lebanese Pound 1,511.75 Lebanese US$1 Pounds = FISCAL YEAR January 1 – December 31 Regional Vice President: Ferid Belhaj Country Director: Saroj Kumar Jha Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager: Andreas Blom Task Team Leader(s): Husein Abdul-Hamid, Mohamed Yassine ICR Main Contributor: Syedah Aroob Iqbal ABBREVIATIONS AND ACRONYMS BCR Benefit to Cost Ratio CBA Cost-Benefit Analysis CERD Center for Educational Research and Development CPF Country Partnership Framework CPS Country Partnership Strategy DA Designated Account DOPS Direction d’Orientation Pédagogique et Scolaire D-RASATI Developing Rehabilitation Assistance to Schools and Teacher Improvement Program EDP2 Second Education Development Project EESSP Emergency Education System Stabilization Project EFO Externally Financed Output ESIA Economic and Social Impact Assessment ESSA Environment and Social System Assessment GOL Government of Lebanon ICR Implementation Completion and Results Report IDA International Development Association IFR Interim unaudited Financial Reports IRR Internal Rate of Return LSCTF Lebanon Syrian Crisis Multi-Donor Trust Fund M&E Monitoring and Evaluation MDTF Multi-Donor Trust Fund MEHE Ministry of Education and Higher Education MOF Ministry of Finance MTR Mid-term Review NPV Net Present Value PAD Project Appraisal Document PDO Project Development Objective PMU Program Management Unit POM Project Operations Manual RACE Reaching All Children with Education REACH Results in Education for All Children REC RACE Executive Committee R4R Research for Results SCD Systematic Country Diagnostic SDGs Sustainable Development Goals S2R2 Support to RACE II TTL Task Team Leader UK DFID United Kingdom Department for International Development UN United Nations UNHCR United Nations High Commissioner for Refugees UNICEF United Nations International Children's Emergency Fund UNRWA United Nations Relief and Works Agency for Palestine Refugees WB World Bank TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 10 II. OUTCOME .................................................................................................................... 13 A. RELEVANCE OF PDOs ............................................................................................................ 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 13 C. EFFICIENCY ........................................................................................................................... 19 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 22 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 22 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 24 A. KEY FACTORS DURING PREPARATION ................................................................................ 24 B. KEY FACTORS DURING IMPLEMENTATION.......................................................................... 25 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 26 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 26 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 28 C. BANK PERFORMANCE ........................................................................................................... 29 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 31 V. LESSONS AND RECOMMENDATIONS ............................................................................. 31 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 33 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 38 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 40 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 41 The World Bank Emergency Education System Stabilization (P152898) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P152898 Emergency Education System Stabilization Country Financing Instrument Lebanon Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Lebanese Republic, Ministry of Finance Ministry of Education and Higher Education Project Development Objective (PDO) Original PDO To support the operational needs of Lebanese public schools and to improve the learning environment in response to the continued influx of Syrian refugee children. Page 1 of 45 The World Bank Emergency Education System Stabilization (P152898) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 32,000,000 32,000,000 32,000,000 TF-18789 Total 32,000,000 32,000,000 32,000,000 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 32,000,000 32,000,000 32,000,000 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 09-Mar-2015 22-Jul-2015 09-Sep-2016 30-Nov-2018 30-Nov-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 19-Dec-2017 32.00 Change in Results Framework Change in Components and Cost KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 17-Jun-2015 Satisfactory Satisfactory 0 02 16-Dec-2015 Satisfactory Satisfactory 7.00 Page 2 of 45 The World Bank Emergency Education System Stabilization (P152898) 03 15-Jun-2016 Satisfactory Satisfactory 13.21 04 15-Dec-2016 Satisfactory Satisfactory 13.21 05 23-Jun-2017 Moderately Satisfactory Moderately Satisfactory 20.12 06 27-Dec-2017 Satisfactory Satisfactory 32.00 07 29-Jun-2018 Satisfactory Satisfactory 32.00 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Primary Education 70 Secondary Education 30 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Education 100 Access to Education 50 Education Financing 50 ADM STAFF Role At Approval At ICR Regional Vice President: Hafez M. H. Ghanem Ferid Belhaj Country Director: Ferid Belhaj Saroj Kumar Jha Senior Global Practice Director: Claudia Maria Costin Jaime Saavedra Chanduvi Practice Manager: Safaa El Tayeb El-Kogali Andreas Blom Noah Bunce Yarrow, Husein Husein Abdul-Hamid, Task Team Leader(s): Abdul-Hamid Mohamed Yassine ICR Contributing Author: Syedah Aroob Iqbal Page 3 of 45 The World Bank Emergency Education System Stabilization (P152898) Page 4 of 45 The World Bank Emergency Education System Stabilization (P152898) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. The ongoing conflict in the Syrian Arab Republic and the Syrian refugee crisis have inflicted an unprecedented socio-economic burden on Lebanon. The significant influx of Syrian refugees has raised the population of Lebanon by 25-30 percent1. In 2014, Lebanon was hosting 1.15 million Syrian refugees officially registered with the United Nations High Commissioner for Refugees (UNHCR), placing Lebanon in first place in terms of countries with the highest concentration of refugees per capita in the world – at a rate of one in six people being a refugee. This high influx of refugees severely strained public resources and services’ capacity to meet livelihood, health, and education needs of both refugees and the Lebanese people. Since 2011, GDP per capita in Lebanon has been declining and by 2014, the Syrian conflict had cost the Lebanese economy an estimated US$7.5 billion in forgone output, widened the fiscal deficit by US$2.6 billion, and contributed to pushing an additional 170,000 Lebanese people into poverty. 2. The influx of Syrian refugees had also drastically increased the number of school-age (3-18 years old) children in the country and strained the existing education infrastructure. In 2014, Lebanon was hosting 481,905 school-age (3–18 years old) Syrian refugee children, while the Lebanese public education system was small in size, serving only 237,872 Lebanese school-age children. This significant increase in the number of school-age children in Lebanon implied that the public education system needed in a very short timeframe, to more than double its capacity while maintaining the quality of education to be able to educate refugee children. Ensuring the education of Syrian school-age refugees would be vital in laying the foundation for their longer-term prosperity, while a lack of schooling would likely contribute to a life of poverty and struggle, exacerbating the risk of future conflict and destabilization in the region. The need for a systematic approach to expand education access to Syrian school-age refugees prompted the Ministry of Education and Higher Education (MEHE) to launch in 2014 an impressive 3-year program called ‘Reaching All Children with Education (RACE) in Lebanon’. The program intended to provide equitable access to quality formal and non-formal learning opportunities to an average of 413,000 Syrian refugees and vulnerable Lebanese school-age children (3-18 years) per year at an overall cost of US$634 million over the three years. 3. The Emergency Education System Stabilization Project (EESSP) was designed to support the immediate need of educating school-age Syrian refugee children in Lebanon. In 2014, the World Bank (WB) was providing support to the education sector in Lebanon through a US$40 million IBRD loan for the ‘Second Education Development Project’ (EDP2), which became effective on November 29, 2012 and closed on June 30, 2018. The EDP2 project focused on longer term strengthening of teaching quality, governance and managerial capacity of schools. However, the large influx of Syrian refugees required a faster response to the challenge of providing education to Syrian refugee children and vulnerable Lebanese children, while maintaining the quality of the learning environment in public schools. MEHE and the WB therefore designed 1 Population estimate from UN population division and UNHCR registered Syrian refugees, November 2015. Page 5 of 45 The World Bank Emergency Education System Stabilization (P152898) a targeted and dedicated emergency response project, the US$32 million ‘Emergency Education System Stabilization Project’ (EESSP), to meet the immediate needs of schools enrolling vulnerable children. The project was aligned with the Government program, supporting the implementation of components of the RACE program. 4. EESSP was designed in full alignment with the Government’s priorities and is informed by the WB’s study on the ‘Economic and Social Impact Assessment (ESIA) of the Syrian conflict in Lebanon’. The report concluded that external financing from the international community would be required to relieve the strain on public services in Lebanon due to the large influx of Syrian refugees. The report also proposed a roadmap of recommendations to mitigate the impact of the Syrian conflict on Lebanon. A Multi-Donor Trust Fund (MDTF), the Lebanon Syrian Crisis MDTF (LSCTF), was set up to finance projects under the proposed roadmap and provide an avenue of collaboration between the Government of Lebanon (GOL), the World Bank, the United Nations (UN), and donor countries. EESSP was fully financed from the Lebanon Syrian Crisis MDTF (LSCTF). 5. EESSP was also aligned with the Lebanon’s Systematic Country Diagnostic (SCD (Report No. 97498)), and the objectives of the Country Partnership Framework FY 17-22 (CPF (Report No 94768-LB)). The Country Partnership Strategy (CPS) 2011-2014 was developed in 2010 prior to the start of the crisis in Syria and therefore concentrated primarily on long-term development goals for the country. In 2013, the Progress Report of the Lebanon CPS 2011-2014 noted that the displacement of a large number of Syrian refugees into Lebanon was exacerbating Lebanon’s existing socio-economic challenges and the country’s ability to provide public services necessary for human development, including health and education. Lebanon’s SCD, also highlighted the influx of Syrian refugees as an important constraint with a large and long-term impact on the twin goals of reducing extreme poverty and promoting shared prosperity. Similarly, Objective 2c of the CPF 2017-22 highlighted the need to work with the GOL to respond to the challenges created and exacerbated by the refugee crisis. The EESSP, developed in 2014-2015, therefore responded to the emergent need for ‘mitigating the immediate, and potentially long-lasting impact of the Syrian crisis’ on the education system in Lebanon, while maintaining overall education quality in the country. Theory of Change (Results Chain) 6. The Project Development Objective (PDO) was “to support the operational needs of the Lebanese public schools and to improve the learning environments in response to the continued influx of Syrian refugee children.” The PDO can be parsed into two main objectives: (a) To support the needs of the Lebanese public schools in response to the continued influx of Syrian refugee children (b) To improve the learning environment in response to the continued influx of Syrian refugee children 7. To attain the twin objectives of the project, the project was divided into three main components: a) Support to Schools; b) Learning Environment Quality; and c) Project Management and Monitoring. The component ‘Support to Schools’ aimed to cover the operational expenses of public schools as they were enrolling more students. This support came in the form of financial transfers to the School Funds (US$100 per student) and Parent Council Funds (US$60 per student), covering the total per-student enrollment fees. With Page 6 of 45 The World Bank Emergency Education System Stabilization (P152898) this support, parents were waived from paying any fees for enrolling their children in public schools, thus decreasing the cost of education for vulnerable and refugee families. The contributions were used by the schools to cover their basic operational expenses (e.g. cleaning and upkeep, electricity, water, heating and cooling, security). The second component ‘Learning Environment Quality’ aimed to improve the quality of the learning environment through school rehabilitation, supplemental school financing, provision of textbooks, and support to the Direction d’Orientation Pédagogique et Scolaire (DOPS)2. There is established literature on the positive impact of the included project activities on the learning environment quality. Research has shown that basic infrastructure ensuring safe drinking water, functioning toilets, classroom visibility, electricity, accessibility for people with disabilities and classroom furniture have positive effects on student learning environment3. Similarly, the availability of textbooks is a key prerequisite for an effective learning environment. The Support to DOPS activities included operational support to DOPS counselors to enable them to provide pedagogical, health, and psycho-social counseling to teachers, students, and school staff, and thereby improve the quality of education environment in schools. Support from DOPS included guidance to school staff on working with highly vulnerable children, including support to instruction and pedagogy in the classroom, psychosocial support to children, support on health aspects, and guidance for working with children with special needs. Finally, the supplemental school financing activity provided school grants for targeted activities that would improve children’s sense of belonging to the school such as field trips, school activities, and financing sport, arts, and extracurricular equipment and activities. The third component, ‘Project management and monitoring’, supported the financing of six key staff positions for the RACE Program Management Unit (PMU) to allow effective implementation of the project. 2 Direction d’Orientation Pédagogique et Scolaire (DOPS) is a department comprising counselors within the Directorate General of Education which provide pedagogical, health, and psycho-social counselling to other teachers and students in the public school system. 3 (O’Reilly et al, 2008; Barde et al, 2013, Urquiola, 2006; Chingos, 2013; Jespen, 2009; Angrist and Lavy, 1999 Page 7 of 45 The World Bank Emergency Education System Stabilization (P152898) Table 1: Results Framework Project Development Objectives (PDOs) 8. The PDO was to “to support the operational needs of the Lebanese public schools and to improve the learning environments in response to the continued influx of Syrian refugee children.” Key Expected Outcomes and Outcome Indicators 9. The key expected outcomes were the following: (a) Support for the operational needs of the Lebanese public schools in response to the continued influx of Syrian refugee children, with the following PDO-level indicators: 1. Number of students benefiting from school fund support Page 8 of 45 The World Bank Emergency Education System Stabilization (P152898) (b) Improvement of quality of learning environment in response to the continued influx of Syrian refugee children, with the following PDO-level indicators: 2. Number of schools benefiting from supplemental school financing 3. Number of schools benefiting from rehabilitation (c) Overall, 4. Direct project beneficiaries (Number) (Core) 5. Percentage of females in the number of direct project beneficiaries Components 10. The project consisted of three components: Component 1 - Support to Schools (Planned - US$13.5 million; Actual – US$15.1 million) – This component financed the operational expenses of public schools by covering the enrollment fees for refugee and Lebanese children. The enrollment fees per student are divided into two funds: the “School fund” (US$100 per student) and the “Parent Council Fund” (US$60 per student). Component 2 – Learning Environment Quality (Planned - US$16.6 million, Actual - $15.8million) – This component financed activities that would improve the learning environment in public schools. It was divided into four sub-component/activities. Sub-component 2.1: School rehabilitation (Planned - US$5.2million; Actual – US$3.038 million) - The project financed the full rehabilitation of the top 10 schools in need of repairs. Rehabilitation works included structural security and support, electrical and mechanical installations for the schools, access to water and sanitation with consideration of girls’ and boys’ needs, as well as improvements to the physical appearance and provision of furniture and equipment to the schools. Sub-component 2.2: Supplemental school financing (Planned - US$5.7 million; Actual US$ 2.0 million) – This sub-component provided small emergency funds to public primary second-shift schools hosting Syrian refugees to finance activities improving the learning environment for refugee students. These emergency grants were used for organizing field trips and extracurricular activities and/or purchasing material and equipment for classrooms. They aimed to enhance students’ motivation and sense of belonging. Sub-component 2.3: National textbook program for public primary schools (Planned - US$5.1 million; Actual US$10.1 million) – This sub-component financed the purchase of national textbook and workbooks for students (grades 1 through 9) enrolled in public schools over three academic years (2015-16, 2016-17 and 2017-18). Sub-component 2.4: Strengthening Direction d’Orientation Pedagogique et Scolaire DOPS (Planned and Actual - US$600,000) – This sub-component financed provision of resources and equipment for DOPS staff to efficiently and effectively carry out visits to Page 9 of 45 The World Bank Emergency Education System Stabilization (P152898) schools and provide academic, psychosocial, health, and special education support to schools’ leaders, staff, teachers, and students. Component 3 – Project Management and Monitoring (Planned - US$1.9 million; Actual – US$1.1 million) – This component financed the operational expenses of the RACE PMU, which was managing the entire RACE program and coordinating all donor support in response to the Syrian crisis, including EESSP. The component financed six key positions (e.g. financial management, procurement, monitoring and evaluation, engineers). 11. The project components were closely tied with the RACE initiative launched by the GOL in May 2014. The overall objective of RACE was that “vulnerable school-aged children (3-18 years) affected by the Syria crisis are able to access quality formal and non-formal learning opportunities in safe and protective environments.” The proposed activities under EESSP, in fact, drew directly from the RACE components and sub-components as listed by MEHE. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 12. The PDO was not revised during restructuring. However, target values for three outcome indicators and one intermediate indicator were revised under the restructuring dated December 19, 2017. Revised PDO Indicators 13. End targets for the following outcome and intermediate results indicators were revised under the restructuring: (a) Number of students benefitting from school fund support was increased (from 30,000 to 190,000) (b) Number of schools benefitting from supplementary financing was decreased (from 1700 to 400) (c) Number of direct project beneficiaries was increased (from 300,500 to 720,000) (d) Intermediate indicator: number of textbooks and workbooks distributed was increased (from 4,500,00 to 6,000,000) Revised Components 14. Components of the project were not revised. However, funds were reallocated between project components thereby leading to a change in the targets in the results framework. Other Changes 15. Funds for component 1 were increased from US$13.5million to US$15.12 million. Similarly, funds for sub-component 2.3 were increased from US$5.1 million to US$10.1 million. In contrast, funds for sub-components 2.1 and 2.2 were reduced from US$5.2 million to US$3.038 million and from Page 10 of 45 The World Bank Emergency Education System Stabilization (P152898) US$5.7 million to US$2.002 million respectively. Similarly, funds allocated to component 3 were reduced by US$0.8 million. Rationale for Changes and Their Implication on the Original Theory of Change 16. The primary rationale for restructuring was to immediately support the enrollment of 16,322 Syrian refugee children in schools. In Academic Year 2016/2017, MEHE faced a financing gap of US$9.8 million in student enrollment fees, as some donor commitments failed to materialize. This financing gap corresponded to the enrollment of 16,322 Syrian refugee children (US$600 per student includes US$160 in enrollment fees, US$340 in teacher salaries, and US$100 in depreciation costs)4. Consequently, to support the enrollment of these students, EESSP was restructured to reallocate financing between components and increase financing for component 1 to cover the contributions to the School Fund (at US$100 per student) and the Parent Council Fund (at US$60 per student) for these 16,322 students. The target of the corresponding indicator (Indicator #1) was also increased to match the increase in financing for this component. 17. Along with this primary rationale, there were four other reasons for reallocation of funds and modification in target indicators. These were: the delay in intergovernmental transfer of funds, the adoption and implementation of the School Grants Program under the Second Education Development Project (EDP 2), the cost savings on the rehabilitation component, and the support to the national textbooks program for one additional year. 18. The project experienced significant delays in the transfer of funds from the Ministry of Finance (MOF) to MEHE, reducing the number of rounds for grant awards that MEHE was able to launch. During the lifetime of the project, it was planned that MEHE would launch three rounds for grant awards, one per project implementation year. However, due to the slow transfer of funds from MOF to MEHE, only two rounds were conducted. In the first year of the project, MEHE approved grants for 160 schools; but the grants were transferred to schools during the second year. MEHE launched a second round of grants during the second year, but also due to the slow transfer of funds and thus unavailability of funds at MEHE, the grants were transferred to schools during the third year. The slow transfer of funds mostly impacted the school grants component as MEHE was able to mitigate its impact on other components. For example, for the enrollment of students (Component 1) and the purchase of textbooks (Component 2.3), MEHE worked with the schools so that they directly incur the expenses and implement the activities, with an agreement to be reimbursed at the end of the academic year as soon as funds were received from MOF. For the rehabilitation component (Component 2.1), the WB, on an exceptional basis, processed six direct payments to cover the rehabilitation contracts even though the contract values were below the direct payment threshold. 19. The establishment of the school grants program under EDP2 led to the need to harmonize all school grants under one institutional framework and the phase-out of the EESSP school grants. During the preparation of EESSP, the WB’s EDP2 was not performing satisfactorily and had only reached a 4 4EEESSP was designed to only cover contributions to the School Fund (US$100 per student) and the Parent Council Fund (US$60 per student). Other costs incurred to enroll Syrian refugee students included contributions to teacher salaries and depreciation costs, which were covered by other donors. Page 11 of 45 The World Bank Emergency Education System Stabilization (P152898) percent disbursement rate given the increased focus of MEHE on the Syrian refugee crisis response. While an institutional framework for school grants had been planned under EDP2, there was a high risk that it would not be able to achieve this objective. Consequently, a school grants component was included in EESSP given their importance in supporting schools in the implementation of activities tailored to their needs to improve the learning environment. During the first year of implementation of EESSP, MEHE resumed the work under EDP2 to establish an institutional framework for grants, which was finalized in December 2016. In response to those changes, the scope of school grants under EESSP was first reduced to cover only second-shift schools (i.e. schools hosting only Syrian refugee children). This reduced the number of eligible schools from approximately 1,275 schools to approximately 300 schools. Furthermore, in 2017, school grants under EDP2 started being extended to schools under the newly established institutional framework. Since the component was being implemented effectively, EDP2 was restructured in May 2017 to increase the number of schools receiving grants from 300 to 440 schools and increase the allocation to the grants component under EDP2 by US$1 million. In light of the EDP 2 restructuring and to strengthen the recently established institutional framework by avoiding duplication of efforts and reducing confusion within MEHE created by having two different projects supporting school grants with two different mechanisms and different supervising entities, MEHE proposed to phase out the EESSP grants component and thus reduce the target for Indicator #2 (number of schools receiving supplemental school financing). 20. The rehabilitation of ten schools was completed efficiently by 2017 with substantial savings of US$2.16 million when compared to the initial budget allocation. The remaining funds were therefore re- allocated from rehabilitation of schools to other activities under EESSP. 21. EESSP provided textbooks and workbooks to cover all students enrolled in public schools during Academic Years 2015-2016 and 2016-2017, and financing for this component was increased to provide textbooks for an additional year (Academic Year 2017-2018). During project design, the plan was for the project to support the provision of textbooks for two academic years only due to limited financing. However, given the savings from the rehabilitation component and the use of EDP2 funds for school grants, financing from subcomponents 2.1 and 2.2 were redirected to subcomponent 2.3 to cover an additional year of textbooks for public school students5. 22. The restructuring had no impact on the original theory of change. All activities were implemented, and the restructuring only affected the scale of some activities, with two of them implemented at an expanded scale (student enrollment and textbooks), and a third one at a reduced scale (school grants). 5During the third year (Academic Year 2017-2018), EESSP funds covered textbooks for all non-Lebanese students in second shifts and for around 75 percent of non-Lebanese students in morning shifts. The remaining cost of textbooks was covered by UNICEF for 25 percent of non-Lebanese students in morning shifts and by the Lebanese Government for all Lebanese students. Page 12 of 45 The World Bank Emergency Education System Stabilization (P152898) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 23. As mentioned above in paragraphs 2 to 5, the PDO was highly relevant to the emergency needs of the country and remained highly relevant under the CPF FY2017-22. Under the second area of focus for the CPF FY2017-2022 – “expand economic opportunities and increase human capital”, the third objective is ‘improved delivery of education services and skills development” and the rationale is that ‘the influx of Syrian refugees has put enormous pressure on the education system at all levels. These stresses are having an impact on access, retention, and quality of education for Lebanese and refugee children. The project therefore sought to address this by aligning with Lebanon’s strategy – Reaching All Children with Education in Lebanon (RACE) – to respond to the challenges created and exacerbated by the refugee crisis.’ 24. The PDO was also closely aligned with the GOL’s own initiative, RACE. The overall objective of RACE was to ensure that ‘vulnerable school-age children (3-18 years) affected by the Syrian refugee crisis are able to access quality formal and non-formal learning opportunities in safe and protective environments.’ The number of Syrian refugee children remained high during EESSP implementation, as well as their need for education and to maintain a quality learning environment for all public-school children. 25. In times of forced displacement, education services are crucial for refugee children. Education not only contributes in building the skills of children that are essential for pursuing productive lives, it can also address psychosocial needs, develop socio-emotional skills, foster social cohesion, provide access to life- saving information, and offer a stable and safe environment for those who need it most. It also helps people to rebuild their communities. 26. Considering these factors, the relevance of the operation’s development objectives is High. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 27. The project achieved or exceeded all the PDO indicators and intermediate results indicators included in the EESSP Results and Monitoring Framework after the restructuring dated December 2017. The project implemented the following activities, as initially planned: (i) support to schools through financing of students’ enrollment fees; (ii) school rehabilitation; (iii) supplemental school financing; (iv) national textbook program for public primary schools; and (v) strengthening the Direction d’Orientation Pédagogique et Scolaire (DOPS). To monitor and evaluate achievements of the activities under the project, relevant indicators were included in the results framework: three activity-specific project development objective indicators and three intermediate results indicators. All the end targets set after the restructuring of the project were met or exceeded (see Table 2). Page 13 of 45 The World Bank Emergency Education System Stabilization (P152898) Table 2: Results: Target and Actual Values of Project Indicators Revised Actual End Value Baseline Target Value (Implementation Initial End Indicator Value (2017 Status and Target (PAD) (2015) Restructuring Results Report – Paper) June 2018) Project Development Objective Indicators Students benefiting from school fund 0 30,000 190,000 201,851 support Number of schools benefiting from 0 1,700 400 410 supplementary financing Number of schools benefiting from 0 10 10 10 rehabilitation Intermediate Results Indicators Number of textbooks distributed 0 4,500,000 6,000,000 8,560,680 Data collected on use of supplemental N Yes Yes Yes school financing and perceived impacts School improvement activities displayed 0 60% 60% 100% on beneficiary school walls 28. The number of direct beneficiaries for the project exceeded both the initial target value as included in the Project Appraisal Document (PAD) and the revised target value as included in the 2017 restructuring paper. The target value for direct project beneficiaries was revised (more than doubled) once under the restructuring dated December 2017 and was exceeded by the end of the project. Slightly more than 50 percent of these project beneficiaries were females. Table 3: Overall Project Indicators – Target and Actual Values Revised Actual End Value Target Value Baseline Initial Target (Implementation Indicator (2017 Value (2015) Value (PAD) Status and Results Restructuring Report – June 2018) Paper) Number of direct project beneficiaries 0 300,500 720,000 1,220,825 of which percentage of female NA 50 50 50.81 29. The project development objectives were effectively met. The project development objective as stated in the PAD can be parsed out into two objectives: (a) To support the operational needs of the Lebanese public schools in response to the continued influx of Syrian refugee children (b) To improve the learning environment in response to the continued influx of Syrian refugee children Page 14 of 45 The World Bank Emergency Education System Stabilization (P152898) 30. The project effectively ‘supported the operational needs of the Lebanese public schools in response to the continued influx of Syrian refugee children’ and directly financed the enrollment of 201,851 children in Lebanese public schools. Schools’ operational needs and expenses were supported through two funds: the School Fund (at US$100 per student) and the Parent Council Fund (at US$60 per student). These funds were financed through enrollment fees levied from parents. However, instead of having parents pay for enrollment fees and school operational expenses, the Government through the RACE initiative and using donor financing (including the WB’s EESSP) covered the full enrollment fees, decreasing the cost of education for vulnerable and refugee households. EESSP was designed to directly support the enrollment of 30,000 students (initial target for indicator #1), which is equivalent to a maximum of 90,000 student enrollment fees for supporting these students over three years. The project greatly exceeded this target by covering 201,851 student enrollment fees over the three years of implementation, more than double the initial target. The project was able to exceed the target through additional financing allocated to this component, in addition to operational efficiency by covering students who were only partially funded by other donors (i.e. donors would have paid for either the School Fund or the Parent Council Fund contribution and EESSP would cover the remaining contribution to ensure enrollment of the students in schools). 31. The project was an essential contribution to the overall RACE objective of providing access to education to Syrian refugee children and vulnerable Lebanese children. In the year 2014-15, the Lebanese public schools educated 347,415 students (including 237,872 Lebanese and 93,627 Syrian students). By the end of the project in the year 2017-2018, the public schools in Lebanon educated 483,054 students (including 264,364 Lebanese and 209,721 Syrian students)6. This implies a 39 percent increase in the capacity of the Lebanese public school system over the three years of the project. This significant increase in the capacity of the public-school system in Lebanon is a major achievement of the RACE initiative. MEHE was able to achieve and exceed its enrollment targets thanks to its coordination efforts with all donors to cover enrollment fees. Under RACE, EESSP financed 201,851 enrollment fees for Lebanese and non- Lebanese students; these constitute approximately 18 percent of all enrollment fees covered by the entire RACE program in three years. Additional enrollment fees were covered primarily through the United Nations International Children's Emergency Fund (UNICEF). 32. In addition to supporting the enrollment of Syrian refugee students, EESSP contributed in supporting the stabilization and then increase in the number of Lebanese students in public schools. During EESSP design stage, the enrollment of Lebanese students in public schools was still in decline, a trend that had been observed for more than a decade and was exacerbated by the Syrian refugee crisis. For example, at the beginning of the crisis in Academic Year 2011-2012, the number of Lebanese students in public schools was 248,209; by 2014-2015, that number had decreased to 237,872 students. Schools reported that the reason for Lebanese children’s flight from public schools was due to the worry of Lebanese households that the quality of education in public schools would deteriorate following the enrollment of Syrian refugee students. This perception was further validated by a WB perception study under the Research for Results (R4R) program, which found that 56 percent of Lebanese households attributed negative effects on the quality of teaching and learning, school facilities and equipment, and relationships among students, to the Syrian refugee crisis. By supporting the operational needs of schools to expand enrollment and improve the learning environment and by supporting the enrollment fees of both Lebanese and refugee 6 Figures from the Center for Educational Research and Development (CERD). Page 15 of 45 The World Bank Emergency Education System Stabilization (P152898) students, the project contributed to stabilizing the enrollment of Lebanese students in public schools and then reversing this trend by increasing enrollment of Lebanese students to 264,364 students by the last year of implementation (Academic Year 2017-2018). The increase in enrollment of Lebanese students is also evidence that the project contributed to improving the learning environment in schools hosting Syrian refugee students, thus halting and reversing the flight of Lebanese students from public schools. 33. The project activities effectively improved the learning environment in response to the continued influx of Syrian refugee children. For this project development objective, the project implemented four main activities: (i) school rehabilitation; (ii) supplemental school financing; (iii) national textbook program for public primary schools; and (iv) strengthening DOPS. All four activities were effectively carried out and met or exceeded their revised targets. There is substantial evidence in the education literature linking the above activities to improved learning environment. 34. The project rehabilitated ten schools that were in extremely bad conditions. Statistics provided by the D-RASATI survey (2011) and updated under EDP 2 showed that among the 1,275 existing public schools, 75 percent needed repairs. Of these, 25 percent needed major repairs. EESSP financed the full rehabilitation of 10 schools from the priority list. Rehabilitation works included structural improvements to buildings to ensure building security and safety, improved access to clean water and sanitation, electrical and mechanical installations repairs at the schools, refurbishment of the physical appearance of buildings and classrooms, improved accessibility of the school, and provision of new furniture and equipment for teaching and learning. The MEHE and WB teams visited the schools and inspected the rehabilitation works which were deemed of high quality and provided a welcoming environment for vulnerable and refugee children. 35. Supplemental school financing helped improve the learning environment by fostering social cohesion between different student communities through extra-curricular activities. Of the 410 schools receiving supplemental school financing, 70 percent used the financing for extra-curriculars; for example, acquiring sports/music equipment or arranging field trips for students. 20 percent of the schools acquired additional instructional materials while another 10 percent carried out small rehabilitation activities. There is an international evidence base that supports the use of school grants to improve quality of the learning environment in schools.7 As part of the project’s monitoring and evaluation, the RACE PMU collected from schools evidence on the impact of the grants in supporting the learning environment for Syrian refugee students. School principals indicated that the extra-curricular activities improved the learning environment for students by making schools more welcoming learning places. This was especially important for refugee children who needed to be re-engaged in the education environment after the substantial gap in their education. During implementation and as part of the ICR process for EESSP, the WB team visited five schools across Lebanon: two in Beirut, two in Mount Lebanon, one in the Beqaa, and one in the South to obtain feedback on the project activities. The team received positive feedback for the activities from the school directors. Schools were also instructed to include results of their grant activities on the school walls to raise confidence and improve the perception that Lebanese and refugee parents have of public schools. 36. Even though the number of school grants was reduced from 1,700 to 410, this reduction in scale was mitigated through the use of the grants mechanism under EDP 2 and the focus of EESSP funds on second-shift schools hosting Syrian refugee children. As stated previously in paragraph 19, the WB’s EDP 2 7 Murnane, Willet and Cardenas (2006), Gertler, Patrinos, and Rodriguez-Oreggia 2012) and Bruns, Filmer and Patrinos (2011) Page 16 of 45 The World Bank Emergency Education System Stabilization (P152898) successfully established an institutional framework for a school grants mechanism which was finalized and approved by MEHE in December 2016. To avoid duplication of efforts and the weakening of national systems, the scope of EESSP grants was adjusted to focus solely on the approximately 300 second-shift schools hosting Syrian refugee children, instead of the entire public-school population of 1,275 schools. Out of the target of approximately 300 second-shift schools, EESSP covered 160 schools during the first round of grant awards and 250 schools during the second round, providing school grants to more than 85 percent of second-shift schools at least once. In parallel, EDP 2 provided additional school grants to 440 schools. Consequently, while the number of school grants planned under EESSP was reduced, MEHE was still able to cover a majority of schools with school grants, including more than 85 percent of second-shift schools hosting Syrian refugee children, which were the primary target of the project. 37. EESSP financed the national textbook program for three academic years from 2015 to 2018. To support the expansion of the education system to enroll refugee children, the project supported the provision of a set of textbooks and workbooks for every child in Lebanon. This activity was planned to support textbooks for two academic years but was expanded to support a third academic year8. Consequently, the target was exceeded thanks to savings from other components, which were reallocated to this activity to ensure three years of support. The textbook quantity and quality study, which included spot checks to schools to ensure the availability of textbooks showed that in February 2018, more than 90 percent of students in basic education had individual textbooks during the classroom visit and that the quality of textbooks was acceptable. The textbooks allowed children to better follow the classroom lessons in class and study at home. As the project was designed in response to the country’s emergency needs, the project used the existing MEHE book purchasing procedures at the time of project design, but also conducted a detailed analysis of the textbook supply chain, providing recommendations for improving the effectiveness and efficiency of the national textbooks program. 38. EESSP covered the operational costs for all DOPS regional centers for the academic year 2016-2017 and provided equipment and material to DOPS staff. DOPS staff provide tailored coaching to teachers on pedagogy, health and psycho-social issues and child protection. The work of DOPS staff in schools is crucial especially with the enrollment of a large number of Syrian refugee children who have moved to Lebanon from war-torn Syria and are in need of psycho-social support and a welcoming/nurturing environment to feel a sense of belonging in their new schools and re-engage in education. The project covered operational expenses for all 16 DOPS regional centers, as well as procurement of equipment and material for DOPS staff. DOPS counselors provided continuous support to schools, as evidenced by the DOPS coaches’ school visits schedule. During school visits, school principals and teachers reported that they were highly satisfied with the support provided by DOPS, highlighting its critical importance for improving the learning environment, especially on the pedagogical side through regular classroom observations and feedback and guidance provided to teachers, and the psychosocial side through their socio-emotional support and counseling to vulnerable and refugee children. 39. The Ministry of Education has shown evidence of improvement in the learning environment leading to higher education quality and learning for vulnerable children, including Syrian refugee children. Available learning data showed that non-Lebanese students have gradually achieved higher results in Grade 8During the third year (Academic Year 2017-2018), EESSP funds covered textbooks for all non-Lebanese students in second shifts and for around 75 percent of non-Lebanese students in morning shifts. The remaining cost of textbooks was covered by UNICEF for 25 percent of non-Lebanese students in morning shifts and by the Lebanese Government for all Lebanese students. Page 17 of 45 The World Bank Emergency Education System Stabilization (P152898) 9 national exams since academic year 2015/16. Results for non-Lebanese in both the morning and afternoon shifts rose from 59.8 percent in 2015/16 to 72.6 percent in 2017/18, despite a 70 percent increase9 in Grade 9 enrollment during this same period. For children in the second shift taking end of school year exams in 2017/18, data showed that participation and pass rates have improved from earlier years of the project. Moreover, a perception survey conducted by the WB under the closely-linked R4R program showed that Lebanese and Syrian parents had positive views on education quality in Lebanon with 76 percent of parents rating education quality in Lebanon as good or excellent, and 59 percent believing that it is improving. Furthermore, the R4R program measured student learning in first-shift and second-shift public schools, as well as in private schools, for Grades 4 and 7 to study progress of student learning outcomes over the course of one academic year (2016-2017). The results showed that second-shift Syrian refugee students start with the lowest baseline learning scores in all subjects (Arabic language, Foreign Language, and Math) at the beginning of the year, but show the greatest improvement in learning outcomes, reaching similar scores to first-shift public school students by the end of the year. Justification of Overall Efficacy Rating 40. The overall efficacy rating is Substantial. EESSP effectively supported the operational needs of schools hosting Syrian refugee children, allowing MEHE to exceed the targets and increase enrollment in public schools of both Lebanese and Syrian refugee children. The project also effectively improved the learning environment in public schools in response to the refugee crisis by supporting improvements to the physical environment (rehabilitation), the provision of learning material (textbooks and workbooks), and the socio-emotional wellbeing of children (extracurricular activities financed by school grants and DOPS counseling support). The overall efficacy rating would have been High if the target for school grants was not reduced resulting in a smaller number of grants provided to schools by the project. The impact of this reduction in number of grants was mitigated through a focus on second shift schools (more than 85 percent covered) and through financing from other programs (EDP2). 9 RACE Factsheet. November 2018. Grade 9 enrollment increased from 1,706 in 2015/16 to 2,894 in 2017/18. Page 18 of 45 The World Bank Emergency Education System Stabilization (P152898) C. EFFICIENCY Assessment of Efficiency and Rating 41. The design of the project established an efficient model for providing education services to Syrian refugee children and achieving the project outcomes. The project strengthens and builds on the existing education system infrastructure to expand services to Syrian refugee children, instead of building a parallel system for refugee education; the latter was an approach adopted for decades to provide education services for Palestinian refugee children through the United Nations Relief and Works Agency for Palestine Refugees (UNRWA). To compare the operational efficiency of these two approaches, the WB, under the R4R program conducted a public expenditure review of the education sector in Lebanon. The unit cost for educating a child in public school in Lebanon is estimated by the WB to be approximately US$2,022 per student when taking into consideration all expenses spent by the GOL on public education10. Conversely, the project used a marginal cost approach for expanding education services to Syrian children by computing the additional cost incurred by adding one student into the existing public education system, either in first shift or in second shift. The marginal cost is the unit cost used by this project and all other donors financing the RACE program to provide education to Syrian refugees; it is equal to US$363 per student in first shift (composed of US$160 in enrollment fees and US$203 in teacher salaries) and US$600 per student in second shift (composed of US$160 in enrollment fees, US$340 in teacher salaries, and US$100 in assets depreciation). By establishing with MEHE this financing model for refugee education, the project introduced substantial efficiency to the RACE program by educating Syrian refugees at approximately 25 percent the unit cost needed if a parallel education system (similar to the Lebanese public system) needed to be created. This approach is also considered the latest best practice in terms of refugee education11. 42. The school rehabilitation activity was completed on time with savings of US$2.16 million. During project preparation, a detailed needs assessment and cost analysis was conducted for the rehabilitation component. The top ten schools in extreme need for rehabilitation were selected and the cost attributed to the component based on rehabilitation cost benchmark in Lebanon multiplied by the size of the school (computed in terms of square meters). Nonetheless, to increase the efficiency of the rehabilitation works, the planning and design phase, as well as the supervision of works phase were not outsourced and were instead executed by in-house engineers at the RACE PMU. Only the works and the procurement of furniture and equipment were contracted to firms. This approach allowed for large cost savings in the implementation of the component. The savings were then re-allocated to school funds and the provision of textbooks. 43. Another operational efficiency lies in the adoption of the existing textbook distribution system for the provision of textbooks. In order to provide textbooks and workbooks to all children in public schools, the project used the existing textbook distribution system by reimbursing the cost of textbooks purchased by schools through the existing supply chain, rather than commissioning a separate procurement for this activity. The result was that textbooks purchased under this project had a low unit cost compared to benchmarks in the region, as shown in the textbook study conducted by the WB. For example, Grade 4 public school textbooks in Lebanon had a unit cost of US$0.87, compared to US$0.91 Page 19 of 45 The World Bank Emergency Education System Stabilization (P152898) in Jordan and US$2-4 in Turkey. Note that in Jordan, 2 textbooks are needed per subject per year, doubling the total cost for purchasing all textbooks. 44. Staff turnover was limited for the staff managing the project at the RACE PMU, improving efficiency in project management and monitoring. Six members of the RACE PMU were hired using EESSP funds and continued their employment through the end of the project. Staff retention is key to a project’s success as it allows the staff to incorporate lessons learned in the initial years into the subsequent years and avoid potential lags in delivery of project targets. 45. The slow transfer of funds from the MOF to MEHE was the major bottleneck that reduced the efficiency of the project. Transfer of funds from the WB to the GOL used an expenditure-based financing approach. Once MEHE has spent funds from the project Designated Account (DA), they would request the WB to replenish the DA with the same amount spent. Funds are disbursed from the WB to the MOF and then transferred from MOF to MEHE. It is the latter transfer between Government agencies that faced significant delays due to internal bureaucratic systems. One of those delays - the longest - was around six months for the third tranche to be transferred from MOF to MEHE. These delays reduced the efficiency of the project. The issue of transfer of funds was largely mitigated by MEHE for most activities. For example, in the case of school enrollment, schools enrolled children at the beginning of the academic year with the assurance that MEHE committed to reimburse enrollment fees as soon as funds were made available. Similarly, for textbooks, school purchased and distributed textbooks at the beginning of the year and were reimbursed months later by MEHE. This approach ensured that activities would not be discontinued while waiting for funds to be available. Nevertheless, the school grants component was affected by the slow transfer of funds, as MEHE was able to launch only two rounds of grants rather than three as originally planned due to the delays in funds availability. 46. The project initiation phase was fast compared to other WB projects in Lebanon and the project was implemented within the allotted timeframe with no need for extensions. Economic analysis 47. Due to the emergency situation in Lebanon and the condensed preparation timeline, a prospective economic analysis with cost benefit ratios through assumed impact and projected beneficiaries was not conducted at appraisal. Rather, a descriptive economic analysis was conducted, and the number of beneficiaries were laid out. For this reason, a direct comparison of the benefit-cost findings from the Implementation Completion and Results Report (ICR) analysis with the findings at appraisal is not possible. 48. A cost-benefit analysis (CBA) was conducted for the ICR to assess the returns on investment, in terms of internal rate of return (IRR), estimates of net present value (NPV) and the benefit to cost ratio (BCR), of the Emergency Education System Stabilization Project (see Annex 4 for details). Standard methods were used to estimate the economic benefits generated by “supporting the operational needs of the Lebanese public schools and to improve the learning environments in response to the continued 10 Figure calculated by the World Bank team using financial data from MOF and MEHE. These numbers are not official numbers endorsed by the Government of Lebanon. 11 UNESCO Global Education Monitoring Report 2019. Retrieved from: https://en.unesco.org/gem-report/report/2019/migration Page 20 of 45 The World Bank Emergency Education System Stabilization (P152898) influx of Syrian refugee children” and whether the investment is justified on economic grounds. The CBA followed a three-step process: first, it estimated the change in student enrollment that can be attributed to the project; second, it translated the increase in enrollment into educational benefits; and third, it assigned a monetary value to these benefits. 49. For the project, we assumed benefits were achieved through increasing enrollment of vulnerable Lebanese and Syrian children in the public education system in Lebanon. The first component of the project, Support to School fund, directly supported enrollment of vulnerable Lebanese and Syrian refugee children in public schools in Lebanon. As the other components of the project, i.e., supplemental school financing, rehabilitation of schools and strengthening of DOPs were also intended to provide a more conducive learning environment for vulnerable Lebanese and Syrian refugee children, we will assume that the benefits of these other components contribute to increased enrollment of children in the public education system in Lebanon. The project’s impact on enrollment was defined by the number of vulnerable Lebanese and Syrian students supported by the project over the three years. Existing evidence was used to translate the benefits from the increased number of children school-years into preserved work productivity and future wages (Montenegro and Patrinos 2014). 50. All costs related to the project were included in the analysis. The impact of educational attainment of project beneficiaries was then translated into economic benefits in terms of average preserved future earnings of an individual (gross domestic project – GDP) per capita, adjusted for the percentage of GDP from wages in Middle East and North Africa (Lubker, 2007). All benefits and costs were discounted at 10 percent rate, the current interest rate from Banque Du Liban. Finally, the costs were compared to the benefits and the IRR and the BCR were calculated. 51. The total investment through the project was of US$32 million over 3 years (2015-2018). The number of students for whom the yearly enrollment cost was subsidized is 201,851. The project yielded an IRR of 12 percent and generated economic benefits with a net present value (NPV) of US$15 million. With an IRR higher than the current cost of capital in Lebanon, the project yielded an economic return higher than the return on investment from alternative capital investments in the financial market. The project had a good benefit to cost ratio of 1.43, suggesting that each US dollar invested yielded an economic return of US$1.43. 52. Considering the above, the efficiency is rated Substantial. There were significant operational efficiencies in the design, including most importantly the financing model for refugee education. There were also significant savings in the rehabilitation activity and in the use of the existing textbook distribution system for the provision of textbooks under the program. The efficiency would have been high if it was not for the slow transfer of funds between Government agencies, which created delays in implementation, most notably for the school grants component, leading to the reduction of school grant rounds from three to two rounds. Page 21 of 45 The World Bank Emergency Education System Stabilization (P152898) D. JUSTIFICATION OF OVERALL OUTCOME RATING 53. As per Bank Guidance on ICRs, the overall outcome rating is Satisfactory (Table 4).12 Table 4: Justification of Overall Outcome Rating Sub-Ratings Overall Outcome Relevance Efficacy Efficiency Rating High Substantial Substantial Satisfactory13 E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 54. The operation preparation had ensured that the project outcomes will be evaluated for both males and females. The project development indicators therefore included an indicator on the percentage of females in the number of total project beneficiaries. The target value for this indicator was 50 percent, ensuring that the project should benefit both genders equally. As shown in the results matrix, this target was met and slightly exceeded. Institutional Strengthening 55. The project contributed to institutional strengthening through capacity building activities for the RACE PMU staff leading the Ministry’s response to the Syrian refugee crisis. The RACE PMU was established by MEHE to lead the response to the Syrian refugee crisis. Given the large number of Syrian refugee children requiring education services, the unit was established to provide fast response to the crisis and ensure leadership of the RACE program by coordinating with donors and partners. Throughout the project, efforts were undertaken to develop the capacity of the RACE PMU. The project financed six key positions at the PMU, including in monitoring and evaluation, financial management, and procurement functions. The project also financed training opportunities for the PMU staff. The Program Manager, the Planning and Monitoring & Evaluation Officer and the Procurement Officer attended three training courses at the International Labor Organization (ILO) in Turin, Italy: “Change Management for Public Sector Institutions”, "Project Cycle Management" and "Works Procurement Management". Similarly, a 3-day safeguards training was conducted at MEHE in August 2016. In addition, the procurement officer attended a 5-day training provided by the Office of the Minister of State for Administrative Reform (OMSAR) in September 2016. This training covered the principles of procurement, planning, processing of National Competitive Bidding (NCB) and consulting firm selections, as well as contract management. Institutional strengthening will depend on the retention of the staff engaged at the RACE PMU and the possible integration and harmonization of the RACE PMU processes into the regular work of the MEHE. 56. In addition to the yearly financial audit requirement, the project included a special technical audit of school level expenditures, which analyzed fiduciary processes at the school level. Within EESSP, most 12 For details about how to derive the Overall Outcome Rating, see World Bank Group 2017f, and in particular Appendix H. 13 Split evaluation is not used for this project as the restructuring did not change the Project Development Outcome. Page 22 of 45 The World Bank Emergency Education System Stabilization (P152898) activities were implemented by schools: these included financial contributions to the school fund and parent council fund to cover the operational needs of schools, the school grants, as well the provision of textbooks, which were entirely bought by schools to be delivered to students. Moreover, the DOPS activity consisted of providing financial contributions to DOPS operating expenses. Under EESSP, MEHE and the WB conducted a yearly independent technical audit that looked at how school-level procurement and financial management processes were conducted. This exercise was the first of its nature even though all donor financing were also channeled to schools. The audit identified areas of improvement in school-based management’s capacity in financial management and procurement. The audit also highlighted the necessity for more unified procedures across the different regions of Lebanon. Trainings at the regional level on financial management and procurement processes were rolled out in response to these findings. The audit was thus an opportunity to meet fiduciary requirements for the project while also aiming for institutional strengthening in the longer-term. 57. EESSP included a study on the textbook supply chain to inform improvements in the existing textbook distribution system. EESSP financed the national textbook program using the existing financing procedures of the GOL. Nonetheless, under EESSP, the WB agreed with MEHE to conduct a full study of the textbook supply chain, analyzing quantities of textbooks available in schools at different points in time, as well as the quality of those textbooks. The analysis also looked at the entire supply chain, the actors involved and the processes, as well as the costing. Several recommendations on improving the textbook supply chain were provided in the study report and discussed with MEHE and the Center for Educational Research and Development (CERD). These discussions were very timely as the GOL is preparing to conduct a first and full review of its curriculum after 22 years, with the design and rollout of new textbooks in the coming three to five years. 58. Closely linked to EESSP, the WB, MEHE, and DFID established a research program - Research for Results (R4R) – to inform implementation and longer-term sector planning. During the design of EESSP, MEHE and the WB working with other donors identified the need for rigorous evidence on the education system in Lebanon, especially as it relates to the impact of the Syrian refugee crisis on the sector. The United Kingdom Department for International Development (UK DFID), which was a major donor to the LSCTF financing EESSP, agreed with the WB and MEHE to allocate additional financing for the implementation of a collaborative research program - Research 4 Results (R4R) – in parallel with EESSP to inform implementation and longer-term sector planning, especially as it relates to the impact of the Syrian refugee crisis on the education sector. R4R was implemented by the WB and covered system-level research such as a public expenditure review of education, a political economy analysis, and a public perception study, as well as service delivery research, which included school management analysis, classroom observations, student assessments, and a special focus on vulnerable Lebanese and Syrian children. The research was conducted in public schools (both first shift and second shift schools hosting Syrian refugee children), as well as in private schools. R4R allowed to create a platform for research in the sector led by the Government and convening a large number of actors such as the private sector, academics, and donors. A first volume of findings was disseminated in April 2018 and the second volume will be disseminated by August 2019. The evidence from R4R has already been integrated in the Government’s education plan and vision 2030 for the achievement of the Sustainable Development Goals (SDGs). 59. Despite being an emergency project, the EESSP allowed the WB to use its convening power to work with MEHE and donors to think beyond the emergent needs and to build development policies with the Page 23 of 45 The World Bank Emergency Education System Stabilization (P152898) Government and international counterparts. The project led to a broader systems-level project “Support to RACE II” (S2R2) in 2016, financed by the WB from three financing streams: US$100 million allocation from the International Development Association (IDA), US$100 million grant from the LSCTF, and a US$4 million from the Results in Education for All Children (REACH) Trust Fund. S2R2 aims to promote equitable access to education services, enhance quality of student learning, and strengthen the education systems in the Recipient’s education sector in response to the protracted refugee crisis. It also builds on the research evidence produced under the R4R program. Mobilizing Private Sector Financing 60. Not applicable. Poverty Reduction and Shared Prosperity 61. As stated in the PDO, the project intended to extend education services to Syrian refugee children. Syrian refugees are highly vulnerable and by providing access to education for Syrian refugee children, the project helped to preserve their productive potential contributing to future poverty reduction and shared prosperity. Other Unintended Outcomes and Impacts 62. There were several unintended outcomes and impacts of EESSP, especially on the institutional strengthening part, which have been detailed in the corresponding section, and include identification of capacity needs in financial management and procurement at the school level and provision of capacity building to regions and schools. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 63. The project design was strongly aligned with the Government program. The project supported the GOL’s own RACE initiative and the project components were very well aligned with the component activities under RACE. Therefore, there was strong ownership from Government counterparts. 64. EESSP was designed with strong coordination with other development partners. The 3-year RACE program, launched by MEHE in response to the Syrian refugee crisis, was designed in coordination with the international development community. MEHE has set up a RACE Executive Committee (REC) chaired by the Minister of Education and Higher Education and which includes the Director General of Education, the CERD President, and the RACE PMU Program Manager, as well as all donor countries and international partners financing the RACE program. The REC was to support in steering the implementation of the program, addressing roadblocks and challenges, and ensuring close coordination and alignment between all partners. EESSP design and implementation were under the RACE program and were governed by the same coordination system. Page 24 of 45 The World Bank Emergency Education System Stabilization (P152898) 65. EESSP helped establish the financing model for Syrian refugee enrollment in public schools. Working with the MEHE, the WB completed, during the preparation of EESSP, the necessary analytical work to devise the financing model for enrollment of Syrian refugee children in Lebanese public schools. The analysis identified the average marginal cost for enrolling an additional child by looking at the financial impact of the increase in enrollment on teacher salaries and operational expenses, as well as depreciation of school physical assets and increased need for maintenance. The result of the analysis was the establishment of a unit cost for Syrian children enrolled in both the first shift (at US$363 per student) and the second shift (at US$600 per student). MEHE and the REC adopted this unit cost, which became the model all donors used to finance enrollment of Syrian refugees. B. KEY FACTORS DURING IMPLEMENTATION C. 66. During implementation, when challenges emerged, MEHE and the WB largely addressed them in a timely manner when they were under their control. The WB provided support to the project through its Beirut-based co-Task Team Leader (TTL), as well as financial management and procurement staff. Four formal implementation support missions were conducted to supervise the implementation of the project throughout the project timeline – two in 2016, one in 2017 and one in 2018; and the project benefited from daily follow-up and oversight by the co-TTL based in Beirut. 67. The major challenge faced during implementation was the slow flow of funds from MOF to the Project’s DA, which led to liquidity shortages at MEHE. These delays were mainly due to intra- governmental administrative and bureaucratic processes. Nevertheless, the WB country management worked with the MOF to streamline these processes as much as possible and reduce these delays. These efforts were successful by reducing the delays in transfer of funds from six months for Tranche #3 to two months for Tranche #414. 68. The WB showed flexibility during project implementation to mitigate the impact of the slow transfer of funds, giving approval to use the direct payments mechanism even though payment amounts were below the regular threshold. Six direct payments were processed by the WB with a total value of US$1,426,996, mostly to contractors that implemented the rehabilitation activities to ensure timely payment for these activities and avoid any penalties that could be incurred due to delays in payment. 69. Coordination with other development partners remained strong during project implementation. As the implementation of RACE involved many donors, donor collaboration and flexibility were important for the success of the project. The REC was the governing body for all financing under the program. Moreover, donors held a monthly donor coordination meeting to share information and harmonize efforts. The WB education team conducted regular briefings for all donor-partners before and after major missions. Given the challenges in flow of funds from the MOF to the Project’s DA, strong coordination between MEHE, the WB, and other donors allowed for a reallocation of funds to cover financing gaps. For example, during the first year of EESSP implementation and given initial delays in flow of funds, the WB and UNICEF agreed to a parallel financing arrangement in which UNICEF covered the remaining cost for Lebanese and non- Lebanese students in the first shift in academic year 2015-2016 for a total of US$5.13 million, which were previously planned to be financed using WB-EESSP funds. Conversely, during the second year of 14For Tranche #5, the delay of transfer of funds from MOF increased to four months, but part of this delay was attributed to the closing of Fiscal year 2017 in December 2017. Page 25 of 45 The World Bank Emergency Education System Stabilization (P152898) implementation, MEHE faced a financing gap of US$9.8 million for student enrollment fees, which EESSP stepped in to cover around US$2.6 million (US$160 per student) through reallocation of EESSP project funds. 70. Lack of long-term financing commitments from donors increased risks of financing gaps to cover all Syrian refugee children. During the project design, the team identified the risk that “substantial external resources needed to cover the costs of Syrian enrollment year by year may not materialize”. Indeed, overall during the lifetime of the project, MEHE was able to raise additional financing for enrollment, however this trend was not consistent from year to year due to the lack of long-term financial commitments from donors. During the second year of project implementation, MEHE faced a financing gap that impeded it from covering enrollment fees for all students enrolled during that year. The WB team and MEHE thus reallocated EESSP financing during that year to partially cover this financing shortfall. The flexibility in re-allocation of funds by the Bank was important to successfully finance the different components of the project and ensure that the project development objectives are achieved. 71. RACE financing gaps. Throughout the implementation of RACE, donor support remained strong. However, donor commitments were primarily short-term and were short of the total financing needs for the program. This led to large financing gaps for RACE and little predictability for long-term financing to the sector. For example, by November 2016, the financing gap for the overall RACE program for academic year 2016-2017 was US$85 million. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 72. Being an emergency project, the design of the Monitoring and Evaluation (M&E) system was simple and closely aligned to project outputs but was limited in its ability to measure impact. The project’s results framework consisted of five project development objective indicators and three intermediate results indicators. Three of the five project development objective indicators were directly tied to the activities under the sub-components. One of the intermediate results indicators, the ‘Number of textbooks distributed’, could have also been a project development objective indicator as it was directly tied to the output of the activity ‘National Textbook Program’. However, the other two intermediate results indicators were not very effective in measuring the impact of the project. The indicator, ‘Data collected on use of supplemental school financing and perceived impact’ was partially a process indicator and partially a results indicator. Data on the use of supplemental school financing was collected as grant proposals were collected from schools and receipts and schools’ bank statements were collected to verify expenditure on the grant items. The data on perceived impact could have been a stronger results indicator for the second development objective of improved learning environment. Similarly, the intermediate results indicator, ‘School improvement activities displayed on beneficiary school walls’ was a process indicator and its effectiveness seemed limited compared to the time needed to collect credible information on this indicator. Page 26 of 45 The World Bank Emergency Education System Stabilization (P152898) M&E Implementation 73. M&E implementation was moderately satisfactory to satisfactory during the lifetime of the project. An M&E specialist, funded by EESSP, was hired at the RACE PMU to work closely with the MEHE central, regional, and school teams to collect data on project activities and was responsible for preparing progress reports. At the design stage, it was decided that brief, standardized quarterly reports would be produced by each department based on the progress made in the implementation of the work program. The M&E specialist at the PMU would compile these reports and submit to the Bank and the RACE Executive Committee. Regular progress reports were produced that laid out in detail the progress towards each activity and sub-component. The progress reports also detailed challenges faced, and measures needed to resolve the challenges. 74. All implementation support visits systematically included a thorough review of implementation status, including joint sessions to update the target values for project development indicators. These efforts were described in detail in the missions’ aide-memoires and the subsequent Implementation Status and Results (ISR) reports. Hence, evidence of achievement of both project indicators was regularly monitored by the RACE PMU and the WB. In addition, independent financial audit was conducted for the overall project to verify that the expenses were made for the intended purposes as laid out in the PAD. To closely monitor the effective implementation of the work program, the WB gathered and reviewed documentation on the different project activities. For example, for the Support to Schools activity, the WB reviewed a sample of bank statements from 52 schools benefiting from EESSP funds to verify that the funds were received by the schools as part of the project. In addition, a technical auditor was hired to conduct an independent evaluation of school-level expenditures and the implementation of the program at the school level. Similarly, a detailed review of textbook expenditures was conducted to verify the quantities and quality of textbooks received by schools and children, as well as to analyze the supply chain of textbooks provision and how it could be strengthened for higher efficacy and efficiency. 75. Data availability for monitoring and evaluation was adequate but needed to be strengthened. The RACE PMU was able to report on targets and implementation progress but experienced delays in accessing the figures. Rapid access to key figures was required to facilitate informed planning regarding any changes in the work program. The PMU set up systems for acquiring data on the project targets. For the morning shift schools, the school directors sent the enrollment numbers to the regional directors. The regional directors sent the enrollment numbers to the central MEHE office and the PMU received the enrollment numbers for the morning shift from the basic education and secondary education directorates. For the second shift, the PMU had recruited 14 field coordinators who regularly visited second shift schools to report enrollment numbers in the second shifts. These field coordinators also conducted spot checks in the morning shift schools to verify the enrollment numbers obtained through the school and regional directors. During the academic year 2016-2017, the PMU started implementing a new online platform “Compiler/Attendance Application’ whereby trained schools’ IT personnel upload all data concerning students (age, date of birth, etc.) in the second shift schools, as well as register their attendance records on the tablets that were distributed to all 314 second shift schools. This platform became fully operational in the academic year 2017-2018 and enabled monitoring of student attendance daily. Similarly, the PMU was encouraged to request that schools report gender disaggregated data on enrollment and project beneficiaries to determine male to female ratios (PDO indicator # 5). Page 27 of 45 The World Bank Emergency Education System Stabilization (P152898) M&E Utilization 76. Data on performance and results progress was used to inform project management and decision- making, particularly at mid-term review (MTR) and for restructuring. During the MTR, the Government and the Bank made the decision to strengthen data collection activities to better inform future operations and to restructure the project and revise certain project indicators based on the accomplishment of many project indicators by the mid-term review. For example, efforts were made to set up more efficient systems to obtain enrollment data faster and with higher accuracy, in addition to the collection of gender disaggregated data on beneficiaries. The PMU frequently reported RACE program progress data to the REC, through several forms, including the production of periodic factsheets. Justification of Overall Rating of Quality of M&E 77. The rating of quality of monitoring and evaluation is Substantial. As the project was an emergency project, and the preparation phase was expedited, there were moderate shortcomings in the design of the monitoring and evaluation system and linking the indicators to the overall project development outcomes. However, a review of the entire implementation process through progress reports, financial reports, technical and financial audits show that the implementation of all activities was regularly and effectively tracked. Also, efforts were made to improve systems and assess the efficacy and efficiency of the activities for example through the Lebanon Textbook Quantity and Quality Analysis and the additional technical audits. The M&E system as implemented was sufficient to assess the successful implementation of project activities. Though there were some challenges faced in data collection, independent technical audits were conducted for the different project components to overcome any shortcomings. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 78. Compliance with environmental and social safeguards was satisfactory. The WB safeguards specialists conducted regular missions with MEHE. The WB environmental specialist conducted visits to rehabilitation sites to ensure compliance with safeguards. A 3-day safeguard training for the engineers at the RACE PMU was led by a WB Environmental and Safeguards Expert in Beirut. The training focused on implementing the recommendations from the Environment and Social System Assessment (ESSA). The training was timely as the PMU engineers got a deeper understanding of the ESSA requirements and visited schools for ensuring compliance with environmental and social safeguards. 79. Procurement processing was satisfactory at the level of the Ministry with shortcomings observed at the level of schools. The WB team in Lebanon conducted two post procurement reviews and out of a total of five (5) awarded contracts (US$2.15 million) eligible for post review, three (3) (60 percent) contracts of an amount of $US1.37 million were reviewed; additionally, a field visit was conducted to two school sites for physical inspection of rehabilitation activities. In both post procurement reviews, the procurement processing (procurement planning, publication, bidding, evaluation and awards), was found satisfactory. Procurement guidelines for the project were outlined in detail in the Project Operation Manual (POM). At the level of the schools, the project included a special technical audit of school-level fiduciary processes and found shortcomings in procurement handling at the school-level. There were differences in how procurement processes were implemented from one school to another and from one region to another. There were also problems of procurement documentation at the school level. The Ministry used these findings to work on Page 28 of 45 The World Bank Emergency Education System Stabilization (P152898) unifying the national procurement guidelines and to provide tailored training to regional and school staff on procurement. 80. Financial Management was satisfactory with delays experienced in the submission of two IFRs and audit reports. The RACE PMU complied with financial management requirements. Detailed modalities and arrangements for financial management were outlined in the Financial section of the project operational manual (POM) developed by the PMU in collaboration with the WB. A financial officer was appointed to ensure proper financial management. The duties of the financial officer included the management of the designated account, honoring the entity requests for issuing payments, consolidating the Project accounts and issuing quarterly Interim unaudited Financial Reports (IFRs) to reflect the Project’s overall financial position. A financial management analyst was also available at the Bank office in Beirut to help the PMU in compliance with the Bank’s guidelines for financial management. There were, however, delays in submission of quarterly IFRs and the yearly audits. The delays in submission of IFRs were due to the time that was needed for the PMU to finalize the installation of a new financial management software: ‘Sage’. Delays in IFRs submission by the end of the project were greatly reduced. Delays in the yearly audits were mostly due to the size of the audit and different donor requirements since the project agreed to joint annual audits covering all financing streams (including EESSP and UNICEF). C. BANK PERFORMANCE Quality at Entry 81. As discussed above, despite the short preparation period, the Bank managed to successfully coordinate with the GOL, identify, facilitate the preparation of and appraise the operation, including the following aspects: (a) The objective of the operation was clearly set out and was closely aligned to the WB’s broader strategy in the region and the country. (b) The design of the operation was kept simple and highly aligned with GOL’s own initiative RACE. Although the operation required coordination with other donors, effective arrangements were in place to ensure successful collaboration between donors and partners. Project oversight was done through the RACE Executive Committee (REC), and day-to-day coordination of project activities was undertaken by the RACE PMU. (c) The approach for refugee education adopted by the project to integrate refugees within the national public education system was appropriate and reflected latest international evidence in refugee education. The approach was also effective and highly efficient. (d) The design of the project was done in close consultation and coordination with other development partners. The WB staff assisted MEHE in calculating the marginal cost per student and designing a simple and cost-effective mechanism to supporting schools with meeting the increased demand. This model was validated by all other development partners supporting RACE and was used for financing the Government program. (e) The monitoring and evaluation indicators were closely aligned with project activities and therefore made monitoring and evaluation simpler. (f) The project was designed within a short timeframe to meet the emergency need and the project initiation phase was short compared to other projects in Lebanon. Page 29 of 45 The World Bank Emergency Education System Stabilization (P152898) (g) The activities under the school grants could have been better defined and better linked to existing initiatives. The team at entry did not determine the linkages between the EESSP school grants component and the institutional framework for school grants that was being established by a concurrent World Bank-financed project (EDP2). However, these linkages were determined later during implementation by adopting the school grants mechanism under EDP2 for schools in first shift and focusing the EESSP financing for schools running a second shift to support aspects of increasing social cohesion and sense of belongingness for Syrian refugee students. 82. The operational risk assessment framework in the PAD was comprehensive. The different risk categories and possible mitigation measures were identified. However, a few risks were identified, but mitigation measures were not proposed due to the difficulty of managing certain risks given the context in Lebanon. For example, the risks of delays in transfer of funds from the MOF to the MEHE was identified, but mitigation measures were not incorporated. This was because of the difficulty in identifying plausible mitigation measures to avoid lengthy approvals at ministerial and Cabinet levels. Earlier WB operations have experienced implementation delays because of the same reason and EESSP had trouble in maintaining financial liquidity due to the delays in transfer of funds. Quality of Supervision 83. The quality of Bank’s supervision was satisfactory. The Bank effectively tracked the progress on the target indicators and as mentioned earlier conducted post procurement reviews, instituted independent technical audits on school-level expenditures, as well as a study on textbook availability and analysis to effectively supervise and monitor the attainment of project development outcomes. 84. The Bank team showed high flexibility and responsiveness to the challenges faced by MEHE. The Bank provided the RACE PMU with support over the implementation period through formal implementation support visits and availability of in-country WB staff. The Bank’s team was responsive to the challenges the PMU faced in financial management and liquidity. For the academic year 2016/2017, MEHE faced a financing gap of US$9.8 million for student enrollment fees. To resolve this serious issue, the WB and MEHE restructured the project and re-allocated EESSP funds, adding around US$1.6 million to Component 1, to meet a portion of the financing gap. Similarly, due to delay in transfer to funds from the MOF, the PMU faced liquidity issues and was unable to pay the contractors on time. The WB processed six direct payments over the course of the project when the payment to contractors was overdue and the Project’s DA did not have sufficient funds available. These payments required the WB team to seek exceptional approval since the payments were below the threshold allowed for direct payments from the WB. 85. The Bank’s team was effective in using the project as a platform to support MEHE in moving from an emergency response to a longer-term development of the sector. The analytical work conducted to audit and measure project impact such as the textbook study and the school-level technical audits were effectively used by the Bank team to support MEHE in strengthening national systems. The Bank team also partnered with MEHE and UK DFID, the main donor to the MDTF financing EESSP, to establish a research program – Research for Results (R4R) – to better inform project implementation and longer-term sector planning. Among other studies, R4R measured teacher classroom practices and student learning outcomes in first and second shift schools. These activities were used to provide capacity building for DOPS counselors. For example, DOPS coaches were trained on a standardized observation tool to assess teacher performance, a pre-requisite for Page 30 of 45 The World Bank Emergency Education System Stabilization (P152898) providing relevant guidance and feedback to improve teacher effectiveness in the classroom. The R4R program also included a public perception study which measured the opinions of Lebanese and Syrian households about education service delivery in Lebanon. Another study followed a cohort of vulnerable Lebanese and Syrian children to analyze factors affecting dropout or persistence in schools. The Bank’s team and MEHE integrated those findings in the design of a longer-term operation (S2R2) that would support MEHE in shifting from an emergency response to long-term sector development. Justification of Overall Rating of Bank Performance 86. Based on the above, the rating of Bank performance is satisfactory since there were only minor shortcomings in the identification, preparation and appraisal at entry. D. RISK TO DEVELOPMENT OUTCOME 87. The risk to development outcome is low. At the time of completion, the Government had achieved the target values for all project development indicators and was committed to continuing the improvements attained through the project. As EESSP was an emergency response project, MEHE and the WB designed a second operation to ensure continued and systems-level improvement in the education sector in Lebanon through a five-year Program for results (PforR) operation ‘Support to Reaching All Children with Education 2’ (S2R2) that became effective in July 2017. S2R2 aims to promote equitable access to education services, enhance quality of student learning, and strengthen the education systems in the Recipient’s education sector in response to the protracted refugee crisis. Accordingly, the technical risk that outcomes that were achieved may not be sustained is low since the S2R2 operation had already commenced its operations and had encompassed among its different activities all the development outcomes of EESSP. Nevertheless, while S2R2 provides essential financing, additional financing will need to be mobilized from other donors, to sustain enrollment of Syrian refugee children in Lebanese public schools. Donor commitment to RACE remain strong but the risk of financing falling short is substantial. V. LESSONS AND RECOMMENDATIONS 88. It is important for an emergency operation to have a realistic objective and align the project with the Government’s own initiatives. Setting the objective at a realistic level of ambition and closely aligning the project activities with the GOL’s own RACE initiative was key to the successful implementation and outcomes of this operation, namely supporting the operational needs of the public schools in Lebanon and improving the quality of learning environment in response to the continued influx of Syrian refugees. A thorough consultative process was adapted to design the project which was very important for the successful implementation of the project. As the project was aligned with the Government’s RACE initiative, the project benefited from the existing mechanisms and from effective international coordination led by the Government. As mentioned earlier, MEHE was the implementing agency for the RACE project and headed the RACE executive committee. As MEHE was able to raise more funds for enrollment, the project contributed to the enrollment of a greater number of students as compared to the targets in the PAD. Page 31 of 45 The World Bank Emergency Education System Stabilization (P152898) 89. The donor coordination mechanism in the Lebanon education sector is a prime example of how donors and international partners can come together under one Government program, especially in a time of emergency. All donor financing to the sector was aligned under the RACE program with the establishment of the RACE Executive Committee (REC) and the RACE Program Management Unit (PMU). The coordination within the REC mechanism allowed for more effective and efficient mobilization of funds with several examples of successful reallocation of funds between donors to cover the most urgent and immediate financing gaps. The alignment under the REC also allowed the reduction of transaction cost in reporting requirements for MEHE to better focus on the crisis response by unifying the financial audits and the program progress reporting requirements. 90. The simplicity of the project design is also important, especially in the context of a crisis response plan. The simplicity and effectiveness of the marginal cost formula for the first component, Support to School Funds simplified the financing needs for meeting the increased demand on public schools in Lebanon. Similarly, the activities under the second component, Improving the Learning Environment Quality, were interventions that were targeted and proven to be successful in the international education literature. 91. During implementation, both flexibility and Bank support are critical. Being able to restructure in a timely manner was crucial to resolving the financing gap faced by the Government for the academic year 2016- 17 to cover the student enrollment costs. Similarly, the possibility to use direct payments was important to pay contractors on time when the delay in the transfer of funds limited the project’s financial liquidity. For Lebanon projects in particular, proactive intervention by the team and the WB senior management with the MOF was critical to improving the transfer of funds between Governmental entities. 92. EESSP provides a good example of how the WB could support countries in emergency and humanitarian situations such as the Syrian refugee crisis, while also integrating the WB’s expertise and added value in analytical work and systems development. The use of the additional technical audits15 to assess fiduciary capacity needs at the school level and the textbook supply chain study have allowed the project to meet its fiduciary assurance requirements, while also providing key evidence on the system for institutional strengthening. Similarly, using the project as a platform for partnership, the WB team agreed with MEHE and the main donor financing EESSP (UK DFID) to allocate additional financing for a research program – Research for Results (R4R) – to be executed by the WB as an Externally Financed Output (EFO) linked to EESSP. The R4R program generated evidence on the education sector in Lebanon to inform project implementation and longer- term sector planning. It presents a good example on how the WB can integrate research and analytical work into an emergency response project. . 15While the MEHE conducted the yearly financial audit, an additional special audit exercise was also conducted at the school level for the purpose of strengthening school-level fiduciary capacity. Page 32 of 45 The World Bank Emergency Education System Stabilization (P152898) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Support for the operational needs of the Lebanese public schools in response to the continued influx of Syrian refugee children Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students benefiting from Number 0.00 30000.00 190000.00 201851.00 school fund support 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 300500.00 720000.00 1220825.00 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Female beneficiaries Percentage 0.00 50.00 50.00 50.81 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Page 33 of 45 The World Bank Emergency Education System Stabilization (P152898) Comments (achievements against targets): Objective/Outcome: Improvement of quality of learning environment in response to the continued influx of Syrian refugee children Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of schools Number 0.00 1700.00 400.00 410.00 benefitting from supplementary financing 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Schools benefitting from Number 0.00 10.00 10.00 10.00 rehabilitation 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 300500.00 720000.00 1220825.00 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Page 34 of 45 The World Bank Emergency Education System Stabilization (P152898) Female beneficiaries Percentage 0.00 50.00 50.00 50.81 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): A.2 Intermediate Results Indicators Component: Improvement of quality of learning environment in response to the continued influx of Syrian refugee children Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of textbooks Number 0.00 4500000.00 6000000.00 8560680.00 distributed 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Data collected on use of Yes/No N Y Y Y supplemental school financing and perceived 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 impacts Comments (achievements against targets): Page 35 of 45 The World Bank Emergency Education System Stabilization (P152898) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion School improvement Percentage 0.00 60.00 60.00 100.00 activities displayed on beneficiary school walls 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): Component: Support for the operational needs of the Lebanese public schools in response to the continued influx of Syrian refugee children Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of textbooks Number 0.00 4500000.00 6000000.00 8560680.00 distributed 01-Jan-2015 09-Mar-2015 31-Oct-2017 30-Nov-2018 Comments (achievements against targets): Page 36 of 45 The World Bank Emergency Education System Stabilization (P152898) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Support for the operational needs of the Lebanese public schools in response to the continued influx of Syrian refugee children, with the following PDO level indicators: 1.Number of students benefiting from school fund support Outcome Indicators Intermediate Results Indicators Key Outputs by Component 1. Number of students supported for schooling (linked to the achievement of the Objective/Outcome 1) Objective/Outcome 2: Improvement of quality of learning environment in response to the continued influx of Syrian refugee children 1.Number of schools benefiting from supplemental school financing 2.Number of schools benefiting from rehabilitation Outcome Indicators 3.Number of students grades 1-9 benefiting from textbooks Number of DOPS (MEHE) staff benefitting from funds 1. Number of textbooks distributed 2.Data collected on supplemental school financing and perceived Intermediate Results Indicators impacts 3.School improvement activities displayed on beneficiary walls 1.Number of schools rehabilitated 2.Number of schools receiving grants Key Outputs by Component 3.Number of textbooks distributed (linked to the achievement of the Objective/Outcome 2) 4. Number of DOPs staff equipped with resources and equipment to successfully accomplish their tasks Page 37 of 45 The World Bank Emergency Education System Stabilization (P152898) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Husein Abdul-Hamid, Mohamed Yassine Task Team Leader(s) Lina Fares Procurement Specialist(s) Rima Abdul-Amir Koteiche Financial Management Specialist Haneen Ismail Sayed Team Member Piers E. Merrick Team Member Chaogang Wang Social Safeguards Specialist Emma Paulette Etori Team Member Pierre Joseph Kamano Team Member Alaa Ahmed Sarhan Social Safeguards Specialist Andrianirina Michel Eric Ranjeva Team Member Christine Makori Counsel Rock Jabbour Team Member Amal Gerges Chaoul Team Member Jocelyne Jabbour Team Member Shafick Hoossein Environmental Safeguards Specialist Mirvat Haddad Team Member Page 38 of 45 The World Bank Emergency Education System Stabilization (P152898) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY15 27.382 165,665.43 Total 27.38 165,665.43 Supervision/ICR FY15 3.905 18,521.86 FY16 78.593 35,320.59 FY17 64.570 311,440.83 FY18 43.738 214,051.64 FY19 11.675 112,637.78 Total 202.48 691,972.70 Page 39 of 45 The World Bank Emergency Education System Stabilization (P152898) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Support to Schools 13.5 15.12 42.2 Learning Environment 16.6 15.78 51.9 Quality Project Management and 1.9 1.10 5.9 Support Total 32.00 32.00 100 Page 40 of 45 The World Bank Emergency Education System Stabilization (P152898) ANNEX 4. EFFICIENCY ANALYSIS Introduction and Purpose 93. The economic analysis conducted for the ICR of the EESSP followed the most recent Bank’s guidelines (2014) and it included, in addition to a standard economic analysis, a financial analysis and a discussion of the operational efficiency of the Project. 94. An economic analysis aims to assess whether the dollar benefits of a program/intervention outweigh its dollar costs. By comparing the pros and cons of policies and programs, it helps policymakers to identify the most valuable options to pursue. The main advantage of the cost-benefit analysis (CBA) is that it monetizes all major benefits and all costs associated with a project so that alternative projects can be directly compared with each other. For this reason, CBA is often considered the “gold standard” method for evaluating programs. 95. Standard CBA provide three measures of the economic value of a project: the benefit to cost ratio (BCR), the internal rate of return (IRR), and the net present value (NPV) of the economic benefits resulting from the project. The BCR is defined as the ratio between the monetized benefits and the costs, both expressed in discounted present values. A BCR greater than 1 indicates that the NPV of the project benefits outweighs the NPV of the costs and it assesses the economic return generated for each dollar spent. Another very important output of the CBA is the IRR, i.e. the discount rate at which the NPV of all project cashflows (benefits minus costs) is equal to 0. The IRR is compared to the costs of capital to assess the project’s profitability. Finally, the NPV of the project is expected to be greater than 0 for a project to be economically attractive. Detailed Write-up on Methods Project Development Objective and Link with the Economic Evaluation 96. The project PDO was to “to support the operational needs of the Lebanese public schools and to improve the learning environments in response to the continued influx of Syrian refugee children” as a result of the crisis in Syria. To this purpose, the project aligned with the GOL’s own initiative RACE to extend education to all children in Lebanon, irrespective of nationality. The high-level outcome of the project was to preserve critical human capital of Syrian refugee children and avoid a lost generation of Syrian children. 97. The economic analysis built directly on the PDO. The project included two major components: (i) support to school funds which was used to subsidize enrollment costs for vulnerable Lebanese and Syrian refugee children; and (ii) Learning Environment Quality. The second component included four activities: rehabilitation of schools; supplemental school financing; strengthening of DOPs; and provision of textbooks for all primary grades at a national level. All the activities of component 2 were aligned to provide a supportive learning environment. 98. There are three key steps to undertake when conducting an economic analysis. First, the economic analysis needs to be linked directly to the PDOs, to the extent possible, because the PDOs define the key objectives of the project. Second, it requires assessing what portion of the changes in the PDOs can be attributed to the project, and what portion instead resulted from factors beyond the project’s control. The Page 41 of 45 The World Bank Emergency Education System Stabilization (P152898) PDOs tracked during the project do not always imply attribution, and therefore necessary steps were taken to adjust for this difference (see the methods section). Third, these benefits need to be translated into monetary value. 99. For this project, the quantifying the impact was simpler – the number of children for whom the project subsidized the enrollment. An economic evaluation generally compares an intervention approach (such as this project) with the status quo, i.e. what would have happened without the project. In this CBA, the situation at the end of the project (2016/17) was compared to the situation before the project’s implementation (2014). As other international development organizations also supported the enrollment costs of the children, we will evaluate the percentage cost supported by the EESSP and then assign the similar percentage of benefits to the project. Project Costs 100. The costs included in a CBA should reflect the marginal costs of the program compared to the status quo. Costs should include all the resources added to the system, in this case the entire amount of the initial grant. The project costs thus reflected the actual total disbursement in US$ million (US$ M) per year, as shown in Table 5. The total costs were US$32 million, and the present value as at the end of 2018 of the total disbursement is US$35.46 million. Table 5: Total Project Disbursement in US$ Million per Year 2016 2017 2018 Total Disbursement 13.2 14.1 4.7 32 (US$ M) Project Beneficiaries and the Effectiveness 101. The effectiveness of a program is often measured through the PDOs as the change in the rate of service utilization. For example, the EESSP tracked the number of vulnerable Lebanese and Syrian refugee children supported by the project for enrollment into public education. Duration 102. When conducting a CBA, the year in which costs occur and interventions are deployed has an impact on the CBA results, due to the discounting of costs and benefits. For this analysis, costs and interventions’ deployment were assigned to the year in which disbursement occurred, based on the Bank’s operations system. As the project was supporting GOL’s own initiative in its operational costs, interventions were assumed to be deployed for three years from middle of the academic year 2015 – 2016 (i.e., end of 2015 – the time of first disbursement) to end of 2018 (the closing of the project). For each year, the number of children supported by the project for enrollment was taken from the periodic Implementation Status and Results Report as laid out below in Table 6. It was possible for EESSP to support enrollment of such numbers as MEHE was able to raise significant funds from other donors. The marginal cost for each additional student was approximately US$363 at the time of project appraisal. If the marginal cost kept constant throughout the three years of the project (the assumption itself might not be justifiable, however, as we are interested in only procuring the Page 42 of 45 The World Bank Emergency Education System Stabilization (P152898) percentage of total benefits attributable to the project, the actual numeric value of the cost does not matter.) EESSP overall contributed US$15.13 million over the three years for enrollment of students costing US$80.4 million, i.e., the project contributed 18.8 percent of the total enrollment cost of the students over the two academic years. Table 6: Number of vulnerable Lebanese and Syrian refugee children supported for enrollment by EESSP and the funds used for Support to School. Number of Academic Total Component Per Student Support Students Year Expenses Supported 54,444 Lebanese US$60 (Parent Council Fund) students in US$3,264,463.71 morning shift 13,838 Non- 2015-2016 US$60 (Parent Council Fund) Lebanese students US$829,726.85 in morning shift 13,871 Non- US$160 (School Fund: US$100 and Lebanese students US$2,217,881.42 US$60 Parent Council Fund) in second shift 103,376 Lebanese US$60 (Parent Council Fund) students in the US$6,198,427.73 morning shift 2016-2017 US$160 (School Support Fund: 16,322 Non- US$100; Parent Support Fund: Lebanese students US$2,609,780.15 US$60) in the second shift 201, 851 Lebanese and non-Lebanese Overall students in US$15,120,279.86 morning/afternoon shift Conversion of the Project’s Benefits in Monetary Terms 103. The last step of the CBA consists in converting benefits (preserved productivity of enrolled students) into (present) monetary value. The assumptions for the main results (base-case scenario) were summarized hereafter. Page 43 of 45 The World Bank Emergency Education System Stabilization (P152898) 104. Benefits yielded were measured in terms of preserved increases in wages. The monetary benefits from the increased school participation were calculated as follows: for each additional child () enrolled in primary school over the years (), the number of additional children in school was multiplied by the per capita GDP ( ) in Lebanon over the years of productive life (), adjusted by the percentage of GDP from wages in Emerging Markets and Developing Economies like Lebanon is (% = 37.5) (IMF 2017), multiplied by the economic return from each additional year of schooling (), which was shown to be 5.5 percent in Lebanon (Montenegro and Patrinos 2014). _ℎ = ∑ ∗ % ∗ ∆ℎ ∗ ,, 105. Once benefits are translated into monetary value, three key assumptions need to be made to estimate the NPV, i.e. the value of the costs and benefits at project start given that benefits are yielded much further out in the future. These assumptions defined the base-case scenario: (a) Time horizon of the CBA: benefits were accrued over the productive life of the beneficiaries. Productive life is assumed from the age of 18 to 65. We assume that on average the students will start earning 5 years from now, and all the costs and benefit monetary values are calculated as of December 2018. (b) Economic growth: using a conservative approach, it was assumed that there would be an economic growth equal to 2.0 percent, the forecasted annual growth rate for Lebanon (World Bank). (c) Discount rate: this is the rate at which benefits and costs are discounted over time. In the base-case scenario of this CBA an assumption was made of a discount rate of 10 percent, the current interest rate from Banque Du Liban, as this reflected the current opportunity costs of capital investments in the country. CBA results 106. Under the base-case scenario, the EESSP yielded a ratio of benefits to costs of 1.43, suggesting that for every dollar invested, the project yielded an economic return of 1.43 dollars (Table 7). The investment of about US$32 million generated economic benefits with a net present value of US$15.1 million. The internal rate of return was 12 percent, higher than the current costs of capital in Lebanon (10 percent). Table 7: Results from Cost Benefit Analysis BCR IRR NPV Base-case scenario 1.43 12% US$15.1 M Limitations 107. Given that a prospective CBA was not conducted at appraisal due to the emergency situation in the country, the findings from the CBA for the ICR cannot be directly compared with expected results at appraisal for this specific project. Furthermore, there are no assumptions to be validated in terms of expected impact/benefits. Page 44 of 45 The World Bank Emergency Education System Stabilization (P152898) Operational Efficiency Aspects of Design and Implementation 108. The operational strength of the project lay in the fact that the project was closely aligned with the GOL’s own initiative RACE. The RACE PMU, which was already coordinating GOL activities and other donor efforts, also managed the EESSP. Time to Effectiveness, Disbursement, and Staff Turnover 109. The time from approval to effectiveness (4.5 months) and from effectiveness to first disbursement (4.0 months) was below or about the same as the average in other projects implemented at the Bank. The total time from concept to first disbursement was 11.8 months, significantly lower than 24 months on average for the Bank. In part, the shorter time reflected the fact that this was an emergency project. Disbursement was also smooth, and no significant hurdles were faced in disbursement. Table 8: Time for Project approval, effectiveness, and first disbursement compared to Bank’s average time Concept-Approval This Project 3.7 Bank 14 110. There was limited staff turnover in the RACE PMU over the years of the project. Operating Costs 111. The total operating costs from the Bank’s side for preparing and supervising the project amounted to US$857,638.13 (see Annex 2), which represents 1.8 percent of the total project’s costs. In addition to that, the in-country operating costs to implement the project were about US$1.102 million and made up 3.44 percent of the total costs. Therefore, the operating costs of the project were well below the expected costs for the Bank’s operations (5-10 percent). Page 45 of 45