Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Report Number: ICRR0022236 1. Project Data Project ID Project Name P125127 MZ-Education Sector Support Program Country Practice Area(Lead) Mozambique Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-49300,IDA-51000,IDA-60750,IDA- 31-Jul-2015 363,139,851.12 D0840,TF-99811,TF-A0429 Bank Approval Date Closing Date (Actual) 28-Apr-2011 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 71,000,000.00 147,900,000.00 Revised Commitment 365,853,113.04 145,870,000.00 Actual 363,139,851.12 145,869,999.68 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Eduardo Fernandez Joy Maria Behrens IEGHC (Unit 2) Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (page 4) and the Project Appraisal Document (PAD, page 7), the project objectives were as follows:  To support the Recipient's efforts to improve access to, and quality and equity of education. Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) The project objectives remained the same throughout the project period, however, targets for key indicators were revised for each of the project objectives (each at a different project restructuring) and therefore the split rating methodology is applied. The ICR (page 18) noted that the project was assessed according to two phases (project start to the July 2015 restructuring; July 2015 to project closing) "because of the changes made to the project scope and design, including the results framework and project components." However, IEG determined that these changes made to the project scope and design (including as reflected in key project indicators) warranted a split rating for each of the project restructurings in April 2012, July 2015, and May 2017. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 05-Apr-2012 c. Will a split evaluation be undertaken? Yes d. Components 1. Improving Access to Education (Appraisal: US$ 52.4 million; AF1: n/a; AF2: n/a; AF3: n/a; Actual: US$ 52.4 million): This component aimed to increase the level of access to primary and secondary education through accelerated construction of classrooms and the recruitment of additional teachers. Classroom construction was to be accelerated through strengthened community involvement and the use of local contractors, relying on large national or international suppliers to take advantage of economies of scale. 2. Improving the Quality of Education (Appraisal: US$ 89.1 million; AF1: n/a AF2: US$ 57.7 million; AF3: US$ 21.0 million; Actual: US$ 165.8 million) This component aimed to improve the quality of education through teacher training, provision of textbooks, school grants, and subsidies for adult literacy workers. Activities included: support to teacher training institutes to provide pre-service and in-service training, distribution of free textbooks (including identifying ways to reduced costs); reforms in primary and secondary education curriculum to focus on the development of basic skills in speaking, reading, writing and numeracy in the first years of school; support to the school grant program, which transfers per capita funds directly from the central level to schools for the purchase of basic materials to support teaching and learning; and subsidy payments to adult literacy workers. 3. HIV/AIDS Prevention and Mitigation (Appraisal: US$ 3.7 million; AF1: n/a; AF2: n/a; AF3: n/a; Actual: US$ 3.7 million): This component aimed to respond to the 11.5% HIV prevalence rate among adults. Activities included mainstreaming HIV/AIDS prevention and mitigation activities into existing school programs. 4. Strengthening Management of the Education Sector Administrative System/ Local Governance and School-Based Management (Appraisal: US$ 15.8 million; AF1: n/a; AF2: US$ 50.2 million; AF3: US$ 27.0 million; Actual: US$ 93.0 million): This component aimed to strengthen institutional capacity at all levels to Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) improve management of the education sector administrative system. Activities included: integration of the education planning and monitoring system, including fiduciary systems; and upgrading of the education management information system. The following components were added through project restructurings: Supporting Community-Based Early Childhood Development (ECD) Activities (Appraisal: n/a; AF1: US$ 40.0 million; AF2: n/a; AF3: US$ 7.0 million; Actual: US$ 47.0 million): This component aimed to expand access to quality ECD programs among children under six years of age living in rural communities and to establish the foundations of a community-based ECD service delivery system. Activities included: provision of a package of basic ECD services through community-based programs implemented by a third-party provider (with payments disbursed to the third party providers according to a results-based disbursement framework); and strengthening of institutional capacity in the areas of policy, implementation capacity, and oversight (including an impact evaluation and a study of ECD models). Institutional Strengthening and Support of Strategic Thinking (Appraisal: n/a; AF1: n/a; AF2: n/a; AF3: US$ 4.0 million; Actual: US$ 4.0 million): This component aimed to support the Ministry in evaluating the impact of activities and informing future education sector strategy. Lastly, original components 1 (Improving Access to Education) and 3 (HIV/AIDS Prevention and Mitigation) were dropped during the 2015 project restructuring. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost  The total appraised project cost (original and additional financings) was US$ 367.9 million. The actual project cost was US$ 365.9 million. Financing  The project was initially financed by an IDA Credit of US$ 71 million and an Education For All grant of US$ 90 million.  Additional Financings in the form of IDA Credits were approved for US$ 40 million and US$ 59 million and an IDA grant of US$ 50 million, and a Global Program for Education grant of US$ 57.9 million.  Project financing was channeled to the common donor fund called the Education Sector Support Fund (FASE), which supports the government's implementation of its education sector strategic plan through a sector wide approach. All FASE expenditures are agreed upon by the government and donors on an annual basis. Borrower contribution  There was no planned Borrower contribution. Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Dates  April 2012: Additional Financing of US$ 40.0 million was approved to add a new component on Early Childhood Development (ECD). The key project indicator target for "primary completion rate" was revised downward from 60.8% to 54%, due to updated population data which showed that the number of primary school students had increased, and six new intermediate results indicators were added to measure the new ECD component.  July 2015: Additional Financing of US$ 50.0 million (IDA grant) and US$ 57.9 million (Global Partnership for Education grant) was approved to continue support/ increase focus on early learning. The key project indicator target for "net enrollment rate of six-year olds in Grade 1" was revised downward from 86% to 83% (but later revised upward to 88%). The key project indicators on primary completion rate and increase transition rate were dropped, and Disbursement-Linked Indicators were introduced into the project. Components 1 (Improving Access) and 3 (HIV/AIDS Prevention and Mitigation) were dropped, the former was still being supported through the ECD and pre-service training activities. The project closing date was extended from July 2015 to December 2018.  May 2017: Additional Financing of US$ 59.0 million was approved to add a new component on analytic and strategic development activities, as well as to sustain key activities that were threatened by the ongoing financial instability. The key project indicator target for "parity index of the retention rate until Grade 3 between the highest and lowest provinces" was revised downward from 0.64 to 0.62. The project closing date was extended from December 2018 to June 2019.  June 2019: The project closing date was extended from June 2019 to December 2019, as part of the emergency support from the Bank following two cyclones that hit Mozambique. 3. Relevance of Objectives Rationale Mozambique emerged from a period of civil war and unrest that lasted until 1992, into a period marked by overall macroeconomic stability, sound policy reforms, agricultural growth, post-war reconstruction and economic expansion. These have led to decreases in the poverty head count index, although the country still remains one of the poorest in the world, ranking 165th out of 169 countries on the 2010 Human Development Index. Progress has been made in reducing child mortality and achieving universal primary education, but the country still faces significant challenges in increasing access to and quality of basic services, such as basic education, particularly in rural areas. At the time of project appraisal, net enrollment in primary education had reached 95% (2010), with 70% net enrollment rate for 6-7 year olds, but the completion rate to grade 7 was only 50.8%. Gender equality indicators were improving, with 47.3% of students in primary schools that were girls. The project objectives were highly consistent with the existing National Education Policy and related Strategic Plan For Education and Culture (2006-2010/11), which focused on increasing access and improving quality of primary education, as well as strengthening institutional capacity. An previous grant from the Education For All (EFA) fund approved in FY09 (P112052) provided financing for implementation Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) of the strategic plan, with the grant funds channeled through the Mozambique education sector donor funding pool (FASE). The project objectives were also highly consistent with the Bank's country strategies. The Country Partnership Strategy (FY 2012-15) at the time of appraisal identified a better educated, skilled and healthier workforce as a key outcome. The current Country Partnership Framework (FY17-21) includes "enhancing the skills base" as one of the objectives, which includes increasing access to preprimary education, reducing disparities in access to basic education, and improving learning outcomes. Key indicators include primary completion rate and parity index of retention rate until grade 3 between the highest and lowest provinces. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve access to education (original targets) Rationale The theory of change for this objective was overall sound. The project activities to construct classrooms, train new teachers, and establish early childhood development programs in rural areas (under the Additional Financing) were likely to contribute to the objective of increasing access to education, particularly at the primary level. Note: All project funds were channeled through the common donor sector fund FASE. In 2011, FASE represented 59% of external funding in the education sector; in 2019, it represented 90%. According to the project team, the FASE covered 90% of non-salary expenditures in the education sector and represented almost all of external financing. 40-50% of total FASE spending, over the entire project period, was financed by the Bank (IDA and GPE); hence the Bank was a significant contributor to the education sector. Outputs  Construction of 2,906 primary classrooms (target: 3,200) and 400 secondary classrooms (target: 800). The ICR (page 19) noted that classroom construction was not continued as a project intervention following the 2015 Additional Financing, however, school construction still continued under FASE and led to construction of a total of 4,603 primary classrooms and 859 secondary classrooms by project closing. During this time, the Bank team continued in its oversight role over the quality of construction as it was still responsible for procurement and safeguards in FASE. Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127)  Training of 44,734 new teachers (target: 44,000). Outcomes  The net enrollment rate for six-year olds in Grade One increased from 70% in 2011 to 93% in 2019. This surpassed the original target of 86%. See note above on contribution of Bank support to national education outcomes. Rating Substantial OBJECTIVE 1 REVISION 1 Revised Objective To improve access to education - NO CHANGE Revised Rationale n/a Revised Rating Substantial OBJECTIVE 1 REVISION 2 Revised Objective To improve access to education (revised target - July 2015) Revised Rationale In addition to the above outputs and outcomes: Outputs  Conducting of parenting information sessions on the importance and benefits of ECD. 49,551 parents/caregivers enrolled in the sessions (target: 42,000).  Establishment of 350 early childhood development centers in rural communities in five provinces, of which 100% were functional by project closing. This was designed as a pilot initiative with the 2015 Additional Financing. The ECD centers offered a basic package of services, the key features of which included strong community involvement, teachers recruited at the community level, low cost technology, well-defined protocols, ongoing monitoring and training by the third-party providers, Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) regular communication with parents and parenting classes, linkage with primary schools, and partnership with local authorities. Outcomes  50,742 children aged 3-5 years benefitted from the ECD centers over the project period (target: 49,000). 90% of the schools had school attendance above 80 percent and the ratio of children per teacher was 15.1 to 1. According to the ICR (page 24), these project-supported ECD centers represented 46 percent of community preschools in the country and nearly one-fourth of all available preschool programs.  The net enrollment rate for six-year olds in Grade One increased from 70% in 2011 to 93% in 2019. This surpassed the revised target of 83%, and also the second revised target (May 2017 restructuring) of 88%. See note above on contribution of Bank support to national education outcomes.  The net intake rate for six-year olds in Grade One in ECD participating districts increased from 71% in 2011 to 81% in 2014. This achieved the target of 80%. (Note: this indicator was no longer tracked following the 2015 restructuring). This outcome can be strongly attributed to Bank support as the project funded all aspects of ECD program including salaries. Revised Rating Substantial OBJECTIVE 1 REVISION 3 Revised Objective To improve access to education - NO CHANGE Revised Rationale n/a Revised Rating Substantial OBJECTIVE 2 Objective To improve quality of education (original targets) Rationale The theory of change for this objective was overall sound. The project activities to train teachers (pre-service and in-service), improve teaching practices, update the curriculum, increase availability of textbooks, and improve local school management capacity and access to resources were likely to contribute to the objective to improve quality of education, particularly at the primary level. Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Outputs  Piloting of new pre-service teacher training program at six institutes, with a new curriculum which included a stronger practicum component. The program increased from one to three years of pre- service training, with two years of theoretical content and one full year of teaching practice. A new 12+3 teacher training model aimed at improving quality was rolled out. The ICR (page 26) noted that this was a critical reform in the education sector and a notable achievement of the project.  Provision of school grants to all school nationwide, which were the main source of funding per student managed at the school level, used to purchase basic materials to support the learning process and student attendance.  Establishment of a system for learning assessment at the primary level. While the learning assessments were administered, the system is not yet providing useful information for education policy and practice. A Service Delivery Indicators (SDI) survey was also conducted in 2018 on a sample basis, designed to ensure national representativeness and comparability with the previous SDI (2014).  Development of a textbook strategy for provision of textbooks to primary schools, which addressed writing and printing of textbooks, copyright ownership of workbooks for grades 1 and 2, and the transition from negotiated purchase to competitive procurement of textbooks and teacher's guides for grades 3-7. Licenses for the Grades 1 and 2 books are now owned by the Ministry, reducing the unit cost of the books, as they are now printed on a competitive basis.  Provision of adult literacy workers, who reached 1.3 million beneficiaries (target: 2.4 million)  Revision of the curriculum, according to changes introduced by the Education System Law. Grade 1 revisions were introduced in schools in 2017, Grades 2 and 3 revisions were introduced in 2018 and 2019. Grades 4-6 revisions were being finalized. These included HIV/AIDS awareness and reproductive health modules into pre-service training programs, a social action plan to retain and/or reintegrate civil servants who were affected by HIV/AIDS, and implementation of a social assistance program that included the creation of HIV/AIDS committees in all districts, the creation of a new social assistance function, training for civil servants on HIV/AIDS prevention, and production of sensitization materials.  Training of district supervisors on use of the school supervision manual, which included provision of tablets to collect and upload school data to the web-based school monitoring system. 6,153 primary school directors participated in management training (target: 3,500).  Conducting of studies on the following: assessment of competencies of tenth graders, research on high failure rate in mathematics and natural sciences. Outcomes  The transition rate EP2/next level post primary rate increased from 79% in 2011 to 92% in 2014, although falling short of the target of 95%.  The retention rate to Grade 3 increased from 69.8% in 2014 to 77% in 2019, achieving the target of 75%.  The dropout rate in primary education decreased from 7.8% in 2011 to 7.4% in 2014 and to 5.2% in 2018. This achieved the target of 6.0%. For girls, the dropout rate decreased from 8.1% in 2011 to 5.6% in 2018. Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127)  The availability of trained teachers increased, through training of 44,734 new qualified teachers (target: 44,000) and in-service training for 55,085 teachers in 1st cycle (grades 1 and 2) (target: 27,500).  The availability of school materials increased, through the purchase and distribution of approximately 40 million textbooks. The percentage of Grades 1 and 2 students with Portuguese and math textbooks decreased from 86.2% in 2014 to 79.7% in 2019, falling short of the target of 92%. According to the ICR (page 26), between 2014 and 2018, the "indicator was being met"; however, in 2018, the number of students entering grades 1 and 2 exceeded the expected enrollment, therefore the number of textbooks printed in 2017 was not enough to increase or maintain the ratio of textbooks per student. The 2019 estimation was expected to maintain the 1:1 textbook ratio, however, two cyclones hit the country which impacted the distribution of textbooks in the areas affected by the cyclones.  The percentage of school grants disbursed according to school improvement plans increased from 95% in 2011 to 100% in 2019, achieving the target of 99%. The percentage of primary schools with grants available before February 28 of each year (beginning of the school year) - a measure of the efficiency of grant disbursement - was 97% in 2019, achieving the target of 95%. The percentage of primary schools that comply with standards of transparency was 80% in 2019, achieving the target of 80%. This meant that 80% of primary schools have functioning school councils that comply with standards of transparency and accountability at the school level (i.e. school councils are elected according to the school council manual, there is an approved school development plan, information is public displayed).  The percentage of primary schools supervised by the education district authorities was 85.1% in 2019, surpassing the target of 80%. According to the ICR (page 27), the project contributed to systematization of district school supervision, which correlated with lower teacher absenteeism.  As reported in the ICR (page 22), the HIV/AIDS activities had been completely mainstreamed by the time of the 2015 AF and therefore, the component did not need to be continued. However,  The primary completion rate initially decreased from 47% in 2011 to 45% in 2014, then increased to 48% in 2018. This fell short of the original target of 60.8%. The ICR (page 29) noted that the initial decrease was explained, in large part, by the fact that primary enrollments surpassed government's expectations (primary gross enrollment rates increased from 114.4% in 2011 to 135% in 2014) and that this contributed to challenges in improving completion rates because overaged children entered the school system leading to higher repetition and drop outs, which impacted the completion rate. This mid-course outcome contributed to the government's decision to re-focus the project and FASE primarily on early learning. Achievement of this objective, under the original targets, is rated Modest due to evidence of increased availability of textbooks, increased local governance of schools, and increased supervision of schools, which likely contributed to the improved retention, drop out, and retention rates. However, there was a significant shortcoming in achieving the target for improved primary completion rate. Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Rating Modest OBJECTIVE 2 REVISION 1 Revised Objective To improve quality of education (revised targets - April 2012 restructuring) Revised Rationale Outputs See outputs reported above Outcomes In addition the outcomes reported above:  The primary completion rate initially decreased from 47% in 2011 to 45% in 2014, then increased to 48% in 2018. This fell short of the revised target of 54%. Achievement of this objective is rated Substantial due to evidence of increased availability of textbooks, increased local governance of schools, and increased supervision of schools. While the key indicator - primary completion rate - only increased from 47% to 48% and still fell short of the target of 54%, the other relevant outcomes reported above on retention rate, transition rate, and drop-out rate, point to overall substantial improvements in the quality of education. Revised Rating Substantial OBJECTIVE 2 REVISION 2 Revised Objective To improve quality of education - NO CHANGE Revised Rationale n/a Revised Rating Substantial Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) OBJECTIVE 2 REVISION 3 Revised Objective To improve quality of education - NO CHANGE Revised Rationale n/a Revised Rating Substantial OBJECTIVE 3 Objective To improve equity of education (original targets) Rationale For this objective, the ICR (page 9) discussion of the theory of chain referred to disadvantaged or vulnerable populations (rural areas, HIV/AIDS-affected). The project activities to train teachers on HIV/AIDS education, improve local school management capacity and access to financial resources, and establish early childhood development centers in rural areas were likely to contribute to the objective to improve equity according to disadvantaged populations. Outputs  Establishment of 350 early childhood development centers in communities in five provinces.  Provision of school grants, which were used to provide additional resources (i.e. uniforms, learning materials) for vulnerable districts to help retain disadvantaged students. Outcomes  The gender parity index improved from 0.90 in 2011 to 0.95 in 2014 and remaining at that level in 2019. This achieved the target of 0.94. Note: This indicator was dropped at the July 2015 restructuring.  The gender parity index upon Grade 1 entry for project districts improved from 0.92 in 2011 to 0.95 in 2018, achieving the original target of 0.94 but falling short of the revised target of 0.96.  The parity index of the retention rate (until Grade 3) between the highest and lowest province decreased from 0.61 in 2014 to 0.51, falling short of the original target of 0.64. According to the ICR (page 28), although all provinces improved retention rates, the Southern provinces (with higher rates) showed a larger increase than the North and Central provinces (with lower rates), thus worsening the parity index.  The number of districts with a pupil:teacher ratio over 80 decreased from 17 to 9 by project closing. However, the ICR (page 29) noted that this was largely due to the fact that the administrative division Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) (or district lines) were re-drawn, which resulted in an increase of the baseline number of districts with high pupil: teacher ratios from the original 12 to revised 17. Achievement under the original targets is rated Modest due to mixed evidence of improved equity, as gender parity improved but other aspects of equity (disadvantaged areas) did not improve. Rating Modest OBJECTIVE 3 REVISION 1 Revised Objective To improve equity of education - NO CHANGE Revised Rationale n/a Revised Rating Modest OBJECTIVE 3 REVISION 2 Revised Objective To improve equity of education - NO CHANGE Revised Rationale n/a Revised Rating Modest OBJECTIVE 3 REVISION 3 Revised Objective To improve equity of education (revised target - May 2017) Revised Rationale In addition to the outputs and outcomes reported above:  The parity index of the retention rate (until Grade 3) between the highest and lowest province decreased from 0.61 in 2014 to 0.51, falling short of the revised target of 0.62. Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Achievement under the original targets is rated Modest due to shortcomings in improved equity in disadvantaged areas. Revised Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy, under the original targets, is rated Modest due to Substantial efficacy of the first objective to increase access to education, but Modest efficacy of the second and third objectives to improve quality and equity of education. Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Following the April 2012 restructuring, Overall Efficacy, under the revised targets, is rated Substantial due to Substantial efficacy of the first and second objectives to increase access and improve quality of education, but Modest efficacy of the third objective to improve equity of education. Overall Efficacy Revision 1 Rating Substantial OBJR2_TBL OVERALL EFFICACY REVISION 2 Overall Efficacy Revision 2 Rationale Following the July 2015 restructuring, Overall Efficacy, under the revised targets, is rated Substantial due to Substantial efficacy of the first and second objectives to increase access and improve quality of education, but Modest efficacy of the third objective to improve equity of education. Overall Efficacy Revision 2 Rating Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Substantial OBJR3_TBL OVERALL EFFICACY REVISION 3 Overall Efficacy Revision 3 Rationale Following the May 2017 restructuring, Overall Efficacy, under the revised targets, is rated Substantial due to Substantial efficacy of the first and second objectives to increase access and improve quality of education, but Modest efficacy of the third objective to improve equity of education. Overall Efficacy Revision 3 Rating Substantial 5. Efficiency A cost benefit analysis was not prepared fort project appraisal. Instead, the PAD (pages 21-22) provided an economic rationale for the project, which included the significant positive externalities that arise from investing in primary education. Also, the private and social rates of returns to education were expected to be high, leading to significant benefits in health, productivity and consumption. A cost benefit analysis was conducted at project closing (ICR, Annex 4). Benefits were estimated according to increased wage levels from completing primary education and higher labor earnings from higher quality of primary education received. The number of students benefitting was estimated using project administrative data. Costs were calculated according to project investment costs. The net present value was estimated at US$ 196.0 million, with an internal rate of return at 13%. The ratio of benefits to costs was calculated as 1.54. Textbook unit costs decreased due to the government obtaining of licenses for grades 1-2 textbooks, which reduced unit costs by 17% between 2011 and 2015. Unit costs per classroom were appraised at US$ 16,000, while actual unit cost (based on actual number of classrooms constructed) was $15,850. Lastly, the ICR suggested that the use of multiple Additional Financings (rather than new operations) contributed to overall lower transaction costs. However, there were some initial implementation delays due to funding structure (pooled donor fund) and procurement problems. In 2013, classroom construction was slowed down due to funding delays from one of the bilateral partners. In 2016, predictability of disbursements was negatively affected by hidden external borrowing, which led to the suspension of budget support from donors. There were also procurement challenges with the ECD activities, due to the unforeseen length of procurement procedures for high value contracts, also contract reviews related to construction quality and durability, which led to delays in the pilot program and thus a lower number of participating communities. Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 100.00 ICR Estimate  13.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Overall outcome is determined for each of the following project phases, and then weighted according to the amount of the loan disbursed): Project under original targets (April 2011 to April 2012) - Moderately Satisfactory; 0% disbursed Relevance is rated High due to strong alignment with country conditions, country education strategy, and Bank strategy. . Efficacy is rated Modest due to Substantial efficacy of the second objective to increase access to education, but Modest efficacy of the second and third objectives to improve quality and equity of education. Efficiency is rated Substantial for the entire project period, due to evidence of implementation efficiency and substantial rate of return on investment, although there were some inefficiencies due to implementation delays. Project under first revised targets (April 2012 to July 2015) - Satisfactory; $104.02 million out of $363.14, or 28.6%, disbursed Relevance is rated High due to strong alignment with country conditions, country education strategy, and Bank strategy. . Efficacy is rated Substantial due to Substantial efficacy of the first and second objectives to increase access and improve quality of education, but Modest efficacy of the third objective to improve equity of education. Efficiency is rated Substantial for the entire project period, due to evidence of implementation efficiency and substantial rate of return on investment, although there were some inefficiencies due to implementation delays. Project under second revised targets (July 2015 to May 2017) - Satisfactory; $30.0 million out of $363.14, or 8.3%, disbursed Relevance is rated High due to strong alignment with country conditions, country education strategy, and Bank strategy. . Efficacy is rated Substantial due to Substantial efficacy of the first and second objectives to increase access and improve quality of education, but Modest efficacy of the third objective to improve equity of education. Efficiency is rated Substantial for the entire project period, due to evidence of implementation Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) efficiency and substantial rate of return on investment, although there were some inefficiencies due to implementation delays. Project under third revised targets (May 2017 to December 2019) - Satisfactory; $229.12 million out of $363.14, or 63.1%, disbursed Relevance is rated High due to strong alignment with country conditions, country education strategy, and Bank strategy. . Efficacy is rated Substantial due to Substantial efficacy of the first and second objectives to increase access and improve quality of education, but Modest efficacy of the third objective to improve equity of education. Efficiency is rated Substantial for the entire project period, due to evidence of implementation efficiency and substantial rate of return on investment, although there were some inefficiencies due to implementation delays. Approval to April 2012 0% at MS (0 x 3) 0 April 2012 to July 2015 28.6% at S (.286 x 4) 1.144 July 2015 to May 2017 8.3% at S (.083 x 4) 0.332 May 2017 to Closing 63.1% at S (.631 x 4) 2.524 TOTAL 4.000 (Satisfactory) a. Outcome Rating Satisfactory 7. Risk to Development Outcome Implementation capacity is likely to be sustained through the continuation of Bank operations that will deliver similar interventions (for example, Emergency Resilient Recovery project, Disaster Risk Management and Resilience project, and Improving Learning and Empowering Girls project). Legislation in support of critical educational reforms, including ECD, will also help to sustain sector achievements. However, macroeconomic challenges entail significant risk to development outcomes. The public fiscal situation is precarious, with budget allocations below sector needs and modest prospects for economic growth in the near term, particularly due to the COVID pandemic. 8. Assessment of Bank Performance a. Quality-at-Entry Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) The project design was based on the government's education sector plan, which drew upon well- established interventions known to improve learning, and the project objectives were in full alignment with the government sector objectives. The pooled donor fund, FASE, further reflected the alignment with and coordination of donor efforts, including in monitoring and evaluation arrangements. According to the project team, as the Bank was the supervising entity for FASE, it played a strong role in ensuring alignment of activities and prioritization of activities according to the government sector strategy. This key role of the Bank also provided assurance to donors to participate in the FASE. The theory of change and results framework were overall sound, with appropriate and realistic targets given local capacity constraints. One shortcoming in the results framework was choice of the key indicator for the objective to "improve equity," which focused on gender parity although the theory of chain and project preparation document did not appear to include targeted interventions to increase girls' enrollment in school. The risk assessment identified low implementing agency capacity, in the area of fiduciary management, as a medium risk, and therefore included mitigation measures to support fiduciary functions at the local level. M&E arrangements were clearly delineated, in coordination with donors, and included both monitoring and evaluative activities. Quality-at-Entry Rating Satisfactory b. Quality of supervision Project supervision was marked by proactive engagement by the Bank team, in response to changing country conditions and sector priorities. The Bank team responded to the mid-term learning assessment results (at the time of the Mid-Term Review), which showed that learning outcomes were very low in the early grades, and thus adjusted the project design accordingly in line with the government's renewed focus on early learning. The Bank team also utilized the project restructurings to refine the results framework including revising targets (based on updated data) and wording of indicators to better capture intended outcomes. As noted in the ICR (page 35), the project restructurings were proactive responses to changes in the context and sector priorities, rather than any challenges with the original project design. The agreement to channel project funds through FASE facilitated donor coordination on sector activities and monitoring; however, it also led to unavoidable delays in project implementation progress when specific donors were delayed in disbursing funds. The Bank team was particularly integral in supporting fiduciary performance, given its lead role in this area for the FASE, by providing intensive supervision mission support including hiring of specialized staff (ECD, textbooks) and developing local capacity. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was robust for the first two objectives to "increase access" and "improve quality," with key outputs identified that were likely to lead to the intended outcomes, monitorable indicators, and realistic targets. However, the results chain for the third objective to "improve equity" had some weaknesses, as the selected key indicator measured gender parity while project outputs did not appear to target female beneficiaries. The subsequent disbursement-linked indicator (DLI) related to that objective - number of districts with high pupil:teacher ratios - aimed to allocate existing resources more equitably and efficiently and was more relevant; however, the results framework still focused on gender parity. Monitoring arrangements included joint monitoring of sector-wide performance, with a sub-set of indicators tracked for the project. Evaluative activities included learning assessments and an ECD impact assessment. b. M&E Implementation Monitoring of implementation progress and achievement of development objectives was regularly conducted. Annual joint reviews were also carried out as planned by the FASE donors and government. The results framework was revised on several occasions to refine indicators, both targets and definitions. Two Service Delivery Indicator surveys were completed (2014 and 2018) to assess learning outcomes, and a process evaluation of the ECD program was completed to inform future programs in the expanding ECD sub-sector. An impact evaluation of the ECD program was ongoing at the time of ICR preparation. c. M&E Utilization Project indicator data were used on a regular basis to track project implementation progress, identify implementation bottlenecks, and measure progress towards achieving outcomes. These monitoring data were used to inform project decisions, in particular raising awareness of low learning outcomes in the early grades and shifting government prioritization on early grades and teacher absenteeism. The ICR (page 38) also noted that the results of the ECD process assessment contributed to the establishment of ECD as a sub-sector in education and highlighted key lessons that will be used to inform the next phase of the expansion of the preschool system. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category "B" project due to potential environmental impacts from construction and rehabilitation of classrooms. Safeguard policies on Environmental Assessment (OP/BP Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) 4.01), Pest Management (OP 4.09), and Involuntary Resettlement (OP/BP 4.12) were triggered. An Environmental and Social Management Framework was prepared, including a pest management approach due to the potential need to purchase pesticides to control for termites. A grievance redress mechanism was also established. According to the ICR (page 39), safeguards were monitoring consistently during project implementation (including 11 joint safeguards supervision missions between 2015 and 2019), with ratings in the satisfactory range. The one exception in satisfactory ratings occurred in 2011 due to the fact that safeguards staff had not yet been hired and there was limited safeguards communication with contractors. b. Fiduciary Compliance The Bank was responsible for all fiduciary functions for FASE, on behalf of all donors. Financial management: According to the ICR (page 39), financial management performance was rated satisfactory throughout the project period. Interim financial reports were submitted in a timely manner. Audits were also submitted regularly and opinions were unqualified. The final audit report was not yet submitted at the time of ICR preparation. Procurement: According to the ICR (page 39), procurement performance was rated moderately satisfactory throughout the project period. Despite training provided by Bank staff to implementing units to ensure compliance with procurement guidelines, local capacity and coordination amongst the various departments within the Ministry of Education remained a challenge and contributed to implementation delays. However, prior and post reviews of contracts did not reveal any procurement irregularities, according to the project team. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Quality of ICR --- Substantial 12. Lessons Lessons drawn from the ICR, adapted by IEG:  The sector-wide approach, while complex, can be an effective mechanism for support to an entire sector system. In the case of this project, there was strong coordination among donors to avoid overlap of interventions, clear alignment with sector strategy and priorities, clear monitoring arrangements to assess progress and make adjustments throughout the project period, and a strong supervising role of the Bank to ensure quality of implementation.  Third-party (private) parties can play a critical implementation role in establishing new programs. In the case of this project, the third party providers introduced ECD interventions that represented a new sub-sector for the Ministry of Education, thereby ensuring effective implementation that achieved results, yet also supported capacity development such that ECD could become institutionalized as a sub-sector within the Ministry. The third-party providers enabled the Ministry to gain valuable experience in implementing ECD in rural communities, curriculum development, institutional structures, community mobilization, and ECD center construction. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well-organized and results-oriented, despite a complex operation with a number of restructurings. The theory of change was clear, particularly given the sector-wide approach, such that contribution of the Bank support (IDA and GPE) could be well-established. The ICR was concise and adhered to guidelines overall, although the approach to the split rating methodology was incorrect. The ICR (page 18) noted that "the changes made at the time of the July 2015 AF are considered substantial enough to view the project in two distinct phases and therefore, a split evaluation is judged to be appropriate." However, the determination on whether to apply a split rating should have been based on the fact that the key project indicator targets were revised downward (see IEG's application of the approach above). a. Quality of ICR Rating Substantial Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ-Education Sector Support Program (P125127) Page 21 of 21