Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004482 IMPLEMENTATION COMPLETION AND RESULTS REPORT Credit Number H354-AF and ARTF 093962 ON A GRANT IN THE AMOUNT OF SDR 19.9 MILLION (US$ 30 MILLION EQUIVALENT) AND A GRANT IN THE AMOUNT OF US$ 408 MILLION TO THE ISLAMIC REPUBLIC OF AFGHANISTAN FOR THE SECOND EDUCATION QUALITY IMPROVEMENT PROGRAM July 31, 2018 Education Global Practice South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective March 22, 2018) Currency Unit = Afghani Af 48.50= US$1 US$ 1.358 = SDR 1 FISCAL YEAR March21 – March 20 Regional Vice President: Hartwig Schafer Country Director: Shubham Chaudhury Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager: Cristian Aedo Task Team Leader(s): Nathalie Lahire, Matiullah Noori ICR Main Contributor: Sandra Beemer ABBREVIATIONS AND ACRONYMS ACER Australian Council for Education Research AF Additional Financing ARTF Afghanistan Reconstruction Trust Fund BESST Building Education Supply Systems for Teachers Program CPF Country Partnership Framework DED District Education Department DTE Diploma in Teacher Education DT3 District Teacher Training Teams EA Environmental Assessment EMIS Education Management Information System EQRA Education Quality Reform in Afghanistan EQUIP I and II Education Improvement Program I and II ESSF Environmental and Social Safeguards Framework FM Financial Management GDP Gross Domestic Product GOA Government of Afghanistan GRM Grievance Redress Mechanism HR Human Resources HRMIS Human Resources Management Information System ICR Implementation Completion and Results Report IDA International Development Association IDP Internally Displaced Persons IFR Interim Financial Report INSET In-service Education for Teachers ISD Infrastructure Service Department ISM Infrastructure Service Management ISMIS Infrastructure Service Management Information System ISN Interim Strategy Note ISR Implementation Status and Results Report PDO Project Development Objective M&E Monitoring and Evaluation MoE Ministry of Education MDGs Millennium Development Goals MIS Management Information Systems MRRD Ministry of Rural Rehabilitation and Development MTR Mid-term Review NATO North Atlantic Treaty Organization NESP National Education Strategic Plan NGO Non-Governmental Organization OPCS Operations Policies and Country Services RF Results Framework SDR Special Drawing Rights SIGAR US Special Inspector General for Afghanistan Reconstruction SIP School Improvement Plan SMD Social Mobilization Directorate SABER System Approach for Better Education Results SMC School Management Committee SMT School Management Training Programs SMU Social Mobilization Unit TA Technical Assistance TED Teacher Education General Directorate TF Trust Fund TMIS Teacher Management Information System TTC Teacher Training College WB World Bank UN United Nations USAID United States Agency for International Development TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 12 II. OUTCOME .................................................................................................................... 18 A. RELEVANCE OF PDOs ............................................................................................................ 18 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 18 C. EFFICIENCY .......................................................................................................................... 26 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 27 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 27 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 29 A. KEY FACTORS DURING PREPARATION ................................................................................ 29 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 31 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 33 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 33 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 35 C. BANK PERFORMANCE ........................................................................................................... 36 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 38 V. LESSONS AND RECOMMENDATIONS ............................................................................. 38 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 41 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 58 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 61 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 62 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 69 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 70 ANNEX 7. ADDITIONAL INDICATOR DATA............................................................................. 72 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Afghanistan - Second Education Quality Improvement P106259 Program Country Financing Instrument Afghanistan Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Islamic Republic of Afghanistan Ministry of Education Project Development Objective (PDO) Original PDO The project development objective is to increase equitable access to quality basic education especially for girls through schoolgrants, teacher training and strengthened institutional capacity with support from communities and private providers. Revised PDO The objective of the Project is to assist the Recipient in increasing equitable Access to quality basic education, especially forgirls. Page 1 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 30,000,000 29,243,702 28,856,992 IDA-H3540 123,582 115,885 115,885 TF-91943 408,000,000 408,000,000 393,364,888 TF-93962 Total 438,123,582 437,359,587 422,337,765 Non-World Bank Financing Borrower 0 0 0 US: Agency for International 22,000,000 0 0 Development (USAID) Total 22,000,000 0 0 Total Project Cost 22,000,000 437,359,587 422,337,765 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 31-Jan-2008 20-Mar-2008 31-Jul-2011 01-Sep-2012 01-Sep-2012 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 10-Mar-2009 5.71 Additional Financing 12-Jun-2012 28.45 Additional Financing Change in Loan Closing Date(s) 13-Aug-2015 28.86 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Institutional Arrangements Change in Implementation Schedule 07-Dec-2016 28.86 Change in Loan Closing Date(s) 28-Jun-2017 28.86 Change in Results Framework Page 2 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 22-May-2008 Satisfactory Satisfactory 0 02 19-Nov-2008 Satisfactory Satisfactory 5.31 03 15-May-2009 Satisfactory Satisfactory 7.63 04 30-Oct-2009 Satisfactory Satisfactory 13.51 05 30-Apr-2010 Satisfactory Satisfactory 14.23 06 08-Dec-2010 Moderately Satisfactory Moderately Satisfactory 17.38 07 25-Jun-2011 Moderately Satisfactory Moderately Unsatisfactory 19.15 08 23-Oct-2011 Moderately Satisfactory Moderately Satisfactory 23.10 09 01-Mar-2012 Moderately Satisfactory Moderately Satisfactory 28.45 10 20-Aug-2012 Moderately Satisfactory Moderately Satisfactory 28.45 11 02-Jan-2013 Moderately Satisfactory Moderately Satisfactory 28.86 12 17-Jun-2013 Moderately Satisfactory Moderately Satisfactory 28.86 13 26-Nov-2013 Moderately Satisfactory Moderately Satisfactory 28.86 14 04-Jul-2014 Moderately Satisfactory Moderately Satisfactory 28.86 15 24-Dec-2014 Moderately Satisfactory Moderately Satisfactory 28.86 16 29-Jun-2015 Moderately Satisfactory Moderately Unsatisfactory 28.86 17 03-Jan-2016 Moderately Satisfactory Moderately Unsatisfactory 28.86 18 22-Jul-2016 Moderately Satisfactory Moderately Unsatisfactory 28.86 19 16-Dec-2016 Moderately Satisfactory Moderately Unsatisfactory 28.86 Page 3 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) 20 24-Jun-2017 Moderately Satisfactory Moderately Unsatisfactory 28.86 21 18-Jan-2018 Moderately Satisfactory Moderately Unsatisfactory 28.86 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 29 Central Government (Central Agencies) 13 Sub-National Government 16 Education 71 Primary Education 17 Secondary Education 17 Tertiary Education 37 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 101 Gender 33 Education 68 Access to Education 34 Education Financing 34 ADM STAFF Role At Approval At ICR Regional Vice President: Praful C. Patel Hartwig Schafer Country Director: Alastair J. McKechnie Shubham Chaudhuri Senior Global Practice Director: John A. Roome Jaime Saavedra Chanduvi Practice Manager: Michelle Riboud Mario Cristian Aedo Inostroza Page 4 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Nathalie Lahire, Matiullah Task Team Leader(s): Scherezad Joya Monami Latif Noori ICR Contributing Author: Sandra F. Beemer Page 5 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Country Context. In 2008, Afghanistan was a country that had been besieged by civil strife for almost three decades. Conflict had left the country devastated with destroyed infrastructure, fragmented institutions and a lack of basic health, education and sanitation facilities in most parts of the country. During the period of 2003-2008, Afghanistan had achieved results in a challenging context. By 2008, Afghanistan had experienced growth momentum with Gross Domestic Product (GDP) projected to grow at approximately 10 percent from 2008-09 onwards, assuming sustained growth in agriculture, construction and the services sector. Nevertheless, the situation remained very fragile, especially in view of the tense, and in some areas deteriorating, security situation. Notwithstanding the economic recovery and the implementation of a development strategy, the country had not emerged fully from its state of conflict. Although GDP was projected to grow, the government’s ambitious reform agenda had many challenges such as the heavy dependence on international aid, institutional and staff capacity constraints, a low domestic revenue base, persistent insecurity and the threat of large illicit opium activity and poverty. The most urgent interventions included the restoration of countrywide security, rehabilitation of key economic and social infrastructure, and the establishment of functional state institutions across the country. 2. During the period 2001-2008, the Government of Afghanistan (GOA) had taken bold steps to tackle the rehabilitation and development needs of the country. In December 2001, Afghan factions met in Bonn, Germany under UN auspices to agree on a transitional process leading to elections for a “broad-based, gender-sensitive, multi-ethnic, and fully representative Government�. In January 2004, a new Constitution had been approved. By December 2004, there had been the first democratically elected President. The inauguration of the National Assembly in December 2005 marked the formal conclusion of the Bonn process, which was a major achievement. Stabilization and recovery beyond the Bonn process was anchored in the Afghanistan Compact presented and agreed at a high-level international conference in London in January 2006 which covered the period 2006- 2010. This new framework was based on mutual commitments between GOA and the international community, with detailed benchmarks and timelines. The Compact identified three critical and interdependent areas or pillars of activity which were mirrored in the Bank’s Interim Strategy Note for Afghanistan: (i) security; (ii) governance, Rule of Law and Human Rights; and (iii) Economic and Social Development. 3. Sector Context. Afghanistan was one of the poorest and least educated countries in the world. At the time of appraisal, education statistics were relatively unreliable. However, the statistics being reported showed that overall literacy rates in Afghanistan were believed to be about 33 percent, with very large gender disparity and a significant difference between rural areas (where possibly 75 percent of people live) and urban centers. The Constitution of the Page 6 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Afghan year 13821 (2003) mandated that the State would provide free and compulsory education for all children from Grade 1 to Grade 9 and a free education to the completion of Grade 12. Although further legal and regulatory work was required, in general terms, the Constitution and the structure of the government (Ministries) assigned the responsibilities for all education within those ranges to the Ministry of Education (MoE). This included provision of schools; teachers and materials; regulation of the private education providers; preprimary education; development of curriculum standards setting and monitoring; training of teachers other than in universities; technical and vocational education; and special education. The MoE employed approximately half the entire civil service of the Afghanistan - there were an estimated 143,000 teachers, and an additional 38,000 staff in administrative and other categories. 4. The demand for education after the fall of the Taliban in 2002 had continuously exceeded expectations and the capacity of supply for educational services. Since 2002, Afghanistan had had considerable success in increasing school enrollment. There were approximately six million children enrolled in school (Grades 1-12), with nearly 35 percent being girls compared to a little more than one million in the previous five years, with almost no girls. These enrollment figures were by far the largest in the history of Afghanistan. Beyond the major achievement of enrolling almost six million children in school, the challenges remained daunting. One of the major bottlenecks that had affected the pace and scope of reconstruction was a massive skill deficit that cut across all institutions in Afghanistan, from principals to teachers, and from managers to skilled labor force needed for the bulk of the reconstruction work and its maintenance. Not least of the issues was the continued conflict in some parts of the country. Specifically for the education sector the challenges included the following: (i) a little less than half of the school age population remained out of school with significant gender and provincial disparities; (ii) while the number of teachers increased, only 22 percent met the minimum qualifications of Grade 14 and only 28 percent were female and located primarily in urban areas; (iii) there was a new secondary school curriculum but new books had yet to be developed; (iv) only 25 percent of schools had usable buildings with lack of access for thousands of communities; (v) nearly 6 percent of schools were burned or closed due to terrorism; (vi) 30,000-50,000 students graduate from high school yearly and only one third of them are admitted to universities; (vii) an estimated 11 million Afghans were illiterate (approximately 28-37 percent of the population); and (viii) urgent need for civil service reform of education ministry personnel including principals and teachers. 5. It was recognized by the government and the donor community that investment in the education sector was an essential component for the future. Quality education was a critical ingredient for poverty alleviation and economic growth in Afghanistan. The Government of Afghanistan (GOA) had developed a strategic vision for education articulated in the Ministry of Education’s (MoE) National Education Strategic Plan 2007-2010 (NESP 2007-2010)2 which was endorsed by all stakeholders in the sector. The NESP was a five-year roadmap for immediate educational development activities in the country and provided a detailed discussion of the interventions that would begin the process that would lead the country to achieving the Compact benchmarks and Afghanistan’s MDGs. Educational development in Afghanistan for the five-year period was prioritized into eight programs-literacy, curriculum, teacher education, construction, public administration and management, vocational education and general education. The GOA’S international commitment to achieving the Millennium Development Goals (MDGs) was ambitious especially in light of the emergency context in the country but the MoE has launched an aggressive campaign to work towards their achievement with the development and the implementation of the priority projects in the NESP. The strong leadership at the MoE and the development of 1 This is the year based on the Islamic calendar which is a lunar calendar consisting of 12 months in a year of 354 or 355 days. 2 The NESP was revised in 2010 and became NESP II for 2010-2014. Page 7 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) the NESP and its subsequent endorsement by the GOA leadership and lead education donors in the sector – the United States Agency of International Development (USAID) and International Development Association (IDA) – generated interest amongst donors who had previously remained outside of the Afghanistan education arena. 6. Project Context. The Second Education Quality Improvement Project (EQUIPII) was a follow-on project to the IDA-supported Education Quality Improvement Project (EQUIP) (2004-2009). EQUIP II, like EQUIP I, was prepared based on “OP/BP 8.00 Rapid Response to Crisis and Emergencies� and was co-financed by the Afghanistan Reconstruction Trust Fund (ARTF)3 as well as USAID4. The project was implemented over a nine-year period. The original allocation was US$158 million from ARTF, US$30 million equivalent from IDA and US$22 million from USAID. In 2012, the project received additional financing (AF) from the ARTF for US$250 million. Between 2002 and 2010, school enrollment increased from 0.8 million to over 8 million, life expectancy had increased from 41 to 61, and annual growth averaged 9.4 percent between 2003 and 2012. However, by 2011, economic and social progress began to slow down with the withdrawal of international security forces5. By 2014, just after the AF, the diminution of economic and social progress was exacerbated after the 2014 presidential elections and the subsequent political transition. Internal divisions within the National Unity Government slowed vital reforms and delayed the appointment of key officials, undermining policy certainty. During this period, it is estimated that the Taliban began to take control of approximately 45 percent of Afghan districts.6 Afghanistan started to experience a humanitarian crisis arising from large numbers of returning refugees (approximately 800,000 in 2016) and a large and growing internally displaced population (1.2 million). Armed conflict in Afghanistan claimed the lives of 26,512 civilians and injured 48,931 others between January 1, 2009 and June 30, 2017 and continues to cause severe harm to civilians across Afghanistan.7 Moreover, it is important to highlight that in Afghanistan the education sector is highly secularized which goes beyond Afghanistan being a conflict state. The Taliban directly targets education by threatening teachers, bombing schools, poisoning school wells, and targeting girls at every stage. These are the conditions under which the project was being implemented which make delivery of education services uniquely challenging. 7. The project results chain is presented in Figure 1. It illustrates the key inputs, activities, outputs and outcomes by project component. The long-term outcome to which the project hopes to contribute in the future is a skilled and competent citizenry. In order to eventually realize this long-term goal, the first step is to improve equitable access to schools and improve the quality of education. The project sought to achieve these outcomes by tailoring activities to achieve key outputs necessary to facilitate the related outcomes of increased student enrollments and students completing the last grade of each stage. In terms of access, the theory of change was that boys’ and girls’ enrollment 3 The ARTF is a multi-donor trust fund for Afghanistan that is administered by the World Bank. It is one of the main instruments available to the GOA to finance key recurrent expenditures (salaries, capital expenditures, operations and maintenance of key government institutions) as well as finance urgent investments and reconstruction activities planned in the country’s economi c recovery program. The ARTF can either finance free-standing projects or co-finance IDA or other donor-funded project. Modalities have been developed by IDA, as the ARTF administrator, to allow donors to express preferences for a priority program or project included in the GoA multi-year development budget, and for which the GoA has requested ARTF funding. 4 USAID funding was specifically allocated for the teacher education component of the project. 5 The number of NATO troops declined from about 130,000 in 2011, to around 15,000 by end-2014. 6 This is estimated by the office of the US Special Inspector General for Afghanistan Reconstruction (SIGAR), the US Congress mandated oversight body of American supported activities in Afghanistan. 7 Afghanistan Protection of Civilians in Armed Conflict Midyear report 2017, United Nations Human Rights https://unama.unmissions.org/protection-of-civilians-reports. Page 8 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) would increase if they had schools they could attend within a reasonable distance from their school with the water points. To attract girls in particular, schools were constructed with gender segregated latrines. Also, to improve the learning environment school grants were provided to school committees so that each community could finance what was appropriate to facilitate children’s attendance in school. For instance, female sanitary products could be provided to make it easier for girls to attend. The second key project outcome was improved quality of education throughout the system—not just for girls. The project included learning assessment to measure quality because assessment is a key component to learning and its associated feedback is essential to student learning. A relevant proxy indicator was used to measure this based on student performance in terms of their completion rates of the last grade in each learning cycle. The logic is that improved student completion rates would be an observable outcome of improved education quality. This was to be achieved by training of teachers, improving the teacher college training curriculum and developing standardized assessments and evaluation systems for teachers to improve their skills. The project sought to improve the quality of education by enhancing the qualifications of teachers both existing and new teachers. Learning materials were also provided to schools such as textbooks and science labs. In short, the project sought to contribute towards more competent learned citizenry in the future through a two pronged approach: (i) by making education more accessible to students through building schools and providing grants to give children what they needed in the given context and social mobilization to raise awareness of the importance of school; (ii) enhance educational quality by enhancing teachers’ qualifications through improved teacher college curriculum, needs assessments, self-evaluations, and in-service training. Page 9 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Results Chain Figure 1: Results Chain Project activities under Short-term benefits Medium-term benefits Long-term benefits components Increased enrollment Improved basic School grants for Improved student Improved learning education quality infrastructure facilities learning outcomes rehabilitation and construction; emphasis on Improved teaching and Increased education girl’s access educational quality equity (Component 1) Increased number of female students and Teacher and principal teachers training; in-service education for teachers; school management training programs; and gender grants for teacher training (Component 2) Sample check of school EMIS carried out; quality enhancement grants provided to schools; grant; schools infrastructure enhancement grants; school Shuras established; and spot checks conducted by third party to validate school level activities (Component 3) Project Development Objectives (PDOs) 8. The project PDO statement as presented in the original documentation was inconsistent between the Project Appraisal Document (PAD) and financing agreement (FA). However, these differences did not reflect a change in the meaning of the PDO. For the purpose of the ICR, the PDO statement that will be used is the one stated in the FA based on the Implementation Completion and Results Report (ICR) guidelines (2017) that say that the accepted PDO statement is the one found in the FA since it is the binding document. The two PDO statements were: (i) Financing agreement. The PDO as stated in the financing agreement (FA) for the IDA grant and the ARTF grant was “to assist the recipient in increasing equitable access to quality basic education, especially for girls.� Page 10 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) (ii) Project Appraisal Document (PAD) and Implementation Status and Results Reports (ISRs). The PDO was stated in the PAD and some project ISRs was “to assist the recipient in increasing equitable access to quality basic education, especially for girls through school grants, teacher training and strengthened institutional capacity with support from communities and private providers.� Key Expected Outcomes and Outcome Indicators 9. The original key project outcome indicators were (i) percentage increase in enrollment—girls; (ii) percentage increase in enrollment—boys; (iii) student learning outcome—reading test and numeracy test. The original PAD also listed “increase in girl/boy ratio from 56 percent in 2007 to 67 percent in 2011� in Annex 2 : Results Framework and Monitoring. However, the indicator was not transferred to Annex 2: Arrangements for Results Monitoring (Cumulative Targets) nor was the indicator listed in aide-memories or Implementation Status Results Reports (ISRs) during the period 2008-2011. During the 2012 additional financing (AF) the indicator was added to the results framework and was monitored during supervision missions and reported on in ISRs. Components 10. Component 1: School Grants (IDA US$7.0 million, ARTF US$13.1 million)8. There were two complementary objectives of the school grants component: (i) to support the improvement of teaching and learning by facilitating the creation of enabling school environments; and (ii) support the improvement of basic school facilities at existing government registered primary, middle and secondary schools with teachers on payroll. The sub components in this component were: (i) Sub Component 1.1 School Grants for Quality that were to develop linkages between schools communities and local government while supporting an empowering schools and communities as local education managers; (ii) Sub Component 1.2 School Grants for Infrastructure that were to rehabilitate and construct schools using community participation and community-based decision making; and, (iii) Sub Component 1.3: Social Mobilization and Awareness were to mobilize and strengthen school shuras9 to manage their schools based on local priorities through social mobilization activities for communities to implement project components with support from NGOs; development of training modules to support DEDs, PED and provincial departments of construction and MoE; and organization of community and provincial dissemination of project information. 11. Component 2: Teacher and Principal Training and Education (IDA US$16.7 million; ARTF US$32.0 million). The objective of this component was to create sustainable systems that would increase the level of professional knowledge and skills of educators throughout Afghanistan. This component was to provide training to teachers and principals and over time create and develop local professional networks that would allow educators to share ideas, problems, and solutions with their peers. The component was to be delivered in partnership with USAID. USAID funding covered eleven provinces and EQUIPII covered the remaining twenty-three. There was also a provision to increase the number of females entering Teacher Training Colleges (TTCs), teaching in TTCs and the provision of high school teachers in underserved high schools around the country. The sub components in this component were: (i) Sub Component 2.1: Support to District 8 The component costs are stated as they were stated in the original costs from the Project Appraisal Document. The overall allocations for the components with all of the IDA+ARFT+USAID allocations are provided in Annex 3. 9 Shuras are method of community-based decision making in Islamic society and is common and in widespread use in Afghanistan. Page 11 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Teacher Training Teams; (ii) Sub Component 2.2: Development of School Principals; and, (iii) Sub Component 2.3: Support to Increase Female Teachers. 12. Component 3: Monitoring and Evaluation (IDA US$4.6 million; ARTF US$5.0 million) The objective of this component was to: (i) build on and support the existing structure from the EQUIP project in MoE and (ii) establish and implement a practical monitoring and evaluation system for the project so that lessons can be drawn in a timely fashion to facilitate project improvement as well as policy making. The sub components were: (i) Sub Component 3.1: Project Management and Coordination; (ii) Sub Component 3.2: Monitoring; and, (iii) Sub Component 3.1: Evaluation. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 13. The PDO of the project was not revised during the project period. During the August 13, 2015 restructuring, the Bank team clarified the PDO by using the PDO language as stated in the financing agreements for both the IDA grant and the ARTF grant within the restructuring paper. Until that time, the project ISRs used the PDO wording from the PAD, rather than the FA. Revised PDO Indicators 14. PDO indicators were modified during the project period. The evolution of the indicators was as follows: Table 1. Original Key Performance Indicators (as stated in Project Appraisal Document) Indicator Baseline (2007) Percentage increase in enrollment (girls) 2.2 million-number (monitoring will include % increase) Percentage increase in enrollment (boys) 3.9 million-number (monitoring will include % increase) Student Learning outcome: (Reading Test and Numeracy No Test)--(learning levels regularly and adequately monitored) Table 2. Key Performance Indicators—2010 (IDA core Indicators were added to the ISR as required in 2010 for IDA projects—there was no restructuring for this addition, only added to the ISR at the time.) The inclusion of the IDA core indicators was a Bank-wide requirement for all IDA supported projects at the time. Indicator Comments Increase in girls’ enrollment No change Increase in boys’ enrollment No change Student Learning outcome: (Reading Test and No change in the indicator—became a core indicator Numeracy Test) Number of additional qualified primary teachers New—IDA core indicator added resulting from project interventions Number of additional classrooms built or New—IDA core indicator added rehabilitated at the primary level resulting from project interventions Page 12 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Table 3. Key Performance Indicators—Modified Indicators Based on 2012 Additional Financing Project Paper Indicator Comments Enrollment of girls Modified to include grade breakdown and target for overall enrollment (Policy Domain-Access) modified (broken down by grades (Grades 1-3; 4-6; 7-9; and 10-12) Enrollment of boys Modified to include grade breakdown and target for overall enrollment (Policy Domain-Access) modified (broken down by grades (Grades 1-3; 4-6; 7-9; and 10-12) Ratio of Girls to Boys New indicator—broken down by grades (Grades 1-3; 4-6; 7-9; and 10-12) (Policy Domain-Equity) Students completing the last grade of a New indicator—broken down by grades (Grades 3; 6; 9; 12) stage (Grades 3, 6, 9, 12) (Policy Domain-Quality) Learning Assessment No Change (IDA Core Indicator) (Policy Domain-Quality) Number of additional qualified primary No Change (IDA Core Indicator) teachers resulting from project interventions Number of additional classrooms built No Change (IDA Core Indicator) or rehabilitated at the primary level resulting from project interventions Table 4. Key Performance Indicators— Modified Indicators Based on 2015 Restructuring Results Framework Indicator Comments Enrollment of girls (broken down by grades Baselines adjusted to reflect updated MIS data. Targets updated to (Grades 1-3; 4-6; 7-9; and 10-12) reflect the closing date extension and latest projections for school year 2016. Enrollment of boys (broken down by grades Same as above (Grades 1-3; 4-6; 7-9; and 10-12) Ratio of Girls to Boys (broken down by grades Same as above (Grades 1-3; 4-6; 7-9; and 10-12) Students completing the last grade of a stage Same as above (Grades 3, 6, 9, 12) Learning Assessment No change Project Beneficiaries (% female) New indicator Number of additional qualified primary Moved to intermediate level indicator teachers resulting from project interventions Number of additional classrooms built or Moved to intermediate level indicator rehabilitated at the primary level resulting from project interventions Page 13 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Revised Components 15. The project components were not revised. However, at the time of the June 2012 Additional Financing and the August 2015 restructuring, additional activities were introduced and in 2015 some were dropped. Some of the activities added during the 2015 restructuring had begun during implementation and the restructuring formalized the activities. The modifications are detailed in Tables 5-7 below. Table 5. Additional Project Activities Based on June 2012 Additional Financing Component New Project Activities Component 2: Teacher and The USAID requested the MoE, through EQUIP-II, to absorb 11 provinces that had Principal Training and been funded under the USAID BESST project for in-service teacher training. EQUIP- Education II was already providing pre-service and in-service training in 23 provinces and agreed to add the additional 11 provinces under the AF. Component 3: Project Three pilots were added to address key gaps that had been identified under the Management and Monitoring original program. The pilots were: (i) strengthening sub-national planning and and Evaluation budgeting; (ii) conditional cash transfers for retention to low income households with girls in grades 6-12 when dropouts of girls increase; and (iii) introduction of an operations and maintenance process that included role out and implementation of an asset maintenance strategy and fund repairs and maintenance of schools by the Infrastructure Service Department (ISD). Table 6. Additional Component Activities Based on August 2015 Restructuring Component New Project Activities Component 1: School Grants (i) Model School grants provided to 70 schools to establish a creative learning environment to provide a demonstration effect on other schools (ii) Evaluation of lessons from the QEGs and prepare a new strategy for the future (iii) Development of a strategy and completion of needs assessment for human resources (HR) required to deliver HR services Component 2: Teacher and (i) Develop teacher management information system (TMIS) to manage Principal Training and teacher related data and link professional development and career advancement Education for teachers (ii) Develop TTC curriculum including the printing and distribution of the remaining subjects particularly math and science; and develop textbooks for revised pedagogy courses on the syllabi linked to the national teacher standards and competencies (iii) Develop a standardized assessment and evaluation system for the TTCs for individual specialties and for overall learning outcomes Component 3: Project (i) Evaluation of training needs for capacity building and provision of training Management and Monitoring for qualified MoE staff in-house or abroad and Evaluation (ii) Development of a comprehensive Human Resource Management Page 14 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Information System (HRMIS) (iii) Development of an infrastructure Service Management Information System (ISMIS) to manage the functions of the Infrastructure Service Department (ISD) (iv) Develop a Web Based Geo-Referencing Information System to function as an on-line education atlas of Afghanistan. The atlas will support informed decision making by presenting key education statistics and indicators on on-line maps. (v) Expansion of the Contract Management Database to include information up to the school level (vi) Establishment and Expansion of an Education Management Information System (EMIS) and Monitoring and Evaluation (third-party monitoring of EMIS data) (vii) Textbook delivery of 3.7 million books. (viii) Establishment of a gender unit and a working policy to address gender related issues. (ix) Development of a learning assessment that included: (i) survey, assessment and dissemination of Grade 3 assessment; (ii) development of Grade 6 assessment, printing of test items, and pilot/field test; (iii) grade 6, data analysis and preparation of report of the grade 9 assessment; (iv) printing of final grade 9 report Table 7. Component Activities Dropped Under ARTF Additional Financing Component Project Activities Component 3: Project Development of an asset management strategy (to be considered in next project) Management and Monitoring and Evaluation Girl’s retention conditional cash transfer for girls in grades 6-12 Design and implementation of a pilot Operation and Maintenance System Enhancement of MoE’s project management capacity through the provision of training and the carrying out of workshops and supporting day to day implementation of the project. (This activity was amended under Component 3:(i) above) Establishment of a monitoring and evaluation system that will enable communities, NGOs and third-party entities to monitor the achievement of progress under the project against a defined set of indicators and disseminate such data at the district, provincial and national level. (This activity was amended under Component 3: (iv) above) Support the verification of information collected at the community level and measurement of the achievement of the key performance indicators of the project to all for better planning and programming of the project. (This activity was amended under Component 3: (iv) above) Page 15 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Other Changes Table 8. Other Changes to Project During Implementation Date Stated Changes to IDA and ARTF Financing Agreements April 12, 2009 Amendment to IDA No. H354-AF financing agreement to modify the institutional arrangements by reassigning two functions of the Grant Management Unit (responsible for implementation) related to financial management and procurement to two newly established departments within MoE: finance and procurement departments. These new department were eligible for project training activities. March 10, 2009 Originally the project was approved for IDA US$30 million; USAID co-financing of US$22 million; and an ARTF blanket proposal for US$158 million. In March 2009, there was a need to request funds from the ARTF to fill the financing gap of US$158 million to support the last 4 years of implementation. The Ministry of Finance submitted a request for financing. The Bank submitted a formal request to the ARTF Management Committee for the full US$158 million with a first tranche of US$35 million. The subsequent tranches were: US$50 million on June 27, 2010; US$50 million July 24, 2011, and US$23 million May 13, 2012. There were no modifications to the results framework or project components. June 12, 2012 An additional financing (AF) proposal was submitted to the ARTF Management Committee for US$250 million which was approved by the committee. The funds were to be allocated in two tranches of US$125 million each. The additional financing changes were: (i) extension of project closing date from September 1, 2012 to August 15, 2014; (ii) modifications of project costs based on the additional resources; (iii) expansion of coverage of teacher training (see Table 5); (iv) development of a strategy by MoE for implementation in insecure provinces recognizing that “one size fits all� had serious constraints given the levels of insecurity; (v) modification of the results framework; and (vi) introduction of pilot activities (see Table 5). September 1, 2012 IDA project closed and all funds disbursed. March 6, 2014 Letter of Amendment—to the ARFT Grant in the amount of US$125 million bringing the total additional financing to the agreed US$250 million and extended the project closing date to August 15, 2015. August 13, 2015 There was a Level 2 restructuring that (i) modified the PDO indicators (as described in Table 4 above) and some project intermediate indicators; (ii) modified the project activities under Components 1-3 (as described in Table 4 and Table 5 above); (iii) eliminated the girl retention cash grants category of US$1.5 million because the activity was cancelled (moved to new project currently being developed) and moved to the US$1.5 million to the main disbursement category; (iv) extended project closing date from August 15, 2015 to December 31, 2016; and (v) modified the institutional arrangements by moving the responsibility for school Shuras from the EQUIP coordination unit to the Social Mobilization Directorate (SMD) base on a presidential decision. December 2016 Closing date extend from December 31, 2016 to December 31, 2017 (No cost extension) Page 16 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Rationale for Changes and Their Implication on the Original Theory of Change 16. The modifications did not affect the original theory of change but rather enhanced the ability to measure progress toward achievement of the theory of change. At the time of the 2012 Additional Financing (AF), the implementation teams introduced two additional indicators to provide an additional measure of equity and quality, as indicated in Table 3 above. The 2012 AF and 2015 restructuring also provided opportunities for further project impact with: (i) the expansion of pre- service and in-service teacher training from 23 to 34 provinces, (ii) increased access through the completion of remaining school construction; (iii) improved measures of quality with the completion of Grade 3 and Grade 6 assessments; (iv) additional measures to increase girls participation in school and teacher training; and (iv) additional studies to inform future programs and projects. These were all important additions to the ongoing priorities of the GoA to bringing educational services to communities nationwide. Page 17 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 17. The PDO was highly relevant to the country’s sectoral needs when the project was developed. The objectives fit into the government’s strategic vision for education articulated in the MoE’s NESP 2007-2010 which were endorsed by all stakeholders in the sector. It also fit into the Bank’s Interim Strategy Note for Afghanistan that focused on: (i) security; (ii) governance, rule of law and human rights; and (iii) economic and social Development. The objective also took into consideration the Operations Policy and Country Services (OPCS) guiding principles for results frameworks and operational design under OP/BP 8.00 which among other things was to keep the project simple, realistic and be candid about the situation on the ground. At the time of the 2012 AF, the PDO remained fully aligned with the three priority areas in the updated NESP II 2010-201410 and NESP III 2017-2021 as well as with the National Priority Program—Education for All. Moreover, it was consistent with the Bank’s Country Partnership Framework (CPF). It remains consistent with Objective 3.1 of the current Bank Country Partnership Framework (CPF) 2017–2021 on improved human development, which aims to support improved access to quality education. The intended outcomes by the end of the CPF in education are improved primary school completion rates, a halt in the slide of the ratio of girls to boys attending primary schools, and an improvement in the quality of graduates. In addition, EQUIP II remains aligned with the World Bank’s Education Strategy 2020 (Learning for All - Investing in People’s Knowledge and Skills to Promote Development). 18. The design was also focused on community participation, school inputs (grants and infrastructure), improved human resources, policy direction, and monitoring and evaluation which were in line with the priorities of the GoA and MoE. The design feature to support inputs and processes as a medium-term quality education strategy was also relevant given the fragility of education institutions and ongoing basic school needs in Afghanistan. The results framework (RF) was simple and included measures for access, equity and quality using measurable indicators available at the time of project design. The revisions in the RF, following the AF, provided the opportunity to adjust the indicators using data available at the time and to introduce two new indicators that could provide more information on equity and quality: ratio of girls to boys by grade and students completing the last grade of a stage. The objective was outcome-oriented and was appropriately pitched for the government capacity, data availability, and development status of Afghanistan. Therefore, the relevance of the PDO is high. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 19. EQUIP II had the objective of increasing equitable access to quality basic education, especially for girls that was to be measured by six PDO indicators (three original PDO indicators, two added at the time of the additional financing and 10 The three priority areas were: (i) to provide adequate learning environment for children, especially girls, to improve access and retention; (ii) upgrading of teachers’ skills, qualifications and quality of teaching and learning in the classroom; and (iii) strengthen the institutional ability to develop and implement sound policies, as well as to effectively manage the human, financial and physical resources required to achieve results. Page 18 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) one at the time of restructuring). There were 17 intermediate indicators that were also monitored. As indicated previously, the project PDO was never modified. However, the project indicators were revised and baselines as well as targets were revised based on data available at the time of the AF and restructuring. Related to project attribution and achievement of the PDO, it should be noted that EQUIP I and II were two of the main contributors to the construction of schools in an effort to increase enrollment but not the only contributors. Other donors and organizations such as UNICEF, USAID, Denmark, non-governmental organizations, the international military forces through the Provincial Reconstruction Teams (PRTs) also contributed to school construction programs throughout Afghanistan. 20. PDO 1: Increasing Equitable Access, Especially for Girls. Achievement of this was measured by: (i) increased enrollment11 for boys and girls and (ii) ratio of girls to boys (broken down by grades (Grades 1-3; 4-6; 7-9; and 10-12). Over the nine-years of implementation, there were three enrollment baselines and targets for girls and boys. Prior to the 2012 additional financing, (2009-2012) boys’ enrollment exceeded the 2012 target by 80,000 students for a 17.4 percent increase. During the same period there was a 32 percent increase in girl’s enrollment from the 2007 baseline which was substantial given the gender context of Afghanistan. When the project received additional financing the targets for enrollment were increased as indicated in Table 10 below. Between July 2012 and the August 2015, enrollment for girls increased from 2.9 million to 3.3 million exceeding the target by approximately 400,000 for a 13.79 percent increase over the newly established 2012 baseline. During the same period, boy’s enrollment exceeded the target by approximately 700,000 for a 15.55 percent increase over the 2012 baseline. Enrollment targets by grades for this period were: Grades 1- 3 were met and Grades 4-6, 7-9; and 10-12 were exceeded. (See Annex 7 for details on grade enrollments.) 21. Beginning in 2015, Afghanistan was experiencing heightened conflict and there were serious security concerns throughout the country. Despite this, overall enrollment indicators saw positive trends. By 2017, overall girls’ enrollment increased to 3.4 million (following a slight drop from 2016) missing the final target of 3.8 million. Although the target of 3.8 million was missed, over the life of the project, girls’ enrollment increased from 1.9 million12 to 3.4 million for a 79 percent increase. This overall increase is significant given the serious security concerns for sending girls to school and the ongoing conflict in Afghanistan. By 2017, boys’ enrollment had increased to 5.5 million exceeding the target of 5.2 million. Over the life of the project, boy’s enrollment increased from the revised 2008 baseline of 3.4 million to 5.5 million for an increase of approximately 61.8 percent. (See Annex 7 for details.) 22. As indicated above, the indicator to measure the ratio of girls to boys by grade was added at the time of the AF. Between 2012-2015 the targets for this indicator were slightly missed for Grades 1-3 and met or slightly exceeded for the Grades 4-6, 7-9; and 10-12 (see Table 10 for details). By project closing, the overall ratio of girls to boys was 60.75, missing the target of 66.75 by 6 percent (see Annex 7 for individual grade ratios). Although the final target was missed, girls’ enrollment did substantially increase, albeit at a slower pace than boys’ enrollment, which led to the relatively flat achievement in the overall girls to boys’ enrollment ratio. 23. Based on the achievements outlined above, the overall rating for access, especially for girls, is judged to be substantial. 11 Enrollment in Afghanistan is defined as children that are registered but it does not mean that students are regularly attending. This definition was established during the project as data quality became more robust. 12 Revised baseline at the time of the 2015 restructuring. Page 19 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Table 9. Original PAD Access Indicators (2009-2012) Indicator Baseline Target Actual by Percent Change Percent (2007) (2012) 2012 (Baseline to Achieved Actual) (Actual to Target) Percentage increase in 2.2 million 3.0 million 2.9 million 32.00% 97% enrollment (girls) (36% increase1) Percentage increase in 3.9 million 4.5 million 4.58 million 17.40% 102% enrollment (boys) (15% increase1) 1 The original percent increase in the PAD RF for the 2012 target was miscalculated at 35 percent for girls and 14 percent for boys. Table 10: Additional Financing Access Indicators (July 2012-August 2015) Indicator Baseline Target (2014)2 Actual (2014) Percent Change Percent 1 (2012) (Baseline to Achieved Actual) (Actual to Target) Enrollment of girls (Grades 2,901,801 3.085 (million) 3.3 (million) 13.72% 107% 1-3; 4-6; 7-9; and 10-12) (number) Enrollment of boys (Grades 4,580,701 4.745 (million) 5.2 (million) 13.52% 110% 1-3; 4-6; 7-9; and 10-12) (number) Ratio of Girls to Boys Gr 1-3=71.0 Gr 1-3=73.0 Gr 1-3=70.0 Gr 1-3—-1.41% Gr 1-3—96% (broken down by grades Gr 4-6=64.0 Gr 4-6=64.0 Gr 4-6=66.0 Gr 4-6—3.13% (Not achieved) (Grades 1-3; 4-6; 7-9; and Gr 7-9=54.0 Gr 7-9=57.0 Gr 7-9=57.0 Gr 7-9—5.56% Gr 4-6—103% 10-12) (percent) Gr 10-12=50.0 Gr 10-12=53.0 Gr 10-12=55.0 Gr 10-12— Gr 7-9—100% 10.00% Gr 10-12—102% 1 Baselines reflects 2011-12 MoE data available in 2012 at the time of the additional financing 2 Targets were established by the Bank based on MoE data available in 2012 and were first reported in the May 2013 aide-memoire not in the Additional Financing Project Paper. Page 20 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Table 11: 2015 Restructuring Access Indicators Indicator Baseline Target Actual Percent Change Percent (2008) 1 (2017)2 (2017) (Baseline to Achieved Actual) (Actual to Target) Enrollment of girls (Grades Total = 1.9 Total = 3.8 Total = 3.4 78.95% 89% 1-3; 4-6; 7-9; and 10-12) million million million (See Annex 7 for details) Enrollment of boys (Grades Total = 3.4 Total = 5.2 Total = 5.5 61.67% 106% 1-3; 4-6; 7-9; and 10-12) million million million (See Annex 7 for details) Ratio of Girls to Boys Gr 1-3=69 Gr 1-3=79 Gr 1-3=66 Gr 1-3=-4.35% Gr 1-3=84% (broken down by grades Gr 4-6=56 Gr 4-6=69 Gr 4-6=66 Gr 4-6=17.86% (not achieved) (Grades 1-3; 4-6; 7-9; and Gr 7-9=45 Gr 7-9=62 Gr 7-9=60.9 Gr 7-9=35.33% Gr 4-6=96% 10-12) Gr 10-12=37 Gr 10-12=57 Gr 10-12=47.6 Gr 10-12=28.65% Gr 7-9=98 % Gr 10-12=84% 1 Baseline modified in 2015 to reflect updated MIS data for 2008 2 Targets updated to reflect the closing date extension and latest projections for school year 2016 24. There were 17 intermediate indicators and the results for which can be found in Annex 1. However, related to increasing equitable access, especially for girls there are project intermediate indicators and activities that played an important role in supporting the achievement of the PDO that should be noted. The intermediate results were: • Number of schools established. The original number of schools to be constructed included 486 schools carried over from EQUIP I because they had not been completed and 634 new schools under EQUIP II for a total of 1,120. The additional financing increased the total number to 1,820. After the 2015 restructuring and with on-going security issues, the number of schools was reduced to 1,758 at the request of MoE. By project closing there were 1,602 schools completed13 (most of which were rural schools), missing the target of 1,758. There were 8,541 classrooms constructed nearly matching the target of 8,571. The increase in rural schools supported by the project arguably contributed to the 79 percent increase in girls’ enrollment from 2009 to 2017 as well as the 61.8 percent increase in enrollment in boys during the same period. As indicated, this was all accomplished despite the worsening security condition in the country and curtailment of procurement activities by presidential decree. It should also be noted that the project faced severe challenges with the construction component due to data misreporting and limited oversight of the community contracts. As a result of these weaknesses, the presidential decree mandated that construction activities be shifted to the ministries having core responsibility for construction (i.e., the Ministry of Urban Development and Housing for urban areas and Ministry of Rural Rehabilitation and Development for rural areas). 13The term “completed� refers to completion of a project according to the terms of contract. The contracted components for the 1,602 schools varied in volume as in some cases it included a school building with the necessary components such as boundary walls, latrines and furniture while in other cases, only a school building was provided and the remaining components were categorized as missing components or annexes. Page 21 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) • Social mobilization was also used to develop linkages between schools, communities and local education managers. The project trained 13,000 shura members on how to create functional shuras that could develop school improvement plans and sensitized communities to the importance of education -and particularly for girls. They were able to develop school improvement plans and received the QEGs. The Social Mobilization Unit, between 2009 and 2017, was also able to open 258 closed schools in insecure areas that catered to more than 13,336 students (35 percent female) supporting the goal of increased access. This was an exceptional performance given the poor security condition in the country. • Model Schools. The project supported 73 model schools and there are an additional 127 model schools in all 34 provinces. The model schools were developed to reinforce an active learning environment for students and teachers with the hope of having a demonstration effect for other schools in the community. Each project supported model school was given a grant in the amount of US$15,000 to improve the school environment. An MoE analysis of the model schools, showed that the model schools had an improved school environment, provided more discipline, attracted more students and teachers’ performance was increased. It was also reported that there was a higher level of satisfaction among the students in the model schools. The study also showed that school facilities such as access to toilets, drinking water, play grounds, laboratories, libraries and computer labs were much better in model schools as compared to regular general education schools. This model is being reviewed more extensively prior to further inclusion in the new Bank-supported EQRA project. 25. PDO 2: Increasing Quality Basic Education. The quality aspect of the PDO was to improve quality throughout the system. Therefore, this part of the PDO was not to be evaluated especially for girls but rather the entire system. The quality indicators were: (i) learning assessment completed and (ii) student completion as a proxy for quality. The learning assessment indicator was exceeded. In 2012, MoE partnered with the Australian Council for Education Research (ACER) to assist them in conducting a Grade 6 learning assessment. By March 2013, the first learning assessment ever conducted in Afghanistan was completed and a report had been produced with the first established grade 6 baselines. By project closing, a national grade 3 learning assessment had also been completed with a full report. Given that assessment is the engine that drives student learning because it encourages active learning, this was a tremendous accomplishment. In addition, the project teams were discussing the institutionalization of learning assessment at the national level through the introduction of PISA and TIMMS as well as potential classroom level teacher assessments of students. 26. Another remarkable result, in spite of the worsening security conditions, is the number of students completing the last grade of a stage (Grades 3, 6, 9, and12) which increased between 2008 and 2015, exceeding targets in all grades. In 2014, the Grade 3 target was exceeded by 200,000 (33 percent increase), Grade 6 target was exceeded by 130,000 (34 percent increase); Grade 9 target was exceeded by 42,000 (15 percent increase), and Grade 12 by 130,000 (118 percent increase). By project closing, the overall average of students completing the last grade of a stage had been exceeded (see Tables 12-13 for details). Additional data also shows that between 2008 and 2016, girls’ completion rate increased by 28.8 percent in Grade 3, 34.8 percent in Grade 6, 72.4 percent in Grade 9 and 79.3 percent in Grade 12. For boy’s completion rate increased by 33.9 percent in Grade 3; 28.1 percent in Grade 6, .4 percent in Grade 9, and 70.4 percent in Grade 12. The data shows that, with the exception of Grade 3, girls’ completion growth was proportionally higher than boys. Based on these achievements, the rating for this portion of the PDO (quality) is substantial. Page 22 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Table 12. 2012 Additional Financing Indicators Indicator Baseline Target Actual Percent Change Percent (2012)1 (2014)2 (2014) (Baseline to Achieved Actual) (Actual to Target) Students completing the Gr 3—0.582 Gr 3—0.59 Gr 3—0.79 Gr 3—35.74% Gr 3—134% last grade of a stage Gr 6—0.376 Gr 6—0.38 Gr 6—0.51 Gr 6—35.64% Gr 6—134% (Grades 3, 6, 9, 12 Gr 9—0.265 Gr 9—0.268 Gr 9—0.31 Gr 9—16.98% Gr 9—116% (number) (million) Gr. 12—0.103 Gr 12—0.110 Gr 12—0.24 Gr 12—133.01% Gr 12—218% Learning Assessment N Develop a Completed Achieved national for Grade 6 assessment and framework, preparations conduction were being national made for assessments, and Grade 3 finalize results 1 Targets were established by the Bank based on MoE data available in 2012 and were first reported in the May 2013 aide-memoire not in the additional financing project paper. 2 Bank projections based on MoE data available in 2012 Table 13. Key Performance Indicators— Modified Indicators Based on 2015 Restructuring Results Framework Indicator Baseline Target Actual Percent Change Percent (2008) 1 (2017)2 (2017) (Baseline to Achieved Actual (Actual to Target) Students completing the 0.27 million- 0.45 million- 0.46 million- 70.37% -overall 102% - overall last grade of a stage overall overall overall (Grades 3, 6, 9, 12 Gr 3—0.51 Gr 3—0.78 Gr 3—0.75 Gr 3—47.06 Gr 3—96% (number) (million) Gr 6—0.40 Gr 6—0.50 Gr 6—0.56 Gr 6—40.00% Gr 6—112% Gr 9—0.11 Gr 9—0.34 Gr 9—0.32 Gr 9—190.91% Gr 9—94% Gr.12—0.06 Gr 12—0.20 Gr 12—0.21 Gr 12—250.00% Gr 12—105% Learning Assessment N Y Y Achieved 1 Baseline modified in 2015 to reflect updated MIS data for 2008 2 Targets updated to reflect the closing date extension and latest projections for school year 2016 27. The intermediate indicators and activities that played an important role in supporting the achievement of the Increasing quality of basic education were: • Quality Enhancement Grants (QEGs).14 There were a total of 16,588 schools that benefitted from QEGs based on school improvement plans between 2009 and 2015. The grants were used as follows: 8,466 schools established 14 The objective of QEGs was to develop linkages between schools, communities, and local governments while supporting and empowering schools and communities as local education managers. There were two types of grants supported under the project: first and second-generation grants. First generation grants were provided to schools which had not received any grants through Page 23 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) libraries with an estimated utilization rate of 83 percent in 2015, 5,732 established laboratories with an estimated utilization rate of 86 percent; and 6,275 purchased computers with an estimated utilization rate of 80 percent. All of these achievements exceeded the targets. • Teacher Training. Teacher training activities were to help create sustainable systems that would increase the level of professional knowledge and skills of educators throughout Afghanistan by providing training to teachers and principals. Over the project period, 115,000 additional qualified teachers were trained which led to 58 percent of all teachers reaching full qualification15 (well above the target of 48 percent). The project also provided 11,436 scholarships to females to attend teacher training colleges (TTCs) exceeding the target of 9,500 by 1,936. The third-party review16 of the overall teacher training found that the Teacher Education General Directorate (TED) has visualized a professional development infrastructure at the national and provincial levels related to teachers and education leadership and management. The analysis showed there had been considerable progress toward improving the quality of teachers and education management from a very low baseline. The evaluation of the Diploma in Teacher Education (DTE) was judged to be a well-designed program offered uniformly to all Teacher Training Colleges (TTC). Moreover, when compared to other programs in the South Asian region (India and Pakistan) the DTE had a comparative advantage over similar preservice teacher education17. The evaluation found that the DTE program covered approximately 78 percent of the required competencies under the teacher competency framework. The remaining 22 percent could not be precisely identified and were largely related to utilization of self-evaluations; establishing and maintaining environments that support learning and using self- assessment strategies to help students learn and improve their thinking. • In-service Education for Teachers (INSET). The INSET programs were judged by the I-SAPS review to provide adequate coverage of the competencies and the District Teacher Training Teams (DT3s) were able to meet the quantitative targets for training of teachers and principals. Again, a considerable achievement given the need for in-service training over the project period and the large numbers that were training. Moreover, the project completed a teacher mapping exercise information on specifications of all in-service teachers. Over the life of the project, the targets for INSET I, II, III, IV, V and VI were all exceeded. (See Annex 1 for details.). • School Management Training Programs (SMT). The I-SAPS analysis of the SMT training showed that competencies and responsibilities of principals were at the heart of the SMT program. Moreover, principals began requested training related to completion of forms and, by the end of the project, were requesting training on teacher performance evaluation, parent communication, and student involvement. Principals had also formed principal learning circles (PLCs) to provide a forum of continuous support after the training. During the life of the project, 21, 277 principals and school administrators received training, exceeding the target of 14,000. previous funding and schools that met certain criteria such as the presence of an active School Management Shura (council), sufficient number of students, and a school building. Within this category of schools, priority was given to female schools. Second generation grants were given to schools which successfully utilized first generation grants, met eligibility requirements based on a per capita formula and had developed a follow-up school improvement plan. Efficient grant utilization is measured by keeping a transparent record of all equipment purchased for science and computer labs, libraries and other materials. The formula also accounts for the utilization of equipment by students. 15 The definition of teachers with professional qualification is: “number of qualified (grade 14 or higher) teachers per 1 teach er) 16 The third-party teacher training review was conducted by the Institute of Social and Policy Sciences, Pakistan (I-SAPS) in2015. The I-SAPS, third party review, was a multidimensional study that looked at the teacher training programs, leadership and management, organizational planning, and research monitoring and evaluation of the teacher education infrastructure in Afghanistan. 17 The comparative advantage was reflected in the fact that DTEs allocated more time to practicum, gave much needed attention to subject knowledge deficit in the prospective teachers, and included an action research component to build capacity for reflective engagement with teaching practice. The subject knowledge courses covered the entire spectrum of school subjects both horizontally (i.e., across school subjects) and vertically (coverage of content across grade levels). Page 24 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Justification of Overall Efficacy Rating 28. The overall efficacy rating is substantial. The substantial rating is based on the facts that: the PDO was substantially achieved as measured by the PDO indicators and that the activities supported the theory of change with the construction of gender friendly schools; teacher training for large numbers of teachers in an effort to improve the learning experience; establishment of school shuras to engage communities in the education of their children, both girls and boys; and provision of textbooks for schools nationwide. Early implementation showed that PDO indicator targets were on a path to being achieved or surpassed. This led the GoA to request additional financing from the ARTF to expand and consolidate the progress that had been achieved. With the additional financing, access, measured by enrollment increased for girls and boys; and equity, measured by the ratio of girls to boys (by grade), met or exceeded the 2014 targets. Quality, measured by students completing the last stage of a grade met or exceeded the 2014 targets for each grade; and the first ever learning assessments had been conducted for Grades 3 and 6. These achievements were made based on increased targets because of the extended project closing date. 29. As indicated previously, late-2014 to project closing was defined by worsening country conditions due to increased conflict and security challenges. Despite these conditions, there continued to be a positive trend toward achievement of all PDO indicators. Enrollment of girls slowed by 2017 but, overall, girls’ enrollment saw a 79 percent increase from the adjusted 2008 baseline while boys’ enrollment experienced a 61.8 percent increase during the same period. It should also be noted that with better EMIS data, the MoE was able to further define enrollment in Afghanistan as students that are registered but not necessarily regularly attending. This distinction has only been made over the last couple of years as data quality became more robust. This should be recognized as good practice given that many other developing countries do not report enrollment at this level of precision. Gender parity was a challenge at this time because boys were entering school in larger numbers than girls. However, over the course of the project, there was a focus on increasing girls’ enrollment through school management training of the shuras along with several other activities as outlined in paragraph 37. The project was also successful in contributing to higher equity in project schools. The number of girls enrolled in schools increased by 79 percent in 9 years, with boy’s enrollment growing at approximately 62 percent over the same time period. The project led to a gender parity index of 0.63, an improvement from the baseline index of 0.56. There were over 11,436 female recipients to EQUIP II-administered scholarships for Teacher Training Colleges, exceeding the post-reconstruction target by more than 20 percent. Nearly half of these scholarship recipients were successfully tracked post-graduation to have been hired as public-school teachers. Quality indicators targets were met given that, overall, more students were completing the last grade of a stage and learning assessments had been completed. These are all substantial achievements given the country context. Overall, the project provided the needed ingredients for school effectiveness. It achieved the key components of the PDO in terms of increasing access to quality education as measured by the PDO indicators as well as the critical intermediate results identified in the project’s theory of change. The established targets for achievement of the goals were largely met under difficult conditions. Page 25 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) C. EFFICIENCY Assessment of Efficiency and Rating 30. EQUIP II was implemented in an extremely challenging context of instability, insecurity, and low management capacity due to conflict. EQUIP II experienced three project extensions and missed its target school construction target of 1,758 schools by 156 schools. Out of the 156 incomplete schools, 114 comprised incomplete school buildings at project closing. Detailed discussion of efficiency loss due to construction delays can be found in Annex 3. Extensions were made due to delays in school construction and weaknesses in financial management and procurement. The MoE developed an action plan for the extension period to ensure smooth implementation of ongoing activities and a proper closing of the project. The Bank team continued to support the MoE in ensuring construction activities were completed and in addressing critical procurement and financial bottlenecks. The project closing date extensions were undertaken to allow for EQUIP II to achieve the project’s development objective to improve access and quality of education in Afghanistan. 31. Despite conditions that made the delivery of education services uniquely challenging the project is expected to have a significant development impact through improving educational attainment, and equity. The economic benefits of EQUIP II arise through the increase in the number of skilled and competent citizens to sustain Afghanistan’s social and economic development, as well as the private returns associated with higher productivity and wages for students. These economic benefits are among the associated outcomes of the provision of school grants (Component 1), teacher and principal training (Component 2), and monitoring and evaluation to support effective administrative structures in the MoE (Component 3). Given the project extensions and delays in school construction, the significant development impact of the Project merits a modest overall efficiency rating. 32. Over the course of the EQUIP II project, K-12 enrollment rates in Afghanistan rose by 68 percent. The project built 8,541 classrooms, nearly matching the target of 8,571 classrooms. The project also trained 111,500 teachers (39 percent of which were women), more than triple the target of 30,000. The economic analysis measures efficiency of the project along three dimensions: it estimates the economic benefits of increased educational attainment as a result of increased access, it also estimates benefits associated with improved education quality, and looks at economic and social returns to increased equity for girls’ education. In addition, this analysis will compare unit costs of EQUIP II components with similar education interventions. Details for each of these analyses can be found in Annex 3. 33. The cost-benefit analysis of school access yields an internal rate of return (IRR) between 31 percent and 43 percent, and a net present value (NPV) between US$414,140,796 and US$2,934,781,361 depending on the projected scenario (Table 14). All calculations attempted to be as conservative as possible, given the unstable security situation in Afghanistan. Even the worst-case scenario, which assumes all project schools will close operations immediately after project implementation ends, EQUIP II is still estimated to yield an IRR of 31 percent and a NPV of US$414,149,796. Table 14: Summary of cost-benefit for project direct beneficiaries Scenario NPV of benefits (US$) IRR Worst-case scenario 414,140,796 31% Medium range scenario 1,269,335,687 41% Best-case scenario 2,934,781,361 43% Page 26 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) 34. In addition to increased educational access, literature shows that the types of interventions that EQUIP II supports have positive impacts on student learning. Using a range of proxies from similar interventions, the estimated per student impact on learning outcomes of EQUIP II components ranges from 0.33 to 1.03 S.D. for every US$100 spent. It is likely however, that the overall effect size of EQUIP II will be higher given the interaction effects of school grants (Component 1), teacher and principal training (Component 2), and monitoring and evaluation to support effective school administration (Component 3). Annex 3 discusses a number of additional non-monetary returns not captured by private monetary benefits to students, as well as impact girls’ education on health and social outcomes. D. JUSTIFICATION OF OVERALL OUTCOME RATING 35. The outcome targets were revised upwards during the 2015 restructuring (without any AF) based on the positive trend demonstrated up until the previous year. However, as presented in paragraph 29, by late-2014 project implementation was defined by worsening country conditions due to increased conflict and security challenges. Hence a split rating has been presented in the table below. Table 15. Overall Outcome Rating Objectives after Objectives after Rating Dimension Original Objective 2012 AF 2015 Revisions Relevance of Objectives High Efficacy Objective 1: Increase Substantial Substantial Modest equitable access (especially for girls) Objective 2: Increase access Substantial Substantial Substantial to quality basic education Overall Efficacy Substantial Substantial Modest Efficiency Modest Outcome Rating Moderately Moderately Satisfactory Moderately Unsatisfactory Satisfactory Outcome Rating Value 4 4 3 Amount Disbursed (US$ US$156.19 million US$165.11 million US$91.52 millions) Disbursement (%) 37.83% 40.00% 22.17% Weight Value 1.51 1.6 .665 Total Weights 3.8 or 4.0 rounded Overall Outcome Rating Moderately Satisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 36. EQUIP II was a gender-informed project. In 2012, EQUIP II established a gender unit in the MoE. This marked a new period for EQUIP’s gender mainstreaming approaches. The gender unit provided support to the curriculum Page 27 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) department, TED, Infrastructure Services Department (ISD); and social mobilization. There was also a gender coordinator specifically for EQUIP II. During project implementation the MoE gender unit/team: (i) supported gender training of EQUIP provincial officers which resulted in the hiring of 500 female teachers in three provinces; (ii) installed a helpline for recording harassment cases although an analysis of the helpline remains to be completed; and (iii) used EMIS data to assess barriers to education for girls, reasons they were out-of-school, and published the yearbook with the analysis. The project also supported: (i) gender grants18 for TTC through the teacher and principals training component and (ii) girls’ scholarships19 for TTCs were awarded to approximately 11,436 females. Of these students 5,298 were tracked after graduation and it showed that 46 percent were hired as teachers in public schools. The school management training (SMT) focused on increased female participation in the school management shuras. The project also gave community construction grants to villages with the priorities for awarding the grants being on girls’ schools, schools with both boys and girls sections/shifts; and boys’ schools that planned to open sections/shifts for girls. The district social mobilizers worked with shuras to develop the grant requests and since 2016, the third-party monitoring reviewed the use of the grants at the school level on a sample basis. The project also increased the focus of construction on the inclusion of boundary walls, water, and latrines for schools to provide a safe environment for girls and encourage parents to send their girls to school albeit with challenges as outlined in paragraph 50. However, there is still work to be done given that by the end of the project female beneficiaries represented 39 percent of all beneficiaries falling short of the 42 percent target. Recent analyses show girls enrollment is more of a rural problem and that in lower secondary schools, boys are 1.5 times more likely to attend middle school and in upper secondary level boys are twice as likely to attend. The reasons for girls not attending school are mainly (i) security concerns, (ii) cultural issues of families not allowing girls to attend, and (iii) no school or the school was too far. Institutional Strengthening 37. The project played a role in strengthening social mobilization. The social mobilization unit (SMU) had teams at the national, provincial and district levels. There was cascade style training for the province and districts on the roles of social mobilization related to education. During the project period, districts went from training on completion of forms to training village level SMC on development of school maintenance plans, reporting on textbook availability and identifying dropouts. At the same time, the village SMC learned these techniques and improved their skills. The project also strengthened the ability of the MoE to administer learning assessments. Although the MoE used international technical assistance to analyze the data and prepare the final report, enumerators were trained and they worked with the international organization to develop the test items and methodology. Overtime, technical assistance reduced from over 1,500 in 2015 to 728 at project closing. While there has been institutional strengthening, there are still considerable challenges. Some are: (i) current institutional structures leave little room for provincial education directorates (PEDs) to 18 Gender grants are provided to TTCs based on the number of female students enrolled in TTCs to encourage them to enroll more female students. Gender grants were also used for providing more facilities and better learning environment to female students. Quality grants in the amount of US$10,000 were provided to every TTC annually through 2015 regardless of the number of female students. Quality grants are used for small repairs and maintenance of TTC facilities and for small procurement enabling them to provide a better teaching and learning environment 19 Girls’ scholarships were awarded based on attendance and academic performance. The aim was to give scholarships in 25 provinces with low female enrollment levels. The program also introduced innovative methods of community participation by engaging elders, fathers and other stakeholders in an orientation of the scholarship program. The result of this out-reach was a threefold increase in female enrollment in 2013. Page 28 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) innovate or adapt to the diverse range of local contexts; (ii) MoE has a large mandate spanning from general, vocational, and Islamic education, adult literacy; and teacher training. The MoE has also faced cumbersome internal processes, lack of departmental coordination and limited procurement and financial management capacity; (iii) continued reliance on technical assistance albeit at a reduced scale; and (iv) corrupt practices at the sectoral level particularly with regard to teacher recruitment. The new proposed Bank-supported EQRA project will continue to build on the institutional strengthening under EQUIP II and work to address some of the remaining challenges. Mobilizing Private Sector Financing Not Applicable Poverty Reduction and Shared Prosperity 38. The EQUIP II was a national project that focused on urban and rural communities. Many of the schools completed were built in the poor villages using community construction grants to provide schools for village children and support increased enrollments. The teacher and principal training provided scholarships for girls from poor families not only for gender equity but also for investing in knowledge to promote development and ensuring that the system provided opportunities for all. The long-term benefits of this are related to the project results chain as well as poverty reduction efforts. Other Unintended Outcomes and Impacts Not Applicable III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 39. Lending instrument. All IDA grants at the time of preparation were prepared under OP8.0 guidelines as stated in the Afghanistan Interim Strategy Note (ISN) because of the continued conflict and persistent fragility of the state institutions. The instrument was an Emergency Recovery Loan that allowed for provision of needed technical assistance and institutional strengthening schemes necessary to build capacity. 40. Results framework. The design took into consideration the Operations Policy and Country Services (OPCS) guidance note on guiding principles for results frameworks and operational design under OP 8.0. The principles supported developing a simple, realistic and candid results framework while continually reassessing as project conditions change. As indicated above, at the time of the additional financing the MoE had a more robust EMIS system that allowed for more disaggregated data as well as inclusion of additional indicators that would provide proxy measures of quality basic education and gender equity. Page 29 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) 41. Use of ARTF. The use of the ARTF provided through EQUIP II, was an important change in the financing of education sector projects in Afghanistan. The ARTF was a pooled source of on-budget financing for the education sector. Previously, the education sector investment was donor driven and fragmented—much of the financing was off budget with a plethora of actors being financed by various donors. The on-budget financing of EQUIP II provided MoE with an opportunity to build capacity and demonstrate that they could deliver education services on a large scale. The credibility of the MoE increased and donors were more willing to work directly with the MoE which was evidenced by the approval of the AF US$250 million from the ARTF. Based on this, the MoE was able to take control of the sector dialogue with donors which, in the context of fragile, conflict and violence (FCV) states is an important achievement. The Bank, through support of the EQUIP II, played a significant role in this achievement. 42. Evidence-based design. The technical design of the project drew from globally recognized approaches for developing effective schools that were aligned with international best practice. These design elements were well-suited to the capacity of the institutions participating in the project. Moreover, the design drew on lessons learned from a study that linked the lessons learned from EQUIP I with EQUIP II that included, but was not limited to: (i) strengthening community participation processes with emphasis on accountability and principal and teacher involvement; (ii) supporting principals and teachers as members of the school management committee (SMC) executive committee; (iii) strengthen the monitoring process of local communities through school improvement plans; (iv) train principals and teachers in a planned and documents way in quantitative and qualitative training; and (v) develop capacity at the national, provincial, district and communities levels to implement programs. 43. Appropriate selection of beneficiary groups to target. The nation-wide focus on students, especially girls, teachers, principals and communities were appropriate and ensured that many of the poorest areas were targeted. The fact that the project included grants and scholarships to poor areas with a specific focus on girls was an appropriate design feature. 44. Implementation Arrangements. While EQUIP I relied on a project implementation unit, EQUIP II comprised coordination unit team within MoE. It had a role of facilitating communication between the MoE and the Bank while most functions were executed by the various MoE departments. The EQUIP team was envisioned to be a small qualified coordination unit which supported the Ministry in incubating new initiatives related to gender, quality enhancement grants, social mobilization, model schools, learning assessment etc. Social mobilization was transferred to a new directorate established in 2015. 45. Appropriate plan for monitoring. Through the results framework, improvements in the EMIS, Grade 3 and 6 learning assessment, and a third-party verification evaluation of project components there were appropriate instruments and activities to assess the achievement of the project development objectives. In addition, the project used a third-party monitoring system which was a novel idea at the time. This was an important tool for ensuring the quality of the project implementation given the country security and conflict issues. 46. Adequacy of risk and mitigation measures identification. Key risks with mitigation measures were identified during preparation. Challenges in the form of capacity, implementation structures, financial management (FM) and country insecurity were anticipated. Mitigation measures were identified with project activities were directly related to risk mitigation through the (i) inclusion of community support to protect schools and teacher training in high risk areas; Page 30 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) (ii) inclusion of social mobilization to maintain stakeholder involvement; (iii) creation of core teacher training teams and DT3 scheme to ensure teacher training targets were achieved; and (iv) provision of direct payments for services to lessen FM risks. B. KEY FACTORS DURING IMPLEMENTATION 47. There were various factors that played a role in the challenges and successes of the project during implementation. 48. Recurrent security incidents. Security issues were present in Afghanistan hence the emergency category of the project. Since 2014, security incidences have been recurrent and have led to prolonged disruption of government functioning and school operations in an increased number of localities. The growing insecurity and consequent lack of mobility in Afghanistan resulted not only in fewer supervision missions by the Bank in the field, it also impacted the MoE’s capacity to monitor service delivery. In 2014, Bank staff were relocated from Kabul to Dubai because of the security concerns. 49. School construction. Delays in school construction along with quality concerns were persistent throughout project implementation. Again, issues of insecurity and the resulting lack of mobility for both the Bank and MoE to monitor progress impacted supervision of construction throughout the project. The mitigation for limited supervision by Bank staff in the field was the use of third-party monitoring that helped ensure completion of most projects with an appropriate level of construction quality.20 While security issues contributed to MoE’s difficulties in supervision, it should be noted that MoE’s overall low capacity for implementing construction as well as a lack of coordination between different departments of MoE involved in construction, contributed to the delays in implementation. As a result, 114 school buildings (61 boys’ schools and 53 girls’ and mixed schools) remained incomplete and are at different stages of construction. There were also 424 schools (213 boys’ schools, 211 mixed and girls’ schools) that had completed school buildings and were functional but had missing items and annexes such as boundary walls, latrines and furniture. There was another category of 415 functional schools that were not completed according to the terms of contract. In these cases, certain elements like the galvanized iron (GI) sheets, furniture etc., were not provided. These projects which were mostly from EQUIP I suffered from inaccurate cost-estimation and increased cost of construction material as the completion period took longer than expected and the allocated amount for these schools was not sufficient. It is important to note that the classrooms at these schools are functional. Between the 424 schools with missing items and 415 schools with incomplete items, there are 162 schools, which have both missing and incomplete items. 50. The limited procurement capacity was also not functional in the last two years as there was a presidential decree that the construction activity would be shifted to the ministries with construction as their core activity (i.e., the Ministry of Urban Development and Housing for urban areas and Ministry of Rural Rehabilitation and Development for rural areas). The MoE was allowed to start procurement with one year remaining in project implementation but its procurement department and the contract management unit were unable to respond to the urgency required to complete the remaining works by December 31, 2017. Following the Bank’s recommendation, a tracking unit was 20The third-party monitoring was conducted by International Relief and Development from September 2011 through September 2015 and by Management Systems International from October 2015 until the project closed in December 2017. Page 31 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) formed in November 2015 to monitor the implementation. The tracking unit was very helpful in identifying the problematic projects and seeking the task force and steering committee’s attention to resolve critical bottlenecks, but senior management did not act on its recommendations. At the end of the project, the GOA had agreed to complete the unfinished school construction and to complete the missing items and annex facilities based on an agreed list with the Bank. 51. Teacher Education. The project was successful in training teachers and, as indicated above, the I-SAPS 2014 evaluation of the pre-service and in-service programs showed that the training programs were relevant, covered competency materials and had a comparative advantage over similar pre-service programs in the region. In addition, the participants judged the programs to be of good quality. The project training was successful in providing an initial foundation for teacher’s qualifications. However, there is evidence from an Early Grade Reading Assessment (EGRA) conducted for Grades 2 and 4 and the SABER that more work is needed in supporting teachers to ensure that they have the competencies and qualifications to teach the subjects that they are teaching. There is also an inadequacy between number of teacher trainees trained in the TTC and those making it into the teaching force. In other words, the number of TTC graduates far exceeded the number of teachers hired. Moreover, teacher recruitment is often prone to corruption and nepotism (MEC report). The follow-on EQRA project will build on these challenges to improve governance of teacher recruitment and improve teacher performance in the classroom. 52. Implementation Arrangements. As indicated above, the implementation arrangements were to include a small coordination unit within MoE to coordinate project activities that were to be mainstreamed within the various MoE departments. While the project did work within the different departments of the MoE there were significant challenges for the MoE in staffing the relatively new ministry due to a limited pool of qualified labor available in the country. This led the project to support a large contingent of technical assistance (TA) within the MoE to carry out daily duties for the MoE. The departments that were supported included: (i) TED; (ii) Procurement Directorate; (iii) Plan General Directorate; (iv) Minister Office; (v) Learning Assessment; (vi) ISD; (vii) Management Internship Program; (viii) Internal Audit; (ix) ICT Directorate; (x) Finance and Accounting; (xi) Directorate of Social Mobilization and Shura; (xii) Curriculum; and (xiii) EQUIP supporting services team (3). The project also supported large numbers of district social mobilizers up to 2015 when the social mobilization aspect of the project was more institutionalized. Overtime the project team, along with MoE, have worked to reduce the TA requirement however, challenges still remained for staffing the MoE. 53. Third-Party Monitoring. Since 2012, the project used a third-party monitoring agency for many aspects of the project. As indicated previously, third party monitoring was introduced to assist the Bank in confirming and monitoring implementation of project activities. The third-party monitor agency relied on local monitors that had more mobility within Afghanistan. These parties provided an invaluable service to the project by ensuring that project activities were implemented as designed given that movement of Bank teams was severely limited. The monitors were able to report monthly on the construction quality of the schools and prepared a list of deviations for MoE to attend to. Beginning in 2016, the third party monitoring supervisory agent also monitored softer components of the project such as: (i) social mobilization progress; (ii) use and delivery of QEGs; and (iii) data collection and verification of education data that has been entered into the EMIS system. The third-party monitoring supervisory agent visited nearly 80 percent of total school sites under the project, some of which even twice. All-in-all, the third-party monitoring system introduced by the project was instrumental in reporting progress and problems to the Bank so that implementation concerns could be Page 32 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) addressed on an ongoing basis. MoE also received the third-party monitoring reports in order to take action on the various issues that were noted. 54. Additional financing and 2015 restructurings. At the time of the additional financing, the Bank and MoE agreed to additional financing, rather than a new project, to help consolidate gains made between 2009 and 2010, strengthen institutional reforms, focus more attention on the insecure provinces, and continue teacher training activities with the inclusion of the 11 provinces that had been previously supported by USAID. Moreover, the additional resources and extension of the project provided more opportunity to learn lessons that could be incorporated into the preparation of the follow-on project. The 2012 additional financing also provided an important opportunity to modify the results framework by adding additional indicators as proxies for improved quality and increased equity thereby improving the project’s ability to measure achievement of the PDO. This was made possible by project investments in the EMIS system and data collection efforts. 55. At the time of the 2015 restructuring, indicator targets were revised upward due an extension of the closing date as well as the fact that more robust data was available over time. The 2015 restructuring also allowed for the completion of the Grade 3 learning assessment, and teacher mapping. As indicated, the lessons learned from the challenging period of implementing EQUIP II, along with the many studies that were completed, have been instrumental in informing the design of the forthcoming EQRA project. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 56. The project M&E design was simple as suggested by OPCS guidance for projects prepared under OP/BP 8.0. The M&E design was developed to monitor and report on progress toward meeting the targets of the PDO and intermediate indicators in the Results Framework as well as monitoring progress through a third-party verification process. Student enrollment and numbers of teachers were specifically collected by NGOs and disaggregated by sex, district and schools and then shared with EMIS, provincial EQUIP offices, PEDs, DEDs, school Shuras and school principals. Additional systems of monitoring were: (i) school Shuras monitored the school grants through the use of checklists that would be publicly posted to facilitate a public audit; (ii) NGOs and their DT3s would monitor teacher and principal training data and NGOs would report data to the national EQUIP coordinator unit; and (iii) TED tracked the support for increases in female teachers and provided data to the EQUIP coordinator unit and NGOs monitored the scholarships for female TTC students, Standardized forms with monitoring protocols were to be developed and provided to NGOs, school Shuras, DEDs and PEDs. M&E Implementation 57. Reporting of data for the results framework indicators was systematically done throughout the life of the project. The MoE/EQUIP coordinator unit received the updated information from the NGOs and reported on results framework Page 33 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) indicator achievement on a semiannual basis and maintained the EMIS database. In addition, the project used a sampling process for a 5 percent data verification. The third-party verification process was implemented and confirmed, that the EMIS data was correct and came from approximately 15,000 schools and encompasses more than 8 million students. Even with the ongoing insecurity issues, the third-party evaluation process judged that the data reporting systems were of high quality. NGOs collected and reported data on the enrollment school grants as well as teacher and principal training. These numbers were consistently provided to Bank teams during supervision missions. In 2013 and 2015, there was third party verification and evaluation of all project activities with full reports provided by the evaluation teams.21 In addition to the two overall project evaluation reports, the project conducted an independent multidimensional study by I-SAPS of the component 2 training activities that looked at: (i) teacher education (pre-service and in-service programs), DT3 and TTCs, TED capacity and needs; (ii) educational leadership and management (including school based management and leadership and TED leadership and management); (iii) organizational planning and management (internal and external relationships of TED departments and divisions across MoE, TED and TTC infrastructure, finances, grants, institution grants TTC enrollment, placement and tracking of graduates) and (iv) research monitoring and evaluation (include general review of girls’ scholarship program, the national teacher examination, and building M&E capacity). As indicated above, the project conducted the first ever national learning assessment for Grades 3 and 6 that provide a solid baseline for future assessments. Moreover, the project developed: (i) a web-based GIS atlas of schools in Afghanistan with key statistics and indicators on on-line maps which can be used for national level and sub-national level information and (ii) developed an enhanced and integrated MoE payroll-MIS system. M&E Utilization 58. EMIS data was used by the MoE, Bank and donors for monitoring progress of the education system and making decisions for areas of investment in the system. The EMIS data was also used for reporting progress toward achievement of the EQUIP II development objectives. The data is also used for publishing education statistical yearbooks but, these were not prepared on a yearly basis. The results of the teacher training evaluation and the SABER survey have been used to develop the follow-on Bask-supported EQRA project both of with provided detailed recommendations for improving the system such as continuing to improve teachers’ ability and skills through training, in-class support, principal support and community participation. These are all substantial achievements. Justification of Overall Rating of Quality of M&E 59. Based on the above, the overall rating for M&E is substantial. All-in-all these are substantial M&E achievements that helped lay a strong foundation for M&E within the sub-sector and considerable achievements given the security issues that persist in Afghanistan 21 The evaluations were done by Particip and Integration firms. In each case the methodology used was: literature review, analysis of statistical data, individual interviews with education stakeholders, visits and discussions with schools depending on security concerns and issues. Page 34 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 60. Safeguards. The project was rated a Category ‘B’ operation, that required an Environmental and Social Safeguards Framework (ESSF). The ESSF was specifically related to the infrastructure grants given to communities for community construction and rehabilitation or new construction of facilities to be constructed on government land. The same ESSF was used for the additional financing. The project included a third-party verification evaluation of civil works to ensure that environmental issues were being monitored. The project did not trigger any additional World Bank safeguards policies that required separate assessments. Environmental and social safeguards were monitored and reported on during the project period. The project faced challenges with safeguards and ratings fluctuating throughout implementation. In 2011 safeguards were downgraded to unsatisfactory after a thorough review by an ISM safeguards team because: (i) the local language versions of the ESSF had not been printed and distributed to provinces/regional offices; (ii) awareness raising programs had not been conducted; (iii) community land donations for schools had not been had not been properly documented and legally transferred; (iv) community school construction did not meet quality requirements and lacked key components of latrines, water points, waste disposal systems etc.; and (v) the existing grievance system needed to be formalized. Adjustments were made and the rating was upgraded to moderately satisfactory. In 2014, a draft Grievance Redress Mechanism (GRM) was developed and a Grievance Redress Committees had been established for more that 50 percent of planned schools and grievances were already being recorded and resolved. However, by 2015 the grievance redress mechanism was showing signs of weakness especially related to land acquisition documentation and ongoing concerns about inclusion of school boundary walls, latrines etc. By project closing, safeguards had been downgraded to moderately unsatisfactory because all grievances were not being logged in; there had been no follow up to complaints and land donations were still not being documented properly. 61. Financial management. FM performance rating fluctuated between Moderately Satisfactory and Unsatisfactory throughout the life of the project. The FM rating was downgraded to unsatisfactory in 2016 because: (i) the Bank had not received responses to FY2014 audit observations; (ii) the internal audit report for 2014 and 2015 had not been shared with the Bank; (iii) a refund of an ineligible expense was overdue for more than a year; and (iv) advances to provinces had been outstanding for three years. The rating at project closing was unsatisfactory because: (i) there were a significant number of outstanding advances to be liquidated and if they were not then they would be declared ineligible; (ii) there was no record of fixed assets, and (iii) the funds disbursed to the contractors and communities exceeded physical progress that will be declared ineligible. The quarterly unaudited interim financial reports (IFRs) were submitted to the World Bank on time, and there were no outstanding IFRs. The required audits were submitted to the World Bank on time in accordance with the Financing Agreement. The final audit report is due June 30, 2018; however, the FM team expects that there will be a delay due to capacity and staffing issues at the Auditor General. 62. It should be noted that the Bank is conducting an in-depth fiduciary review of the project. The objectives of the review are: (i) to verify that the expenditures funded over the project implementation period are eligible as outlined in the Project Appraisal Document (PAD) and grant agreement; (ii) to verity that the funds have been used for the purpose intended with due regard for economy and efficiency, including the identification of indicators of potential fraud, corruption and other misuse of funds; and (iii) to assess that the control and oversight processes used by the Government, third party monitor, and the Bank were adequate and functioned appropriately to identify and prevent problems; in cases where gaps or non-compliance are identified, the Review will also seek to identify the reason for the Page 35 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) control breakdown. The review was planned to be completed by June 30 but due to delays in provisioning of data by the MoE, there will be a delay of 2-3 months. 63. Procurement. During project implementation, procurement was largely rated unsatisfactory to moderately unsatisfactory. The procurement rating was moderately unsatisfactory at the time of project closing. Many of the implementation issues related to procurement stemmed from: (i) overall limited procurement capacity; (ii) an inadequate contract management system; (iii) weak tracking monitoring and reporting system; (iv) inadequate coordination between different units (i.e. EQUIP coordination unit, ISD, procurement, financing, and procurement directorate); and (v) non-compliance with Bank procurement guidelines. These limitations led to the issues such as: (i) full payment of contracts that did not correspond to physical progress; (ii) non-compliance with Bank procedures in the recruitment of individual consultants leading to ineligible expenditures; (iii) use of procurement methods without the Bank’s no objection, and (iv) inability to produce the needed supporting documents related to contracts for post procurement reviews. These challenges were largely responsible for school construction delays that led to project school construction targets being missed. Over time the project did develop a database for managing contracts but there was limited use of the system. There was also a fixed asset register that was maintained in Excel although not appropriate because the level of control was weak. At the time of project closing, procurement capacity remained low. C. BANK PERFORMANCE Quality at Entry 64. The project preparation team ensured that the project design was closely aligned with the NESP 2007-2010, the Bank’s ISN, and the OPCS guidelines for preparation of emergency projects. The original key indicators were appropriate for measuring progress toward achieving the PDO and the medium-term goals outlined in the results chain given the availability of data and the inclusion of learning assessment indicator. Overtime, the education data system evolved allowing the project team to include additional indicators that provided a more robust measure of equity and quality with gender and grade distinction. The design team used a detailed analysis of the lessons from EQUIP I to improve the design of EQUIP II. The team also collaborated with other donors, particularly on the teacher education component, where USAID already had considerable experience. The team also included a third-party verification and evaluation process to ensure that project data and results were being accurately reported. Due to the emergency nature of the project, the preparation team used a parallel project management team for implementation because MoE capacity was limited. The preparation team also identified the appropriate risks and incorporated design features to help mitigate them understanding, that the even with the best mitigation measures, country insecurity could still have a substantial impact on implementation. Based on this, the quality at entry for EQUIP II was Satisfactory. Quality of Supervision 65. There were 18 World Bank review and implementation support missions, which included a mid-term review (MTR). Up to late-2014, the Bank team was based in Kabul and was able to support the MoE on a regular basis in-country. The team was actively engaged in supporting the government in its efforts to implement the project. Whenever Page 36 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) implementation challenges arose, the World Bank team worked with the government to find appropriate solutions. The supervision teams consistently reported on FM and procurement progress during supervision missions and worked with the project unit team to build their capacity in these areas. They also systematically documented project progress in aide- memoires, back-to-office reports, and ISRs, all of which kept the World Bank management informed of progress and provided the foundation for the Implementation Completion and Results Report (ICR) analysis. However, it should be noted that the list of indicators in the ISRs, post AF, was not always accurate--the aide-memoires were accurate but the ISRs were not. The Bank team updated the ISRs at the time of the 2015 restructuring. Key to providing project support was the consistent supervision by the Bank team, both in-country and from Dubai once staff were relocated due to security concerns, and the third-party monitoring. 66. Regarding civil works, MoE had very limited capacity and needed constant support. Thus, the task team was heavily involved on a daily basis to seek solutions to the incomplete and stopped schools. After the civil war the technical capacity in Afghanistan was extremely low and only very basic drawings and broad cost estimates were used when the construction started under EQUIP I and II. But when the AF was approved, the Bank team provided close guidance to the MoE to improve the drawings and set up a cost estimation system for calculating province specific cost estimates. The Bank team continued with its close support when the inaccuracies in data reporting and slow progress at sites continued and helped MoE with setting up a tracking unit and reporting formats. The quality of tracking unit’s reports improved with constant guidance but senior management of MoE did not resolve the issues reported by the Tracking unit in time resulting in many incomplete schools at the project closing. 67. The AF and restructuring allowed the team to make adjustments to the project based on the ever-changing reality on the ground which benefited the project design and improved the results framework. Overall, there was a high level of supervision for this project under extremely difficult circumstances. The quality of supervision of EQUIP II was Satisfactory. Justification of Overall Rating of Bank Performance 68. Based on the analysis above, overall World Bank performance is rated Satisfactory. Page 37 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) D. RISK TO DEVELOPMENT OUTCOME 69. The risk to sustaining the development outcome is substantial. The follow-on Bank-supported EGRA project will continue many of the same activities that were initiated under the EQUIP I and II project and will play a significant role in sustaining the investments. However, substantial risks remain to sustaining the development outcomes. They are as follows: • Recurrent security incidents that have led to widespread and prolonged disruption of government functioning and school operations in an increased number of localities. In an attempt to mitigate these problems, the MoE is instituting a concept of ‘safe havens’, free of all barriers to learning. This includes legislation for demilitarization of schools, addressing the needs of internally displaced persons (IDPs) and returnees, and supporting community school protection mechanisms, that is, support from security forces and use of community negotiation approaches to strengthen the school shuras. • The growing insecurity will also continue to restrict mobility in Afghanistan that will result in limiting the MoE’s own capacity to monitor service delivery. The MoE will continue to use third-party monitoring of service delivery and data gathering to ensure that system is functioning. • Poor governance and accountability throughout the administrative chain continues to persist. There continues to be (i) over-centralization of key administrative functions at the MoE, exacerbated by overall capacity constraints; (ii) inconsistent enforcement of guidelines and regulations, particularly related to teacher recruitment and management; (iii) lack of clarity on responsibility and accountability throughout the administrative chain; (iv) insufficient transparency with regards to budget allocation, execution and monitoring; and (v) poor utilization of existing administrative data. The forthcoming EQRA project will work to mitigate these risks by embedding relevant governance reforms throughout its design on the one hand and including cross-cutting reforms to strengthen transparency on data collection and utilization. In addition, EQRA will strengthen planning and budget execution throughout the service delivery chain. • Continuing quality enhancement of teacher education and recruitment of new teachers face challenges related to widespread politicization and deeply-entrenched corruption in teacher recruitment and management process. While merit-based recruitment regulations exist, there is significant deviation in practice. The integrity of the process is undermined by nepotism, corruption and unlawful interference, resulting in the Afghan people’s diminishing trust in the education system. Going forward, the EQRA project will attempt to mitigate these challenges by strengthening and professionalizing key steps in teacher recruitment and building more transparent management systems. V. LESSONS AND RECOMMENDATIONS 70. Lesson 1. Heavy reliance on technical assistance with a separate coordination unit does not build institutional capacity. EQUIP II established a coordination unit because of the emergency nature of the project and very weak capacity within MoE. this was already an evolution from EQUIP I which had a project implementation unit. Recommendation. MoE will need to develop a permanent cadre of middle level managers with strong capacity development activities to ensure education policies and programs are implemented nationwide. This recommendation Page 38 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) has been taken into consideration in the forthcoming EQRA project where there will be more focus on the oversight responsibilities by Ministry departments as well as more focus on capacity development, governance and empowering PEDs, DEDs and schools in-service delivery. 71. Lesson 2. Third party monitoring that uses local organizations and NGOs is particularly useful in conflict areas. EQUIP II introduced third party monitoring when the security situation in Afghanistan prohibited Bank staff from making site visits to supervise project implementation. The mechanism proved to be invaluable for ensuring regular monitoring and reporting on project supported activities. Recommendation. When security concerns make Bank supervision difficult, look to third party monitors and NGOs for support given their ability to move more freely within the country and report on progress. Learning from EQUIP, the use of third party monitoring will be continued under the EQRA project to monitor the progress of project related activities. 72. Lesson 3. Projects with large construction components need capacity to develop detailed construction and procurement plans as well as strong management and supervision teams to be successful. Under EQUIP II, MoE had: (i) very low capacity for developing construction and procurement plans, (ii) school designs were followed that were not suitable to local conditions; (iii) school drawings and technical specifications were not detailed enough making monitoring difficult; (iv) limited procurement knowledge leading to substantial delays; and (v) very poor construction supervision and its monitoring as most of supervision staff was poorly qualified and to make it worse even they could not access many sites due to security conditions. Recommendation. Build capacity within the construction departments that are responsible for school civil works. This recommendation has been taken into consideration in the EQRA, as the construction will be transferred to the Ministry of Rural Rehabilitation and Development (MRRD) who have successfully implemented the construction activities under NSP project through the Community Development Committees (CDC’s) using locally available manpower. 73. Lesson 4. Increasing girls’ enrollment requires a balanced strategy. EQUIP II included supply and demand features to increase girls’ enrollment with the inclusion of infrastructure investments (i.e. boundary wall, toilet, water) and well as scholarships for girls to attend TTC. The EQRA project will expand on these activities in an effort to further increase girls’ enrollment and retention. 74. Lesson 5. Teacher training that increases the qualifications of teachers is an important component to improving quality of education. The EQUIP project played a significant role in increasing the number of trained teachers and improving the qualification level of teachers. This was a good beginning to improving the quality of education in Afghanistan but only a small proportion of teacher college graduates entered the teaching force. Recommendation. Build on progress that has been achieved by helping teachers through coaching and mentoring to improve pedagogy and subject content. The EQRA project will build on this by providing some additional training related to subject content on a part-time basis to allow existing teachers to continue to teach while simultaneously being trained; piloting of in classroom coaching of teachers, supporting curriculum reform and providing more learning materials for teachers to use in the classroom. 75. Lesson 6. Education statistics systems require ongoing coordination of data as well as a transparent system for all stakeholders. EQUIP II improved the overall EMIS system to the point that MoE could produce regular data which Page 39 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) was annually verified by a third-party audit. Going forward EQRA will support MoE in developing a more transparent system, coordinating data reporting throughout the system and improving capacity for data analysis. 76. Lesson 7: Projects in countries with ongoing conflict and security concerns need flexibility in implementation to adjust to the reality on the ground. EQUIPII had a nationwide focus which presented difficulties for completing civil works and monitoring given the increased security concerns during project implementation. Moreover, the nationwide focus spread resources thinly over the country. Going forward, the new EQRA project will focus resources specifically .on lagging provinces with regard to access interventions where the proportion of out-of-school girls is the highest. Page 40 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Ratio of Girls to Boys in Percentage 69.00 67.00 79.00 66.00 grades 1-3 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 1-3 Percentage 69.00 0.00 79.00 66.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 28-May-2017 grades 4-6 Percentage 56.00 0.00 69.00 66.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): The PAD 2008 original baseline was 56 percent. The targets for the ratio of girls to boys was missed for grades as follows: Grades 1-3 (16%), Grades 4-6 (4%); Grades 7-8 (2%); and Grades 10-12 (16%). Page 41 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Enrollment girls Number 1.90 3.00 3.80 3.40 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 10-12 Number 0.08 0.00 0.40 0.30 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 7-9 Number 0.28 0.00 0.70 0.70 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 4-6 Number 0.69 0.00 1.10 1.10 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 1-3 Number 0.88 0.00 1.60 1.30 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): 2008 PAD refers to a baseline of 2.2 million.Overall enrollment for girls missed the target by 11%. The target was missed by 19% for Grade 1-3; met for grades 4-6 and 7-9 and missed by 25% for grades 10-12. Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Formally Revised Completion Page 42 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Target Enrollment boys Number 2.10 4.50 5.20 5.50 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 10-12 Number 0.22 0.00 0.70 0.63 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 7-9 Number 0.65 0.00 1.00 1.15 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 4-6 Number 1.24 0.00 1.50 1.73 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grades 1-3 Number 1.28 0.00 2.00 2.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): 2008 PAD refers to a baseline of 3.9 million. Overall enrollment for boys exceeded the target by 6%. The target for Grade 1-3 was met; exceeded for grades 4-6 and 7-9 by 15% and missed by 10% for grades 10-12. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 43 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Students completing the last Number 0.06 0.00 0.20 0.21 grade of a stage (grade 12) 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grade 12 Number 0.06 0.00 0.20 0.21 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grade 9 Number 0.11 0.00 0.34 0.32 01-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grade 6 Number 0.40 0.00 0.50 0.56 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Grade 3 Number 0.51 0.00 0.78 0.75 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): The targets for students completing the last grade of a stage was: Gr. 3-exceeded by 5%; Gr. 6 missed by 6%; Gr. 9 exceeded by 12%; and Gr. 12 missed by 4%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion System for learning Yes/No N Y Y Y Page 44 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) assessment at the primary 31-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 level Utility of the learning Number 1.00 0.00 3.00 1.00 assessment system 31-Jan-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Learning assessment indicators was achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 0.00 9000000.00 8900000.00 31-Jul-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Female beneficiaries Percentage 0.00 0.00 42.00 39.00 31-Jul-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): A.2 Intermediate Results Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 45 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Schools established under Number 326.00 0.00 1756.00 1177.00 EQUIP 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): The target was missed by 32.9%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of additional Number 2074.00 0.00 8571.00 8541.00 classrooms built as the result of project intervention 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): The target was missed by 0.3%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Student-Classroom Ratio Number 74.00 0.00 70.00 60.20 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Exceeded the target by approximately 10%. Page 46 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of schools with Percentage 0.00 0.00 33.00 32.00 science labs (secondary) 31-Dec-2008 28-Sep-2012 31-Dec-2017 28-May-2017 Science Labs (secondary) Percentage 0.00 0.00 33.00 32.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Drinking Water Facilities Percentage 50.00 0.00 80.00 65.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Latrines/Toilets/Urinals Percentage 49.00 0.00 70.00 68.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Missed the target by 1%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Share of schools receiving Percentage 0.00 0.00 90.00 99.00 Page 47 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) textbooks as planned 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Exceeded the target by 9%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers recruited or trained Number 0.00 0.00 30000.00 115000.00 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Exceeded the target by 283%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of teachers wtih Percentage 28.00 0.00 48.00 58.00 professional qualifications 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Exceeded the target by 10%. Page 48 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of female Percentage 30.00 0.00 47.00 33.00 teachers 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Missed the target by 29%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of teachers Number 0.00 100000.00 100000.00 100833.00 completing INSET I 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target exceeded. There was a 0.83% increase over the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of teachers Number 0.00 100000.00 100000.00 93985.00 completing INSET -II 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Page 49 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Comments (achievements against targets): Missed the target by 6.01%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of teachers Number 0.00 0.00 100000.00 115814.00 completing INSET - III 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target exceeded. There was a 15.8% increase over the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of teachers Number 0.00 0.00 100000.00 154699.00 completing INSET IV 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target exceeded. There was a 54.6% increase over the target. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 50 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Target Completion Number of teachers Number 0.00 0.00 100000.00 154811.00 completing INSET V 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target exceeded. There was a 54.8% increase over the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of school Number 0.00 7000.00 14000.00 21277.00 administrators and principals completing management 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 training program Comments (achievements against targets): Target exceeded. There was a 51.9% increase over the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of female Number 0.00 5000.00 9500.00 11436.00 scholarship recipients in Teacher Training Colleges 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Page 51 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Comments (achievements against targets): Target exceeded. There was a 20.3% increase over the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 5% sample check of school Yes/No N Y Y Y EMIS carried out 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): The original target from 2008 PAD refers to follow up surveys and evaluations undertaken. Target met. The spot check was conducted with third party verification. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Schools with School Yes/No N Y Y N Information/Report Card piloted 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Timely submission of progress reports was the target mentioned in the 2008 PAD. Report cards were not piloted during the project. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 52 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Target Completion Schools with school Shuras Number 0.00 0.00 13969.00 14992.00 established 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target exceeded. There was a 7.3% increase. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of schools Percentage 26.00 0.00 100.00 99.00 receiving Quality Enhancement Grant 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Original target was 11,000 schools receiving a QEG. Target exceeded. There was a 7.3% increase. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Spot checks conducted in Number 0.00 0.00 3.00 3.00 schools 31-Dec-2008 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target met. Page 53 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Teachers Number 0.00 0.00 73000.00 84831.00 completing INSET-VI 12-Dec-2016 28-Sep-2012 31-Dec-2017 21-Dec-2017 Comments (achievements against targets): Target exceeded. There was a 16.2% increase. Page 54 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 Increasing Equitable Access, especially for girls 1. Enrollment for girls (broken down by grades) Grades 1-3, 4-6, and 10-12) number 2. Enrollment for boys (broken down by grades) Grades 1-3, 4-6, and Outcome Indicators 10-12) number 3. Ratio of girls to boys (broken down by grades) Grades 1-3, 4-6, and 10-12) (percent) 1. Schools established under EQUIP 2. Number of classrooms built as a result of the project 3. Student: classroom ratio 4. Portion of schools with Intermediate Results Indicators science labs (secondary) Portion of schools with drinking water facilities Portion of schools with Latrines/Toilets/Urinals 5. Schools with school shuras established 1. infrastructure grants (Component 1) 2. 13,000 shura members were trained on social mobilization to promote awareness of importance of education especially for girls. Key Outputs by Component (Component 1) (linked to the achievement of the Objective/Outcome 1) 3. Needs assessment for the human resources required to deliver education services completed. Strategy developed for delivery of the services. (Component 1) 4. 26 District Education offices rehabilitated (Component 2) Page 55 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) 5. 8 TTC buildings (Component 2) 6. Web based GIS school atlas was developed (Component 3) that includes education statistics per school. 7. A gender unit was established within MoE to address complaints etc. (Component 3) Objective/Outcome: Increasing Quality Basic Education 1. Students completing the last grade of a stage (Grades 3, 6, 9, 12) number Outcome Indicators 2. Learning assessment completed for Grades 3 and 6 with reports published 1. Share of schools receiving textbooks as planned 2. Teachers recruited and/or trained (full teacher training programs) 3. Proportion of teachers with professional qualifications 4. Proportion of female teachers 5. Number of Teachers completing INSET I, II, III, IV, V, VI (INSET is in- service short-term training) 6. Number of School Administrators and principals completing Intermediate Results Indicators management training programs 7. Number of female scholarship recipients in Teacher Training Colleges 8. Schools with school information report cards piloted 9. 3 spot checks conducted of schools to verify the validity of data collected in EMIS 10. 5 percent sample check of schools 11. Proportion of schools receiving quality enhancement grants 1. 16,588 Quality Enhancement Grants (Component 1) Key Outputs by Component 2. 70 model schools as demonstration effect for establishing a creative (linked to the achievement of the Objective/Outcome 2) learning environment (Component 1) Page 56 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) 3. 3.7 million textbooks were purchased and delivered to middle schools. The monitoring of the delivery was done by a third-party firm. (Component 2) 4. The TED completed a mapping of all teachers in the country in- terms of their qualifications for the Teacher Management Information System that was developed. (Component 2) 5. Teacher training curriculum for TTC was developed. However, the printing and distribution of the textbooks was not completed due to procurement delays. (Component 2) 6. The development of the standardized assessment and evaluation system was begun under EQUIP II and will be completed under the new EQRA project. (Component 2) 7. Study on teacher supply and demand projections was not completed during the project period. (Component 2) 8. EMIS was established and there were spot checks and audits to enhancing monitoring and ensure greater reliability of data. (Component 3) Page 57 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Preparation Name Role Scherezad J. Monami Latif Task Team Leader Venkatesh Sundararaman Lead Economist Irajen Appasamy Senior Operations Officer Mokhlesur Rahman Senior Operations Officer Hiroshi Saeki Senior Economist Hena Mukherjee Consultant - Education Supervision/ICR Nathalie Lahire, Matiullah Noori Task Team Leader(s) Samantha de Silva Task Team Leader (previous) Rahimullah Wardak, Anand Kumar Srivastava Procurement Specialist(s) Syed Waseem Abbas Kazmi Financial Management Specialist Rajiv Aggarwal Team Member Samira Halabi Team Member Mohammad Yasin Noori Social Safeguards Specialist Abdul Hai Sofizada Senior Education Specialist Sayed Mujtaba Shobair Environmental Safeguards Specialist Ahmed Shah Ahmadzai Team Member Ahmad Khalid Afridi Team Member Akbar Ali Mohammadi Team Member Aman Farahi Operations Analyst Fawad Shams Teacher Education Specialist Sabira Qurishi, Najla Sabri Gender Specialist Page 58 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Pradeep Valsangkar MIS Specialist Deepa Sankar Senior Economist Zohra Farouq Financial Management Specialist Asif Ali Senior Procurement Specialist Abdul Mohammed Durani Social Safeguards Specialist Obaidullah Hedayat Environmental Safeguards Specialist Asif Qurishi Team Assistant Francisco Ayala CCT specialist Jose M. Alvarez Senior Public Sector Specialist Basant Chhetri ISD Database Consultant Aimal Sherzad Procurement Specialist Norbert Mugwagwa Operations Advisor HD Henri Aka Operations Officer Ken Okpara Senior Financial Management Specialist Asha Narayan Financial Management Specialist Prasad Mohan Lead Communications Specialist Abdul Mohammad Durani Social Safeguards Specialist Obaidullah Hidayat Environmental Management Specialist Asuranima Bhatnagar, Rebecca Haynes Gender Specialists Kaushik Sarkar Governance Analyst Helge Brochmann NORAD (donor representative) Poul Erik Rassmusan Evaluation/Learning Assessment Yasuhiko Matsuda Program Leader for Governance/Human Development Tariq Ashraf Social Safeguards Specialist Sandra Beemer ICR Author Elaine Yi Zhong Ding ICR Economist Page 59 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY08 1.375 78,050.95 FY16 3.200 31,297.88 FY17 1.700 100,521.63 FY18 1.775 10,801.23 Total 8.05 220,671.69 Supervision/ICR FY08 6.450 48,393.62 FY09 20.033 104,692.54 FY10 25.244 330,870.58 FY11 71.061 396,121.15 FY12 83.350 524,313.73 FY13 78.652 539,917.07 FY14 108.462 491,642.50 FY15 117.422 518,078.41 FY16 121.451 475,009.35 FY17 90.769 274,832.67 FY18 8.152 177,820.48 FY19 0 104,895.00 Total 731.05 3,986,587.10 Page 60 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) School Grants 77.0 0.0 0 Teacher and Principal 96.01 0.0 0 Training and Education Project Management, 15.0 0.0 0 Monitoring and Evaluation Sub-Total (IDA+ARTF) 188.001 0.00 0.00 School Grants 69.30 0 Teacher and Principal 90.10 0 Training and Education Project Management, 90.60 0 Monitoring and Evaluation Sub-Total 250.00 0.00 0.00 (ARTF Additional Financing) Total Amount Approved Actual at Project Percentage of Approval US$ Closing (US$M) (US$M) School Grants 154.25 205.45 133.20% Teacher and Principal 162.70 140.69 86.50% Training and Education Project Management, 120.05 76.20 63.50% Monitoring and Evaluation TOTAL (IDA+ARTF) 438.00 422.34 96.40% 1Numbers have been rounded up and therefore, differ slightly from Table 2 in the original PAD Page 61 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 4. EFFICIENCY ANALYSIS 1. EQUIP II is expected to have a significant development impact through increasing educational attainment, education quality, and equity in education systems in Afghanistan. The PDO as stated in the financing agreement (FA)22 was to “assist the recipient in increasing equitable access to quality basic education, especially for girls through school grants.� To achieve the PDO, the project is structured in three components with activities related to school grants for infrastructure and rehabilitation (Component 1), teacher, principal and school management training (Component 2), and additional activities to strengthen institutional capacity (Component 3). 2. The following economic analysis will measure efficiency of the project along three dimensions: it will estimate the economic benefits of increased educational attainment due to increased access, estimate benefits associated with improved education quality, and look at economic and social returns to increased equity for girls’ education. In addition, this analysis will conduct a cost-efficiency analysis by comparing unit costs of EQUIP II components with similar education interventions. The economic analysis will end with a brief discussion of the social returns to education not captured by private economic benefits to students. 3. The cost-benefit analysis is conducted by comparing the benefits associated with components 1 and 2 with the costs of all components that may have contributed to such benefits. We limit our benefit estimations to components 1 and 2 because they can be (i) more easily attributed to improved access, education quality, and equity, especially in the short run, (ii) more reasonably appraised from existing evidence, and (iii) at the same time represent a conservative measure of benefits, while acknowledging that strengthened system capacity will also contribute to economic benefits in the future. The cost estimate considers all costs under components 1 and 2, as well as Component 3 costs that may have directly contributed to improved educational benefits. In total, 72 staff were involved in the EQUIP coordination unit as a part of Component 3. This unit makes up 15 percent of total Component 3 staff costs. The rest of Component 3 costs were to support the sustainable functions of the Ministry of Education (MoE). As per Figure 1 below, 15 percent of Component 3 costs will be included in the cost- benefit analysis as they are more easily attributed to increased learning outcomes in the short-run. The project’s framework of economic benefits is shown below in Figure 2. In all instances, we attempted to be conservative in our estimates. No economic analysis was undertaken in the PAD due to the emergency basis of project funding under OP 8.0. 22The project PDO statement was found to be inconsistent between the original Project Appraisal Document (PAD) and financial agreement (FA). According to ICR guidelines (2017), the PDO statement found in the FA is considered the accepted statement. See “Project Development Objectives (PDO)� section of this ICR for more of the topic. Page 62 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Figure 1: Dissagregated Costs of Component 3 Component 3 staff and consultant costs EQUIP coordination Support unit (15%) sustainable functions of MoE (75%) Figure 2: EQUIP II Framework of Economic Benefits Project activities under Short-term benefits Medium-term Long-term benefits components benefits/benefits School grants for Increased enrollment estimated for infrastructure Improved student Improved learning Economic Analysis rehabilitation and facilities Improved educational learning outcomes construction; emphasis on girl’s access (Component Improved teaching and attainment 1) educational quality Improved educational Teacher and principal Increased number of quality training; in-service female students and Increased education education for teachers; teachers equity school management training programs; gender grants for teacher training (Component 2) 4. Given the PDO’s emphasis on school access, the cost-benefit analysis begins with estimating future increased earnings because of increased access to schooling. Component 1 provided school grants for infrastructure rehabilitation and construction, thus trying to increase access to schooling for more students. There is a large body of literature that examines the returns to one additional year of schooling on the increase in future annual earnings. This cost-benefit analysis will utilize Afghanistan’s rates of return as estimated by Montenegro and Patrinos (2014). Afghanistan is estimated to have the lowest rate of return to education of all countries and economies for which Montenegro and Patrinos have data (139 in total). This is likely due to the marked slowdown in growth in the Afghanistan economy caused by civil strife and conflict over the past decades. Page 63 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Despite low returns, labor market data from continual surveys on living conditions in Afghanistan show that education attainment is a key determinant of those in wage employment (i.e. individuals employed in the formal sector). The 2013/2014 Afghanistan Living Conditions Survey (ALCS) found that 80 percent of higher education graduates were formal workers, compared to 42 percent of secondary school graduates, and 26 percent of primary education graduates. Table 1 shows rates of return for Afghanistan and in the South Asia region disaggregated by gender as well as means. Table 1: Estimated private returns to an additional year of schooling by country/region and gender Mean rates of return Country/Region to education (latest Source data) South Asia Overall 7.7% Montenegro and Patrinos 2014 Female 10.2% Montenegro and Patrinos 2014 Male 6.9% Montenegro and Patrinos 2014 Afghanistan Overall 1.6% Montenegro and Patrinos 2014 Female 3.0% Montenegro and Patrinos 2014 Male 1.3% Montenegro and Patrinos 2014 5. Comparing future increased earnings to project costs yields a substantial IRR and NPV under several scenarios. Using the mean rate of return estimated by Montenegro and Patrinos (2014) for Afghanistan (1.6 percent), enrollment rates throughout the project lifecycle, and ALCS-reported (2013/2014) average annual income for an unskilled worker (US$1,48523) as the wage base, increased future earnings can be estimated (Table 2). Increased earnings for direct beneficiaries are compared to the total project costs to calculate net present value (NPV) and the internal rate of return (IRR)24. 6. We calculate returns using projected worst-case, medium range, and best-case scenarios. The worst-case scenario assumes that project schools only enroll students for the project implementation period, and operations of all those schools end entirely after project funding ends. This depicts the worst-case scenario and the actual cost-benefit ratio is likely to be higher, assuming that most schools will continue to enroll students after the project end-date. The medium-range scenario estimates that schools will gradually begin to shut down after the funding period, and projects an average additional five years of operation post-funding, which, remains a very conservative estimate of the long-term impact of the project25. In the best-case scenario, we project schools to continue enrolling students for another 10 years after the project implementation period ends. In a further attempt to produce conservative estimates, the benefit estimates do not capture the benefits of the project on students who were already enrolled in schools prior to the 2008 project implementation start (baseline student 23 72,000 Afs converted to US$ using the ICR-consistent currency conversion rate of US$1 to Af 48.50 24 NPV and IRR are calculated to the project start date and represented in US$ 2008 figures. A 5 percent discount rate (WB/IMF standard) is applied. The IRR is calculated by using 2008 baseline data. 25 Assumes operational costs and enrollment growth rate remains consistent Page 64 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) enrollment), and who will likely have also benefitted from updated infrastructure and better school quality. Estimates for NPV and IRR at mean returns to education (1.6 percent) for the worst-case, medium range and best-case scenarios are shown in Table 3. Table 2: Summary of increased earnings for Table 3: Summary of cost-benefit for project direct direct project beneficiaries beneficiaries Scenario Total earnings increase Scenario NPV of benefits IRR ($US) ($US) Worst-case 729,494,748 Worst-case 414,140,796 31% scenario scenario Medium range 1,655,203,295 Medium range 1,269,335,687 41% scenario scenario Best-case 3,391,162,623 Best-case 2,934,781,361 43% scenario scenario Source: Author's calculations, Assumptions: 5% discount rate (WB/IMF standard), further assumptions on cost/benefit modelling, all NPV's in US$ 2008 figures (project start) Note: “Direct project beneficiaries� account for additional students enrolled only. It does not account for students already enrolled and likely increased attainment for those students due to infrastructure and quality improvements 7. In addition to increased educational access, literature shows that the types of interventions that EQUIP II supports have positive impacts on student learning. The estimated impact of the EQUIP II project components can be proxied using existing evidence of similar education interventions from developing countries. Table 4 presents a summary of estimated impact for selected education interventions that are relevant to activities under components 1 and 2 of EQUIP II. Evidence from a meta-analysis by Krishnaratne, White, and Carpenter (2013) is used as a proxy for EQUIP II activities related to school infrastructure grants, administrator/principal training, and school shura26strengthening. The meta-analysis conducted by McEwan (2013) can be used to estimate the impact of teacher training on student learning outcomes in developing countries. The estimated impacts of interventions that focus on building schools, improving learning facilities, and training teachers range from 0.12 to 0.38 standard deviations increase in student learning outcomes, which is equivalent to 5 percent to 15 percent improvement in student learning achievements of project beneficiaries compared to non- beneficiaries27. 26 Shuras are method of community-based decision making in Islamic society and is common and in widespread use in Afghanistan. 27 This is estimated using a Standard Normal Distribution Table (z-score table). Page 65 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Table 4: Relevant studies on impact of education interventions on student learning outcomes Review Type Authors (Year) Category of Effect size Relevant EQUIP intervention (S.D.) II components Buildings and 0.38** Component 1 infrastructure Krishnaratne, White School-based 0.28*** Components 1 and Carpenter management28 and 2 (2013) Meta-analysis of Learning materials 0.16*** Component 2 evidence from developing countries Infrastructure 0.189 Component 1 Conn (2014) Teacher training 0.228** Component 2 McEwan (2015) Teacher quality: 0.12*** Component 2 teacher training *p<.10, **p<.05, ***p<.01 8. Increased student learning can be compared to project costs to yield a cost-effectiveness measure of the EQUIP II project. To yield a cost-effectiveness measure that can be used comparatively, the project impact is presented using the commonly accepted ratio of standard deviations per US $100. The cost for components 1 and 2 combined was US$36.8 per student over the lifespan of the project. As stated above, we estimate the expected effect of project activities on per student learning outcomes to be between 0.12 and 0.38 standard deviations. Comparing these increased learning outcomes with component 1 and 2 costs yield an estimated range of impact of 0.33 to 1.03 S.D. per US$100. The cost-effectiveness analysis is based on effect size proxies for individual components under EQUIP II. Thus, the overall effect size of EQUIP II is likely to be higher, due to interaction effects that are not captured by this analysis. Table 5 shows a cost-effectiveness comparison with other education interventions aimed at improving learning outcomes (World Bank and non-World Bank). Table 5: Comparison of Project Cost-Effectiveness Project Per student impact per US $ 100 (S.D.) Reading, Philippines 1.17 EQUIP II Afghanistan (World Bank) 0.33-1.03 SEQAP Vietnam (World Bank) 0.06-0.23 Cash-transfers, Malawi 0.06 Source: Non-World Bank programs’ impacts as cited in Evans and Popova, 2014 28School-based management is the proxy measurement for the summative effect of training provided to school administrators, as well as community and parental involvement in school decision-making. Page 66 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) 9. The Project’s development objective was also to contribute to more equitable access to education. The increase in rural schools supported by the project arguably contributed to the 79 percent increase in number of girls enrolled in schools increased by 79 percent in 9 years, with boy’s enrollment growing at 62 percent over the same time period. The Project led to a gender parity index of 0.6229 in 2017, a modest increase from 0.56 in 2008. For female project beneficiaries, there are a host of health and social benefits to education that are not captured in the private rates of return analysis. For example, Schultz (1993) found that an additional year of schooling for girls reduces infant mortality by 5 to 10 percent. Summers (1994) also found children of mothers with five years of primary education 40 percent more likely to live beyond the age of 5. There is also evidence of more productive farming methods as well as a significant decline in malnutrition attributable to increased female schooling (Smith and Hadded 1993). EQUIP II provided scholarships to 11,436 female recipients to enter Teacher Training Colleges, exceeding its post-restructuring target by over 20 percent. Of the recipients for which post-graduation information is available, approximately 46 percent were hired as teachers in public schools. It is likely that the impact of these female teachers on female enrollment is significant (See UNESCO 2006 for case studies) and extraneous to the private economic returns calculated in this analysis. In Afghanistan’s National Emergency Rural Access Program survey, one in ten families cited the lack of a school for girl as the biggest reason for not sending their girls to school (EQUIP I ICR, 2008). 10. The EQUIP II project has been cost-efficient based on the comparison of unit costs compared to similar goods and services at market-rate in Afghanistan. The two complementary objectives of the school grants component (Component 1) were: “to support the improvement of teaching and learning by facilitating the creation of enabling school environments�; and “support the improvement of basic school facilities at existing government registered primary, middle and secondary schools with teachers on payroll.� At the beginning of the project period, an estimated 57 percent of school in the country did not have dedicated school buildings (MoE, 2008). In total, the project built 1,602 schools (which constitutes 91 percent of the schools the project targeted to build30) and 10,431 classrooms (which constitutes 122 percent of the classrooms the project targeted to build). 11. The project experienced delays in construction that resulted in efficiency loss. The unit cost of building a classroom for forty students increased by approximately 140% over the course of the project due to the unstable security context and low fiduciary management capacity by the Ministry during this period. This efficiency loss resulted in the project missing its target of 1,758 schools by 156 schools. Of the 1,602 schools that were built, 114 remained unfinished at project completion. Construction delays had significant impact on the efficient use of project funds. Only 75.1 percent of infrastructure development funds were used by the end of the project implementation period. Construction delays account for 73 percent of remaining project balance at the end of the implementation period (approximately US$16 million). 12. The average unit cost per classroom over the implementation period is compared with average classroom unit costs in other development projects to estimate comparative cost-efficiency. The average unit cost per EQUIP II classroom was US$9,800 over the project implementation period. Table 6 summarizes average per unit classroom costs in selected countries with similar GDP per capita (US$ PPP). based on Department for International Development (DFID, 2011) and World Bank (Theunynck, 2009) data. Despite the implementation 29 Total female student enrollment divided by total male student enrollment. Page 67 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) challenges discussed above, EQUIP II managed to keep average unit costs per classroom well within the range of average unit costs of comparable education interventions. Table 6: Average cost per classroom in select education interventions in countries with similar GPD per capita (US$ PPP) $US GDP per Country Average cost per capita (World classroom ($US) Bank data, latest years) South Sudan 30,000 1,808 Mozambique 15,000 1,217 Malawi 12,410 1,169 Sierra Leone 11,000 1,476 EQUIP II Project (Afghanistan) 9,800 1,944 Bangladesh 5,000 3,580 Cambodia 4,084 3,737 Nepal 3,629 2,478 13. The total benefit of the project is likely to be much higher if we account for non-monetary and social returns. Auturupane and others (2013) found that the educational attainment of the household head is positively associated with per capita consumption and probability of women’s labor force participation. The expansion of basic education increases the foundation of people suitable for higher education and provides important social benefits, especially relevant to conflict and post-conflict countries. The social effects may include dimensions such as mortality, health, population growth, and civic life. These benefits are however difficult to quantify in monetary terms but are nevertheless important to consider in evaluating project effectiveness. Page 68 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Pending receipt from the Government Page 69 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) Project Appraisal Document, World Bank, 2008 Aides-memoire and Implementation Status Reports 2007–2017 Linking of Lessons Learned from EQUIP I with EQUIP II, October 2009 Lexow, J., and Murshed, N. Part 2: Education in Afghanistan – Historical Backdrop of EQUIP, 2014 Program Evaluation of EQUIP II, Afghanistan, June 15, 2015 Impact of Financial Aid on University Enrollment, Retention and Completion in Cambodia, 2017 Lumley, Tom; Mendelovits, Juliette; Stanyon, Rachel; Turner, Ross & Walker, Maurice. (March 2015). Class 6 Proficiency in Afghanistan 2013, Outcomes of a Learning Assessment of Mathematical, Reading and Writing Literacy. Melbourne: ACER Anzai, Danielle; Munro-Smith, Pam; Robertson, Sally; Walker, Maurice; Daraganov, Alex. (2018) Class 3 Proficiency in Afghanistan 2015-16, Outcomes of a Learning Assessment of Mathematical, Reading and Writing Literacy. Melbourne: ACER Third Party Review of Pre-service and In-service Teacher Training Programs, Institute of Social and Policy Science (I-SAPS), 2015 Afghanistan Living Conditions Survey (ALCS). 2013/2014. Central Statistics Organization. Islamic Republic of Afghanistan. Conn, K. M. 2014. “Identifying Effective Education Interventions in Sub-Saharan Africa: A meta-analysis of rigorous impact evaluations.� Teachers College, Columbia University. Department for International Development (DfID). 2011. “Delivering cost effective and sustainable school infrastructure (Guidance note).� London, United Kingdom. Evans D. K., and A. Popova. 2014. “Cost-Effectiveness Measurement in Development.� World Bank Policy Research Working Paper, 7027. Washington, D.C.: World Bank. Krishnaratne, S., H. White, and E. Carpenter. 2013. “Quality Education for All Children? What Works in Education in Developing Countries?� New Delhi: International Initiative for Impact Evaluation (3ie), Working Paper, 20. McEwan, P. J. 2015. “Improving Learning in Primary Schools of Developing Countries: A Meta-Analysis of Randomized Experiments.� Review of Educational Research. 85(3), 353-394. Ministry of Education (MoE). 2008. “Schools Survey Summary Report, Year 1387.� Islamic Republic of Afghanistan. Montenegro, C. E., and H. A. Patrinos. 2014. “Comparable Estimates of Returns to Schooling Around the World.� World Bank Policy Research Working Paper, 7020. Washington, D.C.: World Bank. Schultz, T. P. 1993. “Returns to Women’s Schooling.� In Elizabeth King and M. Anne Hill, eds., Women’s Education in Developing Countries: Barriers, Benefits, and Policy. Baltimore: Johns Hopkins University Press. Smith, L. C., and Haddad, L. 1999. “Explaining Child Malnutrition in Developing Countries: A Cross- Country Analysis.� IFPRI Food Consumption and Nutrition Division Discussion Paper No, 60. Page 70 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Washington, D.C. Summers, L. H. 1994. “Investing in All the People: Educating Women in Developing Countries.� World Bank EDI Seminar Paper No. 45. Washington, D.C.: World Bank. Theunynck, S. 2009. “School Construction in Developing Countries: What do we know?� Washington, D.C.: World Bank. UNESCO. 2006. “Advocacy Brief: The Impact of Women Teachers on Girls’ Education.� UNESCO Bangkok. World Bank. 2009. “Education Quality Improvement Project Implementation (EQUIP) Completion Report.� Washington, D.C.: World Bank. Page 71 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) ANNEX 7. ADDITIONAL INDICATOR DATA Table 1. PDO Indicator Details Based on the 2012 Additional Financing Indicator Baseline Target Actual Percent Change Percent (2012) (2014) (2014) (Baseline to Achieved Actual) (Actual to Target) Enrollment of girls Total: 2,901,801 Total: 3.085 (million) Total: 3.3 (million) Total—13.8% Total—107% (broken down by Gr 1-3: 1,262,870 Gr 1-3: 1.4 Gr 1-3: 1.4 Gr 1-3—10.94% Gr 1-3—100% grades (Grades 1- Gr 4-6: 878,963 Gr 4-6: 0.88 Gr 4-6: 1.0 Gr 4-6—13.78% Gr 4-6—114% 3; 4-6; 7-9; and 10- Gr 7-9: 513,320 Gr 7-9: 0.545 Gr 7-9: 0.58 Gr 7-9—12.99% Gr 7-9—106% 12) (number) Gr 10-12: 246,648 Gr 10-12: 0.26 Gr 10-12: 0.33 Gr 10-12—33.8% Gr 10-12—122% Enrollment of boys Total: 4,580,7011 Total: 4.745 (million) Total: 5.2 (million) Total—13.54% Total—110% (broken down by Gr 1-3:1,783,619 Gr 1-3: 1.9 Gr 1-3: 2.0 Gr 1-3—12.13% Gr 1-3—105% grades (Grades 1- Gr 4-6: 1,366,172 Gr 4-6: 1.4 Gr 4-6: 1.57 Gr 4-6—14.93% Gr 4-6—112% 3; 4-6; 7-9; and 10- Gr 7-9: 952,238 Gr 7-9: 0.955 Gr 7-9: 1.01 Gr 7-9—6.07% Gr 7-9—106% 12) Gr 10-12: 478,672 Gr 10-12: 0.49 Gr 10-12: 0.60 Gr 10-12— Gr 10-12—122% (number) 25.35% Number of 1,110 10,150 classrooms 7,883 classrooms 78% additional (2007 baseline) (1,044 schools) (868 schools) classrooms built or rehabilitated at the primary level resulting from project interventions Ratio of Girls to Gr 1-3=0.71 Gr 1-3=0.73 Gr 1-3=0.70 Gr 1-3—-1.41% Gr 1-3—96% Boys (broken Gr 4-6=0.64 Gr 4-6=0.64 Gr 4-6=0.66 Gr 4-6—3.13% (not achieved) down by grades Gr 7-9=0.54 Gr 7-9=0.57 Gr 7-9=0.57 Gr 7-9—5.56% Gr 4-6—103% (Grades 1-3; 4-6; Gr 10-12=0.50 Gr 10-12=0.53 Gr 10-12=0.55 Gr 10-12—10.0% Gr 7-9—100% 7-9; and 10-12) Gr 10-12—102% (percent) Students Gr 3—0.582 Gr 3—0.59 Gr 3—0.79 Gr 3—35.74% Gr 3—134% completing the last Gr 6—0.376 Gr 6—0.38 Gr 6—0.51 Gr 6—35.64% Gr 6—134% grade of a stage Gr 9—0.265 Gr 9—0.268 Gr 9—0.31 Gr 9—16.98% Gr 9—116% (Grades 3, 6, 9, 12 Gr 12—0.103 Gr 12—0.110 Gr 12—0.24 Gr 12—133.01% Gr 12—218% (number) Learning N Develop a national Completed for Achieved Assessment assessment Grade 6 and framework, capacity preparations were building for being made for developing test items Grade 3 and sampling; conduction national assessments, data compiling and analysis. Finally, finalize results Page 72 of 73 The World Bank Afghanistan - Second Education Quality Improvement Program (P106259) Table 2. PDO Indicator Details Based on the 2015 Restructuring Indicator Baseline Target Actual Percent Change Percent (2008) (12/2016) (12/2017) (Baseline to Achieved Actual) (Actual to Target) Enrollment of girls Total: 1.9 million Total: 3.8 (million) Total: 3.4(million) Total—78.95% Total—89% (broken down by Gr 1-3: 0.88 Gr 1-3: 1.60 Gr 1-3: 1.30 Gr 1-3—47.73% Gr 1-3—81% grades (Grades 1- Gr 4-6: 0.69 Gr 4-6: 1.10 Gr 4-6: 1.10 Gr 4-6—59.42% Gr 4-6—100% 3; 4-6; 7-9; and 10- Gr 7-9: 0.28 Gr 7-9: 0.70 Gr 7-9: 0.70 Gr 7-9—150.0% Gr 7-9—100% 12) (number)) Gr 10-12: 0.08 Gr 10-12: 0.40 Gr 10-12: 0.30 Gr 10-12—275.0% Gr 10-12—75% Enrollment of boys Total: 3.4 million Total: 5.2 (million) Total:5.5 (million) Total—%61.76 Total—106% (broken down by Gr 1-3:1.28 Gr 1-3: 2.0 Gr 1-3: 2.0 Gr 1-3—56.25% Gr 1-3—100% grades (Grades 1- Gr 4-6: 1.24 Gr 4-6: 1.5 Gr 4-6: 1.73 Gr 4-6—39.52% Gr 4-6—115% 3; 4-6; 7-9; and 10- Gr 7-9: 0.65 Gr 7-9: 1.0 Gr 7-9: 1.15 Gr 7-9—76.92% Gr 7-9—115% 12) Gr 10-12: 0.22 Gr 10-12: 0.7 Gr 10-12: 0.63 Gr 10-12— Gr 10-12—90% (number) 186.36% Ratio of Girls to Gr 1-3=69 Gr 1-3=79 Gr 1-3=66.0 Gr 1-3—-4.35% Gr 1-3—84% Boys (broken Gr 4-6=56 Gr 4-6=69 Gr 4-6=66.0 Gr 4-6—17.86% (not achieved) down by grades Gr 7-9=45 Gr 7-9=62 Gr 7-9=60.9 Gr 7-9—35.33% Gr 4-6—96% (Grades 1-3; 4-6; Gr 10-12=37 Gr 10-12=57 Gr 10-12=47.6 Gr 10-12—28.65% Gr 7-9—98% 7-9; and 10-12) Gr 10-12—84% (percent) Students Gr 3—0.51 Gr 3—0.78 Gr 3—0.75 Gr 3—47.06% Gr 3—105% completing the last Gr 6—0.40 Gr 6—0.50 Gr 6—0.56 Gr 6—40.0% Gr 6—94% grade of a stage Gr 9—0.11 Gr 9—0.34 Gr 9—0.32 Gr 9—190.91% Gr 9—112% (Grades 3, 6, 9, 12 Gr. 12—0.06 Gr 12—0.20 Gr 12—0.21 Gr 12—250.0% Gr 12—96% (number) Learning N Y Y-learning Achieved Achieved Assessment assessment completed for Grade 6 and Grade 3 Page 73 of 73