Document of The World Bank Report No: ICR00003587 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-12826) ON A GRANT IN THE AMOUNT OF US$10 MILLION UNDER THE MULTI DONOR TRUST FUND FOR KHYBER PAKHTUNKHWA, FEDERALLY ADMINISTERED TRIBAL AREAS, AND BALOCHISTAN TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A PROMOTING GIRLS’ EDUCATION IN BALOCHISTAN PROJECT December 18, 2015 Education Global Practice South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 14, 2012) Currency Unit = Pakistan Rupees (PKR) 1.00 = US$1.51 US$1.00 = PKR 94.36 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS AEPAM Academy For Education Planning and Management ASER Annual Status of Education Report BEMIS Balochistan Education Management Information System BEF Balochistan Education Foundation BESP Balochistan Education Support Project CPS Country Partnership Strategy DFP District Focal Person DP Development Partner ECE Early Childhood Education EFA Education for All EIRR Economic Internal Rate of Return FATA Federally Administered Tribal Area FM Financial Management ICR Implementation Completion and Results report ISR Implementation Status and Results Report GoB Government of Balochistan GPE Global Partnership for Education KP Khyber Pakhtunkhwa MDTF Multi-Donor Trust Fund M&E Monitoring and Evaluation PTSMC Parent Teacher School Management Committee PDO Project Development Objective PGEB Promoting Girls Education in Balochistan PMU Project Management Unit PP Project Paper SED Secondary Education Department Senior Global Practice Director: Claudia Maria Costin Practice Manager: Keiko Miwa Project Team Leader: Umbreen Arif ICR Team Leader: Shinsaku Nomura ICR Author: Shinsaku Nomura PAKISTAN PROMOTING GIRLS’ EDUCATION IN BALOCHISTAN CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes ............................................................................................ 8 4. Assessment of Risk to Development Outcome......................................................... 12 5. Assessment of Bank and Borrower Performance ..................................................... 13 6. Lessons Learned ....................................................................................................... 15 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 16 Annex 1. Project Costs and Financing .......................................................................... 17 Annex 2. Outputs by Component ................................................................................. 18 Annex 3. Economic and Financial Analysis ................................................................. 19 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 23 Annex 5. Result of Beneficiary Consultations.............................................................. 24 Annex 6. Stakeholder Workshop Report and Results................................................... 26 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 27 Annex 8. List of Supporting Documents ...................................................................... 30 MAP A. Basic Information Pakistan: Promoting Country: Pakistan Project Name: Girls' Education in Balochistan (PGEB) Project ID: P128096 L/C/TF Number(s): TF-12826 ICR Date: 12/09/2015 ICR Type: Core ICR Emergency Recovery GOVERNMENT OF Lending Instrument: Grantee: Loan PAKISTAN Original Total US$10.00 million Disbursed Amount: US$9.19 million Commitment: Revised Amount: US$10.00 million Environmental Category: B Implementing Agencies: Secondary Education Department, Govt. of Balochistan Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: Effectiveness: 10/10/2012 09/12/2012 Appraisal: Restructuring(s): Approval: 08/21/2012 Mid-term Review: Closing: 06/30/2015 06/30/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Highly Satisfactory Risk to Development Outcome: Substantial Bank Performance: Satisfactory Grantee Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Highly Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Primary education 100 90 Secondary education 10 Theme Code (as % of total Bank financing) Education for all 70 70 Gender 30 30 E. Bank Staff Positions At ICR At Approval Vice President: Annette Dixon Isabel M. Guerrero Country Director: Patchamuthu Illangovan Rachid Benmessaoud Practice Keiko Miwa Amit Dar Manager/Manager: Project Team Leader: Umbreen Arif Umbreen Arif ICR Team Leader: Shinsaku Nomura ICR Primary Author: Shinsaku Nomura F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the project is to improve access to education and retention of children in schools, with a special emphasis on girls, in Balochistan. Revised Project Development Objectives The PDOs remained unchanged throughout the project. ii (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Number of girls and boys benefitting from newly established schools and Indicator 1: constructed/improved facilities in selected schools Value (Quantitative or 10,500 23,694 – 38,920 Qualitative) Date achieved 03/31/2013 06/30/2015 – 06/08/2015 Comments Target achieved. See section 3.2. for a more detailed discussion. Original (incl. % Project Paper (PP) kept the baseline and targets blank, to be established after achievement) initial survey of schools and their selection. Indicator 2: Retention of children in project schools Value (Quantitative or 70.0% 85.0% – 86.0% Qualitative) Date achieved 03/31/2013 06/30/2015 – 06/08/2015 Comments (incl. % Target achieved (achievement - 102%). achievement) Percentage of schools with functional Parent Teacher School Management Indicator 3: Committees (PTSMCs) involved in monitoring of activities at school level Value (Quantitative or 0 100 – 100 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments (incl. % Target achieved (achievement - 100%). achievement) Number of girls benefitting from newly established schools and Indicator 4: constructed/improved facilities in selected schools Value (Quantitative or 10,000 20,453 33,414 Qualitative) Date achieved 03/31/2013 06/30/2015 06/08/2015 Comments Subindicator for PDO 1. Target achieved. See section 3.2. for a more detailed (incl. % discussion. achievement) iii Number of boys benefitting from newly established schools and Indicator 5: constructed/improved facilities in selected schools Value (Quantitative or 500 3,241 5,606 Qualitative) Date achieved 03/31/2013 06/30/2015 06/08/2015 Comments Subindicator for PDO 1. Target achieved. See section 3.2. for a more detailed (incl. % discussion. achievement) (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Number of shelter-less primary and middle girls' schools provided with adequate Indicator 1: infrastructure Value (Quantitative or 0 130 – 123 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/30/2015 Comments Target achieved (achievement - 95%). This is particularly impressive in light of (incl. % the security situation. Activity canceled in 7 schools due to security problem. achievement) Indicator 2: Number of primary and middle girls' schools provided with missing facilities Value (Quantitative or 0 200 – 226 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments Target surpassed (achievement - 113%). Number increased by using the (incl. % unutilized and saved funds. achievement) Number of candidates for female teacher post identified and successfully Indicator 3: contracted for newly established government, community-supported schools Value (Quantitative or 0 300 – 260 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments Target achieved (achievement - 87%). Seven schools under Component C did (incl. % not become functional due to security problem and some schools had only one achievement) teacher available in the area. iv Indicator 4: Number of children enrolled in the new schools Value (Quantitative or 0 6,000 – 6,809 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments (incl. % Target surpassed (achievement - 113%). achievement) Number of new government primary schools established based on approved Indicator 5: policy and protocol for community-supported schools Value (Quantitative or 0 150 – 143 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments Target achieved (achievement - 95%). Seven schools did not become functional (incl. % due to security problem. achievement) Indicator 6: Teacher attendance rate in community-managed schools Value (Quantitative or 0 90 – 90 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments (incl. % Target achieved (achievement - 100). achievement) Indicator 7: Number of staff and PTSMC members trained through on-job support Value (Quantitative or 0 1,880 – 3,092 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments (incl. % Target surpassed (achievement - 164%). achievement) Performance reports generated including quarterly progress reports and Indicator 8: independent monitoring reports Value (Quantitative or 0 12 – 12 Qualitative) Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments (incl. % Target achieved (achievement - 100%). achievement) v Indicator 9: System established to monitor school construction Value Established and (Quantitative or 0 – – 136 visits Qualitative) conducted Date achieved 09/30/2012 06/30/2015 – 06/08/2015 Comments (incl. % Target achieved. achievement) G. Ratings of Project Performance in Implementation Status and Results Reports (ISRs) Actual Date ISR No. DO IP Disbursements Archived (US$, millions) 1 03/21/2013 Moderately Satisfactory Moderately Satisfactory 0.36 2 10/09/2013 Moderately Satisfactory Moderately Satisfactory 3.74 3 04/24/2014 Satisfactory Moderately Satisfactory 5.07 4 11/11/2014 Moderately Satisfactory Moderately Satisfactory 8.73 5 06/18/2015 Highly Satisfactory Satisfactory 10.00 H. Restructuring (if any) Not Applicable I. Disbursement Profile vi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country and provincial context. By project appraisal in 2012, Pakistan had been exposed to internal and external security and political conflicts for many years. The major provinces/areas affected by these conflicts were and remain Khyber Pakhtunkhwa (KP), Federally Administered Tribal Areas (FATA), and Balochistan (KP/FATA/Balochistan). Multilayers of economic, political, social, ethnic, and religious factors within and surrounding Balochistan contributed to the highly complex landscape of conflicts and heightened security across these geographical areas. The situation was still far from stable with the deteriorating rule of law. The very poor economic and social indicators of Balochistan across all sectors bear witness to the years of poor development. 2. Sector context. Balochistan had always ranked the lowest in education indicators in Pakistan. In 2010,1 national level net enrollment rates were 71 percent for male, 61 percent for female, and 66 percent in total, whereas the corresponding numbers were only 68 percent, 40 percent, and 56 percent in Balochistan. The indicators for female enrolment were exceptionally poor, especially at the rural level—only 33 percent at the primary level as opposed to 56 percent in rural areas at national level. The rural female enrollment dropped to an abysmally low of 7 percent at the middle school level. The situation was seriously offtrack to achieve the internationally committed national Education for All target for primary education. 3. Supply-side constraints, due largely to the vast geographical area and poor public service provision, have been the major educational issue. Almost 40 percent of 22,000 settlements in Balochistan did not have schools. The average school size was small—50 students for primary, 170 students for middle, and 440 students for high school. Multigrade teaching was commonly practiced due to lack of sufficient number of students, teachers, and school facilities. The increasing conflict and worsening security situation was an added constraint in effective education service delivery and children’s school participation in the province. School infrastructure and facilities were largely nonexistent—9 percent schools were without any shelter,2 approximately, 57 percent schools had no drinking water; 46 percent no boundary wall; 52 percent no electricity; and 29 percent without a toilet facility. 4. To overcome the multifold educational difficulties, Balochistan had established a community- driven school development and management model since 1992. The Balochistan Education Foundation (BEF) initiative supported by the World Bank under the Balochistan Education Support Project (BESP), established 649 schools since 2006 through active community participation, engaging community in all aspects of school management. 5. Rationales for Bank assistance. At the request of the government of Pakistan, the Bank agreed to establish and administer a Multi-Donor Trust Fund (MTDF) to respond to the needs 1 Pakistan Social and Living Standards Measurement Survey 2010/11. 2 Education Statistics Report 2008-09, Academy For Education Planning and Management (AEPAM) 1 resulting from the crisis in the northwestern border areas of KP/FATA/Balochistan.3 The Bank has a long-standing partnership with the government of Balochistan (GoB) in education, from the Community Support Process program in 1992 to the recent Balochistan Education Foundation Project (BEF/BESP). The Bank’s Country Partnership Strategy (CPS) 2015–19 supported the priority of the GoB to restore citizens’ trust and improve provision of basic services at the community level, for reducing the longer-term risk of conflict by increased responsiveness of the state. The Bank’s Balochistan Development Needs Assessment, prepared in parallel with the preparation of Promoting Girls' Education in Balochistan (PGEB), also set girls’ education and community partnership as a priority. 1.2 Original Project Development Objectives (PDO) and Key Indicators 6. The objective of the project is to improve access to education and retention of children in schools, with a special emphasis on girls, in Balochistan.4 7. Key performance indicators for the project are (a) number of children (girls and boys) benefitting from newly established schools and constructed/improved facilities in selected schools; (b) retention of children in project schools; and (c) percentage of schools with functional Parent Teacher School Management Committees (PTSMCs) involved in monitoring of activities at school level. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification 8. The PDO, PDO level indicators, or IO indicators were not formally revised. 1.4 Main Beneficiaries 9. The cohort of girls and boys at their primary and middle schools age in the selected schools in the province of Balochistan were to be the main beneficiaries of the project. They would have access to schools with better infrastructure as a result of Components A and B and access to education in newly established schools under Component C, in areas where there were no schools found within a 2 km radius. Communities and female teachers were also considered as beneficiaries as they were to benefit from training and empowerment through the project. Lastly, the officials of the Education Department were to benefit from technical assistance and experience gained during project implementation. 3 The overall objective of the MDTF is to support implementation of a program of reconstruction and development aimed at recovery from the impact of the conflict and reducing the potential for renewed conflict and include projects to promote enhanced, sustainable, and accountable delivery of basic services, including education, health, and administration of justice. In the first phase, the MDTF received US$140 million in pledges from ten donors: Australia, Denmark, Finland, European Union, Germany, Italy, Turkey, United Kingdom, United States, and Sweden, of which US$10 million was allocated for the PGEB. 4 A minor difference between the Project Paper (PP) and the Grant Agreement was observed. The Grant Agreement says ‘in Balochistan’ while the PP says ‘in the province of Balochistan’. There is no substantive difference in the meaning. This Implementation Completion and Results Report (ICR) uses the one from the Grant Agreement. 2 1.5 Original Components 10. The project consisted of the following four components as described in the Grant Agreement and PP.5  Component A (US$5 million) - Construction of building facilities for shelter-less girls schools (primary and middle). Construction of approximately 130 building facilities for selected shelter-less girls’ schools. The list of schools was to be finalized based on the established criteria, after a physical verification of the site and confirmation of availability of teachers and student enrolments. The communities around the new school facilities will be organized as PTSMCs to ensure local support for effective implementation and school improvements.  Component B (US$2 million) - Provision of missing facilities to girls’ schools. Provision of missing facilities and equipment such as toilets, boundary walls, blackboards, furniture, and connection to public services (that is, drinking water and sewage), to approximately 200 girls' schools.  Component C (US$1.6 million) - Establishment of new government primary schools with community participation. Establishment of approximately 150 gender- free primary schools in coordination/cooperation with the communities they will serve, including the merit-based recruitment of local teachers and the establishment of related PTSMCs.  Component D (US$1.4 million) - Technical Assistance to the Education Department for implementation and monitoring at the district level. Establishment of the institutional framework for planning and implementation of project activities, including in relation thereto (a) the operation of the Provincial Coordination Committee; (b) the establishment of the Project Management Unit (PMU), responsible for the recipient's and Project Implementing Entity’s compliance with the project's safeguards, fiduciary, and monitoring of evaluation requirements; (c) the mobilization of PTSMCs as an innovative approach for community-government partnerships; (d) the carrying out of a baseline survey for the verification of schools prioritized for construction and missing facilities; (e) the design and supervision of construction activities under the project; (f) the commission of third-party validation assessments on the progress achieved in implementing the project; and 5 The following descriptions are mainly from Grant Agreement and complemented with additional text from the PP. 3 (g) an impact evaluation of the project interventions. 1.6 Revised Components 11. Components were not revised during the project implementation. 1.7 Other Significant Changes 12. The project was not restructured. However, during project implementation, there were some changes to the originally estimated target numbers of outputs to reflect the reality of what could be reasonably accomplished. Component A coverage was reduced from 130 to 123 due to inability to construct seven schools in specific conflict areas; Component B school coverage was increased from 200 to 225 by utilizing savings6 generated under Component A; and the number of new schools to be established under Component C was reduced from 150 to 143 due to non- availability of qualified female teachers in certain communities or effective community mobilization in certain communities. These adjustments resulted in a net increase in the total number of beneficiary schools to be supported, from 480 schools to 491 schools.7 At closing, a balance of US$806,679.87 remained unused8 and was cancelled. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 13. Quality at Entry. Based on the assessments below, an overall rating for the quality at entry is Satisfactory. The project was well prepared in accordance with the Rapid Response to Crises and Emergency (OP/BP 8.00) to provide timely and needed educational support to the children in Balochistan by addressing the most acute issues and constraints, which were access to education and retention of students in school. 14. Preparation. In anticipation of the fragile and heightened security situation of Balochistan, the project was carefully designed by (a) reviewing the lessons of the past operations in Balochistan; (b) considering the potential leverage and externalities of the impact given the minimal and simple inputs; (c) prioritizing restoration of citizen’s trust as per the objectives of the MDTF; and (d) assessing the relevance and feasibility of implementation, including safeguards and fiduciary aspects as well as risks. Despite an emergency operation, the PGEB preparation was backed by strong assessments, including fiduciary, environmental, economic and financial, and technical assessments before submission to the Board. 15. Design. A simple design was adopted by building on the lessons from former community- led school establishment model. Departing from the BESP model of community engagement, this project also focused on the government’s role in implementation since restoration of citizens’ trust 6 Discussion of savings in sections 2.2 and 3.3. 7 These adjustments did not require any official restructuring from the Bank side. However, the GoB required revision of the PC-1 document since it entailed revision of procurement items. 8 The amount of nondisbursement should be understood in the context of some delays in implementation and generated savings due to cost-efficient civil works and monitoring activities. See sections 2.2 and 3.3 for more details. 4 against the government was one of the key elements of the MDTF. The PGEB set the government as the implementer9 and through the transparent processes and mechanisms of decision making, it aimed to increase the accountability of government operation. Community involvement, as has been used in former projects, was effective for increasing resilience even in areas with security constraints. Preparation also considered lessons from the Punjab Education Sector Project and Sindh Education Reform Project, especially on using multilayered supervision and monitoring support, including third-party validations and mobile technologies to support information collection and analysis. 2.2 Implementation 16. Strong ownership of the GoB and diligent efforts of the PMU contributed to transparent project implementation, with strong community and beneficiary participation. Despite the continuous conflicts and the extremely difficult operating environment, the project was completed without an extension. The highly unstable security situation across Balochistan and the initial low capacity of the newly established PMU contributed to the slow start and the intermittent delays, but these problems were overcome in the end. 17. The transparent and participatory model of implementation was considered as one of the key factors for the high success in the achievement of outcomes (see section 6) but contributed to some delays during implementation. For example, the selection of the construction site was led by beneficiaries according to the project design. This process generated commendable outcomes of improved capacities from the communities with a resilient model of school management. Communities and PMUs had to invest extra efforts during implementation to solve local level disagreements.10 18. By the end of two years, some savings were generated due to (a) lower costs of civil works than original estimates, (b) cancellation of interventions in some conflict areas, (c) additional time required for recruitment of teachers, and (d) exchange rate gains. The Provincial Steering Committee met to take a decision to reallocate the savings for quick but meaningful interventions and increased the number of Component B schools to 226. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 19. M&E design. The M&E design was deliberately kept simple and suitable for this emergency operation.11 Learning from past operations under security constraints, the design put in place a mitigation scheme for alternative monitoring arrangements, including virtual reviews and 9 The BESP model of community schools was implemented by the Balochistan Educational Foundation, a semiautonomous agency. 10 Grievance mechanism was also effective in providing accountability to the decision-making processes. The PMU received about 100 grievance reports from communities during the project life. Consensus building among community members was very important, so this mechanism was effective; however, it delayed implementation in some cases. 11 The Operations Policy and Country Services Guidance Note for Staff, ‘Ensuring a Results Orientation during Project Preparation under OP/BP 8.00’ suggests using simple and actionable activities and in some cases, “it may be that an output is all that can be reasonably expected.” 5 discussions, the use of third-party monitoring and verification, and the counterparts’ missions in Islamabad. 20. M&E implementation. The project effectively built the capacity of the government to monitor project implementation using the government structure and introduced innovative monitoring solutions. However, the implementation of M&E met with some difficulties in ensuring the quality of information. (a) Strengths. The M&E mechanism was designed to enhance the government’s monitoring capacity, by appointing District Focal Persons (DFPs) in each of the target districts from the District Education Office for supervision of schools. This mechanism was effective for the sustainability of M&E beyond project closure and is already being used by another operation. Regular monitoring of PDO and Intermediate Outcome (IO) indicators through quarterly progress reports and verification through Third Party Monitoring (TPM) took place and were published on the project’s website. To overcome the difficulties of supervising the civil works activities in conflict areas during implementation, the PMU used Google Earth to monitor the progress of civil works, which was later verified by TPM.12 Establishment of baseline data during the first year was a major accomplishment by itself given the conflict nature and weak capacity of the government. (b) Weaknesses. Despite the mitigation measures which were planned for M&E, persistent insurgencies in some areas made effective data collection difficult. Low capacities of the key M&E agents, including DFPs, newly established school management, and the PMU, also added to the difficulty of collecting quality information. Newly established schools were not able to record information related to student flow, such as new entrants, repeaters,13 dropouts, and distinction between absenteeism and dropout, but only report gross enrollment.14 Though establishing a baseline data during the first year was a major accomplishment in this emergency operation, there seem to be mismatches in the baseline enrollment as identified by the Baseline Survey and what is reported in the Results Framework due to the delay in the survey. However, this shortcoming is not too detrimental to the evaluation of PDO performance as the credibility of information is ensured by the triangulation of data from various sources by this ICR (see section 3.2). 21. M&E utilization. Third-party monitoring and verification were actively used to verify (a) information provided for new school establishment, (b) female teacher qualifications, (iii) progress of civil works components (Components A and B), (b) enrollment and retention outcomes, and (e) 12 This was possible because of unique design of the newly constructed school buildings with its octagon shape. By this the risk of fake school (or double dipping of schools with schools that other Development Partners (DPs) support) is effectively prevented and monitored. 13 Concept of repetition is very difficult to monitor in most rural schools in Balochistan since they tend to use multi- grade teaching and teachers adjust flexibly depending on the student levels. 14 Retention rates in these schools are therefore one of the indicators which was monitored by using pseudo-retention rate (that is, not separating new entrants and retained students from previous year). 6 verify PTSMC and their activities. Reports were submitted to the PMU for taking corrective actions accordingly.15 2.4 Safeguard and Fiduciary Compliance 22. The environmental category of the project was B, and the project triggered OP/BP 4.01 Environmental Assessment as the proposed construction of new girls’ schools, missing facilities, and community schools may have potentially caused negative environmental and social impacts, such as soil erosion and water and soil contamination. Overall, safeguards were well implemented. Following the Environmental and Social Impact Assessment, an Environmental and Social Management Plan was developed and implemented by the contractors and monitored by the PMU. DFPs regularly assessed and monitored compliance as part of regular monthly supervision visits.16 23. Fiduciary management was considered overall satisfactory although some delays and inadequate capacity were reported, especially at the initial stages. Delays were often related to recruitment of staff, including financial management (FM) staff and internal auditors. In procurement, post- procurement review identified minor issues, 17 but all were immediately corrected. In FM, poor contract management was reported but it improved overtime. Audit reports were submitted on time. Government financing, according to the interim unaudited financial report, was PKR 2.4 billion (about US$23.5 million), including the original commitment of US$4.46 million planned for teacher salaries and other operating costs for project schools. As described in the PP, the funds were channeled through the government accounting system, which could not track school-specific payroll because budget and expenditure are recorded by the spending unit that represents a cluster of schools. The interim unaudited financial report did not specify government spending on project-specific schools, but there were also no funding issues in target schools. 18 2.5 Post-completion Operation/Next Phase 24. Based on the successful implementation of the PGEB, the Global Partnership for Education-Balochistan Education Project (GPE-BEP) for US$34 million became effective on March 25, 2015, and a new European Union trust-funded project is in the pipeline as of October 2015. The design of the GPE project is significantly influenced by the PGEB, and most of the key activities, including establishment of new schools, construction of school buildings and facilities, and recruitment of teachers, will continue with the operational mechanisms developed by the 15 For instance, 260 schools were confirmed by verification survey to have basic criteria for Component C support out of 808 school applications; the appointment of a few female teachers initially selected were canceled after verifying their official certification; and monitoring of PTSMC activities were verified through meeting minutes and when these are not available, the PMU was instructed to improve the PTSMC activity management. 16 Most environmental safeguard issues came up at the time of school site selection since communities tended to donate land that are in flood-prone areas, farmlands, or forests that would require cutting trees. These were carefully handled by the construction firm and the PMU. 17 These include some incidences of missing contract numbers, not prenumbered requests for quotations, advertisement in a relatively minor newspaper, and nondated completion reports by vendors. 18 While the counterpart funding for project schools were not available as discussed in section 2.4., the team was guided by Operations Policy and Country Services to discuss the counterpart funding in project-specific context following the retirement of OP/BP 6.0. 7 PGEB. Since PGEB-supported schools are all government schools, the GPE, which covers all districts in Balochistan, will naturally support the PGEB-supported schools. Institutional changes and new operational models19 will be continued under the government operation and supported by the same PMU. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 25. The overall rating for the relevance of project objective, design, and implementation in the present context is High based on the following assessments. 26. Objectives. The project objective was consistent with CPS 2011–14 (53553-PK and 65286-PK) and CPS 2015–19 (84645-PK), which prioritize girls’ access to education. CPS 2015– 19 has sought to address sources of fragility and conflict, with an emphasis on restoring trust between citizens and the governments of KP, FATA, and Balochistan.20 The project was consistent with the Balochistan Education Sector Plan 2013–2018, which bases its strategies on EFA and the Millennium Development Goals and other international commitments of Pakistan as well as the National Education Policy 2009. 27. Design. The components of the project were highly relevant for achieving the objectives. The needs assessment of the Balochistan education sector 21 identified key factors of low enrollment as weak public service provision and lack of physical access to schools and learning environment. Components A, B, and C respectively dealt with different types of constraints: no educational access (by Component C); insufficient and insecure physical learning spaces (by Component A); and poor learning environment and lack of basic facilities (by Component B). These interventions provided access to education and increased retention through increased physical spaces, better learning environment, and increased sense of security and sense of attachment to schools. 28. Implementation. The implementation mechanism, operational arrangement, and targeting mechanism were also highly relevant. The following aspects need to be highlighted as part of the relevance of implementation: (a) targeting of districts and schools were evidence–based and used transparent criteria; (b) new school site selection mechanism ensured community participation, existence of out-of-school children, and sustainability of schools; (c) capacity building is embedded in the processes, such as DFPs for monitoring; and (d) conflict risk mitigation is properly designed by the use of third-party monitoring. 19 See section 3.5 (b) for more information. 20 This is also set as the goal of the MDTF for Khyber Pakhtunkhwa, FATAs and Balochistan. 21 World Bank. 2012. Balochistan Development Issues and Prospects. Funded by MDTF. Islamabad. 8 3.2 Achievement of Project Development Objectives 29. The PDO is to improve access to education and retention of children in schools, with a special emphasis on girls, in Balochistan. The performance of the project is rated as High against its objectives and PDO indicators. 30. Improving access to education. Access to education increased substantially in the target school areas. Total number of beneficiaries (PDO 1) reached 38,920 as opposed to the corresponding baseline of 29,726 students in 2012 in target schools,22 with a net increase of 9,194 students, which is 31 percent increase in all intervention schools (Table 1). Contribution of Component C is the greatest in this respect since almost all the students who were brought into the school under this component were new entrants to schools. 23 By the end of the project, establishment of 143 new schools in former school-less communities led to an increased enrollment of 6,809 students as against the target of 6,000 (Intermediate Results Indicator 4).24 Enrollment increased by 19 percent and 5 percent for Components A and B, respectively. 25 Table 1. Number of Students in the Baseline and End Line by Components Baseline End line Increase (#) Increase (%) Component A 5,841 6,929 1,088 19 Component B 23,885 25,182 1,297 5 Component C 0 6,809 6,809 – Total 29,726 38,920 9,194 31 Source: PGEB monitoring data. 22 PDO 1 targets were not set in the PP or Grant Agreement. After baseline was conducted, this baseline of 10,500 students and target of 23,694 students for PDO 1 was set at The Implementation Status and Results Report (ISR) No. 2. However, the list of baseline schools included 561 shelter-less schools from which Component A schools were selected. The baseline 10,500 was based on this list of shelter-less schools, from which 5,841 was the enrollment selected 130 schools and the remaining about 4,600 students were from the nonselected shelter-less schools. Since one of criteria for selecting Component B schools was proximity to Component A schools, Component B schools were selected after selection of Component A. For this reason, they were not identified at the time of baseline survey. As against the final outcome of 38,920 beneficiaries, corresponding baseline obtained for this ICR, including all Component B schools and schools that are additionally included in the last year after cost saving is identified, is 29,726 students. It is understood by the ICR author that the baseline of 10,500, by taking all enrollment of 561 shelter-less schools, was a proxy for 330 schools in the absence of other valid data (because Component B schools were not selected yet), and using 23,694 for the target was in anticipation of adding Component B schools that could be possibly large schools. While this raises an issue of M&E, it does not undermine the real achievements that the project has made in Balochistan. Therefore, this ICR evaluates the contribution of the project by taking the end line and comparing against the corresponding baseline for the finally selected schools. It also reviews the outcome performance by considering the absolute size of outcome impact by the project and cross-checking against other sources of information to contextualize the outcomes achieved, in addition to targets set for the indicators. 23 There are very few cases where students transferred from a far school to the newly established school, but these are extremely rare. More details are in the beneficiary feedback in annex 5. 24 National EMIS reported that primary school enrollment was 670,143 and the number of out-of-school children was 563,214 in 2013 (EFA report 2014). By using this number, additional 6,809 students means 1 percent increase in primary school enrollment or 1.2 percent reduction in the number of out-of-school children in Balochistan. 25 This method of calculation is used here because the M&E design monitors total number of beneficiaries enrolled in the target schools without counting ‘additional new intake’. 9 31. Improving retention of children in school. Retention rates of children, as monitored by the PDO indicator, increased from 70 percent in the baseline26 to 86 percent by the end of the project (PDO 2). This information is validated by an analysis of the Balochistan Education Management Information System (BEMIS), which shows 85 percent retention rate between 2012/13 and 2013/14.27 Beneficiary feedback pointed out two reasons for increased retention rates: (a) availability of physical space to sit and study and (b) increased teachers’ attendance rates, which resulted from increased PTSMC involvement in school management in 100 percent28 of schools (PDO 3), as verified by the TPM surveys.29 32. Achievement and contributions toward higher-level objectives. The PGEB’s contribution to higher-level objectives, as envisaged by the MDTF financing strategy, is also significant. It demonstrated full government ownership of implementation and institutionalization of evidence-based planning and decision making, rebuilt trust between communities and the government, and contributed to the change in communities’ mindset toward education and gender.30 3.3 Efficiency 33. The overall efficiency of the PGEB intervention was High. In addition to a high Economic Internal Rate of Return (EIRR) (18.7 percent), a number of noteworthy efficiency gains took place under the project. Most importantly, the project was completed on time, within 3 years despite continuous conflicts and insurgencies in the province. This, in itself, was a significant achievement. 34. EIRRs. A cost-benefit analysis conducted at the time of the ICR shows a high economic rate of return to the PGEB project. Considering the benefit generated by increased number of Grade 5 completers (as a result of increased intake and retention) and the reduced inefficiencies of the system (which has been already observed by the project close), the EIRR was 18.7 percent, indicating economic viability of the project (see more details in annex 3). 35. Implementation design and operation. The implementation design of all the activities were well thought out and proved efficient for the conflict situation on account of the following reasons: 26 The baseline for target schools is 70 percent. District-level retention rate was 80 percent, using the annual census (BEMIS) report 2009/10 and 2010/11. 27 Though the primary data source for this indicator is monitoring data, the BEMIS was used for calculation of the baseline in the PP, and hence use of BEMIS data for cross-checking is relevant for ensuring the credibility of results. The BEMIS data for school year 2014/15 was not yet available at the time of the ICR mission in September 2015. Therefore, the retention rate is verified by using 2012/13 and 2013/14 data, which showed 85 percent retention rate in target schools. Considering that construction of school buildings in many schools was completed in 2014/15 school year, the benefit of intervention is likely to be underestimating the actual benefit of the intervention. 28 Some schools were dropped from the list after PTSMC training took place, so it is likely that more than 100 percent of project schools got functional PTSMC if those schools were still monitored after being dropped. 29 Two major contributing factors for increasing teachers’ attendance rates are contract teachers and PTSMCs. The contract teachers in newly established schools showed high motivation and high attendance rate, contributing to consistent enrollment of students in schools. PTSMCs increased the accountabilities of schools and resource management, including the teachers’ attendance. 30 See section 3.5 for more details. 10 (a) Reduced civil works cost. The following two factors contributed to reduction of civil works cost: (a) transparency of bidding process and (b) packaging civil works related components (Components A and B) by geographical areas. The transparent bidding mechanism increased the number of bidders sometimes to over 50 bidders per package. 31 School site selection was efficient—after selection, Component A and Component B schools were selected from the surrounding areas. This contributed to lowering the cost of material transportation, cost of supervision, and labor cost due to shortened time. It lowered monitoring cost as well. (b) Innovative approach to reduce start-up cost of new schools. Unlike the traditional, supply-led selection of school sites, Component C asked communities to apply for school establishment and required them to provide the site and two rooms for the first two years of operation. This eliminated the risk of constructing schools in the wrong place. 32 Around 143 schools were efficiently established in this manner with the Component C fund effectively oriented for improving the quality and ensuring smooth start-up through provision of teacher training, supervision, teacher salary and teaching materials. The actual cost of starting up and initial support for ensuring functional school operation was US$4,900 per school.33 3.4 Justification of Overall Outcome Rating Rating: Highly Satisfactory 36. The overall outcome rating of the project is Highly Satisfactory based on high relevance, high achievement of the PDO, and high efficiency, as discussed above. 3.5 Overarching Themes, Other Outcomes, and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 37. Social development and employment. For all the components, local communities played an important role. Under the civil works activities, the use of local materials and local labor force had a positive impact on the local economy. In addition, 260 female teachers gained employment through the project. 38. Influencing the mindset of communities, especially about gender: Establishment of schools in communities had a significant impact on the mindset of the former school-less communities. Establishment of schools lowered the barriers associated with distance, cost, distrust against governments’ service provision, and mentality of households in sending girls (and boys) to school. Female teachers hired for newly established schools in communities where no schools 31 Anecdotal evidence shows traditional government tender of civil works attracted 2 to 3 bidders. 32 The traditional approach of supply-driven school construction was recognized as a negative lesson in the past. Schools that were constructed in wrong places, such as multiple schools in one community, were often abandoned when there was no practical demand. 33 The final cost of Component C was about US$706,000. The unit cost for each of the 143 established schools is US$4,937. This includes, among other things, site selection and verification, community mobilization, PTSCM training, teacher recruitment and training cost, and running cost for the first few years, including teacher salary, materials, and supervision. 11 existed before created a new role model for girls in the communities, influencing perceptions towards female role and employment. (b) Institutional Change/Strengthening 39. New operational models and culture of evidence-based decision making. The project had a major impact on the decision-making process and created new operational models. Starting with the evidence-based district selection process, the project introduced a series of evidence-based decision making. Application and validation-based school site selection and competency-based (test based) selection of female teachers increased the transparency of selection processes and these became new operational models of the government. The GoB is starting on a recruitment system reform by introducing test-based teacher selection mechanism for their regular operation. (c) Other Unintended Outcomes and Impacts (positive or negative) 40. Uniting communities. The processes of school site selection, school establishment, and PTSMC formation and operation resulted in a stronger unification of communities. Earlier, community members used to have diverse opinions about girls’ education. However, through the series of discussions, they made a joint decision to select school sites, elected PTSMC members, and got involved in school management activities. 41. Increased trust in the government. Increased transparency and accountability of selection processes, including school site selection and teacher recruitment, increased communities’ trust in the government, contributing to achievement of the MDTF and CPS objectives. 42. Setting up the stage for and catalyzing follow-up operations. One of the important contributions of the PGEB is the operational experience of using the government system. This approach strengthened the government’s capacity and generated operational experience and education sector knowledge of Balochistan. Success of the PGEB built confidence in the government and encouraged DPs to increase their support, attracting the subsequent GPE-BEP project and a forthcoming European Union funded project. It also left a substantial stock of operational knowledge such as the pool of qualified female teachers, a list of communities with sufficient number of out-of-school children, and willingness to establish new schools. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 43. A series of beneficiary interviews and anecdotal stories from the field consistently supported the high impact of the project on their lives, especially in communities where no schools existed before. Transparent and accountable school selection process, instead of political decision, made positive changes to their mindsets. Communities are more supportive about girls’ education and more trustful in the government’s new service delivery approach (annex 5). 4. Assessment of Risk to Development Outcome Rating: Significant 44. The risk that development outcomes will not be maintained is rated Significant based on the following assessments: 12 (a) Country risk. Continuous conflict and security issues in Balochistan can affect enrollment in supported schools. (b) Social risk. Communities were strengthened to establish schools and manage them through the PTSMC. This model is inherently risky as it is tied to how effective the communities continue to be. (c) Financing. Proper maintenance of infrastructure and facilities and operation of newly established schools are important for lasting impact on enrollment and retention. The lack of operational budgets at schools pose a risk. The newly established schools under Component C are supposed to graduate from interim arrangement after 2 years of operation, but whether the government can provide infrastructure to these schools is also a risk. Continuing the financing of contract teachers from the government’s own resource would also be a risk. (d) Mitigation of risks. Presence of the GPE-BEP project is the main factor for reducing the risk from High to Significant. The same PMU will continue to provide support to all 12 districts that were supported by the PGEB and can continue monitoring the PGEB-supported schools. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 45. The Bank performance at entry is rated Satisfactory based on the following assessments: (a) The PDO and project design were highly relevant and remained so throughout the project period. The objective was aligned with the CPS, the government’s Education Sector Plan, and the strategic approach of Balochistan. The activities were designed based on the needs assessments and lessons learned. (b) Despite the emergency nature of this operation, the task team conducted timely and thorough assessments of fiduciary, safeguards, technical, economic, and institutional aspects and carefully incorporated lessons learned from the previous operation in Balochistan and other education operations in Pakistan. (c) To mitigate potential risks, alternative supervision arrangements, especially on M&E, were identified and proven effective during implementation. Community-focused and government-led implementation design fostered resilience to external disturbances and built capacities of communities and the government. (b) Quality of Supervision Rating: Satisfactory 46. The quality of supervision is rated Satisfactory based on the following: 13 (a) Regular supervision missions by an appropriate skill mix (education, procurement, FM, environment, and social) of experts and continuous interaction between the Islamabad- based task team and the government team in Quetta even during the time of heightened security problem the ISR ratings reflected the slow progress in the project during the first two years. (b) Effective and timely supervision and implementation supports of fiduciary and safeguards. (c) Close involvement of other DPs in the dialogue (United Nations International Children Education Fund, Australia - Department of Foreign Affairs and Trade); and (d) Some weakness in M&E support, especially in the support to conduct an impact evaluation and to collect more detailed M&E data for accurate calculation of indicators. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 47. The overall Bank performance is rated Satisfactory based on ratings for both quality at entry and of supervision. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 48. The GoB, which acted through the Secondary Education Department (SED), 34 continuously showed ownership and commitment to the project. Steering Committee and Project Coordination Committee meetings were held regularly and upon to provide necessary guidance to the PMU and to facilitate coordination among stakeholders, such as, for reallocation of the savings generated. All the legal covenants were complied with. The SED has internalized the new mechanism of school identification and test-based teacher selection, which were initiated by the PGEB, showing ownership and sustainability of the project. The GoB made a commitment to pay the salary of 260 contract teachers from their own resources and will reflect this in the Supplementary Budget for FY2015–16. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 49. The PMU in the Ministry of Education was defined as a responsible implementing agency under the guidance of the SED. The project implementation was rated moderately satisfactory by the first four ISRs and satisfactory by the fifth (final) ISR. Despite a slow start, the pace of 34 The Grant Agreement was signed with the Economic Affairs Division of the government of Pakistan, and the Project Agreement was signed with the GoB, by defining the GoB as the project implementing entity. Due to the decentralized nature of administration and budgeting authorities, the GoB through the SED, acted on behalf of the recipient as a responsible agency. Hence, the SED’s performance is evaluated here. 14 disbursement was largely consistent with the original forecast, and the quality of implementation, including procurement, FM, and safeguard were rated ‘satisfactory’. Timely discussion was initiated for reallocating funds between categories and quick action for amending the government’s project document (PC1) was taken. Implementation of M&E was a positive achievement (that is, the government’s monitoring capacity is built, baseline survey was completed during the first year, and all the monitoring took place despite the conflict situation); however, difficulties associated with baseline data collection and selection of baseline and target values can be considered as one observed shortcoming (see section 3.2.). (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 50. The overall borrower performance is rated Satisfactory based on the satisfactory ratings of the performance of the government and the implementing agency. 6. Lessons Learned 51. Lessons for improving girls’ education. The communities’ strong sense of ownership of schools contributed to enhancing girls’ education. Recruiting female teachers from the own community contributed to increased comfort among community members and developed a great role model for girls. 52. Lessons for improving the targeting of service delivery. A community-led model of needs identification provides a cost-effective solution to new school establishment, which can be adopted in similar, sparsely-populated places. In Balochistan, where low population density and sparse population is a constraint for service delivery, the PGEB tackled the problem by soliciting applications from communities and surveying communities to verify the actual needs of the set criteria. 35 This approach enabled fast establishment of schools in communities, which have commitment and willingness to contribute to education, and increased access of children to school. 53. Lessons for emergency operations. The PGEB provided good lessons for operating under emergency and high security constraints: (a) A simple, clear, and well-focused project design is key to success, especially in an emergency operation and, for low capacity countries. (b) Implementation by the government, as opposed to nongovernmental organizations or other parties, greatly contribute to the sustainability of operations and building trust among citizens. (c) Preparation of alternative monitoring arrangements is important for continuous implementation and monitoring in emergency operation. The PP included some fallback plans for monitoring in case of deteriorating security, and these proved to be very effective. 35 Including existence of at least 20 students in the community, nonavailability of schools within 2 km radius, donation of the land and 2 temporary rooms for 2 years, and availability of qualified teachers in localities. 15 (d) Flexibility of project design toward achievement of objectives and quick decisions would ensure timely implementation and maximization of benefits. When it was found that there would be savings due to cancellation of some activities for security reasons, the PMU took an initiative to revise the procurement plan and the GoB made a quick decision to reallocate funds. Additional outputs, as listed in annex 2, contributed to increase the beneficiaries and project outcomes by efficiently utilizing the available funds. (e) Accountability and transparency are keys to restoring citizens’ trust. Transparent selection of schools and teachers increased trust among citizens toward the government. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies 54. The contents of the ICR were discussed with a range of stakeholders who were involved in the preparation and implementation of the PGEB during the ICR mission in September 2015. The Bank ICR team, the government team, and the project beneficiaries exchanged views on the relevance and impact of the operation and reviewed performance and lessons learned. Subsequently, the draft ICR was shared with the GoB for review and comment. The Bank received a copy of the GoB’s ICR and its key findings are summarized in annex 7. At the time of ICR approval, comments had not yet been received. (b) Cofinanciers NA (c) Other partners and stakeholders NA 16 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$, million equivalent) Appraisal Estimate Actual/Latest Estimate Percentage of Components (US$, millions) (US$, millions) Appraisal Component A 5.00 4.99 99.7 Component B 2.00 2.14 106.8 Component C 1.60 0.82 51.2 Component D 1.40 1.20 85.9 Total Baseline Cost 10.00 9.14 91.4 Exchange rate fluctuation 0.05 Total Project Costs 10.00 9.19 91.9 Counterpart Funding 4.46 23.4836 – Total Financing 14.46 32.67 – (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (US$, millions) (US$, millions) Borrower – 4.46 23.48 526.4 MDTF for Crisis Affected Areas of – 10.00 9.19 91.9 NWFP/FATA/Balochistan 36 As explained in section 2.4, the counterpart funding includes salaries for all schools in target district. The appraisal estimate included estimates only for the project schools that were to be selected. 17 Annex 2. Outputs by Component Table 2.1. Outputs by Component, Based on Indicators Intermediate Results Target Actual Remarks Component A: Construction of shelter-less girls’ schools Activity canceled in 7 Number of shelter-less primary and middle girls’ 1 130 123 schools due to security schools provided with adequate infrastructure problem. Component B: Missing facilities for girls' schools Number increased by Number of primary and middle girls’ schools 2 200 226 using the unused and provided with missing facilities saved funds. Component C: Establishment of new primary schools with community participation Unmet because (a) 7 Number of candidates for female teacher post schools did not become identified and successfully contracted for newly functional and (b) some 3 300 260 established government, community-supported schools only had one schools qualified teacher available. 4 Number of children enrolled in the new schools 6,000 6,809 Target exceeded Number of new government primary schools 7 schools did not become 5 established based on approved policy and 150 143 functional due to security protocol for community supported schools issues. Teacher attendance rate in community-managed Verified by third-party 6 90 90 schools valuation Component D: Technical Assistance to the Education Department for implementation and monitoring at the district level PTSMC of 473 schools Number of staff and PTSMC members trained 7 1,880 3,092 trained. Verified by through on-job support third-party valuation. Performance reports generated including QPR 12 QPR 12 8 quarterly progress reports and independent TPV 2 TPV 2 monitoring reports System establishment was the target. The System established to monitor school Number 136 actual value indicates 9 construction of visits visits usage of the system after establishment of the system. Note: QPR = Quarterly Progress Reports; TPV = Third-party Valuation. Table 2.1. List of Additional Outputs Using the Savings Generated Activities Quantity 1 Tube wells 23 schools 2 School furniture 126 schools 3 Playground landscaping 114 schools 4 Electricity connection 100 schools 5 Early Childhood Education (ECE) training 430 teachers 6 ECE kits 326 schools 7 Student bags 25,000 students 18 Annex 3. Economic and Financial Analysis (a) Economic Analysis 1. The benefits of the PGEB, based on the performance of the three-year implementation, are projected to far exceed the costs, with an EIRR of 19.0 percent, indicating the economic viability of this project. The intervention led to an increase of enrollment and increased retention rates at all grades of primary education. Increased enrollment resulted from the establishment of new schools and provision of female teachers and the improvement of school infrastructure and facilities. Improvement in retention was mainly due to improvement in school learning environment. Assuming a population growth rate of 1.5 percent in the project areas, the intervention will provide opportunities for cumulative 108,680 students in the 143 newly established schools over the next 20 years since the beginning of project, compared to a counterfactual case without the PGEB intervention. In addition, the improvement in retention rate in all beneficiary schools will result in an increased number of Grade 5 completers by 88,475 in comparison with the without-project scenario. 2. The cost-benefit analysis included two streams of economic benefits, which are (a) increased productivity of educated children and (b) reduced inefficiencies and cost-savings due to improved retention rate.37 For the simplicity of the analysis, this does not include other longer- term positive externalities.38 On the cost stream, (a) the PGEB investment cost, (b) increased salary bill for teachers recruited by the PGEB and additional teachers needed for in the future to support increased students, and (c) the maintenance cost of civil works are included.39 Detailed results of simulation and assumptions used for year 4 onwards are described in Error! Reference source not found.Error! Reference source not found. 3. A sensitivity analysis of the economic benefit is also conducted (Error! Reference source not found.), by changing values for two parameters: (a) productivity premium and (b) improved retention rates from year 4 onwards. Productivity assumptions are PKR 3,000 for the base scenario and PKR 1,000 and PKR 5,000 for low and high scenarios, respectively. For improved retention assumption, 1.0 percent improvement per year from year 4 is the base scenario, and 0.5 percent and 2.0 percent are for low and high scenarios, respectively. The EIRR reaches 29.1 percent if both parameters are in high scenario while it goes below 10 percent if both parameters fall under low scenario. The improved retention is not too difficult given the low start of the retention rate and rapidly changing community and school environment as already observed during the first three years of implementation. However, lack of wage employment opportunities in rural villages remains a concern for the productivity premium. While productivity does not necessarily mean 37 Reduced foregone earnings as a result of early entry on time to school and reduced travel time can be also considerable and quantifiable immediate economic benefits that have been reported by beneficiary feedback (annex 5), but these are not included for implication of the assessment. 38 Longer-term positive externalities considered from the interventions include, for example, increased social indictors such as reduced child mortality rate and increased child health attributable to educated mothers, reduced fertility rate, increased social stability and reduced rate of crimes. 39 Cost possibly borne by the communities at the time of school establishment (Component C) is not included since anecdotal stories show that communities usually provided rooms and land that were not regularly used, which means substantial rent cost by donating the spaces was not incurred. Also, counterpart funding is not included as part of the cost since this cost-benefit analysis focuses on additional intake and improved retention based on additional investments. 19 wage income but agricultural productivity and household economic activities, it is known that opportunities for jobs and economic activities are important pull factors for households to send their children to school, and thus, important areas to improve. Table 3.1. Summary of Cost and Benefit Streams and EIRR Years since Project Started Year 1 Year 2 Year 3 Year 4 Year 5 Year 19 Year 20 Program 2013 2014 2015 Total OUTCOMES Number of schools established – 43 100 143 143 143 143 – Number of new students in – 1,720 4,000 5,720 5,720 5,720 5,720 108,680 newly established schools Estimated number of female 30,576 32,879 37,668 41,582 45,011 80,154 82,398 – students [counerfactual] Estimated 30,576 31,847 33,003 33,918 34,513 43,149 43,797 – number of female students w/o project scenario Estimated number of male 1,240 1,998 3,490 5,201 6,558 12,910 13,241 – students [counterfactual] Estimated 1,240 1,310 1,392 1,444 1,473 1,842 1,870 – number of male students w/o project scenario Increased number of Grade 5 – – 134 216 325 9,476 10,118 88,475 graduates (real US$, thousands) BENEFIT STREAM Increased earnings of graduates – – 48 78 117 3,411 3,642 31,851 Reduced cost per graduate due to – – 87 146 228 9,006 9,770 80,229 increased inefficiency Total Benefits – – 136 224 345 12,418 13,413 112,080 COST STREAM Project cost 1,000 4,000 4,200 – – – – 9,200 Increased teacher salary for – – – 180 360 360 360 6,300 contract teachers hired during the project Increased teacher salary due to – – – 126 275 1,083 1,116 14,230 increased students Increased maintenance cost for – – – – 5 5 5 77 civil works Total Costs 1,000 4,000 4,200 306 639 1,447 1,480 28,808 Net Benefits (1,000) (4,000) (4,064) (82) (294) 10,970 11,932 83,272 IRR 19.0% NPV at discount rate of 10% 12,535 Note: For benefit stream: Grade 1 enrollment increases at 1.5 percent natural population growth rate plus the number of seats created by school establishment (40*143 schools) per year. Retention rates to Grade 1–5 improve 5 percent every year based on the improvement from 70 percent to 86 percent according to the PDO; 1 percent thereafter. Difference in productivity between Grade 5 completers and illiterates is worth PKR 3000 (US$30) per month (base scenario). Benefit potentially generated from more number of basic, secondary, and university graduates due to more completers of Grade 5 is not included. (This is a factor for underestimating benefit.) Years invested per student is calculated by standard cohort reconstruction method. 20 Unit cost per student in the project schools is assumed to be (cost of teacher + operating cost which is half of teacher salary) divided by number of students. For cost stream: Annual maintenance cost is calculated as 0.3 percent of total value of the building annually from year 5. Salary includes that for female teachers hired and additional teachers required due to increased number of students (assuming PKR 10,000 per month – US$100 per month – US$1,200 per year). The project will be for 30 months. So, the assumption is, half of the required salary in year 3 will be financed by project. Student teacher ratio is assumed to remain consistent over time. Table 3.2. Sensitivity Analysis of EIRR Productivity Premium Low scenario Base scenario High scenario (PKR 1,000) (PKR 3,000) (PKR 5,000) Low scenario 9.3% 12.4% 15.0% (0.5%) Improved Base scenario efficiency from 16.0% 19.0% 21.5% (1.0%) year 4 High scenario 22.9% 26.2% 29.1% (2.0%) (b) Financial Analysis 4. Public education expenditure in Balochistan is on a gradual rise to 20.8 percent of total expenditure of Balochistan in 2013/14 (Table 3.3). It is expected to be 22 percent of budget in 2015/16 and reached PKR 38.3 billion. The contribution of the PGEB is 0.7 percent to 1.2 percent of total education expenditure between 2012/13 and 2014/15, indicating relatively small fiscal impact. However, its contribution to primary development expenditure is significant – 33 percent of primary education capital expenditure in 2013/14 and 26 percent in 2014/15. The PGEB’s contribution is 10–18 percent of total foreign assistance during this period. In sum, the PGEB had an important contribution to primary education development in Balochistan where foreign support is limited. Table 3.3. Trend of Public Education Expenditure in Balochistan (PKR million) 2010/11 2011/12 2012/13 2013/14 2014/15 2015/16 Revised Actual Actual Actual Actual Budget Budget GoB Total 133,779 145,723 154,727 173,591 220,249 333,686 Revenue expenditure 17,946 21,635 25,540 28,712 38,327 28,955 Capital expenditure 1,581 2,866 2,168 7,364 10,018 7,783 Education Sector Total expenditure 19,527 24,501 27,708 36,076 48,345 36,738 Percentage recurrent in 91.9 88.3 92.2 79.6 78.8 79.3 education expenditure Education in total GoB expenditure (%) 14.6 16.8 17.9 20.8 11.0 22.0 Source: Controller General of Accounts Financial Statements 2010–2013; Department of Finance Budget for 2015/16. Note: 2014/15 Revised budget includes one-off PKR 150 billion development budget transfer from the federal government. 21 Table 3.4. Trend of Public Education Expenditure in Balochistan and Contribution of PGEB (US$, million) 2010/11 2011/12 2012/13 2013/14 2014/15 2015/16 Actual Actual Actual Actual Revised Budget Budget Revenue expenditure 176 212 250 281 284 376 Education Capital expenditure 16 28 21 72 76 98 Sector of which for primary – – – 9 17 19 Total expenditure 191 240 272 354 360 474 Foreign funded development budget for the GoB 21.0 16.5 36.0 33.2 PGEB's contribution Approximate amount (US$, million) 2.0 3.0 4.5 % in GoB’s foreign funded development budget 10 18 12 % in total education expenditure 0.7 0.8 1.2 % in total primary education capital expenditure - 33 26 Source: Controller General of Accounts Financial Statements 2010–2013; Department of Finance Budget for 2015/16. Note: U.S. dollar calculated by using the exchange rate of US$1 = PKR 102, based on Table 3.3. Primary education capital expenditure for 2013/14 is revised estimates. 22 Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team Members Responsibility/ Names Title Unit Specialty Lending/Grant Preparation Umbreen Arif Senior Education Specialist SASED Team leader Surendra K. Agarwal Consultant SASED Anwar Ali Bhatti Financial Analyst SACPK Martin M. Serrano Senior Counsel LEGES Javaid Afzal Senior Environmental Specialist SASDI Mohammad Omar Khalid Consultant SASDI Huma Ali Waheed Operations Officer SASED Ayesha Khan E T Consultant SASED Aristeidis I. Panou E T Consultant LEGOP Samina Mussarat Islam Consultant SASDS Syed Waseem Abbas Kazmi Financial Management Specialist SARFM Ali Ansari Consultant SASED Supervision/ICR Umbreen Arif Team Leader GEDDR Team leader Rehan Hyder Sr. Procurement Specialist GGODR Syed Waseem Abbas Kazmi Sr. Financial Management Specialist GGODR Javaid Afzal Sr. Environment Specialist GENDR Martin M. Serrano Sr. Counsel LEGES Mohammad Azhar Ul Haq Program Assistant SACPK Mohammad Omar Khalid Consultant GENDR Samina Mussarat Islam Consultant CPKI3 Shinsaku Nomura Economist GEDDR ICR team leader (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, thousands (including travel No. of staff weeks and consultant costs) Lending Total: 0 0 Supervision/ICR Total: 2 7,329.60 23 Annex 5. Result of Beneficiary Consultations 1. This annex aims to provide feedbacks from beneficiaries. At the time of the ICR mission in September 2015, the security situation did not permit the members to travel to the field. Consultations with the beneficiaries took place by inviting household members, students, PTSMC members, and recruited female teachers to Quetta, the capital city of Balochistan. In addition, beneficiaries’ views are collected by the PMU and reported in their monitoring reports and videos (http://dai.ly/x2esbvi) are also fed into this summary. 2. Outcomes of the PGEB: (a) Components A and B. Construction of shelter-less schools and facilities gave students physical space and security; better and more comfortable learning environment; and most importantly, stronger sense of attachment. (b) Component C. Establishment of new schools, by using a very transparent mechanism (that is, application by community, rather than politician’s nomination of location), gave communities a strong sense of ownership. Children, if they were permitted by parents, used to start going to school in different villages (2–3 km away) at the age of 8 to 10 years. However, the presence of school in their own community reduced the starting age of school-going children to 4–5 years (including KG class). The number of enrolled children in the community increased from 2 or 3 children to 40 to 50 children and is continuously increasing as older children stay in school and new children are coming in. (c) Component C. Recruitment of female teachers from the community had a major impact because unavailability of ‘trusted’ teachers was always an issue. Communities trust the teacher as they are competitively selected. (d) Challenges and next steps. School teachers and communities started to see next steps. The PTSMC members are keen on increasing enrollment by visiting parents of still out- of-school children (mostly overaged). Teachers are starting to focus on teaching quality, especially in the challenge of teaching in multigrade class. 3. Impact and changes brought to communities: (e) Availability of schools changed life in the communities. Idle children have started coming to school with parents’ mindsets gradually changing to encourage girls’ education. Traditionally, distance to the nearest school, a sense of mistrust against outsiders, and lack of employment prospects were major reasons for not encouraging education by many parents. However, all these barriers are taken away by establishment of a school in the community and having a female teacher there. (f) Girls start seeing a new role model in the female teacher from their own community. Girls might start talking about future dreams as they know education unlocks their future potentials. 24 (g) Trust in the government emerged for the first time. People from marginalized communities, where no schools existed before, had a very low trust in the government. However, transparent school site selection process and regular monitoring and support from government officers (DFPs) started to increase communities’ trust in the government. 25 Annex 6. Stakeholder Workshop Report and Results Feedback from beneficiaries are documented in annex 5. Viewpoints and evidence from implementing agencies, consulting firms and government officials, have been incorporated in the main text. No additional stakeholder workshop were conducted. 26 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR 1. The government’s Project Completion Report embodies the context of the project, achievements of the project including outputs and outcomes, issues and challenges, and key lessons learned from project implementation between September 2012 and June 30, 2015. Achievements 2. The project provided buildings to 123 shelter-less schools and provided missing facilities such as boundary walls, toilets, and water facilities for 226 schools. Around 143 new primary schools were established and 260 teachers recruited with two teachers for each school. PTSMCs for 480 schools were formed and trained to ensure ownership at the community level. 3. Enrollment increased significantly in project schools during three project cycle years. In 2012–13, enrollment under Components A and B schools was 29,726 and in 2013–14, it increased up to 31,101. At the end of the project in June 2015, the total enrollment for Components A and B was 32,111. Under Competent C, 143 new schools were established in 2014–15 with enrollment of 6,101. An increase in enrollment of these schools occurred and the total enrolment was maintained as 6,809 till June 2015 under this component. Indeed, as a whole, the total enrollment in all project schools under all components was maintained at 38,920 till June 2015. 4. Conversely, the project achieved the PDO objectives considerably as the retention rate increased from 70 percent to 86 percent and the enrollment increased from 29,72640 to 38,920. All the PTSMCs were made operational during the course of the project cycle. 5. Overall, the project remained effective as it brought social and developmental changes at the community level. The completion of project activities within proposed time frame is the evidence of the efficiency of the project. However, the effective use of the funds resulted in savings, which enabled the project to work on additional plan for the schools. 6. Under the additional plan, small tube wells were installed in 23 schools; 126 schools were provided furniture; in 126 schools of Component A, landscaping of 114 schools were completed; 100 shelter-less schools were provided with the facility of electricity; and teacher training was conducted for 430 teachers through the Provincial Institute for Teacher Education with the provision of ECE kits to 326 schools. Around 25,000 school bags were distributed in project schools in 12 focus districts. 7. The PMU has made firm progress to accomplish all activities under the project. The PMU developed a very efficient monitoring and complaint redress mechanism to facilitate the complainants and to successfully monitor all project activities. During the course of the implementation of the project, 136 visits were made by the monitoring team. On the other hand, the PMU also launched the project website to upload the progress on it for wider sharing and also to address complaints. 8. As a part of project implementation, a few lessons learned would be expected to guide future reforms across the province as the firm commitment and ownership of the government will 40 See the footnote under section 3.2, which discusses the baseline value. 27 enable projects to accomplish their target within the time frame and in an efficient way. While the innovative design of newly constructed school buildings and the role of DFPs remained dynamic, the formulation of criteria for various educational units, that is, teachers, DFPs, and PTSMCs selection criteria, can be used for any upcoming project. On the other hand, the prioritization of districts based on established criteria and the recruitment of the teachers on merit and on contract enabled the merit and sustainable system in the education department. However, mobilization and engagement of PTSMCs, will definitely ensure governance at community level. 9. Overall, the Bank supervised the project on various aspects of planning, implementation, and monitoring as missions were conducted regularly and recurrent coordination remained with the PGEB staff and other stakeholders. As a whole, the performance of the Bank, the government, and the PGEB were up to the mark. 10. Issues and challenges: (a) Service delivery. The large geographic spread, low population density, poor infrastructure, and weak communication systems created problems for the team to carry out monitoring activities. The PGEB team handled these issues in innovative ways by establishing a mechanism to use the services of district education officials such as DFPs. This mechanism helped the PMU maintain coordination and communication with stakeholders at all tiers of education and communities at the district level. (b) Unavailability of reliable data and information. Unavailability of reliable data was a key issue at project commencement. A baseline survey was conducted to verify the BEMIS report which was used for school selections, but it caused some delays in the initial selection process. (c) Decision-making process. At the start of the project, PC-1 of the project was reviewed and proposed to be revised to align with the scope and cost of the project. (d) Social constraints. Due to the community-driven mechanism under Component C, the project team faced issues in recruitment of teachers and site selection. In various places, the PTSMCs had conflicts among themselves and it created hurdles to complete the tasks on time. (e) Security constraints. Security remained a high risk and the situation affected the progress of project implementation. The project used alternative mechanisms to ensure regular supervision of PGEB activities. 11. Lessons learned: (a) Government commitment and ownership. Strong government commitment and ownership is critical for successful implementation and sustainability of service delivery models which make use of mechanisms outside the public sector. (b) Innovative design for school. The uniquely designed schools have not only provided shelter to children but the beautiful design of the building has attracted the community 28 to enroll their children. Enrollment in the (former) shelter-less schools had been growing significantly. (c) Dynamic role of DFPs. The DFPs had a very important role in the implementation and monitoring of the project. Their involvement in project activities not only made the project successful but also enhanced the capacity of the DFPs to have good knowledge of project implementation and procedures, which helps in capacity building of the SED. (d) Formulation of criteria for various activities. Formulation of school establishment criteria, teacher selection criteria, and various terms of reference are useful and assist in coping with political pressures in selection processes. (e) Prioritization of districts based on established criteria. The SED did not have a process to short list infrastructure needs from a large pool of shelter-less schools. The project identified the set of basic criteria for short-listing districts and schools. Contracts for civil works was packaged with missing facilities in nearby schools to ensure efficiency in construction and monitoring. (f) Establishing a system of community engagement in school siting and identification of teachers. As a result of political interferences and poor planning, schools were often built in wrong locations, teachers appointed were non-local and faced mobility challenges. Nonfunctional and underutilized schools had created a huge fiscal burden on the education sector. The new system of community engagement was effective to reach more deserving and dedicated communities. (g) Recruitment of teachers on contract basis. Politicization of the teaching force had resulted in poor hiring procedures and low accountability of teachers before the project. (h) Mobilizing PTSMCs. Community members were organized as PTSMCs and supported functions of accountability and management of schools at the community level. Such empowerment of communities can lead to high levels of ownership. 29 Annex 8. List of Supporting Documents A. World Bank Project Documents  PGEB Aide Memoires (Oct 2011, Dec 2011, Mar 2013, Nov 2013)  CPS for Pakistan for FY2011–14: Report No. 53553-PK (2010) and 65286-PK (2011); and CPS for FY2015–19: Report No. 84645-PK. (2014)  Balochistan Development Issues and Prospects. Funded by Multi-Donor Trust Fund Islamabad (2012)  Post Crisis Needs Assessment (PCNA) - KB and FATA  MDTF for PK, FATA, and Balochistan 2010-2011 Financing Strategy  Other Project Documents for Balochistan Education: GPE-BEP PAD (Report No. PAD910). PAD and ICR for BESP (Report No. 34999-PK, and ICR00003246)  Emergency PP for PGEB, on June 14, 2012  Grant Agreement, Grant No. TF012826-PK, September 12, 2012, and Project Agreement (negotiation draft)  PGEB ISRs 1–5 (2013-2015)  Operational Policies OP 8.00. (2014) B. Project Documents and Data  Balochistan Education Management Information System (BEMIS) 2012-13 and 2013-14 databases  Centre for Research and Development – Balochistan and Linkages for Resilience and Development Consortium. 2014. PGEB TPG Study (Round 1) Report  Centre for Research and Development – Balochistan. 2013. PGEB Project Baseline Survey Final Report  PGEB PMU. Project Completion Report (Draft)  PGEB PMU. Quarterly Monitoring Reports. 2012–2015  PGEB PMU. Annual Reports. 2013 and 2014  Steering Committee and Project Coordination Committee meeting minutes  Environmental and Social Screening and Assessment Framework for MDTF (2010)  Ex Post Review of Contracts Awarded under PGEB C. Other Documents  Education for All 2015 National Review Report: Pakistan. Islamabad Pakistan. June 2014  Government of Balochistan. 2012. Balochistan Education Sector Plan 2013–2018. Developed by Policy Planning Implementation Unit, Education Department of Government of Balochistan 30 IBRD 33460R PA K I S TA N SELECTED CITIES AND TOWNS MAIN ROADS PROVINCE CAPITALS RAILROADS NATIONAL CAPITAL PROVINCE BOUNDARIES RIVERS INTERNATIONAL BOUNDARIES 65E 70E TAJIKISTAN 75E CHI CH I N A Ka h ra K2 K us NORTHERN k o (Mt. Godwin- r a (8,611 m) Austen) d u ARE AS AREAS m PAKISTAN i n Chitral I nd s u Ra H ng KHYBER e 35N Approximate Line of Control PAKHTUNKHWA Saidu To Muzaffarabad Kargil Kabul Srinagar Peshawar Jammu ISLAMABAD and Kashmir Rawalpindi Kahat FED. CAPITAL AFGH AF ANI G H AN I S TAN Bannu TERRITORY ISLAMABAD lum Jhe Gujrat Gujranwala To Mandi ab D.I. Khan n To Che Kandahar Zhob Lahore Faisalabad vi Ra Chaman Zhob PUNJAB To To Bhatinda Ludhiana Sahlwal 30N Quetta D.G. Khan Multan tlej 30N Su us Ind To Kerman Bahawalpur Nok Kundi B A L O C H I S TA N Surab rt se ISLAMIC D e REPUBLIC INDIA r OF To Ranipur a h ng e Panjgur IRAN Khash T a nR i s hka kra Ma Moro Ma l To Turbat C e ntr a Bela SINDH Kerman To Jodhpur Gwadar Pasni Hyderabad 25N 25N Karachi Thatta Badin us Ind This map was produced by Rann of Kutch the Map Design Unit of The World Bank. The boundaries, Arabian Sea colors, denominations and 0 50 100 150 200 Kilometers any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the 0 50 100 150 Miles legal status of any territory, or any endorsement or acceptance of such boundaries. 65E 70E 75E NOVEMBER 2010