The World Bank Pakistan: Third Punjab Education Sector Project (P154524) REPORT NO.: RES35730 RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF PAKISTAN: THIRD PUNJAB EDUCATION SECTOR PROJECT APPROVED ON JUNE 3, 2016 TO ISLAMIC REPUBLIC OF PAKISTAN EDUCATION SOUTH ASIA Regional Vice President: Hartwig Schafer Country Director: Patchamuthu Illangovan Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager/Manager: Mario Cristian Aedo Inostroza Task Team Leader: Tazeen Fasih, Koen Martijn Geven The World Bank Pakistan: Third Punjab Education Sector Project (P154524) ABBREVIATIONS AND ACRONYMS AEO Assistant Education Officer CPD Continuous Professional Development DSD Directorate of Staff Development DTE District Teacher Educator GoPunjab Government of Punjab NTS National Testing Services PESP-III Punjab Education Support Program - III QAED Quaid-e-Azam Academy for Educational Development SED School Education Department The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Note to Task Teams: The following sections are system generated and can only be edited online in the Portal. BASIC DATA Product Information Project ID Financing Instrument P154524 Investment Project Financing Original EA Category Current EA Category Not Required (C) Not Required (C) Approval Date Current Closing Date 03-Jun-2016 31-Dec-2021 Organizations Borrower Responsible Agency Islamic Republic of Pakistan School Education Department Project Development Objective (PDO) Original PDO The project development objective is to support Punjab province to improve school participation, completion, and teaching-learning practices with a particular focus on low-performing districts. OPS_TABLE_PDO_CURRENTPDO Summary Status of Financing Net Ln/Cr/Tf Approval Signing Effectiveness Closing Commitment Disbursed Undisbursed IBRD-86200 03-Jun-2016 31-Aug-2016 05-Oct-2016 31-Dec-2021 300.00 116.33 183.67 Policy Waiver(s) Does this restructuring trigger the need for any policy waiver(s)? No The World Bank Pakistan: Third Punjab Education Sector Project (P154524) I. PROJECT STATUS AND RATIONALE FOR RESTRUCTURING A. Background 1. The Third Punjab Education Support Program (PESP-III) builds on a history of ambitious school reform projects, and has now been under implementation for two years, since September 2016. The project focuses on two main objectives, namely to improve school enrolment (participation and completion) and improving school quality (i.e. the teaching and learning environment). These goals are supported by 9 disbursement linked indicators that address a wide range of actions, namely (DLI-1) the use of vouchers to enroll more children, (DLI-2) expanding and improving the management of public-private partnerships, (DLI-3) stipends to keep girls in schools, (DLI-4) improved early childhood education, (DLI-5) better teacher recruitment, (DLI-6) improved teaching professional development, (DLI-7) student assessment, (DLI-8) non-salary budgets (NSB) managed at school level, and (DLI-9) data integration to improve the use of evidence in policy-making. 2. Progress towards the achievement of the PDO and implementation progress are currently both rated as Moderately Satisfactory. In the first year of implementation, 7 out of 9 DLIs were met. In the second year, 6 out of 9 Project DLIs were met. In the third year, it is expected that 6 out of 9 DLIs will be met, while an additional DLI from the previous year is also expected to be met. There are some impressive results to highlight. The voucher program has consistently outperformed its targets, and public private partnerships now enroll over 2 million students. Over 560,000 families are receiving stipends to keep their girls in school beyond primary education. Over 7,000 early childhood classrooms have been established, out of which 2,700 are meeting new quality standards. All new public-school teachers are being recruited using meritocratic criteria (including a content knowledge test), and further reforms are being planned to improve merit-based recruitments. All schools in 36 districts are now receiving non-salary budgets that can be utilized to improve infrastructure, purchase teaching and learning material and other immediate needs. Finally, data from many sources has been integrated into a new school information system, while all major policy-makers have been trained on how to use this system to track major indicators on the education system in the province. 3. Two DLIs, 6 and 7, have remained unmet annually during these first years of project implementation. These DLIs have not been achieved due to institutional and governance challenges in SED. Specifically, for DLI 6, (continuous professional development or CPD for teachers) has primarily remained unmet because of a change in institutional structure in the School Education Department. The SED has reformed the Directorate of Staff Development (DSD) to create the Quaid-e-Azam Academy for Educational Development (QAED). As part of the structural change, SED has hired a batch of school-mentors called Assistant Education Officers. These are mostly former teachers, who have been identified because of their expertise in pedagogy. Former District Teacher Educators (DTEs) as well as a set of identified teachers now have a different function, acting as local cluster experts who help teachers with increasing content knowledge. The Bank and SED have agreed on a level 2 restructuring of the project to ensure alignment of the project with the new institutional setup and a resumption of the implementation of activities related to the DLI. 4. DLI 7 on student assessments implemented under Punjab Examination Commission (PEC) had been pending the members of commission being notified and assignment of CEO for PEC. Both these issues appear to be resolved at present time. A third DLI, namely DLI 8 on non-salary budget was not disbursed against in FY18, pending some additional The World Bank Pakistan: Third Punjab Education Sector Project (P154524) information on the quarterly disbursements of funds to schools. These latter issues seem to have been resolved after the election period, and disbursement is expected soon. 5. An additional challenge is that the change in government had left some key positions vacant in SED and its attached institutions. Furthermore, there has been a frequent turnover of high level SED staff including the Secretary SED. However, the Program Management and Implementation Unit (PMIU) has remained with a technically strong Project Director which has reduced the immediate impact of the frequent SED turnover on the project activities. There were also delays in procuring the Technical Assistance firm under the project, that is tasked with providing technical assistance which help in the implementation of the DLIs and in monitoring and evaluation. While it is expected at present that DLIs 7 and 8 will resume implementation, further restructuring may be necessary if implementation does not improve. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) B. Rationale for restructuring 6. DLI 6, linked to the continuous professional development of teachers is being restructured as per the request of the SED and the agreement of the WB. The rationale for restructuring is two-fold. First, the institutional setup of the institution proposed to implement DLI 6 at the time of project preparation, the DSD, was re-structured and transformed in QAED. The functions and role of its personnel was revised accordingly as well as additional staff was brought on board as Assistant Education Officers (AEOs) to monitor and serve as mentors for teachers (changes described in para 3 above). This necessitates a revision of the DLI to ensure that the new operational functions of QAED are reflected in the project. Secondly, a baseline survey to establish the PDO indicator on “Quality score of primary teaching-learning practice” was conducted in Punjab. The findings of the survey have helped identify the training needs of the teachers. Specifically, an estimated 90 percent of public school teachers have the minimum required teaching qualification, and the average public-school teacher scored over 80 percent on an independently administered content knowledge exam. However, fewer than 10 percent of teacher showed good practice in instructional support. While these new teachers understand the content, they do not appear to be ready for the environment they encounter in the classrooms. These teachers have had very little or no pedagogical practice before entering classrooms and have to start teaching without adequate professional support. 7. With the specific training needs of the average teacher identified, the restructuring of DLI 6 is seen as an opportunity to support targeted improvement in the pedagogical practices of the teachers. There is a recognition that many professional development programs around the world have failed, or even had negative impact on teaching. The main element of successful programs is the emphasis on continuous teacher mentoring, based on a regular interaction between the teacher and an experienced mentor. Both the QAED and the new batch of AEOs would be well-placed to implement such a continuous mentoring system. An important element in this is the new ‘Teach’ instrument, which is an innovative classroom observation protocol. The instrument helps mentors to assess teacher performance in the classroom, making sure that any feedback would be based on evidence of what happens in these classrooms. An important element of this tool is the creation of a video library of teacher practices, which can also be used to train teachers on pedagogic practices. Such a video library provides an innovation to professional development programs that would be easier to replicate at a scale such as that of Punjab. 8. ‘Schools of the Future’ is a phrase that the Education Global Practice uses for schools that continuously try to improve what happens in the classroom. Teacher professional development aimed at supporting teachers plays a key role in these schools. Teachers are supported to experiment with new teaching practices, while also learning to refine their practice through sustained interaction with peers and mentors. School administrators hold these schools accountable for learning results, while also providing continuous support to reach those results. Data on students, particularly using formative learning assessments, can be used by teachers to adjust their lesson plans and examples to what students are struggling with (something often referred to as ‘teaching at the right level’). 9. Punjab already has many elements to create ‘Schools of the Future’. Data is being collected at several levels and is currently being combined in an innovative data platform that is easily accessible. The government has recently put technology (tablets) in the hand of each school inspector (AEO) and each school administrator. This is currently being used to collect data on the schools; but the availability of this technology has the potential to be used much more widely. There is a wide contingent of AEOs, administrators and head-teachers that can support teachers, if the system will be aligned to teacher support, rather than control. 10. Changes to Results Framework (RF): The RF is updated to reflect the change to PDO indicator pertaining to quality score of teaching learning practices in line with the change to DLI 6 as stated above. In addition (i) the source for The World Bank Pakistan: Third Punjab Education Sector Project (P154524) the PDO indicator on school participation has been updated, which does not constitute a change in the numbers as such; (ii) baseline and targets have been established for one PDO and one intermediate outcome (IO) indicator (school readiness and budget execution); (iii) IO indicator pertaining to implementation of CPD has been updated in line with the change made to the DLI 6 . Details on these changes are presented in the section below. II. DESCRIPTION OF PROPOSED CHANGES A. Elements to the Restructuring 10. The new results under the DLI will require a change in the behavior of teachers in the classroom. This comes with the realization that behavioral change is one of the hardest things to achieve in education projects. The DLI is therefore rephrased (new language in the Annex), in order to more directly support actions that lead to behavioral change. This new language draws on a recent review of the evidence on successful teacher support programs, as well as the new Education Global Practice strategy for teachers. 11. The first element of this is to reform the school inspection system. The AEOs (or similar mentors) will be at the forefront of this new system; they will need to meet regularly with teachers (at least once per month), observe classrooms using the right tools and provide constructive feedback based on classroom observation. This means that the AEOs will function more as a mentor and colleague, and less as an inspector or an authority. At present, AEOs have been given many administrative duties, primarily to collect data on schools, which mostly a duplication of data that is collected elsewhere. Changing this will require the SED to substantially revise the task description of these school mentors / inspectors. At present, each AEO covers about 15 schools, and can visit one school per day. This means that in a given month, an AEO can visit a given school about once per month, or maximum twice per month. Since an average school has 4 teachers, the AEO will need to spend at substantial time (e.g. 1-2 hours) with teachers during any school visit in order to see each teacher once a month. Beyond school visits, AEOs can use tablets and mobile phones to connect with their circle of teachers and to share good practices. 12. The second element is the development of an innovative package for continuous professional development and piloting this package. This package will provide teachers and AEOs with examples of good practice in teaching, which are contextualized to Punjab. The package will also contain modules on how to provide feedback, which will be aimed at head teachers and the AEOs. The package will be primarily digital, consisting of short videos and background material, and will be made accessible for tablets and cellphones. This package will be piloted in a set of 200 schools, in order to learn lessons about whether it works as intended. Ideally, this will be done in the setting of a randomized controlled trial. 13. The third element is the integration of the innovative package in the mainstream continuous professional development system for teachers across Punjab. The QAED is running a mainstream program in 6 districts in each year that is left of the project (FY20 and FY21), allowing the government to integrate the successful elements of the innovations in at least six districts (during FY21). Rolling out this package will require substantial supervision and coordination from district administrators and district-level AEOs. These will have to be trained, certified and continuously supported in order to ensure a high-quality implementation. Hence, in order to ensure the quality of implementation, this is slightly more limited in scope than initially planned (18 districts were initially targeted). The districts in which the package will be implemented will be selected by QAED, focusing on the districts on which they have strong implementation capacity. This approach will ensure the highest quality of implementation, and building a The World Bank Pakistan: Third Punjab Education Sector Project (P154524) case for scaling up afterwards with lessons learned from initial implementation. Since all districts in Punjab are low performing on teacher practices, this is considered to be fully in line with the PDO. 14. The revamped QAED, which has taken over the previous functions from the DSD, will play the leading role in the implementation of this DLI. QAED has the capacity to implement the fiduciary obligations undertaken by the Government of Punjab, further to any assessments done by the Bank. The SED and PMIU continue to support QAED in the fulfillment of the goals for this DLI, as general coordinating institutions. The QAED will also be supported by the Technical Assistance component of the project as the TA firm is now fully on board. The TA focuses primarily on the content development, implementation and evaluation of the innovative support package. B. Changes to Results Framework The changes to the results Framework are described below which include a number of minor changes to establish base- line and end-line results. In addition, the data in the operations portal has been updated in line with the PAD for some indicators. None of these changes are expected to affect project performance or disbursements.  The PDO Indicator on Quality score of primary teaching-learning practices has been revised in line with change made to DLI 6. The baseline and an endline on teachers’ classroom practices has been established, specifically aiming for a 0.25 standard deviation increase in teacher pedagogical practice. Setting a baseline was already envisaged in the original project structure and is now possible with the completion of the service delivery indicator survey.  The PDO Indicator on school participation. The PAD defined PSLM as the only data-source to track school participation data. In the last years, PSLM has not been implemented, leading to a lack of data to track this indicator. The restructured results framework allows for the use of HIES and MICS data to track school participation, which have a similar sampling strategy and are representative and high-quality household survey data. This does not affect the baseline measure.  The PDO indicator on school readiness. This indicator was supposed to have been defined at baseline. Using new administrative data, this indicator is now updated as the percentage of children who are enrolled in quality ECE environments in public schools, out of the entire population of children enrolled in public pre-school (Katchi, nursery and other forms). At baseline, this share was an estimated 0% as all pre-schools were following an outdated curriculum, and investments in these classrooms were long overdue. At present rates, 2% of children age 3-5 are enrolled in such programs. The government estimates that, by project end, 6% of all children can be enrolled in such quality environment.  Intermediate Outcome indicator on budget execution of the non-salary budget. Similarly, the baseline and end-line for budget execution was underdefined due to lack of data availability at project preparation. The government estimates that at project baseline, 64% of NSB budget was being utilized. In line with the PAD, which anticipated 2% increase over baseline by Year 3 and 4% increase over baseline by Year 5, the government expects this to increase to 66% by Year 3 and 68% by project end.  Intermediate Outcome Indicator on continuous professional development. In line with the new DLI formulation, this has been revised to cover 6 districts in year 4 and another 6 districts in year 5, making the total 12 districts. QAED will select the districts where it is most likely to implement this program successfully (since all districts are low performing on teaching practice, this is in line with the PDO). The World Bank Pakistan: Third Punjab Education Sector Project (P154524) I. SUMMARY OF CHANGES Changed Not Changed Results Framework ✔ Other Change(s) ✔ Implementing Agency ✔ DDO Status ✔ Project's Development Objectives ✔ Components and Cost ✔ Loan Closing Date(s) ✔ Cancellations Proposed ✔ Reallocation between Disbursement Categories ✔ Disbursements Arrangements ✔ Disbursement Estimates ✔ Overall Risk Rating ✔ Safeguard Policies Triggered ✔ EA category ✔ Legal Covenants ✔ Institutional Arrangements ✔ Financial Management ✔ Procurement ✔ Implementation Schedule ✔ Economic and Financial Analysis ✔ Technical Analysis ✔ Social Analysis ✔ Environmental Analysis ✔ IV. DETAILED CHANGE(S) . The World Bank Pakistan: Third Punjab Education Sector Project (P154524) . Results framework COUNTRY: Pakistan Pakistan: Third Punjab Education Sector Project Project Development Objectives(s) The project development objective is to support Punjab province to improve school participation, completion, and teaching-learning practices with a particular focus on low-performing districts. Project Development Objective Indicators by Objectives/ Outcomes RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 To improve school participation Participation Rate (Percentage) 75.80 75.80 75.80 77.00 77.00 79.00 Rationale: Action: This indicator has been There is no change to the indicator name, baseline or targets; however, the data source has been updated. Revised Participation rate, boys (6- 85.40 85.40 85.40 86.00 86.00 87.00 10 years) (Percentage) Rationale: Action: This indicator has There is no change to the indicator name, baseline or targets; however, the data source has been updated. been Revised Participation rate, girls (6-10 77.80 77.80 77.80 79.00 79.00 80.00 years) (Percentage) The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 Rationale: Action: This indicator has There is no change to the indicator name, baseline or targets; however, the data source has been updated. been Revised Participation rate, boys (11- 75.40 75.40 75.40 77.00 77.00 79.00 15 years) (Percentage) Rationale: Action: This indicator has There is no change to the indicator name, baseline or targets; however, the data source has been updated. been Revised Participation rate, girls (11- 64.60 64.60 64.60 66.00 66.00 68.00 15 years) (Percentage) Rationale: Action: This indicator has There is no change to the indicator name, baseline or targets; however, the data source has been updated. been Revised Participation rate, poor- performing districts 68.50 68.50 68.50 70.00 70.00 73.00 (Percentage) Rationale: Action: This indicator has There is no change to the indicator name, baseline or targets; however, the data source has been updated. been Revised To improve school completion (Action: This Objective has been Revised) Completion rate, grade 5 67.00 68.00 70.00 (Percentage) Rationale: Action: This indicator has been There is no change to the indicator name, baseline, targets or evidence protocol. The system has been updated in line with the PAD. Revised The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 Completion rate, grade 5, 66.00 66.00 66.00 67.00 67.00 69.00 girls (Percentage) Rationale: Action: This indicator has There is no change to the indicator name, baseline, targets or evidence protocol. The system has been updated in line with the PAD. been Revised Completion rate, Grade 5, 69.00 70.00 70.00 70.00 70.00 71.00 boys (Percentage) Rationale: Action: This indicator has There is no change to the indicator name, baseline, targets or evidence protocol. The system has been updated in line with the PAD. been Revised To improve teaching-learning practices (Action: This Objective has been Revised) Quality score of primary teaching-learning practices 2.53 2.53 2.53 2.53 2.53 2.65 (Number) Rationale: This description of this indicator has been revised and the baseline and targets have been established. The aim of this indicator is to track how teacher Action: This indicator has been practices in the classroom are evolving over time. The indicator measures practical pedagogy under three domains, namely classroom culture, instructional Revised practice and socio-economic support to students, in line with the recently launched 'Teach' classroom observation protocol. Level of School Readiness 0.00 0.00 0.00 0.00 0.00 6.00 (Percentage) Rationale: This description of this indicator has been revised and the baseline and targets have been established. The aim of this indicator is to track the share of Action: This indicator has been children in high quality ECE classrooms, which is a small minority of pre-school environments. The indicator measures the child's readiness for primary school. Revised The quality of ECE is being increased through the implementation of an ECE policy and a new 2 year curriculum bsaed on play. The government monitors the ECE classrooms through an innovative survey tool, done by inspectors who carry tablets to collect live data. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_PDO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 PDO Table SPACE Intermediate Results Indicators by Components RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 Component 1: Improved access, quality and education system management Number of additional children aged 6-16 years to whom a voucher is provided to pay low- 0.00 0.00 10,000.00 130,000.00 270,000.00 420,000.00 cost private school tuition fees (Number) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Number of additional boys aged 6-16 years to whom a voucher is provided to pay 0.00 0.00 5,000.00 65,000.00 135,000.00 210,000.00 low-cost private school tuition fees (Number) Rationale: Action: This indicator has This indicator has not changed from the PAD. This is just an update in the system. been Revised Number of additional girls 0.00 0.00 5,000.00 65,000.00 135,000.00 210,000.00 aged 6-16 years to whom a The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 voucher is provided to pay low-cost private school tuition fees (Number) Rationale: Action: This indicator has This indicator has not changed from the PAD. This is just an update in the system. been Revised Number of additional schools covered by PPP initiatives supported by PEF under a 0.00 225.00 450.00 675.00 900.00 1,125.00 quality assurance system (Number) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Number of additional children enrolled in PPP initiatives supported by PEF under a 0.00 180,000.00 360,000.00 540,000.00 720,000.00 900,000.00 quality assurance system (Number) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Number of additional girls enrolled in PPP initiatives supported by PEF under a 0.00 90,000.00 180,000.00 270,000.00 360,000.00 450,000.00 quality assurance system (Number) The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 Rationale: Action: This indicator has This indicator has not changed from the PAD. This is just an update in the system. been Revised Number of additional boys enrolled in PPP initiatives supported by PEF under a 0.00 90,000.00 180,000.00 270,000.00 360,000.00 450,000.00 quality assurance system (Number) Rationale: Action: This indicator has This indicator has not changed from the PAD. This is just an update in the system. been Revised Number of children aged 3-5 years enrolled in an ECE 0.00 0.00 30,000.00 75,000.00 135,000.00 210,000.00 classroom that meets quality standards (Number) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Number of girls aged 3-5 years enrolled in an ECE 0.00 0.00 15,000.00 37,500.00 67,500.00 105,000.00 classroom that meets quality standards (Number) Rationale: Action: This indicator has This indicator has not changed from the PAD. This is just an update in the system. been Revised Number of boys aged 3-5 0.00 0.00 5,000.00 37,500.00 67,500.00 105,000.00 years enrolled in an ECE The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 classroom that meets quality standards (Number) Rationale: Action: This indicator has This indicator has not changed from the PAD. This is just an update in the system. been Revised Percentage of new teachers recruited based on merit, for all vacant posts reallocated to 100.00 100.00 100.00 100.00 100.00 100.00 schools as per recruitment policy (Percentage) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Number of districts implementing the 0.00 0.00 0.00 0.00 6.00 12.00 strengthened field-based CPD (Number) Rationale: Action: This indicator has been This indicator has been brought in line with the new DLI and the associated timeline. Revised PEC instruments do not use equated items and Grades 5 and 8 PEC (revised) Grades 5 and 8 PEC Grades 5 and 8 PEC PEC reuslts from previous measure higher-order results are analyzed and widely Revised instruments (revised) administered results are analyzed and year are analyzed and cognitive skills, and Instruments revision disseminated in actionable piloted to all public school reported in actionable disseminated in results are not form (Text) children. form. actionable form disseminated in actionable form The World Bank Pakistan: Third Punjab Education Sector Project (P154524) RESULT_FRAME_TBL_IO Indicator Name DLI Baseline Intermediate Targets End Target 1 2 3 4 Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Number of School Council members reached through 80,000.00 80,000.00 96,000.00 128,000.00 212,000.00 212,000.00 citizen engagement initiative(s) (Number) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised Non-salary budget execution 64.00 64.00 64.00 66.00 66.00 69.00 rate (Percentage) Rationale: Action: This indicator has been Baseline data has become available at the mid-term review, and this is reflected here. Revised Component 2: Capacity Building, Project Management, Monitoring, and Evaluation (Action: This Component has been Revised) Number of EDOs-Education that analyze and act on revised district education report card 0.00 3.00 12.00 36.00 36.00 36.00 generated from integrated database (Number) Rationale: Action: This indicator has been This indicator has not changed from the PAD. This is just an update in the system. Revised IO Table SPACE The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Disbursement Linked Indicators Matrix DLI IN00710578 ACTION DLI 1: Private school vouchers Strengthening program design to expand coverage of tuition-replacement vouchers to DLI 1 children from disadvantaged households Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Outcome No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula • Vouchers to attend private schools provided to 322,679 children from disadvantaged households • The program has undergone rapid expansion, Baseline and present design features need to be revisited to ensure more effective beneficiary selection and monitoring Revised EVS system in place; and PEF initiates 2016-17 8,590,000.00 pilot for EVS 10,000 Eligible Children received vouchers to 2017-18 attend private schools; and PEF’s Board of 8,590,000.00 Directors approved the Revised System for EVS. 130,000 Eligible Children received vouchers to 2018-19 8,590,000.00 attend private schools. 270,000 Eligible Children received vouchers to 2019-20 3,220,000.00 attend private schools. 420,000 Eligible Children received vouchers to 2020-21 3,220,000.00 attend private schools. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00710609 ACTION DLI 2 DLI 2. Public-private partnerships in education Leveraging the private sector to support schools to increase enrollment Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Output No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula Currently 1.3 million children are studying in PEF partner schools through the NSP and FAS. Baseline Capacity to enroll additional children through PPPs remains 2016-17 8,590,000.00 2017-18 8,590,000.00 2018-19 8,590,000.00 2019-20 3,220,000.00 2020-21 3,220,000.00 Action: This DLI has been Revised. See below. DLI IN00710930 ACTION DLI 2 DLI 2. Public-private partnerships in education Leveraging the private sector to support schools to increase enrollment Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Output No Text 32,210,000.00 53.34 Period Value Allocated Amount (USD) Formula Currently 1.3 million children are studying in PEF Baseline partner schools through the NSP and FAS. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Capacity to enroll additional children through PPPs remains 180,000 children are enrolled through Public- Private Partnerships under PEF approved 2016-17 8,590,000.00 programs that have a Strengthened Quality Assurance System. 360,000 children are enrolled through Public- Private Partnerships under PEF approved 2017-18 8,590,000.00 programs that have a Strengthened Quality Assurance System. 540,000 children are enrolled through Public- Private Partnerships under PEF approved 2018-19 8,590,000.00 programs that have a Strengthened Quality Assurance System. 720,000 children are enrolled through Public Private Partnerships under PEF approved 2019-20 3,220,000.00 programs that have a Strengthened Quality Assurance System. 900,000 children are enrolled through Public Private Partnerships under PEF approved 2020-21 3,220,000.00 programs that have a Strengthened Quality Assurance System. Rationale: There is no change to this DLI. Due to system issue the values are missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00710612 ACTION DLI 3. Stipends for secondary school girls Stipends for secondary school girls to increase secondary school DLI 3 participation/retention Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Output No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula Stipends are offered to girls enrolled in secondary grades in 16 districts under the original program. Baseline The stipend amount and conditions have remained unchanged since program inception. SED has strengthened its original program and is 2016-17 8,590,000.00 implementing it in 16 Districts. SED continues to implement the strengthened 2017-18 8,590,000.00 program in 16 Districts SED continues to implement the strengthened 2018-19 8,590,000.00 program in 16 Districts SED continues to implement the strengthened 2019-20 3,220,000.00 program in 16 Districts SED continues to implement the strengthened 2020-21 3,220,000.00 program in 16 Districts Action: This DLI has been Revised. See below. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00713538 ACTION DLI 3. Stipends for secondary school girls Stipends for secondary school girls to increase secondary school DLI 3 participation/retention Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Output No Text 32,210,000.00 53.34 Period Value Allocated Amount (USD) Formula Stipends are offered to girls enrolled in secondary grades in 16 districts under the original program. Baseline The stipend amount and conditions have remained unchanged since program inception. SED has strengthened its original program and is 2016-17 8,590,000.00 implementing it in 16 Districts. SED continues to implement the strengthened 2017-18 8,590,000.00 program in 16 Districts SED continues to implement the strengthened 2018-19 8,590,000.00 program in 16 Districts SED continues to implement the strengthened 2019-20 3,220,000.00 program in 16 Districts SED continues to implement the strengthened 2020-21 3,220,000.00 program in 16 Districts Rationale: There is no change to this DLI. Due to system issue the values are missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00710614 ACTION DLI 4 DLI 4: Early childhood education Improving quality and expanding access to early childhood education Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Outcome No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula ECE is currently being implemented at a small Baseline scale SED has started implementation of its 2016-17 recruitment policy for Merit-Based Recruitment 8,590,000.00 of new teachers and reallocation of posts SED continues to implement its recruitment policy for Merit-Based Recruitment of new teachers and reallocation of posts. Third-party validation of the 2017-18 8,590,000.00 SED’s recruitment policy for Merit Based Recruitment of new teachers and reallocation of posts completed SED continues to implement its recruitment policy 2018-19 for Merit-Based Recruitment of new teachers and 8,590,000.00 reallocation of posts. SED continues to implement its recruitment policy 2019-20 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts. SED continues to implement its recruitment policy 2020-21 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts Action: This DLI has been Revised. See below. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00713539 ACTION DLI 4 DLI 4: Early childhood education Improving quality and expanding access to early childhood education Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Outcome No Text 32,210,000.00 53.34 Period Value Allocated Amount (USD) Formula ECE is currently being implemented at a small Baseline scale SED has started implementation of its recruitment 2016-17 policy for Merit-Based Recruitment of new 8,590,000.00 teachers and reallocation of posts SED continues to implement its recruitment policy for Merit-Based Recruitment of new teachers and reallocation of posts. Third-party validation of the 2017-18 8,590,000.00 SED’s recruitment policy for Merit Based Recruitment of new teachers and reallocation of posts completed SED continues to implement its recruitment policy 2018-19 for Merit-Based Recruitment of new teachers and 8,590,000.00 reallocation of posts. SED continues to implement its recruitment policy 2019-20 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts. SED continues to implement its recruitment policy 2020-21 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts Rationale: There is no change to this DLI, except that the verification protocol now mentions AEOs instead of DTEs, in line with the change in DLI 6. Due to system issue the values are also missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00710616 ACTION DLI 5: Human Resources Management Strengthening HR processes through continuous merit-based recruitment and DLI 5 rationalization of teaching posts Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Process No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula Merit-based recruitment of new teachers for all Baseline vacant posts reallocated to schools based on existing recruitment policy SED has started implementation of its 2016-17 recruitment policy for Merit-Based Recruitment 8,590,000.00 of new teachers and reallocation of posts. SED continues to implement its recruitment policy for Merit-Based Recruitment of new teachers and reallocation of posts. Third-party validation of the 2017-18 8,590,000.00 SED’s recruitment policy for Merit Based Recruitment of new teachers and reallocation of posts completed. SED continues to implement its recruitment policy 2018-19 for Merit-Based Recruitment of new teachers and 8,590,000.00 reallocation of posts. SED continues to implement its recruitment policy 2019-20 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts. SED continues to implement its recruitment policy 2020-21 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Action: This DLI has been Revised. See below. DLI IN00713540 ACTION DLI 5: Human Resources Management Strengthening HR processes through continuous merit-based recruitment and DLI 5 rationalization of teaching posts Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Process No Text 32,210,000.00 53.34 Period Value Allocated Amount (USD) Formula Merit-based recruitment of new teachers for all Baseline vacant posts reallocated to schools based on existing recruitment policy SED has started implementation of its recruitment 2016-17 policy for Merit-Based Recruitment of new 8,590,000.00 teachers and reallocation of posts. SED continues to implement its recruitment policy for Merit-Based Recruitment of new teachers and reallocation of posts. Third-party validation of the 2017-18 8,590,000.00 SED’s recruitment policy for Merit Based Recruitment of new teachers and reallocation of posts completed. SED continues to implement its recruitment policy 2018-19 for Merit-Based Recruitment of new teachers and 8,590,000.00 reallocation of posts. SED continues to implement its recruitment policy 2019-20 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts. SED continues to implement its recruitment policy 2020-21 for Merit-Based Recruitment of new teachers and 3,220,000.00 reallocation of posts. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Rationale: There is no change to this DLI. Due to system issue the values are missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. DLI IN00710618 ACTION DLI 6 DLI 6. Quality in the primary classroom Strengthening field- based CPD for improved teacher performance Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Outcome No Text 32,120,000.00 0.00 Period Value Allocated Amount (USD) Formula Field-based system of CPD has mixed operational Baseline capacity and performance, and poor design match with multigrade and large class teaching. DSD has designed: (a) the elements of the New Mainstream Phase of integrated CPD program 2016-17 and piloted its key components; and (b) an 8,500,000.00 innovative teachers support package and tested its components At least 85% of primary school teachers in 4 Districts covered by the New Mainstream Phase 2017-18 of integrated CPD program; and DSD is 8,590,000.00 implementing the validated innovative teachers support package in at least 100 schools At least 85% of primary school teachers in 8 Districts covered by New Mainstream Phase of 2018-19 integrated CPD program; and DSD continues to 8,590,000.00 implement the validated innovative teachers support package in at least 100 schools At least 85% of primary school teachers in 12 2019-20 3,220,000.00 Districts covered by New Mainstream Phase of The World Bank Pakistan: Third Punjab Education Sector Project (P154524) integrated CPD program; DSD has analyzed and disseminated: (a) the results of the integrated CPD program evaluation and integrated its findings into the CPD system; and (b) the results of the innovative teachers support package evaluation. At least 85% of primary school teachers in 18 2020-21 Districts covered by New Mainstream Phase of 3,220,000.00 integrated CPD program Action: This DLI has been Revised. See below. DLI IN00710550 ACTION DLI 6 DLI 6. Quality in the primary classroom Strengthening field- based CPD for improved teacher performance Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Outcome No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula Field-based system of CPD has mixed operational Baseline capacity and performance, and poor design match with multigrade and large class teaching. 2016-17 0.00 2017-18 0.00 Innovative support package is designed; AEOs task description is revised and AEOs start 2018-19 8,590,000.00 conducting classroom observations through android app. Training of Primary School Teachers (as per 2019-20 revamped CPD design) conducted in 06 Districts 11,810,000.00 Innovative support package is developed, The World Bank Pakistan: Third Punjab Education Sector Project (P154524) validated and pilot initiated in 100 schools; QAED has trained AEOs (or equivalent) on classroom observation and feedback provision. Training of Primary School Teachers (as per revamped CPD design) conducted in 12 Districts. 2020-21 QAED has incorporated successful elements of 11,810,000.00 innovative support package into the regular CPD program; Rationale: The goal of this change is to work with the new institutional structure of Continuous Professional Development and to aim for an improvement in teacher practices. This change is described in detail in the restructuring paper. DLI IN00710619 ACTION DLI 7 DLI 7. Student Assessment PEC and its instruments strengthened within a revised policy framework Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Intermediate Outcome No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula No assessment policy framework; outdated PEC Baseline ISP that has been partially implemented and is in need of revision. 2016-17 8,590,000.00 2017-18 8,590,000.00 2018-19 8,590,000.00 2019-20 3,220,000.00 2020-21 3,220,000.00 The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Action: This DLI has been Revised. See below. DLI IN00711076 ACTION DLI 7 DLI 7. Student Assessment PEC and its instruments strengthened within a revised policy framework Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Intermediate Outcome No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula No assessment policy framework; outdated PEC Baseline ISP that has been partially implemented and is in need of revision. PEC has developed the its assessment policy framework; and The new PEC Institutional 2016-17 8,590,000.00 Strengthening Plan has been approved by the commission overseeing PEC. 80% of ISP targets for 2017-18 are met; and 2017-18 strengthened Grade 5 + 8 PEC instruments are 8,590,000.00 piloted in at least one subject 80% of ISP targets for 2018-19 are met; and strengthened Grade 5 + 8 PEC instruments are 2018-19 8,590,000.00 administered throughout the entire Punjab school system PEC has analyzed and reported Grade 5+8 PEC 2019-20 results from the previous year of Project 3,220,000.00 implementation in Actionable Form. PEC has analyzed and reported Grade 5 + 8 PEC 2020-21 results for the previous year of Project 3,220,000.00 implementation in Actionable Form. Rationale: The World Bank Pakistan: Third Punjab Education Sector Project (P154524) There is no change to this DLI. Due to system issue the values are missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. DLI IN00710620 ACTION DLI 8 DLI 8. School specific NSB Setting and executing school budgets in line with school resource needs Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Output No Text 32,210,000.00 0.00 Period Value Allocated Amount (USD) Formula School-specific NSBs for FY2015/16 prepared in accordance with agreed funding formula, and Baseline submitted for inclusion in FY2016/17 district budgets in 36 districts for all schools, including high/high secondary. SED has prepared School-Specific Non-Salary Budgets for FY2016/17 in accordance with agreed funding formula, and submitted for inclusion in FY2017/18 District budgets in 36 Districts for all schools, including high/high secondary; and 2016-17 Punjab’s Finance Department has dispersed 8,590,000.00 School Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and SCs reconstituted in required elementary/primary schools and constituted in all high schools SED has prepared School-Specific Non-Salary Budgets for FY2017/18 in accordance with agreed funding formula, and submitted for inclusion in 2017-18 8,590,000.00 FY2018/19 District budgets in 36 Districts for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed The World Bank Pakistan: Third Punjab Education Sector Project (P154524) School Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. SED has prepared School-Specific Non-Salary Budgets for FY2018/19 in accordance with agreed funding formula, and submitted for inclusion in FY2019/20 District budgets in 36 Districts for all schools, including high/high secondary; and 2018-19 8,590,000.00 Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and TPV of Year 1 and Year 2 of this DLI completed. SED has prepared School-Specific Non-Salary Budgets for FY2019/20 in accordance with agreed funding formula, and submitted for inclusion in FY2020/21 District budgets in 36 District for all 2019-20 schools, including high/high secondary; and 3,220,000.00 Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. SED has prepared School-Specific Non-Salary Budgets for FY2020/21 in accordance with agreed funding formula, and submitted for inclusion in FY2020/21 District budgets in 36 Districts for all 2020-21 schools, including high/high secondary; and 3,220,000.00 Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. Action: This DLI has been Revised. See below. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00713494 ACTION DLI 8 DLI 8. School specific NSB Setting and executing school budgets in line with school resource needs Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Output No Text 32,210,000.00 26.67 Period Value Allocated Amount (USD) Formula School-specific NSBs for FY2015/16 prepared in accordance with agreed funding formula, and Baseline submitted for inclusion in FY2016/17 district budgets in 36 districts for all schools, including high/high secondary. SED has prepared School-Specific Non-Salary Budgets for FY2016/17 in accordance with agreed funding formula, and submitted for inclusion in FY2017/18 District budgets in 36 Districts for all schools, including high/high secondary; and 2016-17 Punjab’s Finance Department has dispersed 8,590,000.00 School Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and SCs reconstituted in required elementary/primary schools and constituted in all high schools SED has prepared School-Specific Non-Salary Budgets for FY2017/18 in accordance with agreed funding formula, and submitted for inclusion in FY2018/19 District budgets in 36 Districts for all 2017-18 schools, including high/high secondary; and 8,590,000.00 Punjab’s Finance Department has dispersed School Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) SED has prepared School-Specific Non-Salary Budgets for FY2018/19 in accordance with agreed funding formula, and submitted for inclusion in FY2019/20 District budgets in 36 Districts for all schools, including high/high secondary; and 2018-19 8,590,000.00 Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and TPV of Year 1 and Year 2 of this DLI completed. SED has prepared School-Specific Non-Salary Budgets for FY2019/20 in accordance with agreed funding formula, and submitted for inclusion in FY2020/21 District budgets in 36 District for all 2019-20 schools, including high/high secondary; and 3,220,000.00 Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. SED has prepared School-Specific Non-Salary Budgets for FY2020/21 in accordance with agreed funding formula, and submitted for inclusion in FY2020/21 District budgets in 36 Districts for all 2020-21 schools, including high/high secondary; and 3,220,000.00 Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. Rationale: There is no change to this DLI. Due to system issue the values are missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI IN00710622 ACTION DLI 9. Data strengthening and performance management Strengthening management capacity and evidence-based DLI 9 decision-making Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Intermediate Outcome No Text 32,212,000.00 0.00 Period Value Allocated Amount (USD) Formula Last private school census was in 2010-11. Various education sector data are not integrated, Baseline utilization of data remains low and restricted to source departments. 2016-17 8,590,000.00 2017-18 8,590,000.00 2018-19 8,590,000.00 2019-20 3,222,000.00 2020-21 3,220,000.00 Action: This DLI has been Revised. See below. DLI IN00711077 ACTION DLI 9. Data strengthening and performance management Strengthening management capacity and evidence-based DLI 9 decision-making Type of DLI Scalability Unit of Measure Total Allocated Amount (USD) As % of Total Financing Amount Intermediate Outcome No Text 32,212,000.00 53.33 Period Value Allocated Amount (USD) Formula The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Last private school census was in 2010-11. Various education sector data are not integrated, Baseline utilization of data remains low and restricted to source departments. SED has conducted a private school census; and PMIU has prepared and delivered on a semi- 2016-17 annual basis District performance report cards for 8,590,000.00 Year 1 following agreed format and contents, to District Education Administrations. PMIU has integrated Education sector data (from PEC, PMIU, DSD and PEF); and PMIU provided access and orientation to the integrated database to education departments and District education 2017-18 officials in 3 Districts; PMIU has prepared and 8,590,000.00 delivered on a semiannual basis District performance report cards for Year 2 following agreed format and contents, to District Education Administrations PMIU has integrated Education sector data (from PEC, PMIU,DSD, and PEF); and PMIU has provided access and orientation to the integrated database to education departments and District education 2018-19 officials in 12 Districts; PMIU has prepared and 8,590,000.00 delivered on a semi-annual basis District performance report cards for Year 3 following agreed format and contents, to District Education Administrations. PMIU has integrated Education sector data (from PEC, PMIU,DSD, and PEF) ; and PMIU has provided 2019-20 access and orientationto the integrated database 3,222,000.00 to education departments and District education officials in 36 Districts; PMIU has prepared and The World Bank Pakistan: Third Punjab Education Sector Project (P154524) delivered on a semi-annual basis District performance report cards for Year 4 following agreed format and contents,to District Education Administrations PMIU has integrated Education sector data (from PEC, PMIU, DSD, and PEF) ; and PMIU has provided access and orientation to the integrated database to education departments and District 2020-21 education officials in 36 Districts; PMIU has 3,220,000.00 prepared and delivered on a semiannual basis District performance report cards for Year 5 following agreed format and contents, to District Education Administrations Rationale: There is no change to this DLI. Due to system issue the values are missing. Hence, this DLI has been reentered below to bring alignment between the PAD and the system. Verification Protocol Table: Disbursement Linked Indicators DLI_TBL_VERIFICATION DLI 1: Private school vouchers Strengthening program design to expand coverage of tuition-replacement vouchers to DLI 1 children from disadvantaged households Definitions  All targets are cumulative and are in addition to the number of children attending the EVS schools as captured in the baseline. In the LPDs and included in the targets for DLIs 1 and 2 (under the EVS and PPP Programs), the cumulative number of children who were previously out of school will be not less than 150,000 by year 5 Description  Pilot initiated is defined as vouchers distributed to households under the revised system  The revised system includes revised beneficiary targeting and validation system, and improved monitoring system includes quarterly monitoring of partner schools  “Eligible Children” refers to those children who would be able to participate in the EVS as defined by the PEF Board of Directors’ criteria for eligibility. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) PEF Data source/ Agency PMIU Verification Entity Evidence  (a) Database of registered voucher recipients (with unique identifiers) (b) database of partner schools (with unique identifiers) (c) expenditures under the vouchers program, in agreed format Procedure  QAT and monitoring database of schools and beneficiaries with unique identifiers  Approval by the PEF’s board of directors will be evidenced with official board minutes. Evidence provided by the PEF to the PMIU DLI_TBL_VERIFICATION DLI 2 DLI 2. Public-private partnerships in education Leveraging the private sector to support schools to increase enrollment Definitions  All targets are cumulative, and are over and above the 1.3 million baseline. In the LPDs and included in the targets for DLIs 1 and 2 (under the EVS and PPP programs), the cumulative number of children who were previously out of school will be not less than 150,000 by year 5.  PEF-approved programs to include the FAS, NSP, and any newly developed program including the Public School Support Program that is approved by the PEF’s Board and is acceptable to the Bank. Description  PPPs will be between the PEF and the private sector including private school owners, NGOs, civil society organizations, and educational institutions.  Strengthened quality assurance system includes (but is not necessarily limited to) CPD support to teachers; improved QATs; and enhanced monitoring of partner schools to include quarterly monitoring.  Minimum performance standards are those that are contractually agreed to between the PEF and the partner school, and are acceptable to the Bank. TPV, PEF Data source/ Agency Database of beneficiary students, database of partner schools, periodic monitoring reports, TPV reports, and QAT results. Verification Entity The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Approval of a program by the PEF will be evidenced by official minutes of the PEF Board meetings. Evidence provided by the Procedure PEF to the PMIU DLI_TBL_VERIFICATION DLI 2 DLI 2. Public-private partnerships in education Leveraging the private sector to support schools to increase enrollment Definitions  All targets are cumulative, and are over and above the 1.3 million baseline. In the LPDs and included in the targets for DLIs 1 and 2 (under the EVS and PPP programs), the cumulative number of children who were previously out of school will be not less than 150,000 by year 5.  PEF-approved programs to include the FAS, NSP, and any newly developed program including the Public School Support Program that is approved by the PEF’s Board and is acceptable to the Bank. Description  PPPs will be between the PEF and the private sector including private school owners, NGOs, civil society organizations, and educational institutions.  Strengthened quality assurance system includes (but is not necessarily limited to) CPD support to teachers; improved QATs; and enhanced monitoring of partner schools to include quarterly monitoring.  Minimum performance standards are those that are contractually agreed to between the PEF and the partner school, and are acceptable to the Bank. TPV, PEF Data source/ Agency Database of beneficiary students, database of partner schools, periodic monitoring reports, TPV reports, and QAT results. Verification Entity Approval of a program by the PEF will be evidenced by official minutes of the PEF Board meetings. Evidence provided by the Procedure PEF to the PMIU DLI_TBL_VERIFICATION DLI 3. Stipends for secondary school girls Stipends for secondary school girls to increase secondary school DLI 3 participation/retention Definitions  Strengthened program includes additional repetition condition (whereby a student becomes ineligible during a Description repeated grade), biannual stipend payments, and more efficient payment mechanisms.  Ongoing Stipend Program includes the 16 districts where girls enrolled in secondary school have been receiving stipends since 2004. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) PMIU Data source/ Agency Verification Entity Database of beneficiaries in agreed format Biannual program implementation status report with beneficiary identification, confirmation, and disbursement data, in Procedure agreed format DLI_TBL_VERIFICATION DLI 3. Stipends for secondary school girls Stipends for secondary school girls to increase secondary school DLI 3 participation/retention Definitions  Strengthened program includes additional repetition condition (whereby a student becomes ineligible during a repeated grade), biannual stipend payments, and more efficient payment mechanisms. Description  Ongoing Stipend Program includes the 16 districts where girls enrolled in secondary school have been receiving stipends since 2004. PMIU and PSPA Administrative Records Data source/ Agency PMIU Verification Entity Database of beneficiaries in agreed format Biannual program implementation status report with beneficiary identification, confirmation, and disbursement data, in Procedure agreed format DLI_TBL_VERIFICATION DLI 4 DLI 4: Early childhood education Improving quality and expanding access to early childhood education Definitions  All targets are cumulative Description  ECE Policy Framework includes institutional unit and budget planning.  An ECE teacher is defined as a primary schoolteacher who has received the DSD ECE teacher training. The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  An ECE caregiver is defined as an individual recruited with revised Terms of Reference (ToR) and who has received the DSD ECE caregiver training.  ‘Revised’ ToRs mean that the ToRs have been updated and reviewed by the DSD, SED, and PMIU after project effectiveness. ECE Quality Standards for a Classroom and its School  Staffing. One ECE teacher and one ECE caregiver  Training. The following persons have received in-service training on ECE through the DSD: ECE teacher, ECE caregiver, head teacher and 2 School Management Council members, of whom one must be a woman  Content. ECE curriculum and ECE activity guide in use  Materials. ECE toolkit  Monitoring for improvement. DTE (or equivalent) mandate revised to include ECE classrooms. ECE classrooms are monitored once a month by the DTEs (or equivalent) who have received ECE training and provide feedback for improvement to ECE teacher and ECE caregivers.  Parental outreach. Parents have received parent activity guide and follow-up; quarterly parent meetings held at schools. Evidence to be provided by the PMIU with DSD inputs.  The ECE curriculum and activity guide are judged to be in use (or not) by the DTE (or equivalent) during the on-site support visit. Pilot. District health/nutrition officers receive training in early childhood development to provide information on health and nutrition to ECE classes and identify malnourished children to refer to district health centers.  Random sample of reports from the DTEs on classrooms meeting quality standards.  Approved ECE Policy Framework with quality standards signed by relevant authority (Secretary Schools Education Department) Data source/ Agency  Database of schools with ECE classrooms that meet quality standards, in an agreed format  TPV report in an agreed format TPV Verification Entity Procedure The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI_TBL_VERIFICATION DLI 4 DLI 4: Early childhood education Improving quality and expanding access to early childhood education Definitions  All targets are cumulative  ECE Policy Framework includes institutional unit and budget planning.  An ECE teacher is defined as a primary schoolteacher who has received the DSD ECE teacher training.  An ECE caregiver is defined as an individual recruited with revised Terms of Reference (ToR) and who has received the DSD ECE caregiver training.  ‘Revised’ ToRs mean that the ToRs have been updated and reviewed by the DSD, SED, and PMIU after project effectiveness. ECE Quality Standards for a Classroom and its School  Staffing. One ECE teacher and one ECE caregiver  Training. The following persons have received in-service training on ECE through the QAED: ECE teacher, ECE caregiver, head teacher and 2 School Management Council members, of whom one must be a woman Description  Content. ECE curriculum and ECE activity guide in use  Materials. ECE toolkit  Monitoring for improvement. AEOs (or equivalent) mandate revised to include ECE classrooms. ECE classrooms are monitored once a month by the AEOs(or equivalent) who have received ECE training and provide feedback for improvement to ECE teacher and ECE caregivers.  Parental outreach. Parents have received parent activity guide and follow-up; quarterly parent meetings held at schools. Evidence to be provided by the PMIU with QAED inputs.  The ECE curriculum and activity guide are judged to be in use (or not) by the AEO (or equivalent) during the on-site support visit. Pilot. District health/nutrition officers receive training in early childhood development to provide information on health and nutrition to ECE classes and identify malnourished children to refer to district health centers. o Random sample of reports from the AEOs on classrooms meeting quality standards. o Approved ECE Policy Framework with quality standards signed by relevant authority (Secretary Schools Education Data source/ Agency Department) o Database of schools with ECE classrooms that meet quality standards, in an agreed format o TPV report in an agreed format The World Bank Pakistan: Third Punjab Education Sector Project (P154524) PMIU Verification Entity AEOs (or equivalent) observe the classrooms, and report back to the SED/PMIU about their observations These data are Procedure aggregated, analysed and validated by the PMIU. DLI_TBL_VERIFICATION DLI 5: Human Resources Management Strengthening HR processes through continuous merit-based recruitment and DLI 5 rationalization of teaching posts  Merit-based recruitment means hiring of 100 percent newly recruited teachers under initial recruitment quota through competitive tests carried out by a third party as under PESRP II. Vacant posts will be identified by district officials based on the recruitment policy.  Initial recruitment quota is identified as vacant posts, which will be filled by hiring of new teachers and not by promotions or transfers of existing teachers. Description  TPV conducted in 2017–18 will review implementation of recruitment policy with particular focus on merit-based recruitment and reallocation of teaching posts. The TPV review of test-based recruitment will identify potential improvements in test content, design, administration, and use of results. Review of recruitment and reallocation policy will identify the ratio of success in implementation and propose changes to address challenges. TPV Data source/ Agency Third party Verification Entity Evidence Approved recruitment policy, teacher-specific test results, and placement information in agreed format. In 2017–18, the Procedure SED will submit a revised recruitment and reallocation mechanism acceptable to the Bank. DLI_TBL_VERIFICATION DLI 5: Human Resources Management Strengthening HR processes through continuous merit-based recruitment and DLI 5 rationalization of teaching posts  Merit-based recruitment means hiring of 100 percent newly recruited teachers under initial recruitment quota Description through competitive tests carried out by a third party as under PESRP II. Vacant posts will be identified by district officials based on the recruitment policy. The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  Initial recruitment quota is identified as vacant posts, which will be filled by hiring of new teachers and not by promotions or transfers of existing teachers.  TPV conducted in 2017–18, 2018-19 and 2019-20 will review implementation of recruitment policy with particular focus on merit-based recruitment and reallocation of teaching posts. The TPV review of test-based recruitment will identify potential improvements in test content, design, administration, and use of results. Review of recruitment and reallocation policy will identify the ratio of success in implementation and propose changes to address challenges. Approved recruitment policy, teacher-specific test results, and placement information in agreed format. In 2017–18, the Data source/ Agency SED will submit a revised recruitment and reallocation mechanism acceptable to the Bank. TPVs in year 2, 3 and 4. Verification Entity NTS will administer tests for teachers and share the results with the SED. PMIU will submit test data to the Bank after each year where recruitment is complete. Actual recruitment and transfer data are shared by district level CEOs and DMOs. These Procedure are then aggregated, analysed and validated by the PMIU. DLI_TBL_VERIFICATION DLI 6 DLI 6. Quality in the primary classroom Strengthening field- based CPD for improved teacher performance  The new mainstream phase strengthens the existing CPD system for primary teachers, and focuses on literacy, numeracy, multigrade, and large class teaching. It comprises o support to teachers and head teachers on the effective utilization of instructional resources by teachers and students during lessons, as well as the use of formative assessment practices by teachers during lessons to identify and remedy student learning gaps. The support will include in- service training for teachers and head teachers; and data-based supportive feedback to teachers and head teachers provided by the DTEs Description using revised observation and learning assessment instruments. The in-service training and observation instrument will be adapted for multigrade teachers; o logistical allowances for the DTEs and resources for conducting schools visits, PD days, and training weeks; o filling all the DTE (or equivalent) posts on revised merit basis for all clusters based on competency levels; and o use of teacher standards to guide training. The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  The selection criteria for the districts are a majority of the project’s LPDs are covered; CPD - Phase 1 districts are included; the teachers perform relatively well with regard to their mentoring score, including on the activity-based dimension (using the DSD’s measurement); and the students have relatively low levels of learning achievement (based on the DSD’s measurement).  The innovative package comprises o curriculum sequenced by learning steps for Grades 1–3 literacy and numeracy, with learning, assessment and supplementary learning activities for each step; o allocation of adequate instructional resources including teacher guides, learning and assessment activity materials for each step, individual child progress monitoring cards, student workbooks, notebooks, supplementary reading books, and literacy and numeracy materials; and o support to teachers and head teachers on the effective utilization of instructional resources and the sequenced learning steps by teachers and students during lessons, as well as the use of formative assessment practices by teachers during lessons to identify and remedy student learning gaps. The support will include in-service training for teachers and head teachers; and data-based supportive feedback to teachers and head teachers provided by the DTEs using adapted observation and revised learning assessment instruments. Validation means that designers (whether of the new mainstream CPD phase or the innovative package) have reviewed the performance of the phase/package in school settings using the monitoring and feedback mechanism set up as part of the design process; have redesigned or modified elements of the package accordingly; and are satisfied that the phase/package is sufficiently well-designed to proceed with further expansion to more schools. Evaluations are undertaken by a third party. They should be designed during year 1, initiated in year 3, and completed in year 4. The evaluation results must be analyzed and the findings disseminated to stakeholders and discussed. The DSD will modify the mainstream phase or innovative package as needed, in light of the evaluation findings. Evidence to be provided by the PMIU with DSD inputs Data source/ Agency Verification Entity Procedure The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI_TBL_VERIFICATION DLI 6 DLI 6. Quality in the primary classroom Strengthening field- based CPD for improved teacher performance  The new CPD approach taken by QAED strengthens the existing CPD system for primary teachers, and focuses on improving teaching practices, while also improving literacy, numeracy, and large class (incl. if relevant, multi-grade) teaching. o This includes a teacher support package on basic literacy and numeracy covering Grades 1-3, curriculum sequencing of learning steps for Grades 1-3 literacy and numeracy, with learning, assessment and supplementary learning activities for each step. o This also includes a focus on strengthened pedagogic practice for teachers, use of formative assessments, and in-service training for teachers and head-teachers or principals, aligned to the broad agenda of government and stakeholders. This also includes adequate instructional resources including teacher guides, learning and assessment activity materials for each step, individual child progress monitoring cards, student workbooks, notebooks, supplementary reading books, and literacy and numeracy materials;  The main focus of the DLI is the development and piloting of an innovative support package focusing on teacher pedagogic practice, which includes the use of videos in order to show good practice and evaluate practice, and is aimed to improve teaching quality in the classroom in areas such as supportive learning environment, setting positive behavioral expectations, providing feedback, checking for understanding (formative assessment), lesson Description facilitation, and critical thinking.  The approach also includes modules (including an app) to support AEOs (or equivalent), headteachers, CPD experts and teachers on teacher observation and feedback provision with the aim of providing data-driven, actionable and constructive feedback to teachers, linked to standards for teaching.  A revised task description for AEOs (or equivalent) will be developed in order to refocus the system on professional development and support, rather than control, while allowing more time for classroom observation, and the development of a professional mentoring relationship between the AEO and the schools. AEOs and headteachers will also be trained and certified on how to do classroom observation and how to provide feedback.  The innovative package will be validated, which means that the innovative package has been reviewed by relevant stakeholders, has been piloted and evaluated, ideally in the setting of a randomized controlled trial that measures its effect on classroom practices and learning outcomes.  The successful elements of the innovative support package pilot will be identified by QAED and stakeholders, and will be selected for scale up in the new mainstream CPD system. Evidence: The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  Innovative support package published on QAED’s website; AEO training materials and certification results published on QAED website; Pilot report; evidence provided by QAED for training of primary school teachers; PMIU (AEO data on classroom observations, and MEA verification of pilot exercise). Evidence to be provided by the PMIU with QAED inputs Data source/ Agency PMIU MEA monitoring system and SED's AEO system. Verification Entity School inspectors (MEAs) visit schools regularly to check whether actions are being implemented. School mentors (AEOs) Procedure will provide classroom observations and feedback to teachers. DLI_TBL_VERIFICATION DLI 7 DLI 7. Student Assessment PEC and its instruments strengthened within a revised policy framework Definitions, Evidence, and Protocols  The assessment policy framework will be developed based upon a review of extant policy, and the roles, mandates and relative strengths and weaknesses of PEC and other institutions involved in assessment. It will be approved by the Commission overseeing PEC.  The ISP will be based upon a functional review of PEC that examines PEC’s mission within the context of the new assessment policy framework, and will have schedules covering at least the following areas of institutional strengthening: (i) clarification of PEC’s mission; (ii) update of PEC functions, activities and operating procedures; (iii) staffing levels and profiles, and recruitment and training needs; (iv) equipment procurement; (v) budgetary Description requirements, and (vi) capacity building to enhance item/test quality and scoring, and to analyze, report and use data.  The reporting of data in actionable form means that the data are analyzed, tailored and disseminated to relevant stakeholders (teachers/trainers, curriculum and textbook developers, and decision makers) in a form that is useful to them in deciding what actions can be taken to improve the quality of education. This will require doing an upfront focus group with each stakeholder group to get a sense of their data needs. It also will require holding a dissemination workshop with each stakeholder group once the report has been prepared to review the data and actionable recommendations with them. PMIU with PEC input. Data source/ Agency The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Verification Entity Procedure DLI_TBL_VERIFICATION DLI 7 DLI 7. Student Assessment PEC and its instruments strengthened within a revised policy framework Definitions, Evidence, and Protocols  The assessment policy framework will be developed based upon a review of extant policy, and the roles, mandates and relative strengths and weaknesses of PEC and other institutions involved in assessment. It will be approved by the Commission overseeing PEC.  The ISP will be based upon a functional review of PEC that examines PEC’s mission within the context of the new assessment policy framework, and will have schedules covering at least the following areas of institutional strengthening: (i) clarification of PEC’s mission; (ii) update of PEC functions, activities and operating procedures; (iii) staffing levels and profiles, and recruitment and training needs; (iv) equipment procurement; (v) budgetary Description requirements, and (vi) capacity building to enhance item/test quality and scoring, and to analyze, report and use data.  The reporting of data in actionable form means that the data are analyzed, tailored and disseminated to relevant stakeholders (teachers/trainers, curriculum and textbook developers, and decision makers) in a form that is useful to them in deciding what actions can be taken to improve the quality of education. This will require doing an upfront focus group with each stakeholder group to get a sense of their data needs. It also will require holding a dissemination workshop with each stakeholder group once the report has been prepared to review the data and actionable recommendations with them. PEC Data source/ Agency PMIU Verification Entity PMIU will verify PEC data. Procedure DLI_TBL_VERIFICATION DLI 8 DLI 8. School specific NSB Setting and executing school budgets in line with school resource needs Description Definitions and Evidence The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  ‘Necessary support structure’ for execution of the NSB component entails formulation of a steering committee with representation from all stakeholders and the required capacity-building program to engage/mobilize SCs and school administrations. The required capacity-building program for school will be a comprehensive mobilization strategy agreed by all stakeholders. It will also include necessary material aid to SCs, such as SC guidelines. The first year activities will be focused on establishing protocols and strategies, which will be implemented in following years. For high and higher secondary schools, Project to Improve Financial Reporting and Auditing (PIFRA) system will be used to identify disbursed money against Drawing & Disbursing Officer (DDO) codes of respective schools in the first 2 years of the program, while the SCs are being formulated in the schools. In the third year of the program, the project will work to ensure that disbursement of the NSB will be through SCs in high or higher secondary schools, based on consultations and subject to agreement with the FD of Punjab Province and the Accountant General’s Office.  SCs will be reconstituted in primary and elementary schools, which were not part of the capacity-building program during PESRP II. Simultaneously, SC will be formalized in high or higher secondary schools with required guidelines and materials disseminated. Data source/ Agency Third Party Verification Entity The TPV to be carried out in the third year will be against the ToRs as agreed by all stakeholders. The TPV will seek to review the impact) of the NSB component along with the capacity-building program, analyzing the efficacy of the in-place capacity- building program.  Verification protocol. Verification of disbursements of funds to schools will be sought from certificates submitted Procedure by the EDO (or equivalent) office for release of cheques to all primary and elementary schools.  For reconstitution and constitution of SCs, certificates will be sought from district EDO (or equivalent) offices highlighting the detail of SCs formulated according to policy. DLI_TBL_VERIFICATION DLI 8 DLI 8. School specific NSB Setting and executing school budgets in line with school resource needs Description Definitions and Evidence The World Bank Pakistan: Third Punjab Education Sector Project (P154524) 1. ‘Necessary support structure’ for execution of the NSB component entails formulation of a steering committee with representation from all stakeholders and the required capacity-building program to engage/mobilize SCs and school administrations. The required capacity-building program for school will be a comprehensive mobilization strategy agreed by all stakeholders. It will also include necessary material aid to SCs, such as SC guidelines. The first year activities will be focused on establishing protocols and strategies, which will be implemented in following years. For high and higher secondary schools, Project to Improve Financial Reporting and Auditing (PIFRA) system will be used to identify disbursed money against Drawing & Disbursing Officer (DDO) codes of respective schools in the first 2 years of the program, while the SCs are being formulated in the schools. In the third year of the program, the project will work to ensure that disbursement of the NSB will be through SCs in high or higher secondary schools, based on consultations and subject to agreement with the FD of Punjab Province and the Accountant General’s Office. 2. SCs will be reconstituted in primary and elementary schools, which were not part of the capacity-building program during PESRP II. Simultaneously, SC will be formalized in high or higher secondary schools with required guidelines and materials disseminated. PMIU/PIFRA Data source/ Agency Third Party Verification Entity The TPV to be carried out in the third year will be against the ToRs as agreed by all stakeholders. The TPV will seek to review the impact) of the NSB component along with the capacity-building program, analyzing the efficacy of the in-place capacity- building program. Verification protocol. Verification of disbursements of funds to schools will be sought from certificates submitted by the Procedure EDO (or equivalent) office for release of cheques to all primary and elementary schools. For reconstitution and constitution of SCs, certificates will be sought from district EDO (or equivalent) offices highlighting the detail of SCs formulated according to policy. DLI_TBL_VERIFICATION DLI 9. Data strengthening and performance management Strengthening management capacity and evidence-based DLI 9 decision-making Description Definitions The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  Integrated education sector database will include a web-based integrated database at the school level consisting of ASC, MEA data, PEC, DSD, PEF, in agreed format  Access and orientation to the integrated education sector dashboard will involve sharing integrated database with source departments and district education officials, and providing training on utilizing the database, and on data utilization for improving education outcomes  Departments include the SED, PEC, DSD, PMIU, and PEF Indicators in the report cards should include student achievement data from PEC results, teacher and student absence and student-teacher ratios at the primary level. Additional school level indicators can be included if in agreement with the Bank Data source/ Agency Verification Entity  Private school census school-wise database with GPS coordinates  Integrated education database in agreed format Procedure  District level lists with signatures of relevant officials indicating receipt of cards by district education administrations, in agreed format DLI_TBL_VERIFICATION DLI 9. Data strengthening and performance management Strengthening management capacity and evidence-based DLI 9 decision-making Definitions  Integrated education sector database will include a web-based integrated database at the school level consisting of ASC, MEA data, PEC, QAED, PEF, in agreed format  Access and orientation to the integrated education sector dashboard will involve sharing integrated database with source departments and district education officials, and providing training on utilizing the database, and on data Description utilization for improving education outcomes  Departments include the SED, PEC, QAED, PMIU, and PEF Indicators in the report cards should include student achievement data from PEC results, teacher and student absence and student-teacher ratios at the primary level. Additional school level indicators can be included if in agreement with the Bank Various data source, including school census, monitoring data, PEC results, QAED teacher training data, PEF administrative Data source/ Agency record. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) PMIU and PITB to aggregate and validate the data. Verification Entity  Private school census school-wise database with GPS coordinates  Integrated education database in agreed format Procedure  District level lists with signatures of relevant officials indicating receipt of cards by district education administrations, in agreed format The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Annex 1 – Updated Results Framework and DLI Matrix Project Development Objectives(s) The project development objective is to support Punjab province to improve school participation, completion, and teaching-learning practices with a particular focus on low-performing districts. PDO LEVEL INDICATORS Indicator Name DLI Baseline Year 3 End Target To improve school participation Participation Rate (Percentage) 1-4 75.80 77.00 79.00 Participation rate, boys (6-10 years) 1-4 85.40 86.00 87.00 (Percentage) Participation rate, girls (6-10 years) 1-4 77.80 79.00 80.00 (Percentage) Participation rate, boys (11-15 years) 1-4 75.40 77.00 79.00 (Percentage) Participation rate, girls (11-15 years) 1-4 64.60 66.00 68.00 (Percentage) Participation rate, poor-performing 1-4 68.50 70.00 73.00 districts (Percentage) To improve school completion Completion rate, grade 5 (Percentage) 1-4 67.00 68.00 70.00 Completion rate, grade 5, girls 1-4 66.00 67.00 69.00 (Percentage) Completion rate, Grade 5, boys 1-4 69.00 70.00 71.00 (Percentage) To improve teaching-learning practices Quality score of primary teaching-learning 5-7 2.53 -- 2.65 practices (Number) Level of School Readiness (Percentage) 4 0.00 -- 6.00 The World Bank Pakistan: Third Punjab Education Sector Project (P154524) MONITORING AND EVALUATION PLAN 1. Participation Rate Definition/Description of Indicator This is defined as the share of children (in the relevant age-group) enrolled in and completing school (regardless of grade), using a representative household survey, such as PSLM, HIES or MICS that is representative at the provincial level. Frequency Baseline, Years 3 and 5 Data Source PSLM, MICS or HIES; Methodology for Data Collection Sample based household Survey Responsibility for Data Collection Pakistan Bureau of Statistics or other relevant statistical authority. 1.1 Participation Rate, Boys (6-10 years) Definition/Description of Indicator This is defined as the share of boys (age 6-10) enrolled in and completing school (regardless of grade), using a representative household survey, such as PSLM, HIES or MICS that is representative at the provincial level. Frequency Baseline, Years 3 and 5 Data Source PSLM, MICS or HIES; Methodology for Data Collection Sample based household Survey Responsibility for Data Collection Pakistan Bureau of Statistics or other relevant statistical authority. 1.2 Participation Rate, Girls (6-10 years) Definition/Description of Indicator This is defined as the share of girls (age 6-10) enrolled in and completing school (regardless of grade), using a representative household survey, such as PSLM, HIES or MICS that is representative at the provincial level. Frequency Baseline, Years 3 and 5 Data Source PSLM, MICS or HIES; Methodology for Data Collection Sample based household Survey Responsibility for Data Collection Pakistan Bureau of Statistics or other relevant statistical authority. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) 1.3 Participation Rate, Boys (11-15 years) Definition/Description of Indicator This is defined as the share of boys (age 11-15) enrolled in and completing school (regardless of grade), using a representative household survey, such as PSLM, HIES or MICS that is representative at the provincial level. Frequency Baseline, Years 3 and 5 Data Source PSLM, MICS or HIES; Methodology for Data Collection Sample based household Survey Responsibility for Data Collection Pakistan Bureau of Statistics or other relevant statistical authority. 1.4 Participation Rate, Girls (11-15 years) Definition/Description of Indicator This is defined as the share of girls (age 11-15) enrolled in and completing school (regardless of grade), using a representative household survey, such as PSLM, HIES or MICS that is representative at the provincial level. Frequency Baseline, Years 3 and 5 Data Source PSLM, MICS or HIES; Methodology for Data Collection Sample based household Survey Responsibility for Data Collection Pakistan Bureau of Statistics or other relevant statistical authority. 1.5 Participation Rate, Poor Performing Districts Definition/Description of Indicator This is defined as the share of children enrolled in and completing school (regardless of grade), using a representative household survey, such as PSLM, HIES or MICS that is representative at the provincial level. Frequency Baseline, Years 3 and 5 Data Source PSLM, MICS or HIES; Methodology for Data Collection Sample based household Survey Responsibility for Data Collection Pakistan Bureau of Statistics or other relevant statistical authority. 2. Completion Rate, Grade 5 Definition/Description of Indicator An indicator of the extent to which children who enroll in Grade 1 then go on to successfully complete Grade 5. The data will cover only government schools. Frequency Baseline, Year 3 and Year 5 Data Source School census data, which are collected from all government schools using a standardized format at the midpoint of the academic year. Methodology for Data Collection The rate is calculated using the number of children enrolled in Grade 5 as a percentage of the number of children enrolled in Grade 1, four years earlier. Public schools only. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Responsibility for Data Collection PMIU/SED/MEAs 2.1 Completion Rate, Grade 5 (Girls) Definition/Description of Indicator An indicator of the extent to which girls who enroll in Grade 1 then go on to successfully complete Grade 5. The data will cover only government schools. Frequency Baseline, Year 3 and Year 5 Data Source School census data, which are collected from all government schools using a standardized format at the midpoint of the academic year. Methodology for Data Collection The rate is calculated using the number of girls enrolled in Grade 5 as a percentage of the number of girls enrolled in Grade 1, four years earlier. Public schools only. Responsibility for Data Collection PMIU/SED/MEAs 2.2 Completion Rate, Grade 5 (Boys) Definition/Description of Indicator An indicator of the extent to which boys who enroll in Grade 1 then go on to successfully complete Grade 5. The data will cover only government schools. Frequency Baseline, Year 3 and Year 5 Data Source School census data, which are collected from all government schools using a standardized format at the midpoint of the academic year. Methodology for Data Collection The rate is calculated using the number of boys enrolled in Grade 5 as a percentage of the number of boys enrolled in Grade 1, four years earlier. Public schools only. Responsibility for Data Collection PMIU/SED/MEAs 3. Quality score of teaching and learning practices Definition/Description of Indicator The indicator is an index of 9 constructs from the 'Teach' Classroom Observation Instrument. These constructs including (1) Supportive Learning Environment, (2) Positive Behavioral Expectations, (3) Lesson Facilitation, (4) Checks for Understanding, (5) Feedback, (6) Critical Thinking, (7), Autonomy, (8) Perseverance, and (9) Social and Collaborative Skills. Each of these constructs is scored by enumerators on a five point scale (1 being very low and 5 being very high). The final result is a simple average of the score on these five constructs (public schools only). Frequency At baseline and endline. Data Source Sample-based survey of public schools and monitoring data from AEOs, applying the 'Teach' classroom observation instrument. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Methodology for Data Collection The baseline is set using an Independent third party, and is sample-based. The instrument is validated, and aligned with the teaching-learning practices targeted for improvement by the Quaid-e-Azam Academy for Educational Development (QAED). Once the instrument has been designed and validated, and a team of observers has been trained to reliably collect data, the baseline data collection will be undertaken to determine the baseline value. The endline will be collected by school mentors (AEOs) that will be trained on a new classroom observation tool. Responsibility for Data Collection PMIU and QAED 4. Level of School Readiness Definition/Description of Indicator The level of school readiness is defined as the number of children who attend quality ECE that is regularly monitored by the government, divided by all children enrolled in pre-school (ECE, Katchi, nursery, etc.), expressed as a percentage. Frequency Baseline and Endline. Data Source MEA or AEO monitoring data Methodology for Data Collection Regular (monthly) monitoring by school inspectors/mentors (MEAs and/or AEOs) visit school and check the number of enrolled and attending children. Responsibility for Data Collection PMIU and SED The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Intermediate Results Indicators Indicator Name DL Baseline Intermediate Targets End Target I Year 1 Year 2 Year 3 Year 4 Year 5 Component 1: Improved access, quality and education system management Number of additional children 1 0 0 10,000 130,000 270,000 420,000 aged 6-16 years to whom a voucher is provided to pay low-cost private school tuition fees (Number) Number of additional boys 1 0 0 5,000 65,000 135,000 210,000 aged 6-16 years to whom a voucher is provided to pay low-cost private school tuition fees (Number) Number of additional girls 1 0 0 5,000 65,000 135,000 210,000 aged 6-16 years to whom a voucher is provided to pay low-cost private school tuition fees (Number) Number of additional schools 2 0 225 450 675 900 1,125 covered by PPP initiatives supported by PEF under a quality assurance system (Number) Number of additional children 2 0 180,000 360,000 540,000 720,000 900,000 enrolled in PPP initiatives supported by PEF under a quality assurance system (Number) Number of additional girls 2 0 90,000 180,000 270,000 360,000 450,000 enrolled in PPP initiatives supported by PEF under a The World Bank Pakistan: Third Punjab Education Sector Project (P154524) quality assurance system (Number) Number of additional boys 2 0 90,000 180,000 270,000 360,000 450,000 enrolled in PPP initiatives supported by PEF under a quality assurance system (Number) Number of children aged 3-5 4 0 0 30,000 75,000 135,000 210,000 years enrolled in an ECE classroom that meets quality standards (Number) Number of girls aged 3-5 4 0 0 15,000 37,500 67,500 105,000 years enrolled in an ECE classroom that meets quality standards (Number) Number of boys aged 3-5 4 0 0 15,000 37,500 67,500 105,000 years enrolled in an ECE classroom that meets quality standards (Number) Percentage of new teachers 5 100 100 100 100 100 100 recruited based on merit, for all vacant posts reallocated to schools as per recruitment policy (Percentage) Number of districts 6 0.00 0.00 0 0 6 12 implementing the strengthened field-based CPD (Number) Grades 5 and 8 PEC (revised) 7 PEC Instruments Revised Grades 5 and 8 Grades 5 and 8 PEC reuslts from results are analyzed and instruments revision instruments PEC (revised) PEC results are previous year are widely disseminated in do not use piloted administered to analyzed and analyzed and actionable form (Text) equated items all public school reported in disseminated in and measure children. actionable form. actionable form higher-order cognitive skills, and results are not The World Bank Pakistan: Third Punjab Education Sector Project (P154524) disseminated in actionable form Number of School Council 8 80,000 80,000 96,000 128,000 212,000 212,000 members reached through citizen engagement initiative(s) (Number) Non-salary budget execution 8 64 64 64 66 66 69 rate (Percentage) Component 2: Capacity Building, Project Management, Monitoring, and Evaluation Number of EDOs-Education 9 0 3 12 36 36 36 that analyze and act on revised district education report card generated from integrated database (Number) MONITORING AND EVALUATION PLAN 1. Number of additional children of age 6–16 years to whom a voucher is provided to pay low-cost private school tuition fees Definition/Description of Indicator Measures the number of children to whom a voucher is provided to pay for low-cost private school tuition at a school that maintains quality standards as contractually defined between the school and the PEF. This indicator will include only those children who participate in the EVS that uses a strengthened quality assurance system designed under the project; and who are in addition to the children enrolled in the EVS schools as captured in the DLI baseline Frequency Annual Data Source PEF administrative records Methodology for Data Collection PEF regular monitoring officers visit the school and verify enrolment, while the quality assurance system is administered annually by PEF's Academic Development Unit. Responsibility for Data Collection PEF 1.1. Number of additional boys of age 6–16 years to whom a voucher is provided to pay low-cost private school tuition fees Definition/Description of Indicator Measures the number of boys to whom a voucher is provided to pay for low-cost private school tuition at a school that maintains quality standards as contractually defined between the school and the PEF. This indicator will include only those children who participate in the EVS that uses a strengthened quality assurance system designed under the project; and who are in addition to the children enrolled in the EVS schools as captured in the DLI baseline Frequency Annual The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Data Source PEF administrative records Methodology for Data Collection PEF regular monitoring officers visit the school and verify enrolment, while the quality assurance system is administered annually by PEF's Academic Development Unit. Responsibility for Data Collection PEF 1.2. Number of additional girls of age 6–16 years to whom a voucher is provided to pay low-cost private school tuition fees Definition/Description of Indicator Measures the number of girls to whom a voucher is provided to pay for low-cost private school tuition at a school that maintains quality standards as contractually defined between the school and the PEF. This indicator will include only those children who participate in the EVS that uses a strengthened quality assurance system designed under the project; and who are in addition to the children enrolled in the EVS schools as captured in the DLI baseline Frequency Annual Data Source PEF administrative records Methodology for Data Collection PEF regular monitoring officers visit the school and verify enrolment, while the quality assurance system is administered annually by PEF's Academic Development Unit. Responsibility for Data Collection PEF 2. Number of additional schools covered by PPP initiatives supported by PEF under a quality assurance system Definition/Description of Indicator Measures the number of additional schools covered by a PPP initiative and for which there is a quality assurance system strengthened by the project. ‘Additional’ means schools that are in addition to the schools included in the DLI baseline. Frequency Annual Data Source PEF administrative records Methodology for Data Collection Data is provided by partner schools and then validated by PEF. Quality Assurance system is administered by PEF's academic development unit. Responsibility for Data Collection PEF 3. Number of additional children enrolled in PPP initiatives supported by PEF under a quality assurance system Definition/Description of Indicator Measures the number of children enrolled in additional schools covered by a PPP initiative and for which there is a quality assurance system strengthened by the project. These children are in addition to children enrolled in PPP schools already operating at baseline, as captured in the DLI.. Frequency Annual Data Source PEF administrative records Methodology for Data Collection PEF regular monitoring officers visit the school and verify enrolment, while the quality assurance system is administered annually by PEF's Academic Development Unit. Responsibility for Data Collection PEF 3.1. Number of additional girls enrolled in PPP initiatives supported by PEF under a quality assurance system The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Definition/Description of Indicator Measures the number of girls enrolled in additional schools covered by a PPP initiative and for which there is a quality assurance system strengthened by the project. These children are in addition to children enrolled in PPP schools already operating at baseline, as captured in the DLI.. Frequency Annual Data Source PEF administrative records Methodology for Data Collection PEF regular monitoring officers visit the school and verify enrolment, while the quality assurance system is administered annually by PEF's Academic Development Unit. Responsibility for Data Collection PEF 4. Number of children aged 3-5 years enrolled in an ECE classroom that meets quality standards Definition/Description of Indicator Measures the number of children enrolled in the classrooms that meet all the quality standards, as stipulated in DLI 4 Frequency Annual Data Source Regular PMIU monitoring data Methodology for Data Collection PMIU collect monthly monitoring data using a classroom observation tool, which is implemented by MEAs and AEOs who visit the schools in person. Responsibility for Data Collection PMIU and SED 4.1. Number of girls aged 3-5 years enrolled in an ECE classroom that meets quality standards Definition/Description of Indicator Measures the number of girls enrolled in the classrooms that meet all the quality standards, as stipulated in DLI 4 Frequency Annual Data Source Regular PMIU monitoring data Methodology for Data Collection PMIU collect monthly monitoring data using a classroom observation tool, which is implemented by MEAs and AEOs who visit the schools in person. Responsibility for Data Collection PMIU and SED 4.2. Number of boys aged 3-5 years enrolled in an ECE classroom that meets quality standards Definition/Description of Indicator Measures the number of girls enrolled in the classrooms that meet all the quality standards, as stipulated in DLI 4 Frequency Annual Data Source Regular PMIU monitoring data Methodology for Data Collection PMIU collect monthly monitoring data using a classroom observation tool, which is implemented by MEAs and AEOs who visit the schools in person. Responsibility for Data Collection PMIU and SED 5. Percentage of new teachers recruited based on merit, for all vacant posts reallocated to schools as per recruitment policy Definition/Description of Indicator Measures the number of new teachers recruited based on merit for all vacant posts (which are to be filled by initial recruitment of teachers and not by promotions) reallocated to schools based on the recruitment policy, as a percentage of all new teachers recruited for those vacant posts The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Frequency Annual Data Source EDOs/DMOs administrative data. Methodology for Data Collection EDOs/DMOs collect this data in their regular monitoring efforts. The data will be validated in Year 3 by a Third Party Responsibility for Data Collection PMIU 6. Number of Districts Implementing the strengthened field-based CPD Definition/Description of Indicator Measures the number of new teachers recruited based on merit for all vacant posts (which are to be filled by initial recruitment of teachers and not by promotions) reallocated to schools based on the recruitment policy, as a percentage of all new teachers recruited for those vacant posts Frequency Year 4 and Endline Data Source Monitoring data Methodology for Data Collection AEOs (or equivalent) will collect data from the schools during regular school visits. Responsibility for Data Collection PMIU and QAED with coordination from SED. 7. Grades 5 and 8 PEC (revised) results are analyzed and widely disseminated in actionable form Definition/Description of Indicator Revised instruments will include instruments that measure higher-order cognitive skills and contain items that are designed for inter-year equitability. The reporting of data in actionable form means that the data are analyzed, tailored, and disseminated to relevant stakeholders (teachers/trainers, curriculum and textbook developers, and decision makers) in a form that is useful to them in deciding what actions can be taken to improve the quality of education. Frequency Year 4 and Endline Data Source TPV Report Methodology for Data Collection TPV Report Responsibility for Data Collection PMIU and PEC. 8. Number of School Council members reached through citizen engagement initiative(s) Definition/Description of Indicator Measures the number of members of a SC (mostly parents) who are given a forum through either a school mobilization program (on going) based on information and communication technology (ICT) or any other forum to voice their service delivery related needs and/or complaints. This will be measured through the data collected by the PMIU on the SC mobilization program(s). Frequency Year 4 and Endline Data Source TPV Report Methodology for Data Collection TPV Report Responsibility for Data Collection PMIU and PEC. 9. Non-salary budget execution rate Definition/Description of Indicator Measures the number of members of a SC (mostly parents) who are given a forum through either a school mobilization program (on going) based on information and communication technology (ICT) or any other forum to The World Bank Pakistan: Third Punjab Education Sector Project (P154524) voice their service delivery related needs and/or complaints. This will be measured through the data collected by the PMIU on the SC mobilization program(s). Frequency Baseline, Year 3 and Endline Data Source CEO confirmations of transfers to the school, and MEA monitoring data to verify expenditures. Methodology for Data Collection Monitoring officers visit schools monthly to check actual receipts and expenditures. These data are then aggregated and validated by PMIU. Responsibility for Data Collection PMIU 10. Number of EDOs-Education that analyze and act on revised district education report card generated from integrated database Definition/Description of Indicator Measures the number of EDOs-Education that analyze the revised District Report Cards generated from the integrated database, and that take action based on the analysis. Evidence of analysis and action will be based on minutes of the District Review Committee (DRC), using a protocol to be elaborated during the first 6 months of the project implementation period. Only those DRCs will be analyzed that have received training through the project on how to analyze and act on the data in the revised district education report card. ‘Revised’ refers to revisions to be made to the report card as described in annex 2. ‘Integrated’ means that the report card is generated from a master database that incorporates data sets as agreed in the Memorandum of Understanding to be elaborated in year 1 of project implementation. Frequency Annual Data Source PMIU administrative records Methodology for Data Collection PMIU records this data through regular district meetings, usually attended by secretary of education. Usage data is also tracked through the integrated database itself. Responsibility for Data Collection PMIU The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI Matrix DLI 1 DLI 1: Private school vouchers Strengthening program design to expand coverage of tuition-replacement vouchers to children from disadvantaged households Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 53.34 % Period Value Allocated Formula Amount (USD) Baseline • Vouchers to attend private schools provided to 322,679 children from -- disadvantaged households • The program has undergone rapid expansion, and present design features need to be revisited to ensure more effective beneficiary selection and monitoring 2016-17 Revised EVS system in place; and PEF initiates pilot for EVS 8,590,000.00 -- 2017-18 10,000 Eligible Children received vouchers to attend private schools; and PEF’s 8,590,000.00 -- Board of Directors approved the Revised System for EVS. 2018-19 130,000 Eligible Children received vouchers to attend private schools. 8,590,000.00 -- 2019-20 270,000 Eligible Children received vouchers to attend private schools. 3,220,000.00 -- 2020-21 420,000 Eligible Children received vouchers to attend private schools. 3,220,000.00 -- Description Definitions  All targets are cumulative and are in addition to the number of children attending the EVS schools as captured in the baseline. In the LPDs and included in the targets for DLIs 1 and 2 (under the EVS and PPP Programs), the cumulative number of children who were previously out of school will be not less than 150,000 by year 5  Pilot initiated is defined as vouchers distributed to households under the revised system  The revised system includes revised beneficiary targeting and validation system, and improved monitoring system includes quarterly monitoring of partner schools  “Eligible Children” refers to those children who would be able to participate in the EVS as defined by the PEF Board of Directors’ criteria for eligibility. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Data source/ PEF Agency Verification PMIU Entity Procedure Evidence  (a) Database of registered voucher recipients (with unique identifiers) (b) database of partner schools (with unique identifiers) (c) expenditures under the vouchers program, in agreed format  QAT and monitoring database of schools and beneficiaries with unique identifiers  Approval by the PEF’s board of directors will be evidenced with official board minutes. Evidence provided by the PEF to the PMIU DLI 2 DLI 2. Public-private partnerships in education Leveraging the private sector to support schools to increase enrollment Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 53.34 % Period Value Allocated Formula Amount (USD) Currently 1.3 million children are studying in PEF partner schools through the NSP -- Baseline and FAS. Capacity to enroll additional children through PPPs remains 180,000 children are enrolled through Public-Private Partnerships under PEF 8,590,000.00 -- 2016-17 approved programs that have a Strengthened Quality Assurance System. 360,000 children are enrolled through Public-Private Partnerships under PEF 8,590,000.00 -- 2017-18 approved programs that have a Strengthened Quality Assurance System. 540,000 children are enrolled through Public-Private Partnerships under PEF 8,590,000.00 -- 2018-19 approved programs that have a Strengthened Quality Assurance System. 720,000 children are enrolled through Public Private Partnerships under PEF 3,220,000.00 -- 2019-20 approved programs that have a Strengthened Quality Assurance System. 900,000 children are enrolled through Public Private Partnerships under PEF 3,220,000.00 -- 2020-21 approved programs that have a Strengthened Quality Assurance System. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Description Definitions  All targets are cumulative and are in addition to the number of children attending the EVS schools as captured in the baseline. In the LPDs and included in the targets for DLIs 1 and 2 (under the EVS and PPP Programs), the cumulative number of children who were previously out of school will be not less than 150,000 by year 5  Pilot initiated is defined as vouchers distributed to households under the revised system  The revised system includes revised beneficiary targeting and validation system, and improved monitoring system includes quarterly monitoring of partner schools  “Eligible Children” refers to those children who would be able to participate in the EVS as defined by the PEF Board of Directors’ criteria for eligibility. Data source/ PEF Agency Verification PMIU Entity Procedure Evidence  (a) Database of registered voucher recipients (with unique identifiers) (b) database of partner schools (with unique identifiers) (c) expenditures under the vouchers program, in agreed format  QAT and monitoring database of schools and beneficiaries with unique identifiers  Approval by the PEF’s board of directors will be evidenced with official board minutes. Evidence provided by the PEF to the PMIU DLI 3 DLI 3. Stipends for secondary school girls Stipends for secondary school girls to increase secondary school participation/retention Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Output No Text 32,210,000.00 53.34 % Period Value Allocated Formula Amount (USD) The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Baseline Stipends are offered to girls enrolled in secondary grades in 16 districts under the -- original program. The stipend amount and conditions have remained unchanged since program inception. 2016-17 SED has strengthened its original program and is implementing it in 16 Districts. 8,590,000.00 -- 2017-18 SED continues to implement the strengthened program in 16 Districts 8,590,000.00 -- 2018-19 SED continues to implement the strengthened program in 16 Districts 8,590,000.00 -- 2019-20 SED continues to implement the strengthened program in 16 Districts 3,220,000.00 -- 2020-21 SED continues to implement the strengthened program in 16 Districts 3,220,000.00 -- Description Definitions  Strengthened program includes additional repetition condition (whereby a student becomes ineligible during a repeated grade), biannual stipend payments, and more efficient payment mechanisms.  Ongoing Stipend Program includes the 16 districts where girls enrolled in secondary school have been receiving stipends since 2004. Data source/ PMIU and PSPA Administrative Records Agency Verification PMIU Entity Procedure Database of beneficiaries in agreed format Biannual program implementation status report with beneficiary identification, confirmation, and disbursement data, in agreed format The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI 4 DLI 4: Early childhood education Improving quality and expanding access to early childhood education Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 53.34 % Period Value Allocated Formula Amount (USD) Baseline ECE is currently being implemented at a small scale -- 2016-17 SED has started implementation of its recruitment policy for Merit-Based 8,590,000.00 -- Recruitment of new teachers and reallocation of posts 2017-18 SED continues to implement its recruitment policy for Merit-Based Recruitment of 8,590,000.00 -- new teachers and reallocation of posts. Third-party validation of the SED’s recruitment policy for Merit Based Recruitment of new teachers and reallocation of posts completed 2018-19 SED continues to implement its recruitment policy for Merit-Based Recruitment of 8,590,000.00 -- new teachers and reallocation of posts. 2019-20 SED continues to implement its recruitment policy for Merit-Based Recruitment of 3,220,000.00 -- new teachers and reallocation of posts. 2020-21 SED continues to implement its recruitment policy for Merit-Based Recruitment of 3,220,000.00 -- new teachers and reallocation of posts Description Definitions  All targets are cumulative  ECE Policy Framework includes institutional unit and budget planning.  An ECE teacher is defined as a primary schoolteacher who has received the DSD ECE teacher training.  An ECE caregiver is defined as an individual recruited with revised Terms of Reference (ToR) and who has received the DSD ECE caregiver training.  ‘Revised’ ToRs mean that the ToRs have been updated and reviewed by the DSD, SED, and PMIU after project effectiveness. ECE Quality Standards for a Classroom and its School  Staffing. One ECE teacher and one ECE caregiver  Training. The following persons have received in-service training on ECE through the QAED: ECE teacher, ECE caregiver, head teacher and 2 School Management Council members, of whom one must be a woman The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  Content. ECE curriculum and ECE activity guide in use  Materials. ECE toolkit  Monitoring for improvement. AEOs (or equivalent) mandate revised to include ECE classrooms. ECE classrooms are monitored once a month by the AEOs(or equivalent) who have received ECE training and provide feedback for improvement to ECE teacher and ECE caregivers.  Parental outreach. Parents have received parent activity guide and follow-up; quarterly parent meetings held at schools. Evidence to be provided by the PMIU with QAED inputs.  The ECE curriculum and activity guide are judged to be in use (or not) by the AEO (or equivalent) during the on-site support visit. Pilot. District health/nutrition officers receive training in early childhood development to provide information on health and nutrition to ECE classes and identify malnourished children to refer to district health centers. Data source/ o Random sample of reports from the AEOs on classrooms meeting quality standards. Agency o Approved ECE Policy Framework with quality standards signed by relevant authority (Secretary Schools Education Department) o Database of schools with ECE classrooms that meet quality standards, in an agreed format o TPV report in an agreed format Verification PMIU Entity Procedure AEOs (or equivalent) observe the classrooms, and report back to the SED/PMIU about their observations These data are aggregated, analysed and validated by the PMIU. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) DLI 5 DLI 5: Human Resources Management Strengthening HR processes through continuous merit-based recruitment and rationalization of teaching posts Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 53.34 Period Value Allocated Formula Amount (USD) Baseline Merit-based recruitment of new teachers for all vacant posts reallocated to schools -- based on existing recruitment policy 2016-17 SED has started implementation of its recruitment policy for Merit-Based 8,590,000.00 -- Recruitment of new teachers and reallocation of posts. 2017-18 SED continues to implement its recruitment policy for Merit-Based Recruitment of 8,590,000.00 -- new teachers and reallocation of posts. Third-party validation of the SED’s recruitment policy for Merit Based Recruitment of new teachers and reallocation of posts completed. 2018-19 SED continues to implement its recruitment policy for Merit-Based Recruitment of 8,590,000.00 -- new teachers and reallocation of posts. 2019-20 SED continues to implement its recruitment policy for Merit-Based Recruitment of 3,220,000.00 -- new teachers and reallocation of posts. 2020-21 SED continues to implement its recruitment policy for Merit-Based Recruitment of 3,220,000.00 -- new teachers and reallocation of posts. Description  Merit-based recruitment means hiring of 100 percent newly recruited teachers under initial recruitment quota through competitive tests carried out by a third party as under PESRP II. Vacant posts will be identified by district officials based on the recruitment policy.  Initial recruitment quota is identified as vacant posts, which will be filled by hiring of new teachers and not by promotions or transfers of existing teachers.  TPV conducted in 2017–18, 2018-19 and 2019-20 will review implementation of recruitment policy with particular focus on merit-based recruitment and reallocation of teaching posts. The TPV review of test-based recruitment will identify potential improvements in test content, design, administration, and use of results. Review of recruitment and reallocation policy will identify the ratio of success in implementation and propose changes to address challenges. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Data source/ Approved recruitment policy, teacher-specific test results, and placement information in agreed format. In 2017–18, the SED will Agency submit a revised recruitment and reallocation mechanism acceptable to the Bank. Verification TPVs in year 2, 3 and 4. Entity Procedure NTS will administer tests for teachers and share the results with the SED. PMIU will submit test data to the Bank after each year where recruitment is complete. Actual recruitment and transfer data are shared by district level CEOs and DMOs. These are then aggregated, analysed and validated by the PMIU. DLI 6 Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 0.00 Period Value Allocated Formula Amount (USD) Baseline Field-based system of CPD has mixed operational capacity and performance, and -- poor design match with multigrade and large class teaching. 2016-17 -- 0.00 -- 2017-18 -- 0.00 -- 2018-19 Innovative support package is designed; AEOs task description is revised and AEOs -- 8,590,000.00 start conducting classroom observations through android app. 2019-20 Training of Primary School Teachers (as per revamped CPD design) conducted in 06 -- Districts Innovative support package is developed, validated and pilot initiated in 11,810,000.00 100 schools; QAED has trained AEOs (or equivalent) on classroom observation and feedback provision. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) 2020-21 Training of Primary School Teachers (as per revamped CPD design) conducted in 12 -- Districts. QAED has incorporated successful elements of innovative support 11,810,000.00 package into the regular CPD program; Description  The new CPD approach taken by QAED strengthens the existing CPD system for primary teachers, and focuses on improving teaching practices, while also improving literacy, numeracy, and large class (incl. if relevant, multi-grade) teaching. o This includes a teacher support package on basic literacy and numeracy covering Grades 1-3, curriculum sequencing of learning steps for Grades 1-3 literacy and numeracy, with learning, assessment and supplementary learning activities for each step. o This also includes a focus on strengthened pedagogic practice for teachers, use of formative assessments, and in-service training for teachers and head-teachers or principals, aligned to the broad agenda of government and stakeholders. This also includes adequate instructional resources including teacher guides, learning and assessment activity materials for each step, individual child progress monitoring cards, student workbooks, notebooks, supplementary reading books, and literacy and numeracy materials;  The main focus of the DLI is the development and piloting of an innovative support package focusing on teacher pedagogic practice, which includes the use of videos in order to show good practice and evaluate practice, and is aimed to improve teaching quality in the classroom in areas such as supportive learning environment, setting positive behavioral expectations, providing feedback, checking for understanding (formative assessment), lesson facilitation, and critical thinking.  The approach also includes modules (including an app) to support AEOs (or equivalent), headteachers, CPD experts and teachers on teacher observation and feedback provision with the aim of providing data-driven, actionable and constructive feedback to teachers, linked to standards for teaching.  A revised task description for AEOs (or equivalent) will be developed in order to refocus the system on professional development and support, rather than control, while allowing more time for classroom observation, and the development of a professional mentoring relationship between the AEO and the schools. AEOs and headteachers will also be trained and certified on how to do classroom observation and how to provide feedback.  The innovative package will be validated, which means that the innovative package has been reviewed by relevant stakeholders, has been piloted and evaluated, ideally in the setting of a randomized controlled trial that measures its effect on classroom practices and learning outcomes.  The successful elements of the innovative support package pilot will be identified by QAED and stakeholders, and will be selected for scale up in the new mainstream CPD system. Evidence: The World Bank Pakistan: Third Punjab Education Sector Project (P154524)  Innovative support package published on QAED’s website; AEO training materials and certification results published on QAED website; Pilot report; evidence provided by QAED for training of primary school teachers; PMIU (AEO data on classroom observations, and MEA verification of pilot exercise). Data source/ Evidence to be provided by the PMIU with QAED inputs Agency Verification PMIU MEA monitoring system and SED's AEO system. Entity Procedure School inspectors (MEAs) visit schools regularly to check whether actions are being implemented. School mentors (AEOs) will provide classroom observations and feedback to teachers. DLI 7 DLI 7. Student Assessment PEC and its instruments strengthened within a revised policy framework Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 0.00 % Period Value Allocated Formula Amount (USD) Baseline No assessment policy framework; outdated PEC ISP that has been partially -- implemented and is in need of revision. 2016-17 PEC has developed the its assessment policy framework; and The new PEC 8,590,000.00 -- Institutional Strengthening Plan has been approved by the commission overseeing PEC. 2017-18 80% of ISP targets for 2017-18 are met; and strengthened Grade 5 + 8 PEC 8,590,000.00 -- instruments are piloted in at least one subject 2018-19 80% of ISP targets for 2018-19 are met; and strengthened Grade 5 + 8 PEC 8,590,000.00 -- instruments are administered throughout the entire Punjab school system 2019-20 PEC has analyzed and reported Grade 5+8 PEC results from the previous year of 3,220,000.00 -- Project implementation in Actionable Form. 2020-21 PEC has analyzed and reported Grade 5 + 8 PEC results for the previous year of 3,220,000.00 -- Project implementation in Actionable Form. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Description Definitions, Evidence, and Protocols  The assessment policy framework will be developed based upon a review of extant policy, and the roles, mandates and relative strengths and weaknesses of PEC and other institutions involved in assessment. It will be approved by the Commission overseeing PEC.  The ISP will be based upon a functional review of PEC that examines PEC’s mission within the context of the new assessment policy framework, and will have schedules covering at least the following areas of institutional strengthening: (i) clarification of PEC’s mission; (ii) update of PEC functions, activities and operating procedures; (iii) staffing levels and profiles, and recruitment and training needs; (iv) equipment procurement; (v) budgetary requirements, and (vi) capacity building to enhance item/test quality and scoring, and to analyze, report and use data.  The reporting of data in actionable form means that the data are analyzed, tailored and disseminated to relevant stakeholders (teachers/trainers, curriculum and textbook developers, and decision makers) in a form that is useful to them in deciding what actions can be taken to improve the quality of education. This will require doing an upfront focus group with each stakeholder group to get a sense of their data needs. It also will require holding a dissemination workshop with each stakeholder group once the report has been prepared to review the data and actionable recommendations with them. Data source/ PEC Agency Verification PMIU Entity Procedure PMIU will verify PEC data. DLI 8 DLI 8. School specific NSB Setting and executing school budgets in line with school resource needs Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 26.67 % Period Value Allocated Formula Amount (USD) The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Baseline School-specific NSBs for FY2015/16 prepared in accordance with agreed funding -- formula, and submitted for inclusion in FY2016/17 district budgets in 36 districts for all schools, including high/high secondary. 2016-17 SED has prepared School-Specific Non-Salary Budgets for FY2016/17 in accordance 8,590,000.00 -- with agreed funding formula, and submitted for inclusion in FY2017/18 District budgets in 36 Districts for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed School Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and SCs reconstituted in required elementary/primary schools and constituted in all high schools 2017-18 SED has prepared School-Specific Non-Salary Budgets for FY2017/18 in accordance 8,590,000.00 -- with agreed funding formula, and submitted for inclusion in FY2018/19 District budgets in 36 Districts for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed School Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. 2018-19 SED has prepared School-Specific Non-Salary Budgets for FY2018/19 in accordance 8,590,000.00 -- with agreed funding formula, and submitted for inclusion in FY2019/20 District budgets in 36 Districts for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and TPV of Year 1 and Year 2 of this DLI completed. 2019-20 SED has prepared School-Specific Non-Salary Budgets for FY2019/20 in accordance 3,220,000.00 -- with agreed funding formula, and submitted for inclusion in FY2020/21 District budgets in 36 District for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. 2020-21 SED has prepared School-Specific Non-Salary Budgets for FY2020/21 in accordance 3,220,000.00 -- with agreed funding formula, and submitted for inclusion in FY2020/21 District budgets in 36 Districts for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed School-Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution. Description Definitions and Evidence 3. ‘Necessary support structure’ for execution of the NSB component entails formulation of a steering committee with representation from all stakeholders and the required capacity-building program to engage/mobilize SCs and school administrations. The required capacity-building program for school will be a comprehensive mobilization strategy agreed by all stakeholders. It will also include necessary material aid to SCs, such as SC guidelines. The first year activities will be focused on establishing protocols and strategies, which will be implemented in following years. For high and higher The World Bank Pakistan: Third Punjab Education Sector Project (P154524) secondary schools, Project to Improve Financial Reporting and Auditing (PIFRA) system will be used to identify disbursed money against Drawing & Disbursing Officer (DDO) codes of respective schools in the first 2 years of the program, while the SCs are being formulated in the schools. In the third year of the program, the project will work to ensure that disbursement of the NSB will be through SCs in high or higher secondary schools, based on consultations and subject to agreement with the FD of Punjab Province and the Accountant General’s Office. 4. SCs will be reconstituted in primary and elementary schools, which were not part of the capacity-building program during PESRP II. Simultaneously, SC will be formalized in high or higher secondary schools with required guidelines and materials disseminated. Data source/ PMIU/PIFRA Agency Verification Third Party Entity Procedure The TPV to be carried out in the third year will be against the ToRs as agreed by all stakeholders. The TPV will seek to review the impact) of the NSB component along with the capacity-building program, analyzing the efficacy of the in-place capacity-building program. Verification protocol. Verification of disbursements of funds to schools will be sought from certificates submitted by the EDO (or equivalent) office for release of cheques to all primary and elementary schools. For reconstitution and constitution of SCs, certificates will be sought from district EDO (or equivalent) offices highlighting the detail of SCs formulated according to policy. DLI 9 Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 53.34 % Period Value Allocated Formula Amount (USD) Baseline Last private school census was in 2010-11. Various education sector data are not -- integrated, utilization of data remains low and restricted to source departments. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) 2016-17 SED has conducted a private school census; and PMIU has prepared and delivered 8,590,000.00 -- on a semi-annual basis District performance report cards for Year 1 following agreed format and contents, to District Education Administrations. 2017-18 PMIU has integrated Education sector data (from PEC, PMIU, DSD and PEF); and 8,590,000.00 -- PMIU provided access and orientation to the integrated database to education departments and District education officials in 3 Districts; PMIU has prepared and delivered on a semiannual basis District performance report cards for Year 2 following agreed format and contents, to District Education Administrations 2018-19 PMIU has integrated Education sector data (from PEC, PMIU,DSD, and PEF); and 8,590,000.00 -- PMIU has provided access and orientation to the integrated database to education departments and District education officials in 12 Districts; PMIU has prepared and delivered on a semi-annual basis District performance report cards for Year 3 following agreed format and contents, to District Education Administrations. 2019-20 PMIU has integrated Education sector data (from PEC, PMIU,DSD, and PEF) ; and 3,220,000.00 -- PMIU has provided access and orientationto the integrated database to education departments and District education officials in 36 Districts; PMIU has prepared and delivered on a semi-annual basis District performance report cards for Year 4 following agreed format and contents,to District Education Administrations 2020-21 PMIU has integrated Education sector data (from PEC, PMIU, DSD, and PEF) ; and 3,220,000.00 -- PMIU has provided access and orientation to the integrated database to education departments and District education officials in 36 Districts; PMIU has prepared and delivered on a semiannual basis District performance report cards for Year 5 following agreed format and contents, to District Education Administrations Description Definitions  Integrated education sector database will include a web-based integrated database at the school level consisting of ASC, MEA data, PEC, QAED, PEF, in agreed format  Access and orientation to the integrated education sector dashboard will involve sharing integrated database with source departments and district education officials, and providing training on utilizing the database, and on data utilization for improving education outcomes  Departments include the SED, PEC, QAED, PMIU, and PEF Indicators in the report cards should include student achievement data from PEC results, teacher and student absence and student-teacher ratios at the primary level. Additional school level indicators can be included if in agreement with the Bank Data source/ Various data source, including school census, monitoring data, PEC results, QAED teacher training data, PEF administrative Agency record. The World Bank Pakistan: Third Punjab Education Sector Project (P154524) Verification PMIU and PITB to aggregate and validate the data. Entity Procedure  Private school census school-wise database with GPS coordinates  Integrated education database in agreed format  District level lists with signatures of relevant officials indicating receipt of cards by district education administrations, in agreed format