The World Bank Support to Implementation of the Regional Education Strategy (P158836) REPORT NO.: RES31087 RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF SUPPORT TO IMPLEMENTATION OF THE REGIONAL EDUCATION STRATEGY APPROVED ON JULY 5, 2016 TO OECS COMMISSION,MINISTRY OF EDUCATION, DOMINICA,MINISTRY OF EDUCATION, GRENADA,MINISTRY OF EDUCATION, ST. LUCIA,MINISTRY OF EDUCATION, ST. VINCENT AND THE GRENADINES EDUCATION LATIN AMERICA AND CARIBBEAN Regional Vice President: Jorge Familiar Country Director: Tahseen Sayed Khan Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager/Manager: Reema Nayar Task Team Leader: Hongyu Yang The World Bank Support to Implementation of the Regional Education Strategy (P158836) ABBREVIATIONS AND ACRONYMS CLASS Classroom Assessment Scoring System CXC Caribbean Examination Council IP Implementation Progress ISR Implementation Status and Results Report MU Moderately Unsatisfactory IRI Intermediate Results Indicators NPDT National Professional Development Team OECS Organization of Eastern Caribbean States PDO Project Development Objective SIP School Improvement Plan The World Bank Support to Implementation of the Regional Education Strategy (P158836) BASIC DATA Product Information Project ID Financing Instrument P158836 Investment Project Financing Original EA Category Current EA Category Not Required (C) Not Required (C) Approval Date Current Closing Date 05-Jul-2016 30-Sep-2019 Organizations Borrower Responsible Agency OECS Commission,Ministry of Education, Dominica,Ministry of Education, Grenada,Ministry of OECS Commission, Education Management Unit Education, St. Lucia,Ministry of Education, St. Vincent and the Grenadines Project Development Objective (PDO) Original PDO The objectives of the Project are to: (i) use quality learning standards to support evidence based teaching and learning at the primary level; (ii) improve teacher practices at the primary level; (iii) strengthen primary school leadership and accountability and (iv) initiate the strengthening of sector monitoring and evaluation capacity in support of evidence- based strategic management and decision-making, all in the Member Countries. OPS_TABLE_PDO_CURRENTPDO Summary Status of Financing Net TF Approval Signing Effectiveness Closing Commitment Disbursed Undisbursed TF-A2937 05-Jul-2016 27-Jul-2016 27-Jul-2016 30-Sep-2019 2.00 .51 1.49 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Policy Waiver(s) Does this restructuring trigger the need for any policy waiver(s)? No I. PROJECT STATUS AND RATIONALE FOR RESTRUCTURING A. Project Status 1. The Project suffered significant delays at the begging of implementation due to: (i) the nature of project design which requires coordination with four countries, and (ii) lack of experience, and low capacity in project procurement and implementation. The Project was launched nearly one year after the project effectiveness. Key consultancy services were not on board until late 2017. While project activities were delayed, some progress in implementation was noticed in December 2016 with the first disbursement. 2. The implementation further progress after the project launch. By November 2017 all key consultancies had been awarded and have since made substantial progress. Specifically, i. Component 1: Strengthen Curriculum Standards and Assessment Framework. The Caribbean Examinations Council (CXC) was engaged for this consultancy and conducted national consultations in the four countries. A cadre of Curriculum Officers, Education Officers, Teachers, and Principals from the four countries were trained to help lead the process and aligning the learning standards with the curriculum. The publication of the learning standards and assessment framework document is expected in June 2018, with training to follow in July 2018. ii. Component 2: Improve Teacher Training Programs and Strengthen Teaching Practice. Each member state has established a National Professional Development Team (NPDT) tasked to coordinate professional development activities and provide support to teachers during the initial implementation of the program. A consultancy firm specialized in education and training, and capacity building, has produced a review of teacher education, a report outlining gaps in teaching practices in the classroom, and a framework for teacher professional development. Training of teachers has already begun. It is expected that all primary school teachers for the four countries will be completed by July 2019. iii. Component 3: Improve School Leadership and Accountability. Consultants have been selected and have conducted focus group meetings with education leaders on the development of a leadership course. The situational analysis on leadership is expected in June 2018 and a training of trainers in leadership is planned for July 2018. iv. Component 4: Strengthen Sector Monitoring and Evaluation Capacity to Support Evidence-Based Strategic Management and Decision-Making. More than 400 classroom observations were conducted across the four- member-countries using the Classroom Assessment Scoring System (CLASS) in 2017. The consulting firm, Targeted Development Consulting, proposed a regional monitoring and evaluation framework and revised it based on feedback from the Chief Education officers in November 2017. The consulting firm will host a series of focus group meetings with education officials in the four project countries to help assess needs for a sustainable structure for monitoring and evaluation at the national level. This exercise will build capacity in monitoring at the country level. The World Bank Support to Implementation of the Regional Education Strategy (P158836) B. Rationale for Restructuring 3. The proposed restructuring would make several changes to the Results Framework. The Project has 2 Impact Indicators, 5 PDO indicators, and 9 Intermediate Results Indicators (IRIs). 4. Impact Indicators: these indicators measure the percentage of pupils performing at or above minimum standards for their grade level in reading and mathematics in grade 2 by gender and urban/rural residence. Due to problems with the comparability of data across the member states, these indicators cannot be aggregated for regional-level monitoring. In addition, as stated in the Project Appraisal Document, the two Impact Indicators were included to keep the focus on learning, but were expected to be achieved beyond project timeline. Given the situation that these two indicators are included in the Results Framework (RF), It is necessary to delete them from the RF. Otherwise, it will impede the achievement of the PDO and unforeseen challenges with the data. The Bank team will review the annual national assessment data on reading and mathematics to track if the project activities have had some potential impact on student learning outcome. 5. There are five PDO Indicator, after the changes proposed in this paper, there would be four PDO indicators, one corresponding with each part of the PDO. i. PDO Indicator 2 measures the percentage of teachers rated effective on classroom practices. Since the baselines for this indicator were established, a number of challenges have arisen for the member states related to data collection, including costs of the proprietary classroom observation instrument and the frequency of recertification required for classroom observers. The member states are discussing ways to adapt to these challenges, but as a consequence there may not be a valid time series for this PDO Indicator, implying that it would no longer be suitable for tracking project performance. The current IRI indicator 3, “Number of teachers trained and certified as competent” could be adapted to replace this PDO indicator since the certification includes a demonstration of skills (as opposed to simply receiving training). However, the indicator would need to be reworded as the percentage of teachers to better capture system-level progress and should more precisely define “competent” as being focused on classroom practices. ii. PDO Indicator 3, there are two indicators under this PDO indicator according to the PAD: Percentage of primary schools with a School Improvement Plan (SIP) focused on learning outcomes, and Percentage of school leaders reporting on progress of teachers’ practices and students learning to oversight bodies are both associated with part (iii) of the PDO, strengthen primary school leadership and accountability. Given the relatively small size of the Project, there is no need for multiple indicators for one part of the PDO. Since the reporting of progress to oversight bodies is a higher-level activity than the SIPs, it is better suited to be a PDO indicator, and the SIP tracking is better suited to be an intermediate results indicator. 6. Intermediate Results Indicators: i. Related to the ongoing efforts to adapt to challenges described in 6 (i), Intermediate Results Indicator 7 will be reworded to indicate the number of teachers monitored through “a standardized classroom observation tool.” ii. Intermediate Results Indicator 8 is the number of schools analyzing and using student learning data. Since the SIPs incorporate the analysis and use of student learning data, this separate indicator is not necessary. The World Bank Support to Implementation of the Regional Education Strategy (P158836) II. DESCRIPTION OF PROPOSED CHANGES 7. The following two Impact Indicators would be removed: (i) % of pupils’ literacy at or above their grade level at the key stages of grade 2 by gender, urban-rural residence; and (ii) % of pupils by gender and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics. 8. PDO Indicators: i. Current PDO Indicator 2 “Percentage of teachers rated effective on classroom practices” would be deleted. ii. The current IRI 3 would replace PDO Indicator 2 and would be reworded as follows: “Percentage of teachers satisfactorily completing teacher professional development course in classroom practices.” iii. PDO Indicator 3, “Percentage of primary schools with SIP focused on learning outcomes” would be moved to an Intermediate Results Indicator. For consistency with the other IRIs, the word “percentage” would be replaced with “number” and the targets revised to reflect the appropriate units. 9. Intermediate Results Indicators: i. Intermediate Results Indicator 7 would have reworded as follows: “Number of teachers monitored with a standardized classroom observation tool.” ii. Intermediate Results Indicator 8 “Number of schools analyzing and using student learning data” would be deleted. III. SUMMARY OF CHANGES Changed Not Changed Change in Results Framework ✔ Change in Implementing Agency ✔ Change in Project's Development Objectives ✔ Change in Components and Cost ✔ Change in Loan Closing Date(s) ✔ Cancellations Proposed ✔ Additional Financing Proposed ✔ Reallocation between Disbursement Categories ✔ Change in Disbursements Arrangements ✔ Change in Disbursement Estimates ✔ Change in Overall Risk Rating ✔ Change in Safeguard Policies Triggered ✔ Change of EA category ✔ The World Bank Support to Implementation of the Regional Education Strategy (P158836) Change in Legal Covenants ✔ Change in Institutional Arrangements ✔ Change in Financial Management ✔ Change in Procurement ✔ Change in Implementation Schedule ✔ Other Change(s) ✔ IV. DETAILED CHANGE(S) OPS_DETAILEDCHANGES_RESULTS_TABLE RESULTS FRAMEWORK Project Development Objective Indicators PDO_IND_TABLE % of pupils reading at or above their grade level at the key stages of grade 2 by sex, urban-rural residence (This is an Impact Indicator) Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 9.00 9.00 10.00 Marked for Deletion Date 30-Sep-2016 30-Nov-2017 30-Aug-2019 % of pupils by sex and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics (This is an Impact Indicator) Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 10.00 Marked for Deletion Date 30-Sep-2016 30-Nov-2017 30-Aug-2019 PDO 1: Percentage of primary teachers using formative assessment based on learning standards Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 50.00 Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 The World Bank Support to Implementation of the Regional Education Strategy (P158836) PDO 2: Percentage of teachers trained and certified Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 50.00 New Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 Percentage of teachers rated effective on classroom practices Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 70.00 70.00 73.00 Marked for Deletion Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 Percentage of primary schools with a School Improvement Plans (SIP) focused on learning outcomes Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 75.00 75.00 50.00 Marked for Deletion Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 PDO 3: Percentage of school leaders reporting on progress of teachers practices and students learning to oversight bodies Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 50.00 Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 PDO 4: Production of an annual report on plan implementation and performance indicators at the country level Unit of Measure: Yes/No Indicator Type: Custom Baseline Actual (Current) End Target Action Value No No Yes Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Intermediate Indicators IO_IND_TABLE IRI 1: Learner-centered assessment framework and guidelines for using learning standards approved Unit of Measure: Yes/No Indicator Type: Custom Baseline Actual (Current) End Target Action Value No No Yes Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 2: Teacher professional development course developed Unit of Measure: Yes/No Indicator Type: Custom Baseline Actual (Current) End Target Action Value No No Yes Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 Number of teachers trained and certified as competent Unit of Measure: Number Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 540.00 Marked for Deletion Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 3: Number of primary schools with a School Improvement Plans (SIP) focused on learning outcomes Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 170.00 New Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 4: Handbook for School Leaders developed and printed Unit of Measure: Yes/No Indicator Type: Custom Baseline Actual (Current) End Target Action Value No No Yes Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 The World Bank Support to Implementation of the Regional Education Strategy (P158836) IRI 5: School leadership training program developed Unit of Measure: Yes/No Indicator Type: Custom Baseline Actual (Current) End Target Action Value No No Yes Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 6: Number of school leaders certified in competence based standards Unit of Measure: Number Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 125.00 Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 7: Number of teachers monitored with a standardized classroom observation tool Unit of Measure: Number Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 300.00 400.00 Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 Number of schools analyzing and using student learning data Unit of Measure: Number Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 300.00 Marked for Deletion Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 8: M&E Manual Developed Unit of Measure: Yes/No Indicator Type: Custom Baseline Actual (Current) End Target Action Value No Yes Yes Revised Date 30-Aug-2016 30-Nov-2017 30-Aug-2019 IRI 9: Stakeholder feedback surveys taken Unit of Measure: Yes/No The World Bank Support to Implementation of the Regional Education Strategy (P158836) Indicator Type: Custom Baseline Actual (Current) End Target Action Value No No Yes New Date 30-Aug-2016 30-Nov-2017 30-Aug-2019