Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004921 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF0A2937 ON A SMALL RECIPIENT EXECUTED TRUST FUND GRANT IN THE AMOUNT OF US$2 MILLION TO THE ORGANIZATION OF EASTERN CARIBBEAN STATES COMMISSION FOR THE SUPPORT TO IMPLEMENTATION OF THE REGIONAL EDUCATION STRATEGY PROJECT March 26, 2020 Education Global Practice Latin America and The Caribbean Region J. Humberto Lopez Regional Vice President (acting): Tahseen Sayed Khan Country Director: Regional Director: Luis Benveniste Practice Manager: Emanuela Di Gropello Task Team Leader(s): Shawn Michael Powers, Ingrid Bjerke ICR Main Contributor: Nicole Mammoser ABBREVIATIONS AND ACRONYMS CLASS Classroom Assessment Scoring System EDMU Education Development Management Unit (of the OECS Commission) GPE Global Partnership for Education IDA International Development Association IRI Intermediate Results Indicator MOE Ministry of Education M&E Monitoring and Evaluation NPDT National Professional Development Team OECS Organization of Eastern Caribbean States OESS OECS Education Sector Strategy PD Professional Development PDO Project Development Objective RF Results Framework SIP School Improvement Plan TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME .................................................................................................................... 10 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 15 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 17 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 19 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 21 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 30 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 31 The World Bank Support to Implementation of the Regional Education Strategy (P158836) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Support to Implementation of the Regional Education P158836 Strategy Country Financing Instrument OECS Countries Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency OECS Commission, Ministry of Education, Dominica, Ministry of Education, Grenada, Ministry of Education, OECS Commission, Education Development St. Lucia, Ministry of Education, St. Vincent and the Management Unit Grenadines Project Development Objective (PDO) Original PDO The objectives of the Project are to: (i) use quality learning standards to support evidence based teaching and learning at the primary level; (ii) improve teacher practices at the primary level; (iii) strengthen primary school leadership and accountability and (iv) initiate the strengthening of sector monitoring and evaluation capacity in support of evidence-based strategic management and decision-making, all in the Member Countries. Page 1 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) FINANCING FINANCE_T BL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-A2937 2,000,000 1,775,557 1,775,557 Total 2,000,000 1,775,557 1,775,557 Total Project Cost 2,000,000 1,775,557 1,775,557 KEY DATES Approval Effectiveness Original Closing Actual Closing 05-Jul-2016 27-Jul-2016 30-Sep-2019 30-Sep-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 13-Jun-2018 0.51 Change in Results Framework KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) Moderately 01 29-Jun-2017 Moderately Unsatisfactory 0.41 Unsatisfactory Moderately 02 20-Dec-2017 Moderately Unsatisfactory 0.41 Unsatisfactory 03 22-Jun-2018 Moderately Satisfactory Moderately Unsatisfactory 0.51 04 21-Dec-2018 Moderately Satisfactory Moderately Satisfactory 1.24 Page 2 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 05 24-Jun-2019 Moderately Satisfactory Moderately Satisfactory 1.66 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon J. Humberto Lopez Country Director: Sophie Sirtaine Tahseen Sayed Khan Director: Claudia Maria Costin Luis Benveniste Practice Manager: Reema Nayar Emanuela Di Gropello Harriet Nannyonjo, Leandro Shawn Michael Powers, Ingrid Task Team Leader(s): Oliveira Costa Bjerke ICR Contributing Author: R. Nicole Mammoser Page 3 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context 1. The Organization of Eastern Caribbean States (OECS) was formed in 1981 and is dedicated to economic harmonization and integration and encouragement of good governance among its eleven Member States. In 2010, the organization updated its treaty to establish a unified financial and economic space in which goods, people and capital move freely, and Member States adopt a common approach to trade, health, education and the environment, as well as for the development of the agriculture, tourism and energy sectors. The administrative body charged with implementing this regional approach is the OECS Commission. The Commission’s Education Development Management Unit (EDMU) works to ensure integrative solutions in regional education development and reform while promoting the full participation of all Member States. Among other tasks, the EDMU helps coordinate the implementation and results monitoring of the OECS Education Sector Strategy (OESS) 2012-21. Of the OECS Member States, four are eligible for concessional long-term financing through the International Development Association (IDA): Dominica, Grenada, Saint Lucia, and St. Vincent and the Grenadines. 2. At time of project preparation in 2016, these four countries had made significant progress in expanding access to basic education. In 2015, net primary enrollment rates were 96 percent in Dominica, 95 percent in Grenada, and 93 percent in both Saint Lucia and St. Vincent and the Grenadines, with gender parity achieved. Government expenditure on education as a percentage of total public expenditure was 15 percent in both Saint Lucia and St. Vincent and the Grenadines, 14 percent in Dominica and 11 percent in Grenada. However, low learning achievement in the region remained a major challenge. National exam data for 2015-16 showed 44 percent of fourth grade students in Grenada performing below minimum standards in reading, and 67 percent of fourth graders in Dominica performing below minimum standards in math.1 3. There were several underlying causes of low education quality in the OECS. The first was a lack of clear learning standards to specify the learning outcomes expected at every grade level and to guide effective teaching and classroom assessment in order to promptly address learning deficiencies. Second, teachers were insufficiently trained to provide effective instruction: in 2015, the proportion of untrained teachers2 at the primary level ranged from 11 percent to 37 percent across the four countries participating in the Project, and there were limited continuous professional development (PD) opportunities to improve instructional practices. The third cause was the low capacity of school leaders to support teaching and to ensure that teachers were performing at desired levels. 4. To address low learning achievement across its Member States, the OECS developed a regional education strategy, referred to as the OESS, for the period 2012-2021. OECS Member States were charged with aligning their national education plans to the strategy, which presents seven strategic imperatives in response to educational challenges and priorities in the region and defines a framework of related interventions and desired outcomes. 1 Data for the four countries participating in the Project is from the OECS Education Statistical Digest. The online Digest was developed to monitor implementation of the OECS Education Sector Strategy and was first published for the 2012-2013 academic year. 2 Untrained teachers are defined as those lacking a formal teaching credential. Page 4 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 5. In 2015, the World Bank began working with the EDMU and officials from the Ministries of Education in Dominica, Grenada, Saint Lucia, and St. Vincent and the Grenadines to develop a project to support implementation of the OESS. In alignment with this strategy, the Project was designed to address several of the primary causes of low learning achievement in the four countries through the development of regional public goods, including common learning standards for core primary school subjects, a related framework for formative assessment of students to ensure progress toward the learning standards, a teacher training program to promote integration of the learning standards and assessment framework in daily teaching, a framework for teacher professional development to improve instructional efficacy, and a professional certification program for school leaders to improve supervision and support of teachers. These public goods were intended to serve not only the four IDA-eligible member states, but also the rest of the OECS through complementary regional and national activities. 6. In June 2016, the Global Partnership for Education (GPE) approved a US$2 million grant to implement the Support to Implementation of the Regional Education Strategy Project over a three-year period.3 The Project would address four of the OESS strategic imperatives: (i) improve the quality and accountability of leadership and management; (ii) improve teachers’ professional development; (iii) improve the quality of teaching and learning; and (iv) improve curriculum and strategies for assessment. The World Bank was designated as the grant agent and the OECS Commission’s EDMU as the implementing agency responsible for overall coordination, supervision and fiduciary oversight. 7. The Project’s focus on establishing clear learning standards and systematic student assessment, along with providing training to strengthen the skills of both teachers and school leaders, aligned with the GPE’s objectives of improving teacher effectiveness and early grade learning outcomes. The Project was also relevant to the World Bank’s Regional Partnership Strategy for the OECS for the period 2015 -2019,4 which emphasized vocational training for young people to address lack of adequate skills for the workplace. The Project’s interventions at the primary education level would benefit learners in the early stages of skill-building for future employment. Project Development Objectives (PDOs) 8. The objectives of the Project were to i) use quality learning standards to support evidence-based teaching and learning at the primary level; ii) improve teacher practices at the primary level; iii) strengthen primary school leadership and accountability; and iv) initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making, all in Dominica, Grenada, Saint Lucia and St. Vincent and the Grenadines. Key Expected Outcomes and Outcome Indicators 9. The following five indicators were chosen to assess achievement of the PDOs: 3 After joining the GPE in March 2016, the OECS received two GPE-funded grants: an Education Plan Development Grant to provide technical assistance to finalize the OESS Education Quality Framework, and a Program Development Grant to develop an implementation and costing plan for all education sector activities to be implemented as part of the OESS. In 2018, the GPE approved a second Education Plan Development Grant to carry out a Mid-Term Review of the OESS. 4 The OECS Regional Partnership Strategy for 2015-2019 was endorsed by the Bank’s Board of Executive Directors in November 2014. The strategy identified three other areas for support: enhancing productivity, competitiveness and employment, modernizing the public sector, and building climate resilience. Page 5 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) i. PDO Indicator 1: Percentage of primary teachers using formative assessment based on learning standards ii. PDO Indicator 2: Percentage of teachers rated effective on classroom practices iii. PDO Indicator 3: Percentage of primary schools with a School Improvement Plan (SIP) focused on learning outcomes iv. PDO Indicator 4: Percentage of school leaders reporting on progress of teachers’ practices and students’ learning to oversight bodies v. PDO Indicator 5: Production of an annual report on plan implementation and performance indicators at the country level Components Component 1: Curriculum and Assessment (US$396,500 at approval; US$292,868 actual) 5 10. This Component would establish a set of learning standards that clearly specified learning outcomes to be achieved in language arts, mathematics, science and technology, and social studies in primary grades one through six. Formative assessment methodology would also be instituted in order for teachers to routinely monitor and evaluate the degree to which students were progressing toward achievement of the learning standards. 11. Component 1 activities would include engaging in national consultations with education stakeholders in each of the four participating countries to finalize primary grade level learning standards6 and develop a learner-centered assessment framework to monitor achievement of the learning standards. In addition, guidelines for the use of the learning standards and assessment framework would be developed, and primary teachers would be trained to integrate the standards and assessment in their daily teaching. The expected outcome of these activities was that trained primary teachers would use formative assessment to gauge students’ performance in light of the core learning standards. Component 2: Teacher Professional Development (US$1,115,000 at approval; US$976,694 actual) 12. This Component would strengthen teachers’ content knowledge and instructional practices through opportunities for continuous professional learning that would support and further student achievement. 13. Component 2 activities would include reviewing existing teacher training programs and classroom teaching practices in each of the four participating countries to develop a teacher professional development training framework. In addition, competence-based PD training activities and certification would be provided for teachers, and an online community of practice for teachers would be established to ensure access to instructional resources and foster knowledge-sharing related to lesson planning and teaching. The expected outcome of these activities was that trained teachers would effectively carry out classroom instructional practices. 5 Though curriculum development itself was not part of Component 1, the component title references curriculum because a significant activity was the alignment of the four countries’ existing curricula with a set of core learning standards. 6 A set of regional learning standards for core subjects at the primary level had been drafted as an initial step in implementation of the OESS; these standards still needed to be aligned with the existing curricula in place in each of the participating countries. Page 6 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Component 3: Improve School Leadership and Accountability (US$209,000 at approval; US$240,655 actual) 14. This Component would train and certify principals and other school administrators in core management and leadership competencies required in order to better support teachers and increase student learning achievement. 15. Component 3 activities would include development of a program to train school leaders based on agreed professional standards, and provision of competence-based training and certification of school leaders in skills including instructional support for teachers and improved use of data for school planning, management and reporting. In addition, a school leadership handbook and a standardized job description and appraisal instrument for school principals would be developed.7 The expected outcomes of these activities were improved numbers of primary schools with School Improvement Plans focused on learning outcomes and of school leaders reporting on progress of teacher practices and student learning to the relevant oversight bodies. Component 4: Project Implementation, Monitoring and Evaluation (US$279,500 at approval; US$265,322 actual) 16. This Component would support Project implementation, including procurement and financial management, and strengthen monitoring and evaluation (M&E) capacity in the four participating countries in order to progress toward the shared goal of using education sector data for evidence-based management and decision-making. 17. Component 4 activities would include monitoring and reporting on Project activities (teacher training in the learning standards and related assessment, PD activities, the school leadership training program) and establishing methodologies and routines for ongoing monitoring of OESS implementation and of indicators specific to each country’s national education sector plan. The expected outcome of these activities was that each of the four participating countries would produce a plan for Project implementation and would systematically monitor and report on Project indicators. Project Restructuring 18. A Restructuring was approved in June 2018.8 The Project’s original Results Framework (RF) was streamlined by removing three indicators (two high-level impact indicators and one PDO Indicator) that measured results that were unlikely to be observed during the Project’s three-year timeline and thus were not appropriate measures of Project achievements. The Restructuring also aligned the PDO Indicators with the four components of the Project. 7 The leadership training program, principal job description, and principal appraisal instrument developed under Component 3 were aligned with a set of school leadership standards developed by the OECS. 8 The GPE required that the Restructuring had to be endorsed by UNICEF, USAID, Caribbean Development Bank, UNESCO, EU and DFID, that is, all the donors that supported the Project when the grant was approved in 2016. This took some time, and the Restructuring was therefore not approved until mid-2018, with 15 months left in the Project implementation period and 26% of the grant disbursed. Page 7 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 19. At Project preparation, two impact indicators were included in the RF in an effort to ensure that focus was maintained on learning achievement. These indicators would track the percentage of grade 2 students reading at or above their grade level and performing at or above the minimum standards in mathematics. However, during implementation it was determined that any improvements in grade 2 learning outcomes that might be attributable to Project interventions could not be measured during the three-year implementation period, and as such the impact indicators were dropped from the Project’s RF. They are represented in the long-term outcome section in the theory of change to more appropriately reflect the Project’s results chain. 20. The Restructuring also reduced the five original PDO Indicators to four, and ensured one PDO Indicator corresponded to each of the four objectives in the PDO statement: i. PDO Indicator 2, which measured the percentage of teachers rated effective on classroom teaching practices, was dropped from the RF. While the Project focused on generating professional development learning opportunities and plans, meaningful changes in teaching practices resulting from PD require more time to take hold in classrooms, and thus it was not feasible to expect to observe these changes during the Project implementation period. ii. Intermediate Results Indicator 3, which tracked the number of teachers trained and certified as competent in a range of PD activities, replaced original PDO Indicator 2 and was modified to measure a percentage of teachers. This new PDO Indicator was considered an effective measure of the Project’s success in developing a framework for continuous teacher PD, and in training and certifying teachers in related PD activities during the life of the Project. iii. PDO Indicators 3 and 4 were originally included to both measure the impact of the third objective in the PDO statement (strengthen primary school leadership and accountability). In an effort to streamline the RF, PDO Indicator 4 (percentage of school leaders reporting on progress of teachers’ practices and students’ learning to oversight bodies) was maintained as a measure of principals’ compliance with a key aspect of their leadership duties and renumbered as PDO Indicator 3. Original PDO Indicator 3 (percentage of primary schools with a SIP focused on learning outcomes) became an Intermediate Results Indicator. 21. Two Intermediate Results Indicators were also modified at Restructuring: i. Intermediate Results Indicator 7 was reworded as the “number of teachers monitored with a standardized classroom observation tool,� removing the reference to use of the CLASS instrument, given uncertainty at the time about whether the region would continue to use this instrument for teacher classroom observations.9 ii. Intermediate Results Indicator 8, which tracked the number of schools analyzing and using student learning data, was dropped from the RF because the School Improvement Plans tracked under new Intermediate Results Indicator 3 (percentage of primary schools with a SIP focused on learning outcomes) would incorporate analysis of student learning data. 9 The CLASS system was used to establish the baseline for original PDO Indicator 2. The four participating countries were concerned about high licensing costs associated with continued use of CLASS, as well as the requirement that classroom observers be recertified annually. One additional round of classroom observations using CLASS was undertaken after the Restructuring, and the region is now moving toward adopting a version of Teach, the open-source classroom observation tool developed by the World Bank. Page 8 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 22. A split rating is not warranted as the Restructuring did not change the Project Development Objective, the targets for PDO Indicators, or the Project’s scope. Rather, it served to better align the RF to impacts that could be realized during the Project’s relatively brief lifetime and to the four project development outcomes of the Project and thus to better reflect the Project’s results chain. 23. The Project’s Theory of Change is represented in the following results chain: Main activities: Outputs: PDOs/Outcomes: Long-term outcomes: -Establish learning standards -Finalized set of core Teachers use quality learning for primary grades learning standards aligned standards to support -Develop related assessment with curricula of the four evidence-based teaching and framework Project countries learning at the primary level -Train primary teachers to -Guidelines for use of (PDO Indicator 1: Percentage integrate learning standards assessment framework of primary teachers using and assessment in daily formative assessment based teaching on learning standards) ---------------------------------- --------------------------------- ---------------------------------- -Develop a framework for -Professional Development Teacher practices are teacher professional Framework Reference improved at the primary development (PD) training Guide level -Support four Project -Teachers participate in PD (PDO Indicator 2: Percentage countries in development of training activities per of teachers trained and Improved learning their own National PD Plans country-specific National PD certified (in PD activities)) outcomes: students Plans reading at or above ---------------------------------- --------------------------------- ---------------------------------- their grade level -Develop a competence- -Training program for Primary school leadership and based training program for school leaders and accountability performing at or school leaders that is aligned -School Leadership strengthened above the minimum with regional professional Handbook (PDO Indicator 3: Percentage standards in standards -Job description and of school leaders reporting mathematics -Train and certify school appraisal instrument for on progress of teachers’ leaders school leaders practices and students’ learning to oversight bodies) ---------------------------------- --------------------------------- ---------------------------------- -Support M&E teams in four -Training for national M&E Initiated the strengthening Project countries in teams of sector M&E capacity in monitoring/reporting on -Documented routines, support of evidence-based Project methodologies and strategic management and -Establish methodologies, processes for systematic decision making processes, templates for monitoring and reporting (PDO Indicator 4: Production M&E that teams use to track on the education sector of an annual report on initiatives being -M&E Manual (Project) plan implemented as part of the implementation and OESS (regional education performance indicators at strategy) the country level) 24. Several critical assumptions underpin the achievement of the PDO. Given the Project’s focus on producing regional public goods, all of the components were premised on member-state level commitment to the Project and successful implementation of national-level activities such as teacher professional development institutes. Under Components 1 and 2, the key assumption is that teachers use the tools that have been developed, such as the formative assessment based on learning standards and Page 9 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) the skills acquired during PD training, in the classroom. To support this link, school leaders would evaluate teachers to ensure this was taking place, and teachers would receive further training and in-classroom support for their training as needed. Under Component 3, the key assumption is similar, that the School Leaders in turn utilize the tools and techniques learned, and Ministry of Education (MOE) officials would evaluate school leaders and provide support accordingly. For Component 4, the key assumption is that the training is implemented and that the resulting annual report is an accurate and useful input for decision making in the education sector. Finally, the critical assumption linking the PDO outcomes to the higher-level outcomes is that the tools and techniques are effective for increasing student learning. This can be assessed in the longer term through appropriate student learning assessments. II. OUTCOME Assessment of Achievement of Each Objective/Outcome Objective 1: Use quality learning standards to support evidence-based teaching and learning at the primary level 25. The Project’s first objective was to use quality learning standards to support evidence-based teaching and learning at the primary level. Achievement of this objective was measured by PDO Indicator 1: Percentage of primary teachers using formative assessment based on learning standards, with a baseline of zero and a target of 50 percent. This target was partially achieved. In April 2019, principals used an evaluation checklist developed under the Project to observe teachers and found that 24 percent used the formative assessment10 in their classrooms to evaluate students’ progress toward meeting a core set of learning standards. 26. Under Component 1, consultants reviewed a set of grade-level learning standards for primary education in language arts, math, science and technology, and social studies11 in order to develop a formative assessment framework to monitor student achievement of the learning standards. The consultants then worked with Education Officers, Curriculum Officers, principals and teachers from all four countries to align the learning standards with each country’s primary curricula. 27. The consultants also developed a training program to equip a cadre of trainers with the knowledge and tools to support teachers in the integration of the learning standards and assessment framework in daily classroom teaching. In turn, these trainers (education professionals from the four Project countries) led in-country, district-based workshops to train 559 teachers from 50 primary schools across the four countries during July, August and September 2018.12 10 Formative assessment is a process that takes place during the course of instruction to collect information that can be used to make decisions (on instruction modification, effective feedback for pupils, PD activities required to improve teaching practice, etc.). There is a wide range of assessment techniques and methods across subjects that teachers can employ to support learning. 11 Learning standards for core subjects taught in grades 1 through 6 were drafted in 2016. These standards serve as a regional benchmark for the knowledge and skills that a student should attain by the end of each grade level in all of the OECS countries. As part of implementation of the OESS at the national level, countries were charged with conducting curricula reviews to ensure alignment with the standards. Under Component 1 of the Project, the consultants who had previously developed the regional learning standards were engaged to assist the Project countries with this work. 12 Teachers were trained from 17 primary schools in Saint Lucia (12% of all primary teachers), 15 primary schools in Grenada (26% of all primary teachers), 10 primary schools in St. Vincent and the Grenadines (16% of all primary teachers) and 8 primary schools in Dominica 18% of all primary teachers). Curriculum Officers from each country also received training. Page 10 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 28. The Phase 1 rollout of the learning standards and assessment framework at the classroom level took place during the 2018-2019 school year. Teachers utilized manuals developed under the Project for ongoing guidance, and the cadres of trainers provided onsite support during the school year to help teachers with lesson planning that aligned with the new learning standards, and to utilize formative assessment in order to adjust their teaching to better guide learning outcomes toward the standards. 29. During the 2018-2019 school year, most teachers focused on the new task of developing lesson plans that aligned with the learning standards. The mandate to also incorporate formative assessment into the daily teaching routine was challenging for many teachers. Furthermore, only a subset of 559 teachers in select pilot schools were directly trained and there was not sufficient time under the Project to expand training to all teachers. Nonetheless, the 24 percent achievement shows that the training activities were effective, with some teachers having taken an important step towards formulating lessons in light of the new learning standards and conducting formative assessment to make informed decisions about instructional strategies. 30. Although the final target was not achieved during the life of the Project, the percentage of teachers using formative assessment based on the learning standards is expected to increase during the 2019-2020 academic year. Teachers are growing increasingly accustomed to this new way of working, and continued training for teachers in integration of the standards and assessment framework, as well as training to bolster principals’ capacity to provide instructional support to teachers, has been integrated into their professional development going forward. Objective 2: Improve teacher practices at the primary level 31. The Project’s second objective was to improve teacher practices at the primary level. Achievement of this objective was measured by PDO Indicator 2: Percentage of teachers trained and certified (in professional development activities), with a baseline of zero and a target of 50 percent. This target was exceeded, with 67 percent of the teachers who participated in PD activities during the 2017- 2018 and 2018-2019 school years receiving an MOE certification based on a minimum of 18 hours of professional development activity per year, in accordance with the OECS’s Professional Development Program. 32. The PD activities measured by PDO Indicator 2 and implemented under the Project are well aligned with global evidence on the characteristics of PD that empirically lead to improved teaching and learning.13 Through their focus on the learning standards, the PD was grounded in specific subjects rather than general pedagogy. The training included an initial face-to-face training and strong follow-up visits through a structured program of coaching and mentoring by principals and education officers. The training was designed for the local context in terms of the regional learning standards as well as customization in each member state, and engaged teachers for their opinions and ideas. Sample lessons were provided for teachers to practice and work on through follow-up visits. These design features – combined with results from PDO Indicator 1, which shows a measured improvement in teacher practices under the Project (i.e. formative assessment) – provide confidence that the teacher training and certification contributed 13 Popova, Anna; Evans, David; Breeding, Mary E.; Arancibia, Violeta. 2018. Teacher Professional Development around the World: The Gap between Evidence and Practice (English). Policy Research working paper; no. WPS 8572. Washington, D.C.: World Bank Group. Page 11 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) positively to the PDO of improved teacher practices. 33. Under Component 2 of the Project, consultants developed a regional framework for continuous teacher professional development. The framework was informed by a review of existing PD initiatives in each of the four countries, findings from 2017 CLASS teacher observation data that identified gaps in classroom teaching practices, and the new learning standards finalized under Component 1 of the Project. The framework lays out tenets and priorities to guide the development of national PD plans. 14 34. Each of the four Project countries established a National Professional Development Team (NPDT) comprised of Education Officers, Curriculum Officers, teacher educators, principals, and senior teachers.15 During the first half of 2018, the consultants provided technical assistance to these teams as they developed National Professional Development Plans, customized based on each country’s education priorities and practices, desired approaches to PD (workshop-based, job embedded, peer coaching, etc.), and data from national learning assessments. The plans identified the resources necessary to deliver PD training and included cost estimates. 16 35. During the 2018-2019 school year, the NPDTs supported the initial implementation of the National Professional Development Plans by coordinating PD activities, leading workshop sessions for teachers in their areas of competence, recruiting resources to deliver training on topics outside their expertise, and providing the MOEs with feedback as the plans were implemented. The NPDTs were supported in their efforts by a Professional Development Manual developed by the consultants under the Project, which elaborated on all aspects of planning and implementing PD at the national and local levels. 36. During the 2017-2018 and 2018-2019 academic years, 835 primary teachers participated in PD training activities, including those designated in the new National Professional Development Plans. 17 By Project closing, the NPDTs were engaged in the challenging work of developing the next iteration of their PD plans, which aim to continue to move PD training from an effort that is driven by the MOEs to one that is responsive to what teachers are doing in the classroom every day. 18 The activities under Component 2 have helped ensure that evidence-based PD planning becomes rooted in the institutional structure of schools in the four participating countries. 37. During implementation, the EDMU designed a new activity, the provision of school-level Student 14 Tenets include: determine the content of annual PD activity based on evidence of need, introduce Professional Learning Communities for educators to discuss pedagogical concerns and advances, require that School Improvement Plans include pedagogical goals aligned to the vision of respective MOEs, introduce peer coaching to support the introduction of new teaching approaches, and develop a strategy to ensure all teachers are trained in the next eight years (culminating in the provision of teachers’ licenses to currently untrained teachers). Priorities include: help teachers master differentiated practice throug h job- embedded coaching, enhance numeracy instruction, and strengthen formative assessment practices. 15 NPDT members serve a three-year term. Teams will initially be truncated or extended to facilitate a staggered change-over. 16 The National PD Teams undertook asset mapping to identify and document institutions, individuals, and citizen associations existing within communities that are potential resources to be utilized for PD activities and training. 17 Many of the 835 teachers certified in PD training activities under Component 2 were also part of the cohort of 559 teachers trained in the learning standards and related assessment framework in the summer of 2018, prior to the start of the 2018-2019 academic year. The National Professional Development Plans that guided PD in 2018-2019 included the training that took place under Component 1 of the Project. 18 The four National Professional Development Teams cited the burden of producing annual PD plans and intend to move to biennial plans. Training schedules for PD activities will continue to be developed annually. Page 12 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Achievement Grants, to further support Component 2’s objective of improving teacher practices through job-embedded PD. Up to eight grants were available in each country for teachers of any primary grade. The application process required teachers to identify student learning needs, conceptualize an intervention to address those needs, and implement the intervention. Principals submitted grant applications to the NPDTs, and applications were approved by the MOEs. 19 Thirty grants were awarded by the MOEs to support school-based projects that were aligned with the learning standards.20 One grant, for example, supported the establishment of a “learning garden� at a rural school in Grenada, which developed students’ scientific process skills by creating an interactive and sustainable learning environment that included an aquaponics system, bird feeders, and a tortoise hut. The project aligned to the Earth and Space Science strand in the grade 6 learning standards for science and technology. All projects were implemented between January and June of 2019, and project status reports were provided to the MOEs midway through implementation and upon project completion. Objective 3: Strengthen primary school leadership and accountability 38. The Project’s third objective was to strengthen primary school leadership and accountability. Achievement of this objective was measured by PDO Indicator 3: Percentage of school leaders reporting on progress of teachers’ practices and students’ learning to oversight bodies, with a target of 50 percent. At the end of the 2018-2019 academic year, 87 percent of all primary school leaders had reported to MOE officials on leadership duties, teacher practices, and student learning via SIPs, exceeding the target for PDO Indicator 3. 39. At the start of the Project, SIPs were not being prepared by school leaders on a routine basis in the four participating countries, and those that were prepared did not focus on student learning. A template was developed to assist school leaders in reporting on a set of leadership, teacher, and student performance metrics through annual SIPs. These plans would then guide school administrators and MOE officials in school planning, management and supervision. By Project closing, 195 primary schools had an MOE-approved SIP in place, surpassing the target for Intermediate Results Indicator 3, which tracked the number of primary schools with a SIP focused on teacher practice and student learning. 40. Component 3 Project activities also included development of a competency-based Professional Certification Program for Education Leaders, based on OECS School Leadership Standards, for select principals. The six-unit course21 was delivered via a combination of synchronous sessions using the Blackboard Collaborate online platform and face-to-face sessions, with school leaders working on their own to complete one performance task per unit. The consultants also developed an Education Leadership Handbook for course participants, a resource document to guide principals in their day-to-day supervision, management, and leadership duties. The course culminated in National Leadership Forums in each country to facilitate dialogue among education stakeholders on standards for education leadership and their role in achieving school effectiveness, and for course participants to present their final projects, in 19 School Achievement Grants are included in each country’s National Professional Development Plan for 2018-2019. 20 Approximately US$130,000 of Project funds were spent to support implementation of Student Achievement Grants across the four Project countries. An additional US$80,000 was used to pay for Project-related training that was carried out at the 2017 and 2018 Summer Training Institutes, which are organized annually by the MOEs in each of the Project countries. 21 The units covered leadership styles and skills, qualities of high-performing schools, conducting school self-evaluation, using data to support instruction, teacher leadership, and curriculum leadership, instruction and assessment. Page 13 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) which they implemented strategies and demonstrated competencies developed during the training program. 41. The Professional Certification Program for Education Leaders was concluded in April 2019. Intermediate Results Indicator 6 tracked the number of school leaders certified in competency-based leadership standards, with a target of 125. This target was nearly achieved, with 110 principals and Education Officers successfully completing the program.22 Fifteen school leaders who started the program were not certified; while a few had to drop out due to competing responsibilities, the majority of these finished the six course units but were not certified as they did not complete their final project. 42. A job description and an appraisal instrument for principals were also finalized under Component 3. By Project closing, the job description was being evaluated by the four countries, and the appraisal instrument was being piloted in 60 schools. Objective 4: Initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making 43. The Project’s fourth objective was to initiate the strengthening of education sector monitoring and evaluation capacity in support of evidence-based strategic management and decision making. Achievement of this objective was measured by PDO Indicator 4: Production of an annual report on (Project) plan implementation and performance indicators at the country level. This target was achieved, as all four countries developed an implementation plan for Project activities and used a template developed under Component 4 to monitor and report on the set of indicators included in the Project’s RF. 44. Under Component 4, consultants conducted M&E capacity assessments for all four countries. These assessments revealed that while the countries were collecting national data required for production of the annual OECS Educational Statistical Digest, existing M&E systems needed to be strengthened to better monitor and report on a range of indicators related to teacher, student and school performance, as well as on processes (PD activities, teacher classroom practices, etc.) and outcomes from education interventions. The consultants provided training in data collection and analysis methods, data management, process and outcome monitoring, and results reporting. They then supported the M&E teams in a learning-by-doing approach, providing technical assistance as the teams built upon existing M&E systems by developing routines, procedures, and templates that could be used for systematic monitoring and reporting on the education sector beyond the life of the Project. 45. By August 2019, each of the M&E teams within the Ministries had developed a National Monitoring Report that included a comprehensive results framework to monitor indicators related to the regional education sector strategy, as well as descriptions of the methodology used to track and report on the indicators and data collection challenges that remained. Through these reports, the four Project countries are now producing the data-based evidence required by the OECS Commission and the Council of Ministers for planning and decision making. Overall Outcome Rating Rating: Moderately Satisfactory 22 Certification was awarded by the School of Education, University of the West Indies (Mona Campus). Page 14 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 46. The Project was effective in achieving its PDO, as measured by PDO Indicators 2, 3 and 4, two of which exceeded targets. National Professional Development Plans are now in place to guide teacher PD training that will improve teaching practices at the primary level. A training program for education leadership has been delivered and will continue to be used to strengthen primary school leadership and accountability. Technical assistance provided to each country’s M&E team has improved capacity to monitor and evaluate the education sector across the four Project countries so that planning and decision- making at the regional and national levels can be evidence-based. 47. The target for PDO Indicator 1 was not met, although a significant portion of teachers are indeed conducting formative assessment to make informed decisions about instructional strategies that will best guide students to achievement of the learning standards. The percentage of teachers using formative assessment in their classrooms is expected to increase quickly as the training program developed under Component 1 of the Project is expanded. 48. The Project remained relevant to several of the strategic imperatives laid out in the OESS for 2012- 2021. It was also efficient in that a relatively small grant was used to support targeted interventions that positively impacted the capacity of primary education systems in the four participating countries. As these interventions are expanded and built upon going forward, schools will provide better quality primary education, thus helping to address the challenge of low learning achievement in the OECS region. Other Outcomes and Impacts 49. The Project strengthened knowledge networks both between the four participating countries and within each country, down to the school level. For example, teachers and principals developed informal networks to provide support in the rollout of the learning standards and related formative assessment. Teams organized online meetings to share updates on the status of implementation and discussed its challenges and strategies to address them. These teams remain active, and plan to visit each other to spend time in the field observing and monitoring classroom integration of the standards and assessment in order to jointly devise strategies to improve the adoption process. Such knowledge networks will outlive the Project and continue to benefit MOE officials, school leaders and teachers going forward. 50. In addition, the Project was implemented by the OECS Commission’s EDMU, which is responsible for regional coordination and oversight of the OESS implementation process. As the EDMU led Project coordination, implementation and procurement activities across multiple countries, it built capacity that enabled the EDMU to better support education sector initiatives within and across Member States beyond the Project’s lifetime. As a result, the OECS Commission is now in the process of becoming a GPE Grant Agent, which means it can manage funding directly for the Member States, and, as such, potentially increase efficiency and efficacy of development funding. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME Key factors during Project preparation 51. Project activities were designed to be mainstreamed into the work program of the MOEs, with the Ministry units tasked with overseeing curricula, teacher training, school administration and M&E for Page 15 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) implementation of activities under each of the respective Project components. Accordingly, implementation arrangements were intended to work within the capacity constraints in the MOEs in the Project countries, where the small size of the ministries required multi-tasking by limited staff. Project management was centered in the OECS Commission’s EDMU, which already worked with the Member States and had successfully implemented a range of operations, rather than creating an additional unit to support Project implementation. This design streamlined implementation, ensured ownership, and contributed to capacity-building, but lack of a dedicated project implementation unit and capacity constraints of existing staff contributed to implementation delays and the need to create additional support structures during implementation. 52. Project components were designed with a high degree of interdependency. Work under Component 1 to finalize learning standards and an assessment framework merged into teacher training activities in the PD plans developed under Component 2. The training program to strengthen the leadership capacity of principals under Component 3 emphasized instructional support for teachers, so that principals could help them as they worked to integrate standards, formative assessment and PD training into their classroom teaching. As part of capacity building under Component 4 of the Project, M&E teams monitored indicators that tracked progress toward outputs and outcomes for Components 1, 2 and 3. As such, the components mutually reinforced one another for greater effect. Key factors during Project implementation 53. During the first year of Project implementation, the four large procurement activities led to 23 delays. The procurement process was lengthy for each consultancy, technical proposals submitted by firms were more complex than required by the Terms of Reference, and financial proposals exceeded the Project’s allocated budget. This necessitated negotiations to review activities and costs, and resubmission and reevaluation of proposals prior to contract negotiations. The OECS Commission’s procurement approval process, and competing procurement demands, further contributed to the delays. 54. Procurement for the Component 3 consultancy took the longest time. When initial proposals far exceeded the budget, the Bank guided the EDMU in annulling the procurement process and revising the Scope of Work in order to qualify lower-cost firms to bid on a reduced job that limited the number of school leaders to be trained to 125 (from 320 in the Project Appraisal Document). All four consultancies were underway by December 2017. 55. Hurricane Maria devastated Dominica in September 2017, and Project activities in Dominica were suspended until January 2018, as the MOE prioritized getting children back into schools and beginning the process of rebuilding damaged schools. Previous to Hurricane Maria, Hurricane Irma almost completely destroyed Barbuda (not part of the Project, but part of another OECS member state). As such, whilst Project implementation in Dominica ultimately caught up with the other member states, the experience of these hurricanes justifiably elevated the relevance and urgency of disaster resilience and recovery in the EDMU’s workplan and in regional fora, leading to some additional diversion of attention from Project activities. 23While the procurement process was underway, teams within the four MOEs were developing national Project implementation plans and related budgets. This effort was tracked by PDO Indicator 4. Page 16 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 56. The EDMU oversaw Project supervision on a regional level while simultaneously continuing to implement the EDMU’s ongoing workload. The MOE teams at the national level accommodated Project implementation responsibilities alongside their existing duties and activities. Nonetheless, the EDMU and MOEs worked together to support effective Project implementation: i. To avoid duplication of efforts across the Project’s four consultancies, EDMU organized several opportunities for the consultants to meet to review their workplans together during the course of Project implementation in order to identify synergies and avoid conflicts in training schedules in the four countries. ii. To improve implementation speed, a dedicated resource person was assigned in each country to coordinate Project activities with other ongoing initiatives within the MOEs. In September 2018, EDMU hired a one retired education professional in each country, collectively called the Local Support Team, to assist the MOEs. The team was essential in improving logistics and communications, particularly around PD agendas. iii. Following the end of the Component 1 consultancy in December 2018, EDMU established Regional Implementation Teams, comprised of MOE officials, to support classroom implementation of the learning standards and related assessment. The teams were trained in June 2019, and by the start of the 2019-2020 school year were serving as an additional source of onsite support for principals and teachers in their efforts to lesson plan for the new learning standards, conduct formative assessment, and use assessment results to improve teaching plans and methods. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME Bank Performance 57. The Bank worked with the OECS Commission’s EDMU to ensure that Project implementation progressed on schedule and that all financial management and reporting duties were carried out satisfactorily. As described previously, the Bank guided the EDMU through a prolonged procurement process and was proactive in processing a Restructuring to modify the RF to better reflect the Project’s objectives. Bank support helped the EDMU strengthen its capacity in the areas of project management, implementation effectiveness, and procurement, as evidenced by the fact that the OECS Commission is now positioned to become a GPE grant agent. 58. The EDMU notes in its Borrower’s Report (Annex 3) that the turnover of Task Team Leaders (three TTLs in three years) caused some loss in the Project’s overall momentum. The EDMU attests that the turnover occasionally resulted in delays in decision-making and in the Bank’s provision of timely technical and management support, both of which impacted Project implementation. The resulting rating for Bank performance is Moderately Satisfactory. 59. M&E is rated as Substantial. As previously described, the M&E teams in the MOEs were trained in data collection and analysis methods, data management, process and outcome monitoring, and results reporting under Component 4 of the Project. The M&E teams then used a template to monitor and report on the indicators included in the Project’s RF. The templates were passed on to the consultants, who aggregated data to calculate indicator results based on a weighted average of the four Project countries. Page 17 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) All results were subject to EDMU review and approval. 60. The Project’s PDO and related indicators were focused more at the output level (teachers use learning standards and formative assessment in classrooms, teachers undergo PD training to improve instructional practices, primary school leadership is strengthened via development of SIPs, education sector M&E capacity is improved) than at the outcome level (improvements in teaching quality, gains in student learning). This reflected what could credibly be achieved given that the Project focused more on producing regional public goods (learning standards, frameworks for formative assessment and teacher PD training, a training program for school leaders) during its three-year timeline. These public goods are critical outputs to the OECS’s long-term goals of improved teaching and learning outcomes. Compliance Issues 61. The Project did not trigger any of the Bank’s Safeguard Policies. The Project maintained a financial management rating of Satisfactory throughout implementation. All relevant financial management arrangements were in place and deemed acceptable to the Bank, and the EDMU consistently provided accurate and timely biannual Interim Financial Reports for the Project. The final IFR for the period July 1, 2019 to January 31, 2020 is due on March 15, 2020. Audited financial statements for the period July 2016 - June 2018 resulted in a clean report on the Project’s use of funds, internal controls and compliance with the terms of the legal agreement for the GPE grant and applicable laws. A second audit covering the period July 2018-January 2020 is due to the Bank by March 31, 2020. Risk to Development Outcome and Sustainability of Project Achievements 62. Project results are expected to be sustainable for three main reasons: (i) Project interventions supported four of the strategic imperatives encapsulated in the OESS for 2012-2021, which is being implemented by all OECS Member States; (ii) the GPE grant funded activities that strengthened the capacity of education systems in the four Project countries, but do not create recurrent costs for the MOEs going forward; (iii) because the Project was implemented by various units within the Commission and MOEs, the capacity required to sustain activities initiated under the Project is in place. 63. The following additional factors provide evidence of the limited risk to sustainability of Project achievements: 64. Component 1 Curriculum and Assessment: i. The established Regional Implementation Teams are an additional source of onsite support for increased numbers of trained teachers and principals as they implement learning standards and formative assessment in classrooms across the four countries. ii. The Project countries plan to integrate the learning standards and assessment framework with their national curricula to streamline the documentation required for lesson planning and teaching. Dominica has already undertaken this exercise as part of a recent curriculum review, and its new curriculum will serve as a reference. Page 18 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 65. Component 2 Teacher Professional Development: i. By Project closing, the NPDTs were finalizing PD Plans for 2019-2020. The teams are institutionalized and will continue their work with the MOEs, school administrators, and teachers to ensure that capacity-building activities are based on evidence of need and grounded in day-to- day teaching practices. The teams also plan to deepen engagement with the Teachers’ Colleges so that student teachers receive instruction aligned with overall PD objectives as part of their formal training. ii. The EDMU is developing the “OECS Hub,� an online interactive resource that will allow teachers to share lesson plans, teaching materials and assessment templates, and to live stream teaching demonstrations, thus enabling PD among peers. Project funds were used to purchase equipment and networking components for classrooms and MOE offices that will facilitate optimal engagement with the “OECS Hub.� iii. At Project closing, the EDMU and MOEs were preparing to pilot Teach, the Bank’s open source classroom observation instrument. Adoption of Teach will ensure that there is a cost-effective, regionally owned and adapted methodology for monitoring and evaluating teachers as they implement their PD training in classroom teaching. 66. Component 3 Improve School Leadership and Accountability: i. Following completion of the Professional Certification Program for Education Leaders, the four top performers from each country were selected as trainers to support the next phase of leadership training. They finished a five-week training course in July 2019 and delivered a second round of the leadership program to dozens of principals in August-September 2019. 67. Component 4 Project Implementation, Monitoring and Evaluation: i. M&E teams have increased their capacity to collect, compile, and report on data on the education sector. This vital work will continue to be supported and expanded, given its critical importance to the MOEs for national decision-making and to the EDMU for a comprehensive regional view of learning within the OECS. V. LESSONS LEARNED AND RECOMMENDATIONS 68. Realistically assess implementation capacity at the project planning stage and include mitigating strategies to ensure effective implementation. The EDMU struggled with the additional workload brought on by Project activities. At the national level, the MOEs had small teams which, despite the alignment of the Project activities with sector plans, needed to multi-task across competing priorities. These capacity constraints contributed to implementation delays; however, measures taken later in the Project, such as recruiting retired educators as a Local Support Team in the Member States, show that the constraints were surmountable. Supporting a larger project implementation team and implementing strategies such as the Local Support Team could help accelerate implementation in a capacity-constrained environment. 69. When Project components and activities are interrelated, consider consolidating procurement of consulting services. Procurement of multiple, separate contracts and delivery of goods and services to many different countries simultaneously to support each of the four components contributed to implementation delays and necessitated greater efforts by EDMU to ensure coordination and avoid Page 19 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) duplication. In projects where the components are closely interrelated, procurement capacity is limited, and firms exist that can provide services across multiple components, combining assignments and pooling shared requirements for goods can help streamline implementation, reduce workload, improve efficiency and promote coherence and synergies across Project activities. 70. Consider and consult the end user when developing deliverables and timelines. The Project introduced teachers to a number of reference documents for lesson planning, including curriculum guides, unit plan templates, learning standards for four subjects, and the Assessment Manual, which teachers reported were overwhelming. At times, the pace of implementation was also too fast for direct beneficiaries to assimilate the Project interventions. For example, the timeline from training of trainers on the learning standards and assessment framework (June 2018) to training of teachers and MOE Curriculum Officers (July through September 2018) was too brief to allow trainers to assimilate the new material and become confident in their new knowledge. These experiences highlight the need to consider . beneficiary experience as key to the goals of the Project. Page 20 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To use quality learning standards to support evidence-based teaching and learning at the primary level. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion PDO 1: Percentage of primary Percentage 0.00 0.00 50.00 24.00 teachers using formative assessment based on learning 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 standards Comments (achievements against targets): Objective/Outcome: To improve teacher practices at the primary level. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion PDO 2: Percentage of teachers Percentage 0.00 0.00 50.00 67.00 trained and certified 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): Page 21 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) This indicator measured the percentage of teachers trained and certified by MOEs for 18 hours of professional development activity according to the streams included in the OECS’s Professional Development Program (introduced in early 2019). National Professional Development Plans, developed under the Project and rolled out during the 2018-2019 academic year, include PD activities for teachers and school leaders. Objective/Outcome: To strengthen primary school leadership and accountability. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion PDO 3: Percentage of school Percentage 0.00 0.00 50.00 87.00 leaders reporting on progress of teachers practices and 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 students learning to oversight bodies Comments (achievements against targets): Objective/Outcome: To initiate the strengthening of education sector monitoring and evaluation capacity in support of evidence-based strategic management and decision making. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion PDO 4: Production of an annual Yes/No N N Y Y report on plan implementation and performance indicators at 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Page 22 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) the country level Comments (achievements against targets): The words “plan implementation� in PDO Indicator 4 refer to each country’s implementation plan for the Project. These implementation plans included a Results Framework that M&E teams in each country used to monitor and report on Project indicators (the “performance indicators�). A.2 Intermediate Results Indicators Component: Component 1: Curriculum and Assessment. (US$396,500) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 1: Learner-centered Yes/No N N Y Y assessment framework and guidelines for using learning 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 standards approved Comments (achievements against targets): Component: Component 2: Teacher Professional Development. (US$1,115,000) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 2: Teacher professional Yes/No N N Y Y Page 23 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) development course 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 developed Comments (achievements against targets): Component: Component 3: Improve School Leadership and Accountability. (US$209,000) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 3: Number of primary Number 0.00 0.00 170.00 195.00 schools with a School Improvement Plans (SIP) 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 focused on learning outcomes Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 4: Handbook for School Yes/No N N Y Y Leaders developed and printed 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): Page 24 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 5: School leadership training Yes/No N N Y Y program developed 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 6: Number of school leaders Number 0.00 0.00 125.00 110.00 certified in competence based standards 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): Component: Component 4: Project implementation, monitoring and evaluation. (US$279,500) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 7: Number of teachers Number 0.00 0.00 400.00 673.00 monitored with a standardized classroom observation tool 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): Page 25 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) The CLASS tool was used twice during the Project. In 2017, 300 teachers were observed in their classrooms across the four Project countries, and the resulting data informed the 2018-2019 National Professional Development Plans drafted under Component 2 of the Project. In 2019, an additional 373 teachers were observed, thus surpassing the target for IRI 7. At Project close, CLASS certified observers were planning to analyze the 2017 and 2019 data to see if there were any changes in observed teaching practices over the life of the Project. Going forward, the OECS Member States plan to adopt the Bank’s Teach instrument for classroom observation of teachers. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 8: M&E Manual Developed Yes/No N N Y Y 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion IRI 9: Stakeholder feedback Yes/No N N Y Y surveys taken 30-Aug-2016 30-Aug-2016 30-Sep-2019 30-Sep-2019 Comments (achievements against targets): The M&E consultant hired under Component 4 conducted four online surveys over the course of the Project to collect and tabulate data from the four participating countries for the Project’s Results Framework. Page 26 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Page 27 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) B. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1: Use quality learning standards to support evidence-based teaching and learning at the primary level 1. Percentage of primary teachers using formative assessment based Outcome Indicators on learning standards 1. IRI 1: Learner-centered assessment framework and guidelines for Intermediate Results Indicators using learning standards approved 1. Set of core learning standards for primary grades (aligned with curricula of four Project countries) Key Outputs by Component 2. Guidelines for use of learning assessment framework (linked to the achievement of the Objective/Outcome 1) 3. Teachers trained to integrate learning standards and assessment in daily teaching Objective/Outcome 2: Improve teacher practices at the primary level 1. PDO Indicator 2: Percentage of teachers trained and certified (in Outcome Indicators professional development activities) Intermediate Results Indicators 1. IRI 2: teacher professional development course developed 1. Framework for teacher professional development training 2. Professional development framework reference guide Key Outputs by Component 3. National Development Plans for four Project countries (linked to the achievement of the Objective/Outcome 2) 4. Teachers participated in professional development training activities Objective/Outcome 3: Strengthen primary school leadership and accountability 1. PDO Indicator 3: Percentage of school leaders reporting on progress Outcome Indicators of teachers' practices and students' learning to oversight bodies Page 28 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) 1. IRI 3: Number of primary schools with a School Improvement Plan focused on learning outcomes 2. IRI 4: Handbook for school leaders developed and printed Intermediate Results Indicators 3. IRI 5: School leadership training program developed 4. IRI 6: Number of school leaders certified in competence-based standards 1. Training program for school leaders Key Outputs by Component 2. School Leadership Handbook (linked to the achievement of the Objective/Outcome 3) 3. School leaders trained and certified 4. Job description and appraisal instrument for school leaders Objective/Outcome 4: Initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making 1. PDO Indicator 4: Production of an annual report on (Project) plan Outcome Indicators implementation and performance indicators at the country level 1. IRI 7: Number of teachers monitored with a standardized classroom observation tool Intermediate Results Indicators 2. IRI 8: M&E Manual developed 3. IRI 9: Stakeholder feedback surveys taken 1. Training for national M&E teams Key Outputs by Component 2. M&E Manual (linked to the achievement of the Objective/Outcome 4) 3. Documented methodologies and processes for systematic education sector M&E Page 29 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1: Curriculum $396,500.00 $292,868.12 74% and Assessment Component 2: Teacher $1,115,000.00 $976,693.98 88% professional development Component 3: Improve school leadership and $209,000.00 $240,655.18 115% accountability Component 4: Project implementation, monitoring, $279,500.00 $265,321.92 95% and evaluation Total $2,000,000.00 $1775,539.2124 24 The discrepancy between the total cost by component and total amount disbursed in the data sheet is due to an unutilized balance of US$20 not refunded by the project and waived on an exceptional basis by the Bank. Page 30 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Borrower’s Completion Report for the OECS Education Support Project This summary report on the implementation of the OECS Education Support Project (OESP) was prepared by Targeted Development Consulting Inc. for the Education Development Management Unit (EDMU) at the OECS Commission, in accordance with the World Bank guidelines for Borrower Completion Reports. Context and Background The OECS developed and launched a Regional Education Strategy (OESS) with the vision that, ‘Every Learner Succeeds’, to address the common challenge of low learning achievement across Member States. The strategy was endorsed by the OECS Council of Ministers of Education at their annual meeting of May 2012, for implementation across the states. The higher-level objective of the OESS is to improve learning outcomes at the basic education level. In 2016, the OESP was approved by the Global Partnership for Education (GPE) and the World Bank (WB) was assigned as the grant agent to support the implementation of the OESS in Dominica, St Vincent and the Grenadines, St. Lucia and Grenada. Four priority areas of focus were included in the design of the OESP, namely: Curriculum and Assessment, Teacher Professional Development, Leadership and Accountability and Monitoring and Evaluation. Context of Project Objectives According to the appraisal report for the OECS Education Support Project (OESP), the OESP's focus on improving the quality of basic education is aligned with the Education Global Practice goals of "access for all and learning for all". In addition, it is aligned with two goals under the Global Partnership for Education: Goal 2 of improved and more equitable student learning outcomes through quality teaching and learning, and Goal 3 of efficient and effective education systems. Quite importantly, the OESP emerged from the OESS (2012) and as such, was designed to be responsive to the needs and priorities of Member States as captured in the strategy document. The Member States were therefore expected to pursue national actions based on a regional agenda. Thus, the need for the OESP component areas was compelling and already validated by regional stakeholders. The OECS Economic Union requires that the experience of children in one Member State should be consistent with that in another. The OESP, therefore, provided a platform for the harmonisation of standards and processes in the education systems of four OECS Member States, particularly related to three of the seven OESS strategic imperatives: School Leadership and Accountability, Teacher Professional Development and Curriculum and Assessment. Page 31 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Achievement of Outcomes (Based on Evidence) Professional Development The focus of this component was to improve Teacher Training Programmes and strengthen Teaching Practice by developing a professional development programme for primary teachers that is responsive to teaching needs within the four Member States of the OECS. Summary Achievements � 2278 teachers participated in competency-based professional development activities. � 2182 teachers have been certified by the Ministries of Education across the four Member States. � 673 teachers across the Member States were monitored using the CLASS observation tool Member State *Total number of Teachers Trained and Percentage teachers certified Dominica 560 209 37 Grenada 690 498 74 St. Lucia 1044 630 60 St. Vincent and the 970 845 87 Grenadines Total 3264 2182 67% *Based on 2017 statistics of teachers from OECS Regional Statistical Digest Curriculum and Assessment This component was designed to improve the academic performance of pupils at the primary level across member territories through the implementation and eventual adoption of a common core of quality learning standards in four specific subject areas: Language Arts, Mathematics, Science and Social Studies as well as a formative assessment framework that is designed to improve and develop a system of continuous learning assessment. Page 32 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Summary Achievements � 835 education officials across the four Member States were trained to implement the revised standards and assessment framework. � All four (4) of the Member States reported primary school teachers using formative assessment based on learning standards though in varying degrees as follows: Dominica 100% Grenada 11% St. Lucia 10% St. Vincent and the Grenadines 4% � Three of the four (4) Member States provided percentages of Educations Officers trained in the new learning standards as follows: Grenada 55% St. Lucia 63% St. Vincent and the Grenadines 92% Dominica Indicated that “Officers were not trained in the new learning standards, however, they were all sensitized about them�. Dominica explained that they already had curriculum developed for the four core subject areas. Further, Dominica’s curricular were used by the consultants in the development of the learning standards. Dominica’s task after the development of the Standards and Assessment framework was to identify the gaps and integrate into the existing curriculum. Notwithstanding, the curricular (which included the learning standards) were already being implemented by all teachers (100%). Further, it was noted that their EMIS system records results of some formative assessments including class assignment/ homework/projects (Individual and group) and quizzes. Leadership and Accountability This component focused on strengthening primary school leadership and accountability in the Member States building on previous work by the OECS in the development of leadership standards. There were three broad goals that guided this component: 1. Strengthen school leadership capacity to support teachers in improving instructional practices 2. Design, develop and deliver an improved program for training leaders in school management 3. Increase school planning activities that are focused on learning outcomes Page 33 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Summary Achievements � 110 school leaders from the four Member States were certified in competency-based standards while 125 participated in competency-based training provided by the University of the West Indies, Mona. Member State Number of School Leaders Certified Dominica 28 Grenada 27 St. Lucia 28 St. Vincent and the Grenadines 27 Total 110 � All four Member States reported on the number of primary schools with School Improvement Plans focused on learning outcomes as detailed below: Member State Number of primary schools with School improvement plans focused on learning outcomes Dominica 58 Grenada 17 St. Lucia 59 St. Vincent and the Grenadines 61 Total 195 � In terms of reporting on the progress of teachers' practices and student learning to oversight bodies the following was reported: Member State Number of School Number of School Percentage Leaders leaders reporting Dominica 58 58 100 Grenada 57 33 58 St. Lucia 79 68 86 St. Vincent and the 72 72 100 Grenadines Total 266 231 87 Page 34 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Key Factors that Affected Implementation and Outcomes Borrower Performance Consistent with the OECS mandate under the Treaty of Basseterre, the EDMU served as a facilitator for the implementation of the OESP. This role also extended to being an enabler, catalyst and oftentimes a mediator. The critical role played by the EDMU in bringing the four Member States together and providing the necessary support and hand-holding should not be understated. While it is difficult to speculate what implementation would have been with a different execution arrangement, it is clear that the EDMU’s role was pivotal in “making things happen�. Further, the message from the EDMU to the Member States was that they are operating in one economic space and as such, the benefits should allow for the same education experiences to be shared across the economic union. Factors positively Influencing Borrower’s performance:  They had strong relationships with the Member States and was able to provide the necessary guidance and direction as required  They were willing to try new ways of doing things and focused on innovative solutions Factors negatively influencing Borrower’s performance:  Receiving responses from the Member States was slow in some instances, which delayed decision making. Further, reports were sometimes not adequately prepared which caused further delays.  The slow uptake of the national ownership of the OESP. There was often the sense that what Member States were doing, they were doing it for EDMU and not themselves.  There was no dedicated project management team in the EDMU and consequently, the Head of the EDMU, Education Specialist and Administrative Assistant were expected to manage and coordinate the implementation of the OESP along with other projects and initiatives led by the unit. There was also a change in the leadership of the Head of EDMU early in the OESP lifecycle.  There were significant delays in the procurement process which pushed back the start of activities under a number of the component areas, in particular, consultancies.  OECS’s accounting systems were cumbersome and many vendors were inconvenienced with the payment of invoices. Furthermore, wire transfers came at a cost to some vendors.  Took for granted that information would filter down through the Ministries of Education (MOEs) and this did not happen. Teachers and principals, in particular, were left out of the communication loop.  OECS Commission staff had to fill major gaps concerning procurement and accounting. Given their other priorities, this made both these areas challenging to manage. BANK PERFORMANCE The World Bank effectively performed its role as the Grant Agent for the Global Partnership for Education funding to the OECS Commission. Page 35 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Factors positively influencing the Bank’s performance:  The WB team conducted regular supervision missions during which regional project updates were presented by the Member States as well as Consultants.  The WB team was often receptive to the needs of the Member States and worked with EDMU to find solutions to bottlenecks and challenges faced at both the regional and national levels.  Strong guidance was provided on procurement and other technical areas. Factors negatively Influencing the Bank’s performance:  Three WB Task Team Leaders were assigned to OESP over the lifecycle of the OESP which caused some loss in momentum and delays with decision making.  Significant delays with baseline CLASS analysis as the Bank engaged their consultant to undertake this. EDMU then requested the OESP M&E consultants to undertake the analysis given the delays experienced with the WB-engaged consultant.  Significant delays in providing the necessary approvals/no objections for the Leadership and Accountability consultancy.  Technical and management support was often lacking particularly due to delays in decision making which in turn affected implementation. EXTERNAL F ACTORS Natural disasters. Hurricane Maria devastated Dominica in September 2017 and consequently brought all implementation activities to an abrupt halt. While the Chief Education Officer (CEO) reiterated the Government's commitment to the OESP implementation, this desire was trumped by the urgent need of getting children back in school, as many of the schools were severely damaged or destroyed. Government's priorities were therefore around providing the necessary psycho-social support to children and their families and reconstruction. The impact meant that all MOE officials were focused on recovery and many of the planned activities could not be undertaken as and when scheduled. In the meantime, the Member States like St. Lucia tried to assist by accommodating displaced students from Dominica. Changes in leadership. In Grenada, there were many changes in leadership as well as focal points for the component areas. The leadership changes included a change in Minister (following the general elections), two changes in permanent secretary and three changes in the Chief Education Officer. The movement of key personnel resulted in a loss in momentum and uncertainty about the next steps. The focal points for Curriculum and Assessment, Professional Development and Monitoring and Evaluation were also changed. These personnel changes were further compounded by industrial strike action which resulted in schools closing and Ministry of Education officials focused on getting schools reopened. In both SVG and SLU there were also changes with the Chief Education Officers. In fact, in all Member States, many persons in key leadership roles are "acting" in the position and this oftentimes slowed decision making. A summary of all the changes made in the leadership of the four Education Ministries (at the level of Permanent Secretary and Chief Education Officer) is detailed: Page 36 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Dominica 2 Permanent Secretaries Grenada 3 Permanent Secretaries and 3 CEOs St. Lucia 2 Permanent Secretaries and 2 CEOs (with one still pending) St. Vincent and the Grenadines 2 Permanent Secretaries and 3 CEOs Of note, all current CEOs are acting in their position and therefore are not in their substantive position. Sustainability of Outcomes The EDMU has developed a draft sustainability plan for the region which is 80% complete. This will serve as a broader framework for countries to adapt accordingly. However, it should be pointed out that many key elements are already in place that could contribute to varying levels of sustainability being realized at the Member State level. In particular, a significant amount of technical capacity has been developed at the national level and structures have been put in place that can be built on, such as those described below:  Several trained training cadres have been established that can continue and scale up PD activities nationally.  PD teams have been established who will be responsible for coordinating national PD activities and updating national PD plans.  Assets maps have been developed which identify key resources in the MOE that can be used to support teachers in the classroom and or support PD activities.  Grenada and St. Lucia started conducting webinars on their own and have set up the relevant structures to mainstream these. OESP contributed to several changes being realised and these can be sustained through required and focused actions at the national level:  Increasing monitoring and supervision as well as the provision of technical support to principals and teachers. Given that not all teachers participated in the training on the standards, they will need to be trained as well as be provided with hand-holding/coaching to ensure they can adapt accordingly.  Undertaking ongoing and systematic monitoring by various stakeholders (principals, education officers, curriculum officers) of the skills and knowledge acquired during the Summer Institute to ensure implementation objectives are tracked and adjustments made to improve the effectiveness.  Ensuring support is provided to schools that need it and that hard copies of the learning standards are produced so that teachers can use as a resource.  Strategically deploying persons who have been trained under the OESP so they can be used across the education system.  Making participation in Teacher Professional Development compulsory so that it is a requirement for the job as a teacher and a requirement for promotion or increments.  Putting action plans and strategies in place to institutionalize initiatives.  Monitoring, recognizing successes, addressing challenges promptly and holding educators accountable for the implementation of the initiatives. Page 37 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836)  Making some of the activities into policy and then ongoing monitoring and periodic evaluation of these activities.  Ongoing promotion of the PLCs with teachers and school leaders to enable and shape the continuous learning culture at the school level.  Continuing virtual as well as face to face coaching to ensure that the model becomes part of mainstream support for teachers in the classroom. Lessons and Recommendations Several lessons were learnt by the EDMU as follows:  The implementation of a project of this nature required additional human resources particularly in the area of procurement and finance to enhance efficiency.  Broad-based stakeholder sensitisation is key to gaining stakeholder buy-in and receptivity to the project. Sensitisation only took place at mid-term and this should have happened from the outset.  Further to the above, it was assumed that the Member States would have undertaken their sensitisation; however, this did not happen as anticipated. Provision should, therefore, have been made to coordinate this rather than leave to the MOEs.  Given the significant amount of procurement that needed to happen, there should have been a pre-implementation phase which would have given the EDMU time and space to treat with administrative matters before the start of implementation.  Annual reporting at meetings of CEOs was not considered to be adequate. However, due to GPE funding regulations, it was not possible to include the non-benefitting Member States in the regional discourse.  The visibility of the project was lacking and needed to package key messages for dissemination.  The submission of work plans by each Member State was critically important to planning and budgeting for implementation and the submission of these plans should have been a requirement before commencing implementation.  Given the experiences in 2017 with hurricanes Maria and Irma, it is important to plan for risks and how best to manage if a Member State is affected by a natural hazard.  A meeting of consultants emerged organically and was not part of the original project plan. However, given the enormous value such a forum provided, this meeting should have been built into the project implementation plan.  There is a need for careful consideration in the appointment of focal points. TORs should have been developed to present a profile of who can serve in this position and serve well.  St. Lucia developed a team-based approach model to each of the components which allowed for teams to work collectively rather than one person having to carrying the component on their own. This model could have been used from the outset given the efficiencies realised from this approach. At the national level, Member States shared that they learnt the following:  There is enormous value in working in teams rather than as individuals. Teamwork leads to greater effectiveness and efficiency. Page 38 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836)  Communication is critical and should not be taken for granted. While it is often assumed that information is known or already shared, it is important to ensure that stakeholders are engaged, and information is shared continuously.  A phased approach to implementation is important to work out any early kinks and correct any errors made. This saves on the use of critical time and resources.  It is important to delegate responsibilities to competent team members rather than micro- managing. This approach leads to improved results being realized.  Teachers embrace opportunities for further growth even when it is outside of their comfort zone.  Stakeholder buy-in is necessary for effective implementation and sustainability of projects.  High levels of sensitization and participatory consultation are needed to obtain stakeholder buy- in to the process and activities being undertaken.  The effective implementation of any new initiative is heavily dependent on the quality and quantity of support and guidance provided to the implementers.  Continuous professional development is also important for school leaders rather than only teachers.  There is a dire need for additional monitoring and evaluation capacity in the Ministries of Education. Recommendations  A number of the issues with sustainability will rely heavily on the support of MOE leadership at the highest level to sustain results. Thus, MOE leaders need to ensure a coherent and coordinated approach is taken as this can’t be left to committees to make these connections. If top-level MOE leadership does not step in to ensure there are regular meetings of key stakeholders and demonstrate how it connects to their sector plan, it will be difficult to keep the change momentum going.  There is a need to encourage more teacher involvement in school development planning, implementation of standards and assessment framework and professional development. Such activities are still regarded as led by MOE leadership and in many respects top-down rather than bottom-up. Teachers need to understand each component, how they fit together in terms of an education ecosystem and how they all benefit them as teachers and, more importantly, the students. This means having them actively participating in discussions and activities.  There is a need for greater stakeholder engagement around the BIG PICTURE -- EVERY LEARNER SUCCEEDS and more particularly in the continued implementation of the changes. It is important in any change process, that stakeholders understand how what they do will impact other parts of the system. Knowing that they are contributing to something meaningful and something beyond their space can inspire greater buy-in.  A change management team, led by “education change champions", should be established to agree to any changes related to the project and to ensure that a change management lens is used in determining the best course of action. This would include EDMU and World Bank as well as other partners.  There is a need for a fully dedicated project team at the EDMU that would include a Team Leader, Education Officer, Administrative Officer, Procurement Officer and Finance Officer.  There is a need for dedicated and targeted focus at the national (micro) level, given to date, most of the focus has been on building regional structures. This means placing a microscope on each Page 39 of 40 The World Bank Support to Implementation of the Regional Education Strategy (P158836) Member State to identify the main bottlenecks and barriers and how these can be untied or resolved.  To date, EDMU has played many critical roles including coordinating, managing, brokering and driving the processes at both national and regional levels. However, going forward, EDMU should be more of a facilitator with the Member States being the ones leading and driving the process.  Regular dialogue should continue but the format for review meetings needs to be examined so the ownership is turned over to the Member States. They should lead to determine if they need a meeting and then set the agenda. Page 40 of 40