Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) MN Education Quality Reform Project (P148110) EAST ASIA AND PACIFIC | Mongolia | Education Global Practice | IBRD/IDA | Investment Project Financing | FY 2014 | Seq No: 9 | ARCHIVED on 20-Jun-2018 | ISR32357 | Implementing Agencies: Mongolia, Ministry of Education, Culture, Science and Sport Key Dates Key Project Dates Bank Approval Date:06-Jun-2014 Effectiveness Date:17-Aug-2015 Planned Mid Term Review Date:08-May-2018 Actual Mid-Term Review Date:15-May-2018 Original Closing Date:31-Dec-2019 Revised Closing Date:31-Dec-2020 Project Development Objectives Project Development Objective (from Project Appraisal Document) The Project Development Objectives are to improve the quality of education for primary school children in Mongolia, with particular emphasis on improving native language and mathematical skills, and strengthen school-level planning. Has the Project Development Objective been changed since Board Approval of the Project Objective? No PHRPDODEL Components Name Improving learning outcomes:(Cost $14.90 M) Pre- and in-service professional development of teachers:(Cost $3.30 M) Implementation of a school support program:(Cost $6.10 M) System management, monitoring and evaluation:(Cost $5.70 M) Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO  Moderately Satisfactory  Moderately Satisfactory Overall Implementation Progress (IP)  Moderately Satisfactory  Moderately Satisfactory Overall Risk Rating  Moderate  Moderate 6/20/2018 Page 1 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) Implementation Status and Key Decisions Under component 1, a significant achievement to date was the completion of preliminary analysis of data collected under Mongolia’s first early grade reading and math assessments in a nationally representative sample of grade 1 and 2 students. Findings from these assessments fed into development of a follow-up Mongolian language reading intervention by the Mongolian Institute for Educational Research (MIER), including design of teacher materials and aids, in-service teacher training to address learning issues revealed by the early grade assessments and design and use of formative assessments to diagnose learning issues, monitor student progress, and implement corrective strategies. The interventions are on track to being rolled out in September 2018. Equivalent materials will also be developed for faculty at pre-service teacher training institutes. Under Component 2, the review of the Primary Teacher Pre-Service Education Program of the Mongolian National University for Education (MNUE) was completed in December 2017. A brainstorming workshop with MNUE is planned for late June 2018 to explore the feasibility of implementing the recommendations. In May 2018, training was offered under the component to aimag and district-level education specialists in the methodology and use of formative assessments. Under component 3, the third annual cycle of school grants currently being implemented in 450 schools is near completion, and schools are preparing their final reports for submission by end of June 2018. Meanwhile, a third-party assessment found that 52 percent of grant proposals implemented during the school year 2016-17 reached their grant’s development objectives, indicating room for improvement in order to achieve the project's end- target value of 70 percent. Overall, the recently-completed Mid-Term Review for the Project found that the school grant activities in the first three years of EQRP have made a large contribution to the project’s PDO of strengthening school-level planning as a result of the experience which school-level stakeholders acquired through the annual grant proposal preparation and implementation processes, under which teams typically consisting of teachers, the school training manager, and the principal, identify and agree on the key school-level issues to be addressed under the proposal, jointly develop proposal objectives and targets expressed in observable terms, and design, cost, and implement the activities that will help meet the grant objectives within a one-year time frame. A third-party assessment to evaluate performance of the ongoing cycle-3 grants is planned for the first semester of 2019. Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance  --  Moderate  Moderate Macroeconomic  --  Low  Low Sector Strategies and Policies  --  Moderate  Moderate Technical Design of Project or Program  --  Low  Low Institutional Capacity for Implementation and Sustainability  --  Substantial  Substantial Fiduciary  --  Low  Low Environment and Social  --  Low  Low Stakeholders  --  Low  Low Other  --  --  -- Overall  --  Moderate  Moderate Results 6/20/2018 Page 2 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) Project Development Objective Indicators PHINDPDOTBL  The average number of words read correctly with comprehension per minute at end of grade 2. (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 24.70 0.00 24.70 29.64 Date 01-Jul-2014 24-Nov-2015 19-May-2017 31-Dec-2019 PHINDPDOTBL  At the end of grade 2, the average number of correct additions and subtractions per minute. (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 12.72 12.72 12.72 13.99 Date 19-May-2017 19-May-2017 19-May-2017 31-Dec-2019 PHINDPDOTBL  Of the total number of school financing recipient schools in the project, the percentage that have successfully achieved the school proposal objectives and targets agreed in the school financing (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 66.70 52.00 70.00 Date 01-Jul-2014 15-May-2017 18-May-2018 31-Dec-2019 Comments The team is not surprised to see a decline in this indicator value. The first indicator value was based on a random sample of schools drawn from the 200 that participated in the first annual cycle of school grants (C1) under EQRP. The updated value this year is based on a sample drawn from the 450 that participated in the second cycle of school grants (C2). As documented in EQRP aide-memoires, C2 was different from C1. While C1 was competitive, C2 grant awards were treated as entitlements –all schools that prepared proposals, regardless of the relevance or quality of proposals, received grants (a few were asked to revise their proposals based on feedback from the review and selection committee). The decision to proceed on a non-competitive basis undermined the incentive to produce high-quality proposals. This shortcoming of the award process was discussed extensively with the client in every mission since September 2016, and recommendations were offered on how to avoid the same situation in the third and ongoing cycle of school grants, C3, through improved quality assurance mechanisms and re-introduction of competitive selection. Recent missions have confirmed that the client is complying with the team’s recommendations. We therefore expect to see an improvement in this indicator value at its next update in a year from now. Overall Comments 6/20/2018 Page 3 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) Intermediate Results Indicators PHINDIRITBL  Percentage of grade 2 teachers reporting native language assessments results (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value To be collected in Y1 0.00 0.00 Baseline +80% Date 01-Jul-2014 24-Nov-2015 24-Nov-2015 31-Dec-2019 Comments This indicator value can be updated once the EGRA follow-up intervention is rolled out in September, 2018. PHINDIRITBL  Percentage of grade 2 teachers reporting assessment results on: (a) additions and subtractions; and (b) identification of missing numbers in a sequence (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value To be collected in Y2 0.00 0.00 Baseline +60% Date 01-Jul-2014 24-Nov-2015 24-Nov-2015 31-Dec-2019 Comments This indicator value can be updated once the EGMA follow-up intervention is rolled out in September, 2019. PHINDIRITBL  Of the total number of grades 1 & 2 primary school teachers to be trained during the project life, the cumulative % that achieved intended competency levels in native language assessment methodology (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 60.00 Date 01-Jul-2014 24-Nov-2015 24-Nov-2015 31-Dec-2019 6/20/2018 Page 4 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) Comments This indicator value can be updated once the EGRA follow-up intervention is rolled out in September, 2018. PHINDIRITBL  Of the total no. of grades 1 & 2 primary school teachers to be trained during the project life, the cumulative % that achieved intended competency levels in early grade mathematical skills assessment (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 60.00 Date 01-Jul-2014 24-Nov-2015 24-Nov-2015 31-Dec-2019 Comments This indicator value can be updated once the EGMA follow-up intervention is rolled out in September, 2019. PHINDIRITBL  Teachers recruited or trained (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 4,897.00 4,897.00 7,400.00 Date 01-Jul-2014 19-May-2017 19-May-2017 31-Dec-2019 PHINDIRITBL  Of the total number of school financing recipient schools in the project, the percentage that use the proceeds for pedagogical- related activities (as defined in the School Support Handbook) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 40.00 30.00 Date 01-Jul-2014 24-Nov-2015 18-May-2018 31-Dec-2019 6/20/2018 Page 5 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) PHINDIRITBL  Roll out of grade 5 National Assessment (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N N N Y Date 01-Jul-2014 24-Nov-2015 24-Nov-2015 31-Dec-2019 PHINDIRITBL  Average percentage of the school year allocated for instruction in the classroom. (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value To be collected in Y1 0.00 0.00 Baseline + 6% Date 01-Jul-2014 24-Nov-2015 24-Nov-2015 31-Dec-2019 PHINDIRITBL  System for learning assessment at the primary level (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N Y Y Y Date 01-Jul-2014 24-Nov-2015 11-Jun-2018 31-Dec-2019 PHINDIRITBL  Utility of the learning assessment system (Number, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 1.00 1.00 1.00 2.00 6/20/2018 Page 6 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) PHINDIRITBL  Direct project beneficiaries (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 57,557.00 -- 44,000.00 Date 01-Jul-2014 19-May-2017 19-May-2017 31-Dec-2019 PHINDIRITBL  Female beneficiaries (Percentage, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 52.90 -- 52.00 PHINDIRITBL  Students benefiting from direct interventions to enhance learning (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 52,660.00 58,915.00 37,000.00 Date 01-Jul-2014 19-May-2017 31-May-2018 31-Dec-2019 PHINDIRITBL  Students benefiting from direct interventions to enhance learning - Female (Number, Corporate Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 25,781.00 28,868.00 19,240.00 Overall Comments 6/20/2018 Page 7 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed Disbursed P148110 IDA-54880 Effective USD 30.00 30.00 0.00 9.27 18.06 34% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P148110 IDA-54880 Effective 06-Jun-2014 18-Jun-2015 17-Aug-2015 31-Dec-2019 31-Dec-2020 Cumulative Disbursements Restructuring History Level 2 Approved on 11-Jun-2015 Related Project(s) 6/20/2018 Page 8 of 9 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report MN Education Quality Reform Project (P148110) There are no related projects. 6/20/2018 Page 9 of 9 Public Disclosure Copy