The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) Mongolia Education Quality Reform Project (P148110) EAST ASIA AND PACIFIC | Mongolia | Education Global Practice | IBRD/IDA | Investment Project Financing | FY 2014 | Seq No: 17 | ARCHIVED on 29-Jun-2022 | ISR52077 | Implementing Agencies: Ministry of Education and Science, Ministry of Labor and Social Protection, Mongolia Key Dates Key Project Dates Bank Approval Date: 06-Jun-2014 Effectiveness Date: 17-Aug-2015 Planned Mid Term Review Date: 08-May-2018 Actual Mid-Term Review Date: 15-May-2018 Original Closing Date: 31-Dec-2019 Revised Closing Date: 31-Dec-2022 pdoTable Project Development Objectives Project Development Objective (from Project Appraisal Document) The Project Development Objectives are to improve the quality of education for primary school children in Mongolia, with particular emphasis on improving native language and mathematical skills, and strengthen school-level planning. Has the Project Development Objective been changed since Board Approval of the Project Objective? Yes Board Approved Revised Project Development Objective (If project is formally restructured) The Project Development Objectives are to improve the quality of education for primary school children in Mongolia, with particular emphasis on improving native language and mathematical skills, strengthen school-level planning and increase support for child welfare transfers during the COVID-19 emergency. Components Table Name Support for child welfare during the COVID-19 emergency:(Cost $5.00 M) Improving learning outcomes:(Cost $11.40 M) Pre- and in-service professional development of teachers:(Cost $3.10 M) Implementation of a school support program:(Cost $8.60 M) System management, monitoring and evaluation:(Cost $1.90 M) Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO Moderately Unsatisfactory Moderately Satisfactory Overall Implementation Progress (IP) Moderately Satisfactory Moderately Satisfactory Overall Risk Rating Moderate Moderate Implementation Status and Key Decisions The Education Quality Reform Project (EQRP) has disbursed 99% of the US$30 million IDA funding. With the reopening of schools in September 2021 and the gradual relaxation of COVID-19 restrictions, the project picked up its pace in implementing the planned activities crucial to achieving 6/29/2022 Page 1 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) the PDO. The Education Evaluation Center (EEC)—which was commissioned by the Ministry of Education and Science (MES) to develop diagnostic tools to assess learning lags of grade 2–12 students and school readiness of new primary school entrants—administered the diagnostic and school readiness assessments in September–October 2021. The workbooks developed for select subjects have been used to accelerate catch- up learning for students according to their current learning levels. Follow-up online Early Grade Mathematics Assessment (EGMA) training was provided to primary school teachers between September 2021 and January 2022. Over 5,600 teachers have successfully completed the training and received online certificates, and 70% of these teachers improved their knowledge of EGMA methodology skills based on the pre- and post-training rest results. The project is supporting Mongolia’s first participation in the Programme for International Student Assessment (PISA) main study, which has been completed in May 2022. Inclusive education resource rooms equipped with furniture, toys, learning materials, and assistive tools and devices for children with special needs have been set up in each of the 21 aimags and have been fully operating since February 2022. As of June 2022, of the 692 proposals submitted by primary schools, local education and science departments, and parent teacher associations for the fifth cycle of school grants, 630 have been selected to implement activities aimed at addressing learning lags with an integration of education technology. Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance -- Moderate Moderate Macroeconomic -- Moderate Moderate Sector Strategies and Policies -- Moderate Moderate Technical Design of Project or Program -- Moderate Moderate Institutional Capacity for Implementation and -- Moderate Moderate Sustainability Fiduciary -- Low Low Environment and Social -- Low Low Stakeholders -- Low Low Other -- Substantial Substantial Overall -- Moderate Moderate Results PDO Indicators by Objectives / Outcomes Improved learning outcomes IN01159717 ►The average number of words read correctly per minute at end of grade 2 (oral reading fluency). (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 44.00 46.00 46.00 50.00 6/29/2022 Page 2 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) Date 01-Jul-2014 30-Jun-2021 21-Jun-2022 30-Dec-2022 This indicator will only be updated by endline assessment. Comments: IN01159719 The average number of words read correctly per minute at end of Grade 2 by Males (oral reading fluency) (Text, Custom Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 40.00 42.00 42.00 48.00 Date 19-May-2017 30-Jun-2021 21-Jun-2022 30-Dec-2022 IN01159720 The average number of word read correctly per minute at the end of Grade 2 by Females (oral reading fluency) (Text, Custom Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 47.00 50.00 50.00 52.00 Date 19-May-2017 30-Jun-2021 21-Jun-2022 30-Dec-2022 IN01159721 ►The average number of correct additions per minute at the end of grade 2. (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 14.00 14.00 14.00 16.00 Date 19-May-2017 30-Jun-2021 21-Jun-2022 30-Dec-2022 This indicator will only be updated by endline assessment. Comments: IN01159722 The average number of correct additions per minute at end of Grade 2 for Males (Text, Custom Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 14.00 14.00 14.00 16.00 Date 19-May-2017 30-Jun-2021 21-Jun-2022 30-Dec-2022 IN01159723 The average number of correct additions per minute at end of Grade 2 for Females (Text, Custom Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 14.00 14.00 14.00 16.00 Date 19-May-2017 30-Jun-2021 21-Jun-2022 30-Dec-2022 Strengthened school-level planning IN01159718 ►Of the total number of school financing recipient schools in the project, the percentage that have successfully achieved the school proposal objectives and targets agreed in the school financing (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target 6/29/2022 Page 3 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) Value 0.00 55.00 55.00 70.00 Date 01-Jul-2014 30-Jun-2021 21-Jun-2022 30-Dec-2022 The draft third party monitoring report of the School Grants Cycle IV was received in May 2021. Even with the COVID disruptions, the results were remarkable, just missing the annual target by 5 percentage points. Comments: This indicator will be updated when the third-party monitoring of Cycle V School Grants is conducted in September 2022. Support to child welfare funds in response to COVID-19 emergency IN01159716 ►Social Welfare Fund top up allocated to children during COVID-19 emergency (Amount(USD), Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 5,000,000.00 5,000,000.00 5,000,000.00 Date 29-Apr-2020 25-Sep-2020 21-Jun-2022 30-Dec-2022 This indicator has been achieved. Comments: Intermediate Results Indicators by Components Improving learning outcomes IN01159724 ►Number of schools benefiting from full set of reading books available in the classrooms (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 553.00 553.00 550.00 Date 30-Dec-2014 30-Jun-2021 21-Jun-2022 30-Dec-2022 Kazakh language books delivered to additional 42 schools in Bayan-Ulgii. Comments: IN01159734 ►Mongolia participation in PISA 2022 assessment (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value No Yes Yes Yes Date 31-Dec-2018 10-Dec-2020 21-Jun-2022 30-Dec-2022 Mongolia is participating in the OECD PISA assessment that has been postponed to April-May 2022. During this reporting period, the PISA National Center team of the Education Evaluation Comments: Center successfully conducted the computer-based main survey in 195 schools between April 11 and May 13, 2022. IN01159737 6/29/2022 Page 4 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) ►Number of schools received distance learning materials for distribution during COVID-19 emergency (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 300.00 Date 30-Apr-2020 30-Jun-2021 21-Jun-2022 30-Dec-2022 The number of schools whose Cycle V school grants were allocated to supporting the production of distance learning materials (e.g., audio and printed materials) will be employed for measuring the end Comments: result of this intermediate results indicator. Corroborating evidence will be provided as part of the final implementation reports to be submitted by the schools. IN01159738 ►Number of inclusive education resource rooms available (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 22.00 22.00 22.00 Date 29-May-2020 30-Nov-2021 21-Jun-2022 30-Dec-2022 A total of 21 inclusive education resource rooms have been fully set up at all the 21 aimag center schools by November 2021 and started operating since the schools resumed for the second trimester on February 7, 2022 after the winter and Lunar New Year holidays. One remaining resource room was also established Comments: at the Mongolian National University of Education in Ulaanbaatar with financial support from the Asian Development Bank under its ongoing project focusing on teacher training. IN01159739 ►Large-scale primary/secondary learning assessments completed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 4.00 5.00 4.00 Date 01-Jan-2015 20-Oct-2021 21-Jun-2022 30-Dec-2022 Corporate Result Indicator added as per Bank requirements 2021. This indicator would reflect number of large-scale primary assessments supported by the project. Cumulative Assessments supported by the project between 2018 and 2021. The grade 4 national assessment for 2020 was postponed because of COVID-19 school closures. The grade 5 national assessment (which was initially scheduled for May 2021) was also ultimately withdrawn by the MES to prioritize the administration of the school readiness and diagnostic assessments in 2021*. According to the MES, the national assessment is currently under revision to improve its capacity to assess students’ application of knowledge (hence, more aligned with PISA). Once the revision is complete, the national Comments: assessment will be conducted annually in October. *The first round of the national diagnostic assessment was administered to grades 2–12 in September 2021 and the school readiness assessment was piloted for new primary school entrants between September 20 and October 20, 2021. Both assessments were conducted on a sample basis (10 percent of grade 2–12 students for the diagnostic assessment and 30 percent of grade 1 students for the school readiness assessment) (It should be noted that the EQRP supported grades 1–5 only.) The second round of the national assessment was carried out during the last two weeks of May 2022, with a larger sample size of students across grades 1–5 in 9 subjects at 654 schools. Pre- and in-service professional development of teachers IN01159725 ►Percentage of grade 2 teachers reporting native language assessments results (Percentage, Custom) 6/29/2022 Page 5 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 90.00 90.00 93.00 Date 01-Jul-2014 30-Jun-2021 21-Jun-2022 30-Dec-2022 Due to the pause of processing contracts while resolving the issue with a new Minister of Education’s Order—which resulted in the delay in the contracting of the endline survey firm and the postponement of Comments: the endline data collection from June to September 2022—this indicator will be updated in Fall 2022. IN01159728 ►Percentage of grade 2 teachers reporting assessment results on: (a) additions and subtractions; and (b) identification of missing numbers in a sequence (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 86.00 86.00 89.00 Date 01-Jul-2014 30-Jun-2021 21-Jun-2022 30-Dec-2022 Due to the pause of processing contracts while resolving the issue with a new Minister of Education’s Order—which resulted in the delay in the contracting of the endline survey firm and the postponement of Comments: the endline data collection from June to September 2022—this indicator will be updated in Fall 2022. IN01159731 ►Of the total number of primary school teachers to be trained during the project life, the average % that improved their knowledge in native language assessment methodology (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 58.00 58.00 65.00 Date 01-Jul-2014 29-Nov-2019 21-Jun-2022 30-Dec-2022 Due to the delays in the development of EGRA online training modules and subsequent delivery of e- Comments: traing, this indicator will be updated in Fall 2022. IN01159732 ►Of the total number primary school teachers to be trained during the project life, the average % that improved their knowledge in early grade mathematical skills assessment (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 70.00 70.00 65.00 Date 01-Jul-2014 28-Feb-2022 21-Jun-2022 30-Dec-2022 IN01159735 ►Average percentage of the school year allocated for instruction in the classroom. (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 69.00 69.00 69.00 75.00 Date 19-May-2017 30-Jun-2021 21-Jun-2022 30-Dec-2022 This indicator will only be updated by an end line assessment. Comments: Implementation of a school support program 6/29/2022 Page 6 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) IN01159726 ►The percentage of schools under the project, used school grant funding for pedagogical-related activities (as defined in the School Support Handbook) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 33.00 33.00 60.00 Date 01-Jul-2014 30-Jun-2021 21-Jun-2022 30-Dec-2022 Due to the pause of processing contracts while resolving the issue with a new Minister of Education’s Order—which resulted in the delay in conducting the third-party monitoring of Cycle V School Grants—this Comments: indicator will be updated in Fall 2022. IN01159729 ►Beneficiary satisfaction (through survey) with school grant supported activities (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 70.00 Date 30-Jun-2021 30-Jun-2021 21-Jun-2022 30-Dec-2022 Feedback from stakeholders especially related to the impact of school grants and the learning lags activities would be important for the sustainability of the EQRP programs, understanding the scale and Comments: depth of learning improvements at the school/community level and capturing important recommendations from parents and teachers on school grant efforts. System management, monitoring and evaluation IN01159727 ►Direct project beneficiaries (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 1,132,164.00 1,132,164.00 520,000.00 Date 01-Jul-2014 28-Feb-2022 21-Jun-2022 30-Dec-2022 Now that the project is being implemented nationally, the number of beneficiaries includes all primary students and teachers as well as school administrators, local education specialists, and education policymakers. Comments: However, the calculation method adopted by the Project Implementation Unit (PIU) suggests that there is an issue of double-counting (some of the beneficiaries were counted more than once). IN01159730 Female beneficiaries (Percentage, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 51.00 51.00 52.00 IN01159733 ►Students benefiting from direct interventions to enhance learning (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 1,084,499.00 1,084,499.00 506,000.00 6/29/2022 Page 7 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) Date 01-Jul-2014 28-Feb-2022 21-Jun-2022 30-Dec-2022 Number of primary students However, the calculation method adopted by the Project Implementation Unit (PIU) suggests that there is Comments: an issue of double-counting (some of the students were counted more than once). IN01159736 Students benefiting from direct interventions to enhance learning - Female (Number, Corporate Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 531,442.00 531,442.00 240,000.00 Performance-Based Conditions IN01159740 ►PBC 1 Improved learning outcomes (Text, Output, 13,500,000.00, 40.74%) Baseline Actual (Previous) Actual (Current) Dropped Full set of new reading Full set of new reading Value No actions completed books available in 80% of books available in 80% of -- schools in country schools in country Date -- 28-Jun-2019 28-Jun-2019 -- IN01159741 ►PBC 2 Implementation of a school grants program (Text, Output, 6,100,000.00, 67.21%) Baseline Actual (Previous) Actual (Current) Dropped First batch of schools First batch of schools (200) have completed (200) have completed implementation of their implementation of their school proposals. At school proposals. At least least 10% of the first School support 10% of the first batch of batch of schools have completion reports Value No actions completed schools have satisfactorily satisfactorily completed have been completed implementation implementation of their submitted to MES of their school proposal. school proposal. Second Second batch of schools batch of schools (450) (450) are implementing are implementing their their school proposals. school proposals. Date -- 11-Jun-2018 11-Jun-2018 -- Data on Financial Performance Disbursements (by loan) 6/29/2022 Page 8 of 9 The World Bank Implementation Status & Results Report Mongolia Education Quality Reform Project (P148110) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed % Disbursed P148110 IDA-54880 Effective USD 30.00 30.00 0.00 26.82 0.25 99% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P148110 IDA-54880 Effective 06-Jun-2014 18-Jun-2015 17-Aug-2015 31-Dec-2019 31-Dec-2022 Cumulative Disbursements Restructuring History Level 2 Approved on 11-Jun-2015 ,Level 2 Approved on 07-Jun-2020 Related Project(s) There are no related projects. 6/29/2022 Page 9 of 9