The World Bank Implementation Status & Results Report Education Modernization Project (P153496) Education Modernization Project (P153496) EUROPE AND CENTRAL ASIA | Kazakhstan | Education Global Practice | IBRD/IDA | Investment Project Financing | FY 2017 | Seq No: 10 | ARCHIVED on 26-May-2021 | ISR46703 | Implementing Agencies: Ministry of Education and Science, Republic of Kazakhstan Key Dates Key Project Dates Bank Approval Date: 01-Mar-2017 Effectiveness Date: 30-Jun-2018 Planned Mid Term Review Date: 14-Apr-2021 Actual Mid-Term Review Date: 10-Mar-2021 Original Closing Date: 31-Dec-2022 Revised Closing Date: 31-Dec-2022 pdoTable Project Development Objectives Project Development Objective (from Project Appraisal Document) The project development objective (PDO) is to improve quality and equity in primary and secondary education, particularly in rural and disadvantaged schools Has the Project Development Objective been changed since Board Approval of the Project Objective? No Components Table Name Supporting system-wide improvement in primary and secondary:(Cost $9.72 M) Supporting rural and disadvantaged schools to reduce disparities in learning results:(Cost $62.04 M) Supporting citizen engagement, monitoring and evaluation, and implementation:(Cost $5.24 M) Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO Moderately Unsatisfactory Moderately Unsatisfactory Overall Implementation Progress (IP) Moderately Unsatisfactory Moderately Unsatisfactory Overall Risk Rating Substantial Substantial Implementation Status and Key Decisions A mid-term review of the project was carried out from March 10-17, 2021 to assess progress towards achievement of the Project Development Objectives (PDO), review the relevance of project activities, and realign activities to ensure they are relevant, achievable by the project closing date, and sustainable after closing. Based on the mid-term review outcomes, the Government of Kazakhstan and the World Bank agreed to move forward with a partial cancellation of the loan, and a realignment of activities to ensure the PDO is achievable within the remaining timeline. 5/26/2021 Page 1 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance Moderate Moderate Moderate Macroeconomic Substantial Substantial Substantial Sector Strategies and Policies Moderate Moderate Moderate Technical Design of Project or Program Substantial Substantial Substantial Institutional Capacity for Implementation and Substantial Substantial Substantial Sustainability Fiduciary Substantial Substantial Substantial Environment and Social Low Low Low Stakeholders Substantial Substantial Substantial Other -- Moderate Moderate Overall Substantial Substantial Substantial Results PDO Indicators by Objectives / Outcomes The project development objective (PDO) is to improve quality and equity in primary and secondary ed IN00992897 ►Percentage of teachers in project schools who demonstrate improved pedagogical approaches (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 60.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This PDO indicator will not be achieved with the proposed cancellation and will be dropped as part of the Comments: cancellation / restructuring IN00992898 ►Percentage of students from project schools benefiting from instruction materials and multimedia equipment (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 100.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This PDO indicator will not be achieved with the proposed cancellation and will be dropped as part of the Comments: cancellation / restructuring 5/26/2021 Page 2 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) IN00992899 ►Grade 4 EALA (revised) scores in project schools improved, by Reading and Mathematics: boys/girls (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target By subjects: a statistically significant To be measured in year increase in scores as 3 when the assessment compared to baseline Value tool (i.e. EALA) is revised No No in line with the new Boys/girls: reduction in standards. disparity for reading only Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This PDO indicator will not be achieved with the proposed cancellation and will be dropped as part of the Comments: cancellation / restructuring Intermediate Results Indicators by Components Supporting system-wide improvement in primary and secondary IN00992900 ►Number of students enrolled in the new model of pre-service teacher training (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 280.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 The consultancy contract to develop the new program was signed on 22-Oct-2020. Pilot of the new pre- service teacher training model is planned to be completed by May, 2022. New cohort of students will be Comments: enrolling in the new program in September, 2022. IN00992903 ►Assessment instruments are designed in line with the new standards (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target EALA: No EALA: Yes Value No No UNT: No UNT: Yes Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 The consultancy contract to develop the new standards was signed on 17-Nov-2020. Piloting is planned to Comments: start in October 2021 and to be piloted in June 2022. Reviewing is still in process. IN00992906 ►Percentage of textbooks that are appraised by National Textbooks Center using revised quality assurance system (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 100.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 5/26/2021 Page 3 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) The Consultant submitted the second report to MOES on February 28, 2021. MOES reviewed the report and suggested some revisions. The revised report was submitted in April and is currently under MOES Comments: revision. The contract completion date is November 17, 2021. IN00992909 ►Evaluation of the new curriculum pilot completed (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value No No No Yes Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 The Consultant submitted the initial report on April 8, 2021. The report encompasses review of the current documents and procedures of preparing and conducting tests, review of international practices, assessment and monitoring of academic achievements, and recommendations for improving the Comments: assessment system. Training and developing test items are planned for August 2021. The contract completion date is December 31, 2022. IN00992911 ►Percentage of project school administrators who exercise pedagogical leadership, by gender (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 90.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This indicator will not be achieved with the proposed cancellation and will be dropped as part of the Comments: cancellation / restructuring IN00992912 ►Number of project schools equipped with new multimedia equipment (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 5,400.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This IRI will not be achieved with the proposed cancellation and will be dropped as part of the cancellation Comments: / restructuring IN00992913 ►Students benefiting from direct interventions to enhance learning (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 3,014.00 3,014.00 2,600,000.00 Date 21-Dec-2020 12-Jan-2021 21-May-2021 31-Dec-2022 Current value of beneficiaries reached reflects the number of students benefiting from robolabs installed at pedagogical universities which was completed in December, 2019, under Component 1. Comments: The end target reflects the total number of beneficiaries (including secondary school students and students at pedagogical universities) who will benefit from system-level interventions under Component 1. IN00992914 Students benefiting from direct interventions to enhance learning - Female (Number, Corporate Supplement) 5/26/2021 Page 4 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 2,351.00 2,351.00 1,300,000.00 IN00992915 ►Large-scale primary/secondary learning assessments completed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 2.00 Date 22-Dec-2020 12-Jan-2021 21-May-2021 31-Dec-2022 The number of project-supported completed large-scale assessment rounds at the primary or secondary levels. Comments: *This intermediate indicator will not be achieved with the proposed cancellation and will be dropped as part of the cancellation / restructuring. Supporting rural and disadvantaged schools to reduce disparities in learning results IN00992901 ►Percentage of teachers and administrators of project schools received support post training (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 90.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This indicator is likely to be dropped as the proposed training will be dropped in the restructuring Comments: proposal. IN00992904 ►Percentage of teachers from project schools trained and certified in new pedagogical approaches, by gender (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 100.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This indicator is likely to be dropped as the proposed training will be dropped in the restructuring proposal. Comments: IN00992907 ►Number of teachers who pass the proficiency test in English after training (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 450.00 Date 30-Jun-2018 12-Jan-2021 21-May-2021 31-Dec-2022 This indicator is likely to be dropped as the proposed training will be dropped in the restructuring Comments: proposal. 5/26/2021 Page 5 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) Supporting citizen engagement, monitoring and evaluation, and implementation IN00992902 ►Percentage of project schools in which parents report an improvement in the openness and responsiveness of school teachers and management (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 75.00 Date 30-Jun-2018 24-Mar-2021 21-May-2021 31-Dec-2022 Since May 2020, the COVID context has severely affected the activities focused on improving openness and responsiveness of school teachers and management. However the Ministry of Education and Science conducted a citizen engagement survey (for beneficiaries) online through regional department with the following findings (4036 respondents from 14 oblasts and two major cities) :  58% of total schools in the country have responded to the questionnaire.  98% reported that Boards of Trustees have convened on a periodic basis to discuss and provide feedback twice per month. All schools have trust-boxes and questionnaires and separate parents committees. Comments:  96% of respondents reported that they discuss key issues on parent committees.  94% of respondents confirmed their school have a mechanism to track feedback  66% of the school communities provide feedback in written form; 31% through open discussions and meetings; 3 percent through other sources (whatsap, trust-boxes etc.)  General assessment of openness and responsiveness - 40% very good, 58% good, 2% have found it difficult to answer.  The actual (current) result will be updated following MTR. IN00992905 ►Percentage of schools that formulate School Improvement Plans (SIPs) reflecting feedback provided by parents and beneficiaries in project schools (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 75.00 Date 30-Jun-2018 24-Mar-2021 21-May-2021 31-Dec-2022 Given the COVID context and the need to monitor school performance under challenging circumstances, , the engagement on School Improvement Plans was replaced with a Distance Learning Survey. The Distance Learning survey was launched and covered 12,000 students and 6,000 teachers in survey (10% of schools). The task covered following:  Student participation in different modalities of the emergency remote teaching;  Student completion of courses within different modalities of emergency remote teaching;  Learning outcomes on End of Quarter 4 assessment linked to the assessment conducted before Comments: schools closed for COVID-19. Monitoring was conducted through following tools: Kundelik, Bilimal, Imektep, MektepEDU, BilimLand, Daryn. Online; National Educational Database; the viewership of video lessons on ElArna, Qazaqstan and Balapan TV channels. Currently, final results are being discussed by the Ministry of Education and Science prior to disclosure. This indicator will be considered for revision during the Restructuring. IN00992908 ►Third party and/or beneficiary monitoring findings disclosed, discussed at open forum and agreed actions included in national action plans (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Actions included in 2 national action plans Value 0.00 0.00 0.00 and implemented in subsequent year 5/26/2021 Page 6 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) Date 30-Jun-2018 24-Mar-2021 21-May-2021 31-Dec-2022 The contract for the third party and/or beneficiary monitoring stalled during COVID-19. There is currently a discussion as to the importance of independent monitoring with the PIU, and the work involved; a way Comments: forward will be developed in the coming months during mid-term review and following restructuring. IN00992910 ►Number of regions that: (i) publicly display and discuss school budget in meetings with citizens, and (ii) publicly report on meeting discussions and actions to be taken (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 16.00 Date 30-Jun-2018 24-Mar-2021 21-May-2021 31-Dec-2022 Budget disclosure is reported on www.openbudget.kz. The results of the Distance Learning Review are Comments: within Ministry's quality review. Performance-Based Conditions Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed % Disbursed P153496 IBRD-87090 Effective USD 67.00 67.00 0.00 1.79 65.21 2.7% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P153496 IBRD-87090 Effective 01-Mar-2017 21-Aug-2017 30-Jun-2018 31-Dec-2022 31-Dec-2022 Cumulative Disbursements 5/26/2021 Page 7 of 8 The World Bank Implementation Status & Results Report Education Modernization Project (P153496) PBC Disbursement Achievement Disbursed amount in Disbursement % PBC ID PBC Type Description Coc PBC Amount Status Coc for PBC Restructuring History There has been no restructuring to date. Related Project(s) There are no related projects. 5/26/2021 Page 8 of 8