Document of The World Bank FOR OFFICIAL USE ONLY Report No: IMPLEMENTATION COMPLETION AND RESULTS REPORT ON A SMALL GRANT IN THE AMOUNT OF USD(XX) MILLION TO THE Adrian Lupușor FOR Empowered citizens enhancing accountability of Edu reform and quality (P147607) {Date} Education Global Practice Europe And Central Asia Region Note to Task Teams: 1. For date, enter the date the ICR is submitted to SECPO. Please delete this note when finalizing the document. Regional Vice President: Cyril E Muller Country Director: Satu Kristiina J. Kahkonen Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager: Harry Anthony Patrinos Task Team Leader(s): Lucia Casap ICR Main Contributor: Alina Sava ABBREVIATIONS AND ACRONYMS TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME ...................................................................................................................... 4 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME .................................. 5 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .... 5 V. LESSONS LEARNED AND RECOMMENDATIONS ................................................................ 6 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ............................................................. 7 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 14 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 15 ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Note to Task Teams: The following sections are system generated and can only be edited online in the Portal. Please delete this note when finalizing the document. DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Empowered citizens enhancing accountability of Edu P147607 reform and quality Country Financing Instrument Moldova Investment Project Financing Original EA Category Revised EA Category Not Required (C) Organizations Borrower Implementing Agency Adrian Lupușor Expert Grup Project Development Objective (PDO) Original PDO The objective of the Project is to empower the Member Country’s civil society to: (i) engage local, regional and national authorities in evidence-based policy and budget dialogue in the Member Country’s education sector; and (ii) to promote an enabling environment for social accountability. Page 1 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) FINANCING FINANCE_T BL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-15859 696,955 696,955 686,153 Total 696,955 696,955 686,153 Total Project Cost 696,955 696,955 686,153 KEY DATES Approval Effectiveness Original Closing Actual Closing 16-Dec-2013 20-Dec-2013 20-Dec-2018 20-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 16-Mar-2014 Satisfactory Satisfactory 0.21 02 04-Nov-2014 Satisfactory Satisfactory 0.21 03 18-Sep-2015 Satisfactory Satisfactory 0.34 04 10-Apr-2016 Satisfactory Satisfactory 0.34 05 19-Jun-2017 Satisfactory Satisfactory 0.49 06 27-Aug-2018 Satisfactory Satisfactory 0.63 Page 2 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) 07 17-May-2019 Satisfactory Satisfactory 0.69 ADM STAFF Role At Approval At ICR Regional Vice President: Laura Tuck Cyril E Muller Country Director: Qimiao Fan Satu Kristiina J. Kahkonen Senior Global Practice Director: Ana L. Revenga Jaime Saavedra Chanduvi Practice Manager: Alberto Rodriguez Harry Anthony Patrinos Task Team Leader(s): Josef S. Trommer, Anna Olefir Lucia Casap ICR Contributing Author: Alina Sava Note to Task Teams: End of system generated content, document is editable from here. Please delete this note when finalizing the document. Page 3 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Note to Task Teams: This section provides a succinct, coherent, and clear story line and narrative description of the operation’s context and rationale, its objectives or key outcomes, and how these outcomes were envisioned to be achieved. This section also documents, if applicable, any significant changes that were introduced during implementation. This section is descriptive, not evaluative, taken mostly from other documents (e.g., Project Paper/ISR/GRM). Please delete this note when finalizing the document. Context Project Development Objectives (PDOs) Key Expected Outcomes and Outcome Indicators Components II. OUTCOME Assessment of Achievement of Each Objective/Outcome Note to Task Teams: The assessment of PDOs should be organized around each objective or outcome indicated in the PDO statement. Compound PDOs with multiple outcomes should be “unpacked� and each outcome assessed separately. This applies to the case when several outcomes are linked together in a single sentence. Assessment of each objective or outcome should be based on the level of achievement and the directness of demonstrated connections between the activities of the project and the achievements reported. Please delete this note when finalizing the document. Page 4 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Overall Outcome Rating Note to Task Teams: Rate the overall outcome rating, using 6-point scale (Highly Satisfactory (HS), Satisfactory (S), Moderately Satisfactory (MS), Moderately Unsatisfactory (MU), Unsatisfactory (U) and Highly Unsatisfactory (HU)). In projects with multiple objectives or outcomes, provide only a single overall outcome rating covering all objectives or outcomes. Each objective does not need to be separately rated, although each need to be separately discussed. Please delete this note when finalizing the document. Other Outcomes and Impacts Note to Task Teams: (If any, discuss other outcomes and impacts achieved, including unintended, positive or negative. Where relevant, discuss how the project contributed to the development/strengthening of relevant institutions, mobilization of other resources, knowledge exchange, recipient policy/program implementation, replicable best practices). Please delete this note when finalizing the document. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME Note to Task Teams: This section provides an analysis of key factors and events influencing the operation’s achievements or non-achievements during preparation and implementation. This section also discusses actions taken by the Bank and/or the borrower to address and correct each factor undermining project performance. Please delete this note when finalizing the document. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME Note to Task Teams: To the extent not previously covered, discuss other factors (if any) linked to the operation being assessed, including the Bank performance, compliance issues, and risk to development outcome. Please delete this note when finalizing the document. Page 5 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) V. LESSONS LEARNED AND RECOMMENDATIONS Note to Task Teams: Drawing on the descriptions and analysis in the earlier section on key factors that impacted performance and achievement of PDOs (Section III above), this section briefly presents the most significant positive and negative lessons learned from the operation’s experience. These lessons need to be reflected in the arrangements in the post-completion phase, follow-up operations, and/or in any other next steps. Please delete this note when finalizing the document. . Page 6 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS Note to Task Teams: The following sections are system generated and can only be edited online in the Portal. Please delete this note when finalizing the document. A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increased transparency and public participation in the policymaking and budgeting processes in the educational sector at local, regional and national level in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of educational-sector Number 0.00 5500.00 5500.00 14800.00 stakeholders taking part in education budget consultations 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 at local, regional and national level Comments (achievements against targets): Participants at public hearings counted. The target is overachieved. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of users of the Number 0.00 10000.00 10000.00 169600.00 Page 7 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) scoalamea.md 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Unique users counted. The target is overachieved. Objective/Outcome: Higher-quality education policy debates at local, regional and national level in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Quality level of the education Text 5.14 7.64 +2.5 to baseline +2.36 to baseline policy debate 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): The quality level of the education policy debate is calculated based on the experts’ responses for the 23 questions in the questionnaire “How would you rate the level of debate on educational policies in the Republic of Moldova?�, on a scale from 1 (low level) to 10 (high level). The low rate of public debates was caused by frequent changes in the Ministry of Education, Culture and Research leadership (former Ministry of Education) during the last years of the project. The central government administration reform also delayed the implementation of some of the activities. Also, 2014 – Year 1 of the project – was a period of reforms and changes in the education sector; therefore, the score was higher and further expectations of experts in education were also higher. Another aspect of the lower results is the fact that many experts who participated in the survey were part of various governments throughout these years, which may skew the results of the survey. A recalculated indicator for the Y5 survey conducted from November to December 2018, using only the full completed questionnaires and the answers from experts that participated in at least three public events (17 out of 25 received, while methodology was requiring 20) indicates a +2.6, thus reaching and exceeding the target. Objective/Outcome: Higher-quality education policy debates at local, regional and national level in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Level of satisfaction of the Text 8 10 Y5: +0.5 to the current +0.5 to the Year 5 Page 8 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) quality of education services year cohort's baseline, cohort’s baseline, estimated at the estimated at the beginning of the year beginning of the year = 9.5 31-Dec-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): The scoring scale is 1 to 10. The final target is 10 calculated as +0.5 to the year 5 cohort’s, estimated at the beginning of the year. The baseline survey was run in March-May 2018 and established for the fourth round of schools a level of school openness at 9.5, much higher than in the previous years. A repeated survey run in November 2018 resulted in an openness score of 9.98. A.2 Intermediate Results Indicators Component: Building an environment enabling sustainable social accountability in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of regional focal- Number 0.00 5.00 5.00 5.00 points engaged 17-Feb-2014 17-Feb-2014 20-Dec-2017 20-Dec-2018 Comments (achievements against targets): 5 regional focal points engaged- partners in the project Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of local community- Number 0.00 100.00 100.00 100.00 level coalitions mobilized and engaged in project 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Page 9 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) implementation Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Participants' self-assessed Text absent upper-medium upper-medium upper-medium readiness degree 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Overall degree of expertise in implementing the SACc tools is at the upper-medium level (4.5) Component: Applying social accountability tools to enhance accountability of education management and transparency of school budget Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Citizens taking part or Number 0.00 25000.00 25000.00 54900.00 represented by CSOs in participatory budgetary cycle 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Only citizens taking part in participatory budgetary cycle are measured. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Schools' budgets discussed Number 0.00 100.00 100.00 100.00 Page 10 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Feedback reports from the Number 0.00 100.00 100.00 100.00 school management and relevant public authorities on 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Social Accountability Activities Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Active users of the Number 0.00 600.00 600.00 15800.00 www.scoalamea.md 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): An active user spends more than 3 minutes on the website. Component: Integrating Social Accountability activities with national education policy Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Citizens participating with Number 0.00 65.00 65.00 224.00 Page 11 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) meaningful contributions in 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 high-level policy debates (both written and oral) Comments (achievements against targets): School management (directors, deputies, accountants) that meaningfully contributed are counted. Component: Knowledge & Learning Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Knowledge shared with other Number 0.00 200.00 200.00 450.00 stakeholders nationally and internationally as part of 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 annual conferences Comments (achievements against targets): Note to Task Teams: End of system generated content, document is editable from here. Please delete this note when finalizing the document. Page 12 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) A. ORGANIZATION OF THE ASSESSMENT OF THE PDO Note to Task Teams: Organize the indicators and outputs around each Objective/Outcome captured in the PDO statement. Please delete this note when finalizing the document. Objective/Outcome 1 1. Outcome Indicators 2. 1. Intermediate Results Indicators 2. 3. 1. Key Outputs by Component 2. (linked to the achievement of the Objective/Outcome 1) 3. 4. Objective/Outcome 2 1. Outcome Indicators 2. 3. 1. Intermediate Results Indicators 2. 3. 1. Key Outputs by Component 2. (linked to the achievement of the Objective/Outcome 2) 3. 4. Page 13 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) . ANNEX 2. PROJECT COST BY COMPONENT Note to Task Teams: The data in this section has been pre-populated for the first time for your convenience, but it is completely editable. Please delete this note when finalizing the document. Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1. Building an Environment Enabling 0 .23 0 Sustainable Social Accountability in Moldova Component 2. Applying Social Accountability Tools to Empower Local and Regional 0 .18 0 Stakeholders to Increase Policy Accountability Component 3. Integrating Social Accountability with 0 .22 0 Policy and Budget Dialogue Component 4. Knowledge 0 .05 0 and Learning Total 0.00 0.68 0.00 Page 14 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Page 15 of 16 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) Page 16 of 16