Results in Education for All Children (REACH) ANNUAL REPORT 2017 Acknowledgements: This Annual Report was prepared by Jessica Lee, Kesha Lee, Minna Mattero, and Marie Tamagnan. Contributors include Samer Al-Samarrai, Omar Arias, and Patricia da Camara. Photo credits: World Bank/Flickr Contents The Road to Results-Based Financing: Achievements so Far. . . . . . . . . . . . . . 2 Leveraging Results-Based Financing for Strengthening Systems . . . . . . . . . . . 4 Country Program Grant: Nepal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Country Program Grant: Lebanon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Growing the Global Evidence-Base: Knowledge, Learning, and Innovation Grants. 11 Contributions to the RBF Knowledge and Evidence Base . . . . . . . . . . . . . . . . 12 Lessons Learned from the Knowledge, Learning, and Innovation Grants . . . . . . . . 16 Spreading the Word on What Works and What Doesn’t . . . . . . . . . . . . . . . 20 Knowledge, Learning, and Sharing Activities. . . . . . . . . . . . . . . . . . . . . . . . 21 Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Just-in-Time Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 The Path Ahead for Results-based Financing . . . . . . . . . . . . . . . . . . . . . 28 REACH Gross Balance Sheet 2015-2019 Updated (in US$). . . . . . . . . . . . . . . 29 Results Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Annex 1:KLI Call for Proposals Process. . . . . . . . . . . . . . . . . . . . . . . . . 31 Annex 2:Description of KLI Grants from Calls 1-4. . . . . . . . . . . . . . . . . . . . 32 FIGURES 1 Countries Receiving KLI and CPG Grants . . . . . . . . . . . . . . . . . . . . . . 3 TABLES 1 KLI Grants status update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2 REACH Gross Balance Sheet in US$. . . . . . . . . . . . . . . . . . . . . . . . 29 3 Results Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 REACH ANNUAL REPORT 2017 i ii RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Results in Education for All Children (REACH) Annual Report 2017 REACH’s mission is to support efforts toward more and better education services, especially for the most excluded, by helping country education systems focus more sharply on results. In line with the World Bank’s Education Strategy 2020, REACH supports the institution’s efforts to build evidence on RBF in education. To achieve these goals, REACH has established three primary pillars of activities: ■■ Country Program Grants (CPG) – REACH provides larger financing to RBF schemes that strengthen country systems in specific IDA1 countries; ■■ Knowledge, Learning and Innovation (KLI) Grants – REACH provides smaller financing to RBF schemes in IDA and IBRD2 countries that will contribute to the global evidence base; and ■■ Capacity-Building and Learning around RBF – REACH also organizes capacity building and learning events, maintains a website on global knowledge in RBF and provides just-in-time support to WBG teams and clients on projects with RBF components. Demand from WBG teams, country clients, development partners, and other key stakeholders to better understand RBF and its applications in various contexts is high and growing. Both staff and country clients are keen to use RBF to center policy dialogue on results. REACH has been able to offer concrete support to WBG teams and country clients by providing funding and technical assistance to help coun- tries raise the bar toward better education quality, and reaching highly vulnerable children still excluded from education systems. 1 The International Development Association (IDA) is the part of the World Bank that helps the world’s poorest countries by providing loans (called “credits”) and grants for programs that boost economic growth, reduce inequalities, and improve people’s living conditions. IDA countries are low income countries which qualify for IDA assistance. 2 The International Bank for Reconstruction and Development (IBRD) is the original World Bank institution. It provides loans and other assistance primarily to middle income countries. IBRD countries are middle income countries which qualify for IBRD assistance. REACH ANNUAL REPORT 2017 1 The Road to Results-Based Financing: Achievements so Far In 2017, REACH received additional contri- allowed REACH to establish activities that butions from the Government of Norway are assessing the potential of RBF to tackle and the German Federal Ministry for system weaknesses at the school level, Economic Cooperation and Development meso-level (e.g. district/province level), and with continued support from United States national level as well as in different contexts Agency for International Development. (such as low-income, middle-income, and fragile contexts). These grants are part of To date, REACH has achieved the following: larger patterns of research around the world ■■ Financed 30 RBF activities in 22 and help to foster relationships with those countries that cover a range of topics working on RBF both inside and outside of within RBF (See Figure 1). the World Bank Group. ■■ Generated knowledge about RBF REACH only supports activities that are also by organizing 32 learning events with supported by the client government and that roughly 700 total participants and have the potential to be scaled up, ensuring published reports on various RBF-related that the funds invested will pave the way for topics both in print and online. long-term systemic change. REACH is currently funding RBF activities through large country program grants In addition to funding RBF schemes and (CPGs) to Nepal and Lebanon and through activities that add to the knowledge base, several smaller activities under the KLI REACH provides expertise and advisory Grants. All of these grants are supervised services, either from in-house experts or by a World Bank task team in the country by connecting those seeking technical in question. While the CPGs leverage support with leading global experts. This is more money, KLIs are also supported by done through learning events, just-in-time the World Bank’s country teams and are support, and informal networking with aligned with World Bank work programs to task team leaders at the World Bank, the ensure that the funded activities are closely Global Partnership for Education (GPE), aligned with the country client’s national academics, think tanks, and local NGOs. education strategy. This strategic invest- REACH support extends to every stage of ment in a series of smaller activities has the policy process. 2 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) FIGURE 1: Countries Receiving KLI and CPG Grants REACH ANNUAL REPORT 2017 3 Leveraging Results-Based Financing for Strengthening Systems Country Program Grants (CPGs) are awarded on the basis of demonstrating country owner- ship, readiness to implement a large program, a clear financing gap, and strong technical merits of the proposed operation. Country Program Grant: Nepal Snapshot Summary REACH-funded Disbursement Linked Total Value Total Disbursed Status update Indicators* DLI 4: Independent US$2 million US$2 million Achieved verification of EMIS data** (pending) DLI 5: Strengthening of financial management Financial management US$2 million US$0 capacity at the school guidelines revised level Notes: *A disbursement linked indicator (DLI) is a results indicator to which a monetary sum is linked. When that indicator is met and externally verified, the pre-determined sum attached to that DLI is disbursed. **EMIS = education management information system. 4 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Overall Context which the disbursement of funds is condi- Nepal is in a period of transition, following tional on the client achieving certain goals. a 10-year conflict that ended in 2006, a The three main components of SSRP destructive earthquake in 2015, and several phase 4 are: trade disruptions. Situated between two of the world’s fastest growing economies, ■■ (i) to ensure equitable access to and the India and China, Nepal remains among the quality of basic education for all children poorest countries in the world, with a per in the 5 and12 age group; capita gross national income of only US$762 ■■ (ii) to increase equitable access to (2015). However, in recent years, it has made secondary education and to improve its significant progress in terms of poverty equity, quality, and relevance; and reduction and human development. Nepal achieved the first Millennium Development ■■ (iii) to increase the capacity of SSRP Goal of halving extreme poverty ahead of implementation agencies and partners schedule, and the country has also achieved to enhance delivery and monitoring and gender parity in education. evaluation (M&E) services. In providing additional financing for the Discussions about results-based financing SSRP, the GPE and REACH grants are in the Nepalese education sector began in designed to support the entire school 2015, shortly after REACH began its Country education sector (grades 1 to 12), benefit- Program Grants (CPG) funding stream. The ting more than 7 million students and World Bank in partnership with the Global 200,000 teachers in more than 28,000 Partnership for Education (GPE) began community schools across the country. talks with the Government of Nepal about additional financing for its ongoing School The Additional Financing came about as part Sector Reform Program that would include of the new Nepal Sector Plan—the Nepal an element of results-based financing. Nepal School Sector Development Plan (SSDP had already had some experience with 2016/2017-2022/23), which is a long-term results-based financing in other sectors. The strategic plan for the Nepalese education World Bank transport team was in the process sector. The SSDP aims to expand equitable of implementing a Program-for-Results access to good quality education for all (PforR) operation, and the Asian Develop- children by focusing on strategic interven- ment Bank (ADB) had had success with other tions and new reform initiatives to increase results-based financing instruments. the efficiency and improve the management, and governance of the education system. In January 2016, the Government of Nepal and the World Bank signed an agreement Three DLIs are specified in the GPE’s grant: for Additional Financing (AF) grants to ■■ (i) approval and implementation of the ongoing Nepal School Sector Reform subject-based examinations at end of Program (SSRP phase 4) in the amount of grades 10 and 12 (efficiency – DLI1); US$59.3 million from the Global Partner- ship for Education (GPE) and US$4 million ■■ (ii) implementation of school-based from Results in Education for All Children reading assessments (quality – DLI2); and (REACH). The REACH grant and part of the ■■ (iii) the design and implementation of GPE grant were to be provided on the basis an intervention to enroll out-of-school of disbursement-linked indicators (DLIs) in children (equity – DLI3). REACH ANNUAL REPORT 2017 5 These GPE DLIs, valued at US$17.9 that REACH helped to initiate, about 75 million, complement the REACH DLIs, percent of this external donor support was in valued at US$4 million: the form of DLIs. ■■ (i) independent verification of data REACH funds are also supporting two from Nepal’s education management additional activities to support the RBF information system (EMIS) (REACH1, implementation. The first activity, carried DLI4) and out jointly by the Nepal task team and 3ie, ■■ (ii) strengthening of financial an external international organization, is an management capacity at the school impact evaluation of how RBF can support level (REACH2, DLI5). decentralization efforts in Nepal’s education system. This evaluation will assess how REACH1 DLI4 was chosen because effective the REACH funded DLIs were in monitoring systems such as the EMIS strengthening country systems to be able are a necessary precondition for RBF to carry out RBF. Work on the baseline to be effective, and these systems and survey is underway, with randomized the indicators that they track must be credibly monitored and then verified by a control trials being conducted until August trusted source. REACH2 DLI5 was chosen 2018 and a final report due by November because strong procurement and financial 2018. The second activity is a supervisory management systems at the country level budget to provide technical assistance to are necessary to ensure that the education the Government of Nepal to help them to system can handle the demands placed on achieve the DLIs. With this support, REACH them by a results-based approach. aims to demonstrate that that introducing a results-based approach gave the Later in 2016, a PforR in education was government an incentive to strengthen their approved by the World Bank Board of own country systems in order to produce Governors as a result of the experience of better education results. RBF in the Nepal School Sector Reform Program (SSRP phase 4). Thus, the REACH One of REACH’s greatest contributions DLIs helped pave the way for the PforR. (aside from direct financing) to the project has been by providing critical just-in-time Other than having a direct influence on and technical support. REACH worked more financing for the Nepalese education with the World Bank task team in Nepal sector, the impact of the REACH grant and to discuss how monetary incentives could technical assistance also helped to shift the be used to incentivize improvements in policy dialogue with Nepal towards results financial management and monitoring and and systems building. The REACH funds and evaluation systems. During the preparation GPE financing were complemented by phase of the AF grant, REACH helped the funds from other donors, including the task team in Nepal to decide on the design Asia Development Bank (US$120 million), of the DLIs and verification protocols and the European Union (US$7 million), Finland brought in an external academic to help (US$23 million), Norway (US$21 million), them think through how to conduct the UNICEF (US$3 million), Australia (US$3 impact evaluation. million), JICA (US$15 million), and non-Joint Financing Partners (US$14 million). In keeping with the results-based approach 6 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Progress to Date Progress has been slower on achieving the The SSRP project is currently on track to REACH2 DLI5 (strengthening of financial achieving the DLIs associated with the GPE management capacity at the school level). and REACH grants. There has already been An eighth amendment to the Education progress towards achieving DLI4 under the Act was promulgated by Parliament in REACH financing, and activities linked to June 2016, following which the Education DLI5 on financial management are currently Regulation was revised, requiring a revision underway. Initial progress had been slow, to the financial management guidelines. and the entire project was restructured During this time, the Government of to better align it with the new sector Nepal was taking steps to make local plan (SSDP), which led to an extension of governments responsible for basic and the closing date of the REACH Country secondary education as mandated by the Partnership Grant from July 1, 2018 to new Constitution, after the local elections, November 30, 2018. which ended on September 18, 2017. These new developments have delayed Of the two REACH DLIs, DLI4 (independent the revision of the guidelines, which is a verification of EMIS data) has been achieved prerequisite for this DLI, but more progress through a random sample-based survey. is likely to be made now that local elections The government employed an indepen- have ended. dent agency to conduct a random sample of 1,500 schools to collect information on The Nepal case highlights how REACH key indicators to verify that they had been has used funding and technical assistance achieved. This verification made it possible to strengthen fiduciary and monitoring for REACH and the GPE to confirm that the and evaluation systems and government DLIs had been achieved and to disburse the capacity to carry out RBF. The World Bank’s grant funds. In addition, the survey firm has task team in Nepal has indicated that both submitted the final report to the Ministry of types of support were critical to paving the Education with recommendations on how way for the Ministry of Education to accept results-based financing as a way to achieve to improve the EMIS, and the government better results. is now in the process of taking the recom- mended corrective actions. REACH ANNUAL REPORT 2017 7 Country Program Grant: Lebanon Snapshot Summary REACH funded DLI Total Value Total Disbursed Status update DLI 3: teacher Baseline data for impact US$1 million (as an performance measured US$4 million evaluation to begin this advance) and evaluated calendar year Overall Context Syrian children were enrolling in schools in In 2012, amidst the humanitarian crisis in areas that were historically under-privileged. Syria, the Ministry of Education and Higher In 2013, as part of a general move towards Education (MEHE) of Lebanon issued a increasing funding for education in emer- memorandum instructing all schools to gencies, the UN Special Envoy for Global enroll Syrian students regardless of their Education convened a meeting of donors legal status and to waive school and book to discuss the impact of the Syrian crisis on fees. In 2013, the number of Syrian refugee the Lebanese education system. At that children enrolled in Lebanese schools meeting, it was agreed that it was vital for exceeded the total of Lebanese children the international community to step up and in primary and secondary schools, and by provide a joint response commensurate with May of 2014, there were more than 1 million the crisis in Lebanon as an essential invest- Syrian refugees registered in Lebanon ment towards rebuilding Syria. with more awaiting registration. This made Lebanon the country with the largest Within this complex political and emer- refugee intake in the region. Furthermore, gency context, the MEHE developed the 8 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Reaching All Children with Education systems. As in Nepal, these systems need (RACE) Sector Plan to address the refugee to be functioning cohesively for RBF to be crisis in Lebanon. RACE was tightly focused effective. The entire project has nine DLIs on refugee children and education in that fit under these three categories, and the context of the systematic challenges the REACH funds are tied to DLI3: Teacher historically faced by the Lebanese education performance measured and evaluated. The sector. These challenges, which by 2014 funds under this DLI will be disbursed after had been exacerbated by the Syrian crisis, the completion of an impact evaluation included low teacher effectiveness, dispari- study of two teacher support programs. ties between public and private schools, and The first is classroom observation, where the generally poor learning outcomes of teachers are observed in the classroom by public schooling. the academic counselor and the school principal who give them feedback and By 2016, with 480,000 Syrian children and guidance. The second program includes 250,000 Lebanese children enrolled in the same elements (classroom observations primary and secondary schools, the Govern- and guidance) but also includes training ment of Lebanon wanted to take a broader provided to teachers by the Center for approach that would not only benefit Syrian Educational Research and Development children but also Lebanese students and (CERD). The evaluation will study three the Lebanese education system. Therefore, groups of teachers: one group receiving only by the end of the first RACE Sector Plan in observation and guidance, one receiving 2016, the RACE-II Sector Plan (S2R2) was observation and guidance plus CERD developed. The World Bank through IDA training, and one control group who receives prepared a PforR grant of US$100 million no support. The study will start by collecting to complement the roughly US$100 million a baseline set of data at the beginning of that was contributed in a Multi-Donor Trust the year, followed by the implementation Fund (MDTF) by donors such as the United of the interventions, then endline data will Kingdom’s Department for International be collected at the end of the year. Teacher Development (DFID) and the Governments performance will be compared between the of Norway and Denmark. baseline and endline data and among the In addition to the IDA financing, Lebanon three studied groups of teachers. Once this became the second country to receive a DLI is achieved, the associated funds will be REACH Country Project Grant (CPG). The disbursed and will not be earmarked so they preparation of the S2R2 Program for Results can be used for any activities in the sector. operation and the preparation of the REACH This emphasizes one of the main design grant came hand in hand, in large part due benefits of RBF, which is that disbursements to the just-in-time support REACH provided are not linked to inputs. to the Lebanon task team in thinking REACH’s particular added value has been through how RBF could help RACE-II the technical assistance that it provides achieve better results. during initial conversations within the The S2R2 PforR comprises three transfor- development community about education mational activities: (i) monitoring systems issues. Although stakeholders had agreed and external verification, (ii) impact evalua- on the importance of providing funding to tion of teacher support programs to assess the Government of Lebanon in the context cost-effectiveness of interventions, and (iii) of the Syrian refugee crisis, initially they procurement and financial management could not agree on what specifically to REACH ANNUAL REPORT 2017 9 fund. Given the difficult political context, 2017, approvals for the implementation the Government of Lebanon was hesitant decrees for both grants were delayed by to fund and take credit for the education of the Lebanese government due to political Syrian refugees in Lebanon since it wanted challenges regarding the scope of respon- to prioritize providing a quality education sibilities of the RACE project management to Lebanese children first and believed that unit. This was brought to the attention of the the two options were mutually exclusive. In World Bank’s senior managers who, along parallel, the importance that the UK DFID with representatives of the DFID, worked placed on results implied that they would with the Prime Minister, the Minister of no longer be satisfied with supporting Education, and the Minister of Finance to inputs, while the World Bank was exploring resolve the issue. the possibility of introducing a new project that did not cover school construction Despite the political challenges, some since evidence has shown that constructing progress has been made as a result of the schools without trying to strengthen the technical support given to the Ministry of quality of teachers and materials does very Education by the World Bank task team. little to improve learning conditions. As a Initial targets have been met in at least four result, these conversations often fixated on out of the nine DLIs. Once the Ministry of piecemeal issues, such as whether or not to Education hires an independent verifica- build schools, whether or not to continue tion agent to confirm the achievement of funding certain programs, and whether or not to first address Syrian refugees or these results, they will be able to request a Lebanese children. The REACH team helped second transfer of funds. One of the DLIs to steer the conversation to focus first on that has been achieved is the implementa- results and then on how each stakeholder tion of modules for the new in-service could contribute to achieving those results. teacher training program, which will directly help to achieve the REACH-funded DLI3 After the donors came to a consensus to related to measuring and evaluating apply a results-based approach in Lebanon, teacher performance. REACH provided just-in-time support in the form of communication about RBF, particu- As in Nepal, REACH is also funding an larly with the Government of Lebanon, impact evaluation and providing a super- which had worked with the World Bank over visory budget. A principal researcher for the course of many years and was used to the impact evaluation has been recruited, traditional investment project financing and the initial design of the study has been that funded inputs and activities rather than completed. Baseline data collection is results. The importance of continuously expected to take place early in 2018. discussing and explaining RBF with the country client cannot be understated and Once the impact evaluation funded by has repeatedly served as a lesson for teams REACH is completed, further analysis will be preparing RBF operations in other countries. done to investigate how much impact this DLI had in terms of strengthening systems to Progress to Date measure and evaluate teacher performance. The project has only been effective since The case of the country partnership grant in July 2017, and there have already been Lebanon demonstrates that results-based some serious country-related challenges financing approaches can work in all country to achieving the DLIs. In September contexts, even in fragile states. 10 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Growing the Global Evidence-Base: Knowledge, Learning, and Innovation Grants The objective of the KLI grants is to expand defined and including both outputs and knowledge about how RBF can be used to outcomes. The projects chosen to receive strengthen education systems in low-income grants have been implemented in a range and middle-income counties. REACH has of situations from low-income, fragile, and put out five calls for proposals to date, two conflict-affected countries to middle-income general calls in April and October 2015, two economies across multiple regions. They themed calls for textbooks and supplemen- have also covered a number of RBF topics, tary reading materials (REACH for Reading) including performance-based school grants, in November 2016 and November 2017 performance-based contracts, incentives (currently underway), and one themed call for teachers, incentives for parents and/ or students; information and management on the meso-level of the education system in systems for RBF, and preconditions for RBF. July 2017 (see Table 1 for list of grants). At the end of the grant cycle, each research Calls 1, 2, and 4 awarded up to US$200,000 team that received a KLI grant is required to per grant, and Calls 3 and 5 awarded up to produce a policy note or similar knowledge US$500,000 per grant. In every call period, product to contribute to the growing body each application undergoes a rigorous of global evidence and knowledge on RBF selection and interview process. Approval in education. The intention is to compile all criteria include the technical merits of the of the available knowledge from the first two proposed activities, the proposal’s meth- calls for submissions into a compendium odological rigor, its potential contribution to provide some thematic structure to the to global knowledge beyond the proposed general calls. country context, any strategic opportunities for long-term engagement on the topic, More detailed descriptions of the grants can demonstrated commitment to previous be found in the REACH Annual Reports 2015 clients, and the proposed activity’s impact and 2016, and on the REACH website: on country systems. Results can be broadly www.worldbank.org/reach. REACH ANNUAL REPORT 2017 11 Contributions to the RBF Knowledge and Evidence Base For an overview of the KLI review process and brief descriptions of all grants, please see the annexes. TABLE 1: KLI Grant Status Grant Rate of Original Listed Initiation Call Amount Disbursement Closing Closing Deliverable Status Date (USD) (as of Jan 17, 2018) Date Date* CALL 1: APRIL 2015 HAITI: Developing a quality $200,000 86% July 2015 June 2016 June 2018 Due Jan 31, 2018 assurance system for RBF INDONESIA: Piloting perfor- Deliverables moved mance-based contracting in $50,000 100% July 2015 June 2016 Dec 2017 to Call 2 Jakarta schools MOZAMBIQUE: Designing Late due to performance-based school $180,000 24% July 2015 June 2016 Dec 2018 government delays grants MOZAMBIQUE: Testing demand-side incentives to $198,000 100% July 2015 June 2016 Sept 2017 Received keep girls in school (see Box 1) NIGER: Developing a sustainable monitoring and Late due to $100,000 96% July 2015 June 2016 June 2018 evaluation system for future government delays RBF RWANDA: Can pay for performance improve teacher $195,741 94% July 2015 June 2016 Oct 2017 Due Feb 1, 2018 performance and attract top talent? TANZANIA: Does the design of pay for performance $198,500 100% July 2015 June 2016 Oct 2017 Received schemes matter for student learning gains? (See Box 2.) TANZANIA: Do students who set goals for themselves $198,500 100% July 2015 June 2016 Mar 2017 Received perform better? Or do they require financial incentives? VIETNAM: Designing a predictive set of indicators for $200,000 99% July 2015 June 2016 July 2017 Received future RBF CALL 2: OCTOBER 2015 CAMEROON: Pre-piloting a performance-based school $200,000 37% Jan 2016 June 2017 July 2018 On track grant CHINA: Building the founda- tion to incentivize teacher $200,000 99% Jan 2016 Nov 2017 Oct 2017 Received training institutes to deliver better teachers (See Box 3.) 12 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) TABLE 1: Continued (Call 2) Grant Rate of Original Listed Initiation Call Amount Disbursement Closing Closing Deliverable Status Date (USD) (as of Jan 17, 2018) Date Date* COLOMBIA: Creating an RBF monitoring system that $200,000 97% Jan 2016 Jan 2017 Dec 2017 Received covers multiple dimensions of education quality DEMOCRATIC REPUBLIC OF CONGO: Do schools that Received: $200,000 50% Jan 2016 June 2018 July 2018 receive RBF subsidies perform baseline survey better than those that do not? INDIA: Creating a valid Received: teacher assessment for future $200,000 49% Jan 2016 Oct 2016 Dec 2018 revised proposal pay-for-performance schemes INDIA: Evaluating perfor- mance contracts between No progress, REACH second-chance education $200,000 98% Jan 2016 Dec 2017 July 2018 team is following up providers and the Govern- ment of India Received: INDONESIA: Evaluating Paper on evaluation performance-based contracts of performance-based (from Call 1) and perfor- $130,000 85% Jan 2016 June 2017 Mar 2018 school grants; mance-based school grants in Awaiting: Jakarta (See Box 4.) paper outlining the methodology MOROCCO: Developing and piloting performance-based Received: contracts between national, $200,000 81% Jan 2016 Dec 2016 July 2018 draft materials regional, and local education authorities REP. OF CONGO: Increasing accountability through Firm contracted to $100,000 10% Jan 2016 Sept 2017 Mar 2018 open data to inform an RBF begin work program for school finance CALL 3: REACH FOR READING (NOVEMBER 2016) CAMBODIA: Using RBF to increase accountability and $500,000 38% July 2017 July 2019 - On track transparency in the book supply chain SOUTH AFRICA: Testing how incentives work in a private- $500,000 10% July 2017 Aug 2019 - On track public partnership (See Box 5.) ZAMBIA: Incentivizing more $500,000 27% July 2017 Jan 2019 - On track efficient book delivery BANGLADESH: Using RBF to Approved Pending contract create more diverse reading $500,000 0% Sept 2019 - Jan 2018 signing materials in mother tongues Continued next page REACH ANNUAL REPORT 2017 13 TABLE 1: Continued (Call 4) Grant Rate of Original Listed Initiation Call Amount Disbursement Closing Closing Deliverable Status Date (USD) (as of Jan 17, 2018) Date Date* CALL 4: MESO LEVEL OF THE EDUCATION SYSTEM (JULY 2017) COLOMBIA: Piloting an RBF $200,000 0% Dec 2019 - monitoring system in Bogota DOMINICAN REPUBLIC: Designing performance $200,000 0% Dec 2017 Feb 2019 - On track agreements to improve district-level performance INDIA: Incentivizing and empowering district-level $200,000 0% Dec 2017 Mar 2020 - On track officers to collect real-time data PERU: Evaluating perfor- mance-based career path and $200,000 0% Dec 2017 Jan 2019 - On track compensation reforms for school leaders TANZANIA: Using RBF to $200,000 0% Jan 2020 - improve school inspections BANGLADESH, INDIA, TANZANIA: Understanding how district education officers $121,000 0% Dec 2017 Dec 2018 - On track make decisions in order to design better targeted RBF interventions in the future CALL 5: REACH FOR READING (NOVEMBER 2017) Review process ongoing; review committee meeting discussion on January 19, 2018 Notes: *Most REACH grants are delayed on average by 18 months. These delays are mainly a result of country context since the REACH teams must work with country governments and other external partners to complete activities. The REACH team assesses requests for extensions using a protocol that includes the following steps: (i) REACH team investigates the reason for delay and whether it is within the control of the imple- menting team; (ii) if the delay is not within control of the team, REACH agrees to a six-month extension period; and (iii) if the implementing team makes a second extension request, REACH raises the request with both the team leader’s manager and the manager of the global knowledge and innovation unit that oversees REACH. 14 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Calls 1-3 Information about Call for Proposals 1-3 are available on the REACH website under the “Grants” section: www.worldbank.org/reach Call 4: Box 1: Mozambique Meso-level of the education system REACH funded an evaluation that compared Call 4 was designed to fill gaps in knowledge the effectiveness of three different interven- on RBF that remained after the first two calls, tions designed to increase girls' school which were open to all interested parties. attendance in grades 6 and 7 in Mozambique. The goal of the first two calls was to identify The first intervention provided girls with existing RBF activities and interventions and vouchers conditional on them maintaining a to fund promising approaches and evalua- high level of school attendance. The second tions. Given that the third call already had a intervention provided parents with a cash theme around textbooks and supplementary transfer conditional on their daughters’ school materials, the REACH team proposed that attendance. The third intervention simply the fourth call should center on the meso provided students’ parents with weekly report level of education systems, in other words, cards on their daughters’ attendance. These the local authorities that directly oversee interventions were designed to answer the the operations of schools. This proposal following questions: (i) whether providing infor- was made because existing education mation to parents to increase monitoring could research has tended to focus on examining be as effective in increasing school attendance incentives at the front-line of education as financial incentives and (ii) whether financial service provision with little attention to the incentives were more effective when given bottlenecks that occur above the school directly to students or to their parents. level. For example, there is little evidence about interventions that use RBF to improve This evaluation found that all three interven- quality assurance systems for schools, tions significantly increased girls’ school that examine the effectiveness of teacher attendance. While providing information preparation efforts (whether in-service or alone had a substantial effect, this effect was pre-service), that improve the functioning of smaller than the impact of financial incentives. district education offices, or that strengthen However, given the low cost of providing the role of inspectors. The finalists from information and the ease with which it could this call are listed in Table 1, and several of be scaled up, this may be a promising policy them build on previous grants, for example, option for countries with limited capacity. In Colombia where the RBF monitoring system addition, providing financial incentives directly developed as a result of Call 2 will be piloted to students was at least as effective as incentiv- in Bogota and Tanzania where the grant will izing parents and nearly twice as effective as follow on from progress made in the Big providing information alone. Results Now PforR. REACH ANNUAL REPORT 2017 15 Call 5: REACH for Reading Call 5 is the most recent KLI call for proposals, and the review process is still Box 2: Tanzania underway. A total of 190 applications were received, 39 of which met the eligibility criteria, and to date the review committee REACH funded an evaluation that compared has selected 12 finalists to whom they will the effectiveness of two different teacher send follow-up questions. performance-based payment systems in primary schools in Tanzania. The first was a simple system that rewarded teachers based Lessons Learned from the on the number of students who reached Knowledge, Learning, and specific proficiency levels. The second was Innovation Grants a more complex system that rewarded Several lessons have been learned from the teachers based on students’ average learning administration of the KLI grants so far, and gains relative to their initial levels. In theory, these lessons have been documented in rewarding learning gains should produce various ways. better results because it rewards all teachers regardless of their students’ initial learning Working with External Organizations levels and because it incentivizes teachers to and Governments improve learning for all students. All REACH grants (both CPGs and KLIs) are While both systems raised test scores, the overseen for quality assurance and for World evaluation found that, despite the theoretical Bank procedural reasons by a World Bank advantages of the more complex learning task team leader. While REACH has also “gains” system, the simple learning “levels” funded external organizations such as Inno- system was at least as effective in raising vations for Poverty Action and Cordaid, all student learning. Furthermore, the simpler grants are supervised by a World Bank task scheme produced benefits that were more team in the country in question who liaise equitably distributed across all five quintiles with partners such as external organizations, of students, while the more complex scheme research institutes, CSOs, and the govern- benefitted primarily the top quintile of ment to ensure that the grant activities are students. These results highlight the critical closely aligned with country priorities. The importance of design in RBF schemes. By team also helps to build capacity within rewarding teachers for students at multiple country. The majority of these World Bank learning levels rather than just one, the simple task teams work with local or international scheme overcame one of the disadvantages groups and consultants to complete the of other similar systems with minimal added grant activities. Here are some examples. complexity. Further research is needed to determine which other design features may CONGO: The international technology produce optimal results. company BlueSquare has been contracted to design a mobile-based app to enable citizens to report fraud and corruption so that RBF can be applied to a future school finance program. BlueSquare has extensive 16 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) experience in surveying local stakeholders to design technology for RBF, particularly in Africa in the health sector. INDIA (CALL 4): A local Indian tech- nology company, Learning Links, has been contracted to develop enhanced business Box 3: China tools and processes to help school inspec- tors to do their jobs more effectively. The REACH funded a pilot of the Classroom Assess- World Bank task team leader and review ment Scoring System (CLASS) in Guangdong, committee has seen an early design of the China to establish a proof of concept for using mobile app, which will be used in association classroom observations to measure the impact with performance incentives. of teacher training and incentivize training providers within an RBF mechanism. This pilot RWANDA: The fieldwork coordinator for sought to test whether such a tool could be the Rwandan KLI and the principal investi- used to measure teaching practices in the gator for the impact evaluation is Professor Chinese context, inform how to introduce class- Andrew Zeitlin from Georgetown Univer- room observations into a Quality Assurance and sity in Washington DC. He is conducting Monitoring and Evaluation (QAME) system, and research on the effect of teacher incentives establish the preconditions for RBF in China. on recruitment. Professor Zeitlin has worked extensively on economic development in The pilot showed that the CLASS tool could Africa, including agricultural technology yield valid and reliable measures of teaching adoption in Ghana, health insurance in quality in the Guangdong context as it had Kenya, public service delivery in Uganda, in other countries. On average, teachers in and property rights in urban Tanzania. Guangdong scored high on classroom orga- nization but lower on emotional support and ZAMBIA: The Department of Economics at instructional support, which indicated how the University of Zambia has been hired to teacher training can address the most critical conduct the baseline survey for an impact gaps in teaching practices. While there was evaluation that tests what type of RBF substantial variation in performance among design works best in textbook delivery. the teachers, there was only modest variation The World Bank task team will help to build by county, teacher type, grade, and years of local capacity to produce textbooks and experience and between urban and rural areas. will continue to provide the Department of Economics with technical assistance to Lesson learned from this pilot can be used to ensure the study is robust. improve the design of RBF classroom observa- tions in the future. To address discrepancies between raters (the individuals who observe Results-based Financing Around the and rate teachers in the classroom), it may World Note Series be necessary to provide them with additional REACH is producing short, easy to read training or to modify the scoring rubric to notes based on the final deliverables from account for any potential cultural issues. In grantees. These notes answer four main designing an RBF mechanism using classroom questions: (i) Why was the intervention/ observations, it is critical to tailor incentives to activity chosen? (ii) How does the interven- improve those teaching practices where there is tion/activity work? (iii) What are the results? significant area for improvement. REACH ANNUAL REPORT 2017 17 and (iv) What implementation lessons have been learned? The goal of the notes is to not only summarize the research findings for a broad audience but to also to include details of the implementation of the research Box 4: Indonesia process to inform practitioners who are undertaking future RBF projects. Currently REACH funded an evaluation of the initial there are five notes available for the KLIs for impact of a performance-based school grants China, Indonesia, Mozambique (incentives program on student learning in Indonesia. for girls and parents), Vietnam, and Zanzibar. While all government schools in Jakarta These notes will be made available both in receive a fixed-value grant per student, the print and on the web. new program allows the top performing schools to receive a bonus. The evaluation The REACH Compendium of the impact of the first two years of the The concept note for the REACH Compen- new program assessed how the announce- dium has been completed, and the peer ment of the performance-based incentive review will take place on February 15, 2018. affected student achievement in schools and The authors have been consulting with how receiving the bonus affected student various development agencies and World achievement in the top performing schools. Bank task teams to gather practical imple- Announcing the performance incentive had a mentation lessons from RBF in education. different impact on primary schools than on The goal is to produce a volume of work, junior secondary schools. Student test scores with two parts, for a general audience. The improved significantly in all junior secondary first part will look at general lessons across schools, with the largest gains being made in a variety of sub-themes in RBF while the schools that were already performing the best. second part will focus on how the REACH However, in primary schools, the impact on grants have contributed to the larger test scores was negative, albeit small in magni- body of evidence on four specific topics: tude. While there was a modest improvement (i) performance-based school grants; (ii) in student achievement in low-performing teacher incentives; (iii) incentives to others schools, this was offset by losses in the high- (such as students and parents); and (iv) performing schools. While the magnitude of information and monitoring systems for RBF. the gains in low-performing schools was similar The Compendium will include all REACH in primary and junior secondary schools, these grants from Calls 1 and 2 for which there are mixed results highlight the risk that perfor- final research papers. This version will be mance-based competition can actually have completed in late summer/early fall of 2018. a demotivating effect for some schools. On As additional research papers are finalized the other hand, there is little evidence that the from the first two calls, they will be added schools that received performance bonuses to the Compendium. Calls 3 to 5 will have performed better than those that did not. The different analytical products given their effect that the program had on learning was thematic emphasis. largely the result of the change in incentives created by announcing the performance-based grants rather than by the additional grant funding itself. 18 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Box 5: South Africa Two key challenges to increasing access to storybooks for young readers in South Africa are that the book supply chain: (i) fails to provide enough good quality early grade storybooks in underserved languages and (ii) emphasizes textbooks over storybooks for enjoyment. This KLI grant funds a RBF intervention that provided incentives to government and the private sector to create a national public-private partnership to establish best practice recommendations for early grade storybooks, to produce 20 to 25 new titles in five languages (isiZulu, Tshivenda, Siswati, Xitsonga, and Sepedi), and to develop a pooled procurement and “sliding scale” model for book purchases. Building on the South African government’s Read to Lead Campaign (2015-2019), this project sets specific milestones for developing the standards and funding mechanisms to begin creating government demand for better storybook content, while improving the kinds of books that publishers create and sell. To date, the project team has conducted a series of workshops in Pretoria aimed at advising the government on how to build the capacity to publish quality children’s storybooks and develop the public-private working group. Best practice recommendations for early grade storybooks are emerging from these workshops, which are setting the stage for RBF activities (incentivizing publishers and developing the pooled procurement model for book purchases) to be conducted in mid-2018. REACH ANNUAL REPORT 2017 19 Spreading the Word on What Works and What Doesn’t REACH is dedicated to expanding the body In 2017, REACH organized 10 knowledge of knowledge on RBF in education and sharing and learning events, involving close building evidence on how it can strengthen to 400 participants, and published seven education systems. To accomplish these knowledge products on the REACH website. goals, REACH not only focuses on gener- REACH also partnered with internal and ating and acquiring knowledge on RBF external groups to host additional RBF- but also on sharing it widely and building related events. The internal partners have country capacity to implement it. included the World Bank Group’s Trade Its three main knowledge management and Competitiveness unit, Markets and objectives are to: Finance unit, Social Protection unit, Health, Nutrition, and Population unit, the Human 1. Build momentum and share information Development Vice President’s office, with a wide range of audiences about Operations Policy and Country Services where and how RBF has been used (OPCS), Education Staff Development successfully Program (ESDP), and the Strategic Impact 2. Bring together a community of practice Evaluation Fund (SIEF). The external orga- on RBF in education to share their nizations have included NGOs that focus operational knowledge on RBF such as Instiglio and Social Finance UK; international groups such as Building 3. Build national and local capacity to use Evidence in Education BE2 and the Interna- RBF instruments. tional Commission on Financing Education To do this, REACH has developed four types Opportunity; development agencies like of knowledge sharing activities that are the UK DFID, the Asian Development Bank, geared to specific audiences: the Inter-American Development Bank; think tanks such as Brookings and Results 1. Public events on RBF in education for Development (R4D); and civil society featuring a panel of experts on featured organizations such as Cordaid. topics. 2. “RBF for Breakfast,” a closed morning session to share knowledge among RBF practitioners 3. “Quick-and-Dirty Operational Notes,” available online 4. Program-for-Results (PforR) in Education training for World Bank and GPE staff. 20 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Knowledge, Learning, and recorded to make them available to larger Sharing Activities audiences and use online platforms and tools to include virtual presenters and This section gives a short description of each engage global audiences. knowledge sharing and learning activity. Expert Panel on RBF: Silver bullet or scattershot? Public events on RBF in education At the Comparative International Education Society featuring a panel of experts on the topic (CIES) Annual Conference | Atlanta, Georgia | March 5, 2017 This series features a talk show style panel of experts from the RBF community geared to The CIES annual conference draws in over larger audiences to share the latest knowl- 2,500 participants from all over the world. The edge about RBF in education. The panelists REACH team put together a panel featuring a come from research and development orga- frank conversation on the virtues and vices of nizations working on RBF around the world. RBF with representatives of The New School, The topics discussed at these events have the Global Partnership for Education (GPE), ranged from general RBF in education issues and the evaluators of the DFID’s Girls Educa- to more specific topics within RBF, such as tion Challenge from PricewaterhouseCoopers performance-based fiscal transfers, which sharing their views. The event also served as are becoming more popular. Whenever an opportunity to launch and share the World possible, the events are livestreamed and Bank’s Strategy on RBF. REACH ANNUAL REPORT 2017 21 Expert Panel on Innovative sion on Financing Education Opportunity, Financing in Education Foresight Associates, Cordaid, and the Co-hosted with Results for Development (R4D) at the University of Virginia. The panelists show- World Bank headquarters | March 23, 2017 cased new work on results-based financing, The panel discussed innovative financing shared the latest evidence and lessons, and mechanisms in education. The discussion highlighted the benefits, opportunities, chal- highlighted the pros and cons of various lenges, and limits of RBF. The REACH team mechanisms with the aim of creating a presented its strategy paper on RBF. general understanding of the circumstances Expert Panel on Impact Bonds: in which they can be best applied. The Can They Deliver Better Outcomes, panelists were representatives of R4D, the Faster and Cheaper? Brookings Institution, the Center for Global Co-hosted by Education, Finance & Market, and Trade and Competitiveness Global Practices at the Development, the East Meets West Founda- World Bank headquarters | October 4, 2017 tion, and the International Commission on In this panel discussion, experts from Financing Education Opportunity. A video of the Brookings Institution, Instiglio, Social the discussion can be viewed here. Finance UK, and the Global Lead for the Expert Panel on RBF in Education: World Bank’s Finance and Markets Global Financing Results to Strengthen Systems Practice engaged the audience in a conver- At the Building Evidence in Education (BE2) meeting | sation about what has happened since the Florence, Italy | April 5, 2017 launch of the first development impact The BE2 meeting provided REACH and its bond (DIB) in education (Educate Girls in partners with an opportunity to address India), what it means for other sectors, and a group of international researchers and how this new financing model affects our donors working on education. REACH work in international development. hosted a panel on RBF with representation A PowerPoint presentation by the panelists from UNESCO, the International Commis- can be viewed here. 22 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) RBF for Breakfast This series brings together practitioners working on a specific subtopic of results- based financing in an informal setting to exchange operational lessons learned and to identify remaining gaps. The webinar events take place in the mornings (EST) to enable overseas staff to participate. They are “by invitation only” to ensure a more productive, fast-paced, and candid small Indicators/RBF Monitoring Systems group discussion than would be the case if October 5, 2017 I KLI grants: Colombia, Haiti, the participants were presenting to a larger and Vietnam audience with less in-depth knowledge Country teams have specified that having on the topic. The key conclusions of these only limited available data on education as a expert discussions are captured in the key bottleneck to linking financing to results. “Quick-and-Dirty Operational Notes” and RBF can only work if systems are in place then shared with a wider audience. to provide robust and reliable information RBF for Breakfast events are also used to about the results that are required before help further develop the REACH compen- the financing is released. In recognition of dium. Additional events in 2018 will cover this issue, REACH has funded some grants incentives to students and parents, perfor- to encourage country teams to explore how mance-based contracts, and fiscal transfers. to identify the most appropriate indicators and to create data monitoring systems to Performance-based School Grants enable the successful operation of RBF. In June 28, 2017 I KLI grants: Indonesia and Mozambique this event the task team leaders of three Since school grants are a big part of educa- REACH-funded RBF projects shared their tion projects at the World Bank, many teams experiences, and participants were invited are thinking of ways to help governments to provide additional comments, feedback, to make them more performance-oriented, and advice. Link to RBF Quick-and-Dirty with the hope of ultimately achieving more Operational Note on the Lessons Learned learning. The discussion between the by the Task Teams participants of this event highlighted some of the main questions that should be asked Teacher incentives and pay for performance I October 5, 2017 I KLI when starting to design and implement grants: Tanzania and Rwanda. performance-based school grants. Five areas The conversation in this event focused on related to the design and implementation the aspects that need to be considered of performance-based school grants were when designing and implementing teacher addressed: indicators, incentives, sanctions, incentives. The participants discussed the verification, and sustainability. Link to RBF kinds of problems that incentives can and Quick-and-Dirty Operational Note on the cannot solve, including how to diagnose the Lessons Learned by the Task Teams underlying problems and how to design an incentive scheme accordingly. REACH ANNUAL REPORT 2017 23 Box 6: RBF for Breakfast: Designing Quick-and-Dirty Operational Notes and Implementing Each of these notes capture the lessons Teacher Incentives discussed at a REACH event: ■■ Quick-and-Dirty Operational Note #1: The latest RBF for Breakfast event gathered Do Fiscal Transfers Generate Results? together about 20 experts working on teacher incentives from different parts of the ■■ Quick-and-Dirty Operational Note #2: World Bank, including the Strategic Impact Lessons Learned: Implementing GPE’s Evaluation Fund (SIEF) and the Mind, Behavior, Variable Tranche. (Given the detailed and Development Unit (eMBeD), as well as nature of this conversation, this internal representatives from the Center for Global note was directly circulated among WBG Development, GPE, and academics. The KLI and GPE staff.) grantees represented projects in Tanzania ■■ Quick-and-Dirty Operational Note #3: and Rwanda. How Are Schools like Clinics? The participants discussed what to consider ■■ Quick-and-Dirty Operational Note #4: when designing and implementing teacher Things to Consider in the Design of incentives. The discussion centered around the Performance-Based School Grants following key issues: ■■ Quick-and-Dirty Operational Note #5: How to identify the underlying issues and How Can Data Support Results-Based make sure that the solutions address those Financing? issues. Incentives usually need to be paired ■■ Quick-and-Dirty Operational Note #6: with other initiatives such as training or Can Impact Bonds Deliver Better Results improved recruitment practices since teacher Faster and Cheaper? motivation might not always be the only problem. Program for Results (PforR) in What kind of modalities to use. Group incen- Education Training tives may work better to address gaming than REACH partners with the World Bank’s individual incentives. Also, financial and non- Operations Policy and Country Services financial incentives can be mixed, but both have (OPCS) and Education Staff Development to be designed to be sustainable and scalable. Program (ESDP) to organize regular training courses on how to use the World Bank’s How to make the link between effort and RBF instruments in education projects: reward explicit and equitable. Without this, (i) Program-for-Results (PforR) and (ii) Invest- there will be gaming and loss of trust in the ment Project Financing (IPF) with DLIs. scheme. It needs to be clear that rewards go to those who deliver learning. In 2017, REACH held three different PforR How to ensure that data systems are events, two during Human Development adequate to support incentives. Without Learning Week in May 2017 and one in reliable data, no incentive scheme can work October 2017. Human Development Week properly. The right systems must be in place to brings together all Human Development capture student learning (assessments), and to sector staff from across the world to Wash- track teachers (HR/personnel), and there must ington D.C. for a week of knowledge sharing be sufficient capacity to manage these systems. and learning. 24 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) RBF on Trial: Why and When to Use Program for Results (PforR): Scaling Up Results-Based Financing Instruments for Sustainable Results in the Education. Organized jointly with the Social Protection and Organized jointly with the Education Staff Health Global Practices and the Human Development Development Program (ESDP) and Operations Policy Vice President’s office | May 8, 2017 and Country Services (OPCS) I October 26, 2017. In this mock trial, task teams from educa- This event consisted of a one-day training tion, health, and social protection testified session for World Bank and GPE opera- for and against the use of RBF. The pros- tional staff to enhance their knowledge of ecution and defense examined and the PforR instrument and introduce key cross-examined six task team leaders who concepts that are essential to designing and served as witnesses representing projects implementing PforR programs. The training from Albania, the Dominican Republic, the included an operational panel of World Bank Gambia, Moldova, Nepal, and Nigeria. task team leaders sharing their experiences Speaking from their personal experiences and practical lessons learned from PforR in serving clients with RBF instruments, they operations in in the education sector. testified to both the virtues and limitations of the RBF approach. The audience of human development staff served as the jury Communications for the trial, providing a verdict based on REACH revamped its website in 2017 to what they heard throughout the afternoon. make information even more easily acces- sible to a general audience. All of the Operational Panel on Program-for-Results (PforR) in Education: Towards Strengthening information on the site was updated, and Institutions and Financing Results new information is added regularly. A new May 10, 2017 “Resources” page was introduced catego- This session was designed to familiarize rized by RBF sub-topics. task team leaders with the PforR instrument In addition to the REACH website, event in the context of the education sector. The information and knowledge products are task teams for education PforR programs in shared by email invitations with the World Bangladesh, India, Lebanon, Mozambique, Bank staff (those working in education and Nepal, Tanzania, and Vietnam drew from other relevant sectors) and approximately their operational experience in designing 5,000 external contacts who have expressed and implementing the programs to provide an interest in receiving information from the insights and practical tips on how to World Bank Education group. When REACH leverage opportunities and address chal- co-hosts an event with another units or lenges. Discussants from the OPCS and organization, these bodies also share infor- REACH provided the institutional framework mation on RBF with their networks and on and guidelines. their websites when appropriate. Moreover, REACH is making greater efforts to promote its knowledge products during its learning events and presentations. On these occasions, REACH also collects feedback from participants not only about the events but also about their knowledge sharing preferences in general. REACH ANNUAL REPORT 2017 25 Box 7: Results-based Financing on Trial Just-in-Time Support In this mock trial session, six World Bank task Just-in-Time support is an advisory service team leaders served as witnesses representing that REACH offers to World Bank task teams projects from Albania, the Dominican Republic, that are working on RBF. The World Bank is the Gambia, Moldova, Nepal, and Nigeria. the largest external financier of education Speaking from their experiences in serving in the world and has education teams in all clients with RBF instruments, they testified to regions. These teams often need practical both the virtues and limitations of RBF. advice on how to talk to clients about RBF as well as technical advice on how to design The main arguments were: RBF interventions and guidance on how to ensure strong monitoring and evaluation. DEFENSE: RBF strengthens systems in a way that is superior to traditional financing. Just-in-Time support ranges from informal PROSECUTION: RBF is simply more of the phone calls and videoconferencing to same, just renamed. formal peer review, a process where staff members provide informed comments on DEFENSE: RBF aligns actors and forces them project papers. This is captured in internal to focus on results. World Bank quality assurance systems. PROSECUTION: It doesn’t matter if actors are Just-in-Time support very rarely covers aligned if they don’t have the technical capacity travel costs; for the most part it generally to implement RBF or are constrained by external covers a small percentage of staff time. factors beyond the government’s control. However, REACH staff also contribute a significant amount of time for free to the DEFENSE: RBF uniquely institutionalizes tightknit community of RBF practitioners measurement in countries and highlights the who contribute to REACH and participate in importance of evaluation. REACH events and activities. PROSECUTION: Too often, RBF programs rely on parallel data systems to independently Here are some examples of the validate government data systems, thus institu- customized support that REACH tionalizing measurement in the wrong places. provided in 2017. DEFENSE: Using RBF requires recipients to COLOMBIA: REACH served as a peer continue to pursue results regardless of crises or reviewer for the RBF monitoring system and changes in governments and enables them to proposed pilot in Bogota. The tool requires receive donor resources in a predictable way. buy-in from many stakeholders in order for it PROSECUTION: The cost of predictable to produce accurate data. REACH convened financing is that RBF operations sometimes a small clinic to give the project team ideas produce unambitious results that are not worth about how to encourage stakeholders to use what donors pay. the system and at what level to incentivize. The team was concerned that the tool would When the audience was polled, the defense of be used to produce a ranking of schools, RBF prevailed, and participants said that they which would lead to problems with incen- were now more likely to use RBF to push policy tivization at the school level. Ultimately, the reforms and to use RBF in varying contexts from team decided to introduce incentives at the middle-income countries to fragile states. meso level of the education system to avoid gaming and unintended behavior. 26 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) ETHIOPIA: The Bank’s Ethiopia task team in how to design the DLIs that are funded by education was preparing a new operation the variable tranche. The health project has and was unfamiliar with the specific nuances since been approved by the World Bank of various RBF instruments. REACH provided Board of Governors and includes incentives technical advice on how to design a PforR to service providers to reduce childhood and other modalities to support donor stunting. contributions and later served as a formal MOZAMBIQUE: The Bank’s task team peer reviewer. leader for Mozambique recently changed, GUINEA BISSAU: Guinea Bissau is a GPE and REACH provided the new staff member grant recipient, though it is recused from with guidance on how best to work with the implementing the variable tranche given government to revise their existing ineffec- the small size of the grant. However, there tive school grants model and turn it into is a strong desire to push for more results- something more performance-oriented. based approaches within the country given TUNISIA: Preparations for a new project are its history of poor outcomes. REACH was a ongoing in Tunisia. The project will contain peer of the concept note for an upcoming DLIs, and REACH has served as a formal education project that had a component on peer reviewer and as an advisor to the school grants and provided the task team project team, providing suggestions on DLIs, with advice on how to structure the grants the verification protocols, and how to ensure so that they would be most effective. As a effective implementation of the RBF element result, the project has revised the funds flow of the project. for the school grants. UGANDA: New education financing MADAGASCAR: REACH provided some guidelines have been prepared with the technical advice to both the Bank’s educa- Governance Global Practice at the World tion and health teams in Madagascar as they Bank. REACH provided technical support were introducing RBF to the government. on the performance allocation section and REACH was a peer reviewer of the concept made recommendations on how to best use note for an education project that includes the performance aspect of the transfers. This a GPE variable tranche grant. Given that the information was shared with the Uganda task World Bank is implementing five of the six team in education as they are implementing GPE variable tranche grants, World Bank a project that contains fiscal transfers. task team leaders have unique insights into REACH ANNUAL REPORT 2017 27 The Path Ahead for Results-based Financing REACH has a solid conceptual basis, a lean Distilling and Disseminating management approach, and an established Knowledge reputation as the leading operational platform for results-based financing in The KLI Compendium: In 2018, the REACH education. In the future, its aims are to team will publish the Compendium of generate knowledge on specific educa- research funded by KLI grants and will tion issues about which little information is publicly disseminate the findings through a currently available, continue to distill and series of events and webinars. disseminate knowledge on RBF as widely Learning and training: REACH will continue as possible, and use RBF to strengthen to focus on building capacity and knowledge country education systems. on RBF among practitioners, including World Bank teams and other development Growing the RBF agencies such as GPE. Evidence Base Knowledge, Learning, and Innovation Strengthening grants: With REACH set to close in 2020, Country Systems further calls for KLI proposals are unlikely to Partnerships and leverage: REACH has be possible, but other activities to grow the established solid relationships with a number RBF evidence base will be considered during of internal and external partners through the annual REACH donor meeting in late its outreach efforts. As a result, REACH has February 2018. demonstrated its value to operational teams serving country clients that wish to apply an RBF approach in their education systems. 28 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) REACH Financial Statement, FY15-FY18 (in US$) Expense Total Cumulative Disbursements Category Commitments Disbursement FY18 FY17 FY16 FY15 Program Management and $1,850,000.00 $247,673.37 $ 665,115.07 $358,821.42 $62,264.59 $1,333,874.45 Administration* Knowledge $71,502.91 $211,008.55 $109,462.78 $6,487.56 $398,461.80 Sharing & Learning Just-in-Time $32,733.00 $108,636.40 $56,946.52 $1,861.04 $200,176.96 Support KNOWLEDGE, LEARNING AND INNOVATION KLI Call 1 (FY16) $1,657,411.58 $114,085.48 $1,011,635.36 $338,719.92 $1,464,440.76 KLI Call 2 (FY16) $1,609,125.00 $279,550.92 $615,265.78 $81,387.42 $976,204.12 KLI Call 3 (FY17) $2,060,000.00 $426,367.32 $426,367.32 KLI Call 4 (FY18) $1,121,000.00 $14,504.00 $14,504.24 KLI Call 5 (FY19) $4,000,000.00 (estimated amount) Compendium $90,000.00 $22,808.00 $22,808.00 COUNTRY PROGRAM GRANTS Nepal DLIs $4,000,000.00 $2,000,000.00 $2,000,000.00 (Recipient-Executed) Nepal IE & $650,000.00 $65,207.64 $40,127.50 $44,109.04 $149,444.18 Supervision Lebanon DLI $4,000,000.00 $1,000,000.00 $1,000,000.00 (Recipient-Executed) Lebanon IE & $650,000.00 $22,005.30 $2,162.22 $24,167.52 Supervision Total $21,597,536.58 $4,296,438.18 $2,653,950.88 $989,447.10 $70,613.19 $8,010,449.35 Total donor $28,013,669.30 contributions Administration fee* $286,304.15 Remaining funds $6,129,828.57 available Notes: *Administration fee taken from donor contributions made before July 15, 2015. FY = fiscal year, June 30 – July 1. Data as of February 2, 2018. REACH ANNUAL REPORT 2017 29 Results Framework Objective / Indicators (cumulative) Baseline 2016 2017 2018 Actual Actual Planned GOAL OF THE REACH TRUST FUND: To support efforts toward more and better education services, especially to those most excluded, by helping country systems focus more sharply on results. RESULT 1. Country systems and capacity for 0 16 22 25 RBF in education strengtheneda/  inancial and technical support to Country 1.1 F 2 2 2 Programs and Pilots 1.2 Financial and technical support to activities for 18 28 34 Knowledge, Learning, and Innovation (KLI) 1.3 Just-in-time support to WBG program teams, 6 11 14 informing RBF approaches on-the-groundb/ RESULT 2. Global evidence base for RBF in education is developed and made publicly available 2.1 Total number of KLI grant policy notes on RBF 0 0 11 approaches prepared and disseminated 2.2 Total number of impact evaluations [ongoing] 6 8 10 RESULT 3. Evidence about RBF in education is publicly disseminated 3.1 Approach Paper on RBF in education 1 1 1 published and disseminated 3.2 Number of RBF learning events 2 6 10 (open to the public) 3.3 Number of external blogs/articles published 3 3 7 3.4 REACH website operating and regularly 1 1 1 updated with new knowledgec/ Notes: a/ Represents number of countries. Some countries have benefitted from different kinds of REACH support. b/ Countries that are not receiving CPG or KLI grants. c/ Including externally generated evidence. 30 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Annexes Annex 1: KLI Call for Proposals Process REACH team issues call for proposals with background document outlining the theme and criteria for eligibility. REACH also designs a rubric based on those documents as a tool for review committee members to score proposals. REACH launches the call and advertises widely through a distribution list targeted towards internal staff and external organizations who have expressed interest in receiving informa- tion about REACH. The call usually stays open for at least 4 weeks. During this time, REACH continues to advertise the call and responds to requests to set-up meetings, phone calls, and or webinars with interested applicants. REACH also assembles the review committee, which comprises a mix of individuals with different backgrounds – economists, education specialists, reading specialists, and/or others who are well-versed in RBF. The review committee always consists of the REACH program manager and one manager from the Education Global Practice at the World Bank. Once the call for proposals closes, the review committee scores proposals independently. Scores are then normalized and the review committee meets to discuss which ones should be selected for additional follow-up and/or interviews. Once the review committee agrees on the shortlist, the World Bank education management team and task team leaders in the respective shortlisted countries are contacted to ensure that proposed activities align with country priorities. After follow-up and interviews, the review committee meets again to decide on the final grantees. These final grantees are vetted by the REACH technical review committee, which consists of education managers who are in charge of World Bank engage- ment in various regions across the world. Upon final vetting, grantees are contacted and sent an offer letter stipulating the terms and conditions for accepting REACH funds. This includes payment in tranches based on the deliverables. Every grant is required to deliver some type of analytical work or research, along with a mid-term report. For external organizations, they must also pass through corporate procurement processes at the World Bank, which are in a separate unit from REACH. REACH ANNUAL REPORT 2017 31 Annex 2: Description of KLI Grants from Calls 1-4 Call 1: General KLI Call HAITI: From Financing Access Results to Learning Results. The Government of Haiti has successfully increased primary school enrollment through a results-based mechanism called the Tuition Waiver Program, which pays schools fees for enrolling poor children in non- public schools. It intends to build on this achievement by providing financial incentives for schools to improve conditions, instruction, and learning outcomes for poor children in these schools, while reducing grade repetition and dropouts. The KLI grant is enabling Haiti to develop the capacity and systems necessary to develop a functioning RBF mechanism so as to inform future Bank and other donor funding, and, more importantly, national policy. Haiti’s experience of establishing these preconditions will generate knowledge about how to lay the foundations for RBF in low-income, fragile situations. INDONESIA: Piloting Performance-based Contracting in Schools in DKI Jakarta. Education officials in DKI Jakarta face two problems. Firstly, resource allocation in schools in Jakarta can be inefficient. Secondly, a set of National Education Standards governing student learning, teacher competency, and school facilities among others, have been created, but are not yet properly implemented in schools. This activity seeks to address both challenges by establishing a contract between the school and the local government linking key perfor- mance and competency indicators in the National Education Standards to school funding. This would improve the ways in which schools in DKI Jakarta plan their budget by ensuring that school programs and activities are linked to the performance indicators in National Education Standards. The KLI Grant is providing the research team with the resources they need to implement a pre-pilot, ensuring that a design is developed which can be scaled into a regional pilot if effective. The research will generate knowledge about designing incen- tives which link local budgeting to existing national standards. MOZAMBIQUE: Learning from Performance-based school grants. Ministry of Education (MOE) has engaged in a far-reaching reform program to upgrade teachers’ knowledge and performance, and strengthen service delivery at local level. The MOE intends to incentivize school performance by providing direct financial incentives through a performance-based school grants scheme, complemented by other key interventions aiming at improving local and school management. A school-grants scheme is already in place, but has not been as effective as was hoped. The KLI Grant will fund the pilot of a revised school-grants scheme, using lessons learned from the existing scheme to improve the incentive system. The team will simultaneously support the development of management tools for mid-level manage- ment, in particular school directors and district office, to enable them to administer the school grants effectively. This intervention will generate lessons learned about the chal- lenges and successes of implementing school grants, with a focus on how they can support improvement in learning outcomes. It will also provide evidence about the impact of strengthening mid-level school managers (school directors and districts officers) to perform adequate administrative and pedagogical supervision as part of a school grants program. 32 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) MOZAMBIQUE: Keeping Rural Girls in School – Testing the Impact of Cash, Goods, and Information. In Mozambique rates of primary completion are low, especially in rural areas, where only 14 percent of boys and 8 percent of girls complete upper primary school. The KLI Grant will fund research which tests the effect of demand-side incentives on school attendance for girls. The study will compare the impact of providing girls with tokens to buy school-related items such as uniform and supplies, providing households with cash, and providing households with information about school attendance without any financial or in-kind incentive. The research will generate knowledge about whether it is more effective to incentivize students or households, and about whether information is an incentive in itself. NIGER: Resolving the Indicator Bottleneck for Results Based Financing. The Government of Niger has access to a great deal of data about its education system and young population. Available data comprises learning outcomes (PASEC and EGRA), social development indica- tors, a population census from 2012, and household surveys. The Government of Niger is interested in using results-based financing mechanisms in its education system. However, the available data don’t yet translate into meaningful, usable, and reliable indicators that the government and donors can use in order to sharpen a focus on results. The KLI grant is enabling the Government of Niger, in collaboration with the World Bank, to establish a sustainable monitoring and evaluation system. This will lay the foundation for future results- based financing operations in education in Niger, using more effective indicators and best available data sources. The Government of Niger’s experience in moving towards results- based policy in a resource-constrained environment is expected to produce useful lessons in establishing national data systems. RWANDA: Pay-for-Performance for Teacher Recruitment and Retention. The Government of Rwanda has established a system of performance contracts for public-sector employees, which allows for performance-based bonuses averaging three percent of salary. The KLI grant is funding a study which builds on the existing civil-service contracts, by introducing a bonus scheme that rewards teachers who score within the top 20 percent of their district on this performance measure with a merit bonus worth 15 percent of base salary. The research will address two questions. Firstly, whether a pay-for-performance (P4P) scheme can improve teacher performance, and produce student learning gains. Secondly, how effective are P4P contracts at attracting skilled and motivated teachers to undersupplied schools, particularly in rural areas? This research will generate information about how P4P schemes can alter the demographic spread and characteristics of teachers. TANZANIA: Aligning Teacher Pay with Performance of All Students. In 2015 the Govern- ment of Tanzania announced its commitment to improving access and quality in its education system by using innovative approaches to tackle long-standing problems in the quality of service delivery. This research provides teachers with bonus pay based on the learning outcomes of their students. The KLI Grant is funding a study which compares the impact of rewarding relative and incremental gains in student learning against rewarding teachers whose students pass a defined threshold. This research builds on a previous study by Twaweza that sought to test the efficacy of a simple pay for performance scheme that rewarded teachers if their students passed a simple test. By continuing to work in this set of schools, the research will provide insights into the “long term” effect of learning in an REACH ANNUAL REPORT 2017 33 environment where teachers have a performance pay scheme. The research will generate information about how teacher pay for performance programs can be best structured to benefit all children. TANZANIA: Incentives for Students to Stay – and Succeed – in School. In Zanzibar, almost half the students entering secondary school drop out before secondary completion. The Ministry of Education and Vocational Training Zanzibar intends to reverse this trend. The KLI grant is enabling the research team to generate clear guidance to the Ministry on how to design performance-based incentive schemes for students to maximize learning impacts and reduce dropouts at the secondary level. The research will answer questions about whether individual targets or a team tournament is likely to be more effective to incentivize poorly performing students, and how results-based financing can help overcome psycholog- ical barriers that might prevent students from responding to performance-based incentives. This research is expected to inform the body of evidence on how financing demand-side incentives can lead to better results. VIETNAM: Are School Characteristics and Teaching Practices Reliable Proxies for Learning Gains? Vietnam’s Ministry of Education and Training is in the process of improving the general education teacher performance evaluation system, introducing new curricula and methods of instruction, and developing a comprehensive learning assessment system. Vietnam also has a large amount of data available about learning across the whole system. The KLI Grant is funding research which will use existing data to establish the underlying factors which affect school quality in Vietnam. The results will be used to inform the redesign of the general education teacher performance evaluation system using evidence about what works in Vietnam. The research will generate knowledge about Vietnam, and will establish a model which could be adapted by other countries to evaluate the factors which impact learning in their unique contexts. Call 2: General KLI Call CAMEROON: Results-Based Financing for Improved Education Service Delivery Building on the success of RBF in the health sector in Cameroon, the government of Cameroon is keen to experiment with RBF as a tool for increasing girl’s enrollment and improving service delivery in two of its most disadvantaged districts. The KLI Grant will fund a feasibility study and pre-pilot for a performance-based school grants program in the North and far-North of the country. The lessons learned will inform a two-year pilot, which will be scaled up beyond pilot districts if successful. CHINA: Assessment of Teaching Practices for Changes in the Classroom The Guangdong Department of Education is innovating to improve the impact of in-service training towards greater effectiveness of its teaching force. The KLI Grant will fund a pilot intervention to incentivize teacher-training institutions who deliver in-service training that teachers actually use in classrooms to improve student learning outcomes. 34 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) COLOMBIA: Results-Based Monitoring System The Colombian government has ongoing efforts to manage its education system based on results. Since 2014, Colombia has had a “synthetic index of education quality” for all basic education schools, with yearly targets at the school level. However, this index is limited, and more information is needed for policymakers to make well-informed decisions. The KLI Grant will support the development of a results-based monitoring system that covers multiple dimensions of education quality, with a view to targeting fiscal transfers towards improvement in those dimensions. DEMOCRATIC REPUBLIC OF CONGO: Impact Evaluation of RBF approach in South Kivu The Government of South Kivu, in partnership with the Dutch NGO Cordaid, has been piloting RBF to improve education service delivery. As the end of the pilot approaches, the KLI Grant will fund an impact evaluation to assess whether its RBF approach has generated better education results and whether learning outcomes in primary schools receiving RBF subsidies are higher than in primary schools that do not receive them. INDIA: Improving Teacher Performance through Outcome Linked Incentives The Government of Bihar is considering designing a teacher performance-pay program to improve service delivery in its schools using a teacher composite score designed by the Indian civil society organization Pratham. The KLI Grant would fund research to: i) test the validity of an existing teacher assessment/composite score to inform this work; ii) if the assessment is valid, design a pilot for this program. INDIA: New Horizons India’s National Institute of Open Schooling (NIOS) provides second-chance education for religious and other minority students who have dropped out of school. The KLI Grant will fund research to evaluate the design of performance contracts between the Government of India and service providers, as well as the cost-effectiveness of varied levels of monitoring. INDONESIA: Assessing the Impact of Alternative Approaches to Linking Funding to School Performance in Jakarta The two activities under this proposal are part of an overarching exercise by the Jakarta government to experiment with different approaches to linking financing with results: a) the introduction of a performance and equity school grant and b) the introduction of performance contracts with schools. The already implemented performance and equity school grant program looks at whether the announcement of an incentive would be enough to change school behavior and ultimately student performance. The pilot for the performance-based contracts goes further by using the national standards to identify areas of weakness at the school level and developing school improvement agreements between the school and the district office based on this information. The KLI Grant will fund the evaluation of both activities. REACH ANNUAL REPORT 2017 35 MOROCCO: Support to Performance Contracting in Education The Government of Morocco is keen to use performance-based contracts to bring about education system and governance reform. The KLI Grant will support the development and piloting of performance-based contracts between local, regional, and national government, in two regions in Morocco. REPUBLIC OF CONGO: Congo Citizen Voice for Education It is believed that leakage in the education system in Congo-Brazzaville results in only a small proportion of funds disbursed by the Ministry of Finance arriving at schools. The KLI Grant will finance the set-up of an open data system, which will allow community members to report on the funds which arrive at their local school. If the system is effective, it could be used to inform a RBF program for school finance. Call 3: REACH for Reading BANGLADESH: Anuperona Book Challenge (Grantee: IREX) The Anuperona Book Challenge will generate incentives for communities to meet the need for diverse materials in mother tongue languages and align with the Prime Minister’s Access to Information (A2I) project. This intervention is a results-based book creation activity that uses the Bloom software and will train a cadre of local content creators in Bangladesh with the tools and skills to create high quality supplementary readers in mother tongue languages. The resulting content will enhance the supply of books and be made readily available for digital distribution on the national Reading Hub. The KLI grant will fund a longitudinal, attitudinal survey to assess regional/national readiness for a results-based competitive approach to content creation, the implementation of the book challenge, and a cost analysis on the use of results based competitive book challenge in Bangladesh. CAMBODIA: Enhancing Book Distribution in Cambodia (Grantee: World Education) Two key challenges to improving supply chains in Cambodia include (1) end-to-end visibility of data to drive the supply chain and (2) leadership and commitment throughout stake- holder organizations in establishing the importance of data to inform decisions and track performance. This intervention seeks to generate rigorous evidence on the effectiveness of a RBF intervention that incentivizes 400 school communities in 2 provinces and other key players in the supply chain in order to build a robust supply chain for the distribution of books and supplies to classrooms in Cambodia. This intervention will facilitate account- ability and transparency in the book supply chain in Cambodia by increasing the visibility of end-to-end data in the supply chain and enable School Support Committees, school staff, parents, district administration and Ministry staff to better track the ordering and delivery of books. The KLI grant will fund a baseline survey that will define the design of the RBF inter- vention, the implementation of Track and Trace and post surveys to measure the availability of books post intervention.   36 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) SOUTH AFRICA: National Public-Private Storybook Development Initiative in South Africa (Grantee: Room to Read) This intervention will facilitate a public-private partnership to develop national guidelines and create and distribute storybooks in support of the existing government Read to Lead campaign. The KLI grant will fund the creation of a national public-private working group to establish best practice recommendations for early-grade storybooks, the provision of technical assistance and capacity building to publishers of early grade storybooks and the distribution of over one hundred thousand books to government schools through a pooled procurement and ‘sliding scale’ model for book purchases. This intervention seeks to generate rigorous evidence on the effectiveness of a RBF intervention that incentives each of three aforementioned intervention steps that will ultimately benefit the South African government and children in Limpopo, Mpumalanga, and Kwa-Zulu Natal.    ZAMBIA: Evaluating Results Based Textbook Delivery Systems (World Bank) The textbook delivery system in Zambia relies on centralized procurement at the national level followed by the transfer of books to District Education Board Secretaries (DEBS) offices at the regional level and then final delivery by DEBS offices to schools. This intervention will identify and evaluate the provision of results-based delivery stipends to better align incen- tives so that local language textbooks reach their final destinations. The KLI grant will fund the roll out of two randomized results-based delivery stipends to district offices and private publishers aimed at identifying the most effective entity to be incentivized to achieve lower textbook shortfalls at the school level through improved textbook delivery/retrieval systems. Call 4: Meso Level of the Education System BANGLADESH, INDIA, AND TANZANIA: Improving District-Level Decision-Making In most countries, the ability of schools to function smoothly and access resources is mediated by District Education Officers (DEOs). However, very little is known about this level, especially with respect to how decisions are made and how success is evaluated. The KLI grant will fund the development of a novel (three-stage) field-based lab experiment using mobile phones that aims to better understand how DEOs in three different countries make decisions. The experiment will test assumptions about DEOs in order to generate information on how to better align their decision-making with results-based financing prin- ciples and to ensure that future results-based interventions targeted at DEOs are evidence based. COLOMBIA: Using School Information to Improve Service Delivery This KLI grant is a continuation of a previous grant that supported the development of a results-based monitoring system in Colombia by creating a multi-dimensional set of indica- tors that focus on education quality. In this iteration, the system will be piloted in Bogotá, in order to transform three distinct aspects of decision making at the meso level: targeting of programs, targeted support for school improvement plans, and allocation of resources. REACH ANNUAL REPORT 2017 37 DOMINICAN REPUBLIC: Establishing Conditions to Improve the Performance of School Districts The Dominican Republic has spent a decade decentralizing financial resources and responsi- bilities to school districts and schools with the hope of better responding to local needs. As part of this process, the Ministry of Education is seeking to improve sector management at the meso-level, by linking district-level performance agreements with the financial transfers they already receive. The KLI grant will support the design and implementation of these performance agreements. INDIA: Technology-Enabled Strengthening of Elementary School Monitoring Systems In the state of Jharkhand the government has hired a cadre of Cluster/Block Resource Persons (CRPs/ BRPs) across the State as contract staff on a fixed remuneration. These individuals often have minimal professional experience and little oversight for their many tasks, which include visiting schools, acting as mentors to teachers, and galvanizing local participation through school committees. The KLI grant will support the improvement of quality assurance mechanisms by empowering and incentivizing CRPs/BRPs through on-the- job training and providing them with technological tools to collect real-time data. PERU: Evaluating Performance-Based Career Path and Compensation Reforms for School Leaders Improving the quality of the education system has been the focus of ongoing reforms in Peru. One aspect of those reforms has been focused on formulating a performance-based career path and compensation program for managerial positions in schools at all levels of basic education. The KLI grant will fund an impact evaluation to assess the implementation of a 2014 and 2016 system-wide meso-level change in policy which primarily focused on improving the management and organization of schools. The results of the evaluation will inform policy and strengthen the school leader appointment systems and other related education policies in Peru. TANZANIA: Strengthening School Inspections Evidence from SDI surveys in Tanzania indicate that low-cost improvement in school management, such as better teacher attendance and time-on-task, could significantly improve learning outcomes. Tanzania’s national school inspectorate produces detailed information on school performance, but has struggled to reach its target of inspecting 50 percent of schools each year. The KLI grant will provide support and incentives, through results-based financing, to district and sub-district level education officials to help schools implement low-cost recommendations for improved service delivery. 38 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) Results in Education for All Children (REACH) Education Global Practice World Bank 1818 H Street, NW Washington DC, 20433 USA Website: www.worldbank.org/reach Email: reach@worldbank.org