A GUIDE FOR EFFECTIVE RESULTS-BASED FINANCING STRATEGIES A GUIDE FOR EFFECTIVE RESULTS-BASED FINANCING STRATEGIES @2018 The Global Partnership on Output-Based Aid The World Bank 1818 H Street, NW Washington, DC 20433 USA Website: http://www.gpoba.org/ Contact us at: gpoba@worldbank.org All rights reserved This work is a product of GPOBA, part of The World Bank Group. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgement on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Note: All currency amounts are in U.S. dollars unless otherwise noted. This report was authored by Instiglio, Inc. For more information, visit http://www.instiglio.org. Commissioned by the Global Partnership on Output-Based Aid (GPOBA). Rights and Permissions This work is licensed under the Creative Commons Attribution-NonCommercial- NoDerivatives 4.0 International License. To view a copy of this license, visit http:// creativecommons.org/licenses/by-nc-nd/4.0/ You are free to: Share — copy and redistribute the material in any medium or format. The licensor cannot revoke these freedoms as long as you follow the license terms. Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. NonCommercial — You may not use the material for commercial purposes. NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material. No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. All non-Creative Commons images in this publication require permission for reuse. All queries on rights and licenses should be addressed to GPOBA, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; e-mail: gpoba@worldbank.org. Foreword By the Global Partnership on Output-Based Aid At the Global Partnership on Output-Based Aid (GPOBA), we are committed to development financing solutions that link funding to actual results. For 15 years, GPOBA has used results-based financing (RBF) to assure delivery of basic services to those most in need. Through a portfolio of nearly 50 projects across the globe, we have tested the use of RBF in water and sanitation, energy, solid waste, health, education, and telecommunications, demonstrating the efficacy of RBF in responding to global pressures for greater aid effectiveness. GPOBA is a multi-donor trust fund, managed by the World Bank Group, that uses a form of RBF called output-based aid (OBA) to improve access and delivery of basic infrastructure and social services for the poor. While GPOBA was originally established to focus exclusively on OBA, we have found that working in complex and changing environments requires greater flexibility in the financial instruments we use. Therefore, backed by our donors, we are expanding our mandate to include more flexible financing solutions to have a greater impact on our clients—low-income communities in developing countries. This includes not only broadening our menu of RBF instruments, but also leveraging the flexibility of RBF to design development projects which bring together public and private sector investors in order to maximize resources. This guide is a contribution toward that objective, with the vision of becoming not just a center of expertise on OBA, but on results-based approaches more broadly. It is our hope that this publication will provide guidance to your organization in the decision to adopt appropriate RBF approaches to solve development challenges. Shobha Shetty Practice Manager, Partnerships and Resource Mobilization Social, Urban, Rural & Resilience Global Practice The World Bank Group A Guide for Effective Results-Based Financing Strategies 1 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2. A Primer for RBF Practitioners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1 The three key steps of RBF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Categories of RBF instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 RBF trends in low- and middle-income countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3. Overview of the Diagnostic Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1 Contextual Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.2 RBF Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.3 The RBF diagnostic process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4. Diagnostic Component 1: Contextual Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.1 The RBF value-add . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.2 Conditions for RBF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5. Diagnostic Component 2: RBF Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.1 Choosing the RBF instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2 Designing the RBF instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.3 Strengthening and Scaling RBF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Appendix A. Results-Based Financing Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Appendix B. RBF Projects by Instrument Type and Sector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 A Guide for Effective Results-Based Financing Strategies 3 Abbreviations COD cash on delivery DIB development impact bond EG Educate Girls GPOBA Global Partnership on Output-Based Aid NGO nongovernmental organization OBA output-based aid OBD output-based disbursement PBA performance-based aid PBC performance-based contract PBF performance-based financing PBL performance-based loan PBT performance-based transfer PDBD performance debt buy-down PforR Program-for-Results (World Bank) RBF results-based financing SIB social impact bond WSP water service provider A Guide for Effective Results-Based Financing Strategies 5 6 INTRODUCTION A Guide for Effective Results-Based Financing Strategies 1 1. INTRODUCTION Enhancing the effectiveness of development spending is a priority for those engaged in addressing global poverty. This priority has intensified in recent years, driven by ongoing budget pressures, concerns with business-as-usual practices, and the growing attention to achieving measurable results, especially in the context of reaching the Sustainable Development Goals. Results-based financing (RBF) is an approach for driving development effectiveness that responds to this context and which development practitioners are increasingly embracing. In the last decade, at least $25 billion of development spending has been tied to results, an increase from just a few billion the decade before. 1 This surge reflects the growing use of RBF by organizations like the World Bank Group, bilateral aid agencies, and many national governments. The burgeoning interest in RBF is also reflected in the comments of World Bank Group President Jim Yong Kim: “Evidence shows that results-based financing has a significant impact – saving lives and expanding access to quality, essential health services for the poorest women and children in developing countries.”2 However, despite an upward trend in the use of RBF, the practice in certain thematic areas remains emergent and insufficiently documented. In what contexts does RBF work best and which instruments are most appropriate? What does it take to enable the use of RBF? How should RBF be designed to maximize impact? How can RBF approaches be strengthened and scaled over time? Understanding and answering these questions is critical to building a mature practice of RBF that consistently delivers on its potential to improve the results of development funding. The diagnostic tool presented in this report responds to this need. It provides a set of structured questions and frameworks to guide practitioners interested in using RBF to meet their development objectives. The approach outlined in this report draws on lessons from the global experience with RBF and includes insights from the work of Instiglio and the Global Partnership on Output Based Aid (GPOBA) on these issues.3 The diagnostic tool is written for development practitioners, with a specific emphasis on the perspective of development funders. Further, it focuses on issues of specific relevance to GPOBA and its work providing technical input on output-based aid (OBA) and RBF more generally. As such, the focus of the diagnostic tool is understanding where and how funders such as GPOBA can best use different RBF instruments to deliver impact. In addition, the diagnostic tool may also be of Estimate based on the RBF database, discussed in section 2.3. 1 ² World Bank 2013b. ³ Key sources include: Instiglio 2017; World Bank 2017; Clist 2016; NAO 2015; Perakis and Savedoff 2015; DFID 2014; USAID 2010. A Guide for Effective Results-Based Financing Strategies 7 interest to implementers seeking to engage with RBF and other stakeholders such as investors and policy makers. Given the diagnostic tool’s focus on developing country contexts, this report focuses on the use of RBF in low- and middle-income countries.4 The rest of this report is structured as follows: • Chapter 2 is a primer for RBF practitioners, describing what RBF is and why to use an RBF- based approach before providing an overview of how RBF has been used across various sectors and regions. • Chapter 3 introduces the diagnostic tool, with an overview of its purpose, structure, and intended use. • Chapter 4 covers the diagnostic tool’s contextual analysis component, outlining the approach for assessing the RBF value added and understanding the conditions for using RBF. • Chapter 5 presents the key questions that funders must answer to define an RBF strategy: What conditions need to be in place to use RBF? What specific RBF instrument is most appropriate to the context? What core RBF design features would maximize impact? How should the RBF strategy be refined over time to improve impact? The World Bank classification of countries is used in this report: “For the current 2018 fiscal year, low-income economies 4 are defined as those with a GNI [gross national income] per capita, calculated using the World Bank Atlas method, of $ 1,005 or less in 2016; lower middle-income economies are those with a GNI per capita between $ 1,006 and $ 3,955; upper middle-income economies are those with a GNI per capita between $ 3,956 and $ 12,235; high-income economies are those with a GNI per capita of $ 12,236 or more” (World Bank 2018) 8 A Guide for Effective Results-Based Financing Strategies This section introduces the concept of Results-Based Financing (RBF) and explains how it enables development funders to deliver better social results. Following a definition of RBF and an explanation of how it differs from traditional funding mechanisms, the section defines various RBF instruments and demonstrates how RBF is in fact a flexible practice rather than a rigid set of instruments. Finally, this section illustrates how RBF has been used across different developing countries and sectors. It presents the key RBF market trends, along with insights to inform RBF design decisions. 10 A PRIMER FOR RBF PRACTITIONERS A Guide for Effective Results-Based Financing Strategies 2 2. A PRIMER FOR RBF PRACTITIONERS KEY CONCEPTS OF THIS SECTION • Results: A generic term for outputs, outcomes, and impact. • Output: The immediate effect of activities such as tangible products or services rendered. For example, the number of schools that were refurbished or the number of hours of training delivered by teachers are outputs. • Outcome: A change in the ultimate beneficiary’s conditions or state achieved because of outputs, including improvements in beneficiaries’ knowledge, skills, behaviors, or well-being. For example, greater motivation from teachers, improved learning for students, and student lifetime earnings are all outcomes. Outcomes can be immediate, intermediate, and long-term: if teacher motivation, student learning, and life outcomes were all part of the same causal chain, motivated teachers would be an immediate outcome, student learning an intermediate outcome, and lifelong earnings a long-term outcome. • Impact: The desired long-term and sustained effect of a program on the lives of its beneficiaries. • Service provider: The agent that works on the ground to deliver a product or service to the program beneficiaries. It can be a public, private, or nongovernmental organization. For example, a public service provider could be a public utility company or a district health office, while examples of private service providers include private schools or private companies. • Development funder: An organization that finances development programs. Examples include foundations, bi/multilateral agencies, and governments. • Results funder: A development funder that makes payments conditional on the achievement of predefined results. Examples of results funders include governments, foundations, bi/multilateral agencies, and private funders. “Results funder” is a subcategory of funders. • Incentivized agent: The agent whose payments are contingent upon results. • Results-based financing: A financing arrangement in which part of the payments are contingent upon the achievement of predefined and verified results. RBF agreements involve two central agents: the results funder and the incentivized agent. To illustrate, a multilateral agency using RBF, such as GPOBA (the results funder), can engage a service provider (the incentivized agent) to increase access to potable water (a predefined result) and pay only when this result has been achieved. • RBF agreement: The specific configuration of parameters of an RBF instrument, including the selection of the results to be paid for, how results are verified, and the total payment made for the achievement of results. • RBF instrument: The type of RBF agreement signed between a results funder and an incentivized agent. There are four main categories of RBF instruments depending on the type of agent that is being incentivized: performance-based aid, performance-based transfer, performance-based contract, and impact bond. Section 2.3, RBF typology of instruments, presents further details on RBF instruments. • Fee-for-service: A payment mechanism typically used in the health sector in which providers are paid for each service performed. • Fee-for-service-conditional-on-quality: A form of incentive payment mechanism typically used in the health sector where providers are, at least partially, funded on the basis of their performance to meet targets or undertake specific actions. A Guide for Effective Results-Based Financing Strategies 11 In 2015, a rigorous evaluation of a village-level intervention5 called the Poverty Graduation Program6 published in the journal Science showed that it delivers significant impact in certain contexts, with some observers labeling it a “universal method to help the very poor.”7,8 Despite this success, the study also shows large variations in results across contexts, including the program failing to deliver any measurable impact in some contexts. Unsatisfactory results such as these, even with programs based on strong evidence, are all too common in the field of international development, where complex problems require tailored, context-based solutions. These unsatisfactory results can be perpetuated by traditional development funding arrangements where payments are based on inputs and activities.9 This sort of funding can be problematic because the prescription it entails can deprive recipient governments and service providers of the flexibility needed to experiment and adapt their programs to the context. 10 Results-based financing responds to these limitations through a simple but fundamental change in the way programs are funded: moving away from paying for inputs and activities and toward paying at least in part based on measurable results being achieved and verified. RBF thus provides additional guarantee of value for money compared to traditional funding. In the graduation program example, while traditional funding rewards implementers for delivering the activities of the program according to pre-established plans and timelines, an RBF approach instead (at least partly) pays for the achievement of verified results: increased income levels of the beneficiaries. With RBF, payments are closely connected to the intended development result and therefore bridge the gap between good intentions and results. The examples in table 2.1 further illustrate the paradigm shift RBF creates by moving away from paying for inputs and activity to paying for results. By tying payments to results, RBF ensures that funding supports outputs or outcomes. 5 The term “intervention” is used here and elsewhere in the report to refer to a wide range of efforts to address social issues. It includes service delivery, policy reforms, and infrastructure delivery. 6 A graduation program is an intervention used to increase the incomes of people living in extreme poverty. 7 Banerjee et al. 2015. 8 The Economist 2015. 9 Clist 2016; NAO 2015. 10 Birdsall et al. 2011. 12 A Guide for Effective Results-Based Financing Strategies TABLE 2.1 Paying for Inputs and Activities versus Paying for Results PROGRAM PAYING FOR RESULTS PAYING FOR INPUTS AND ACTIVITIES OBJECTIVE (RBF) Improve Funders pay services providers for: Funders pay services providers for: electricity • Hiring staff to develop strategies for • Every new connection for customers access the distribution and installation of who did not previously have access to electricity networks. electricity. • Purchasing equipment needed to • Number of months of sustained install new connections. electricity delivery after one year. Decrease Funders pay services providers for: Funders pay services providers for: unemployment • Hiring trainers to deliver skills training. • The number of people who completed the training. • Purchasing training materials. • The number of people who found a job. Reduce Funders pay services providers for: Funders pay services providers for: deforestation • Delivering skills training for forestry • Every ton of CO2 emission reduced. workers. • Building community awareness about deforestation. While the practice of RBF remains nascent, evidence of RBF’s impact is emerging. For instance, consider the case of a child and maternal health care program in Rwanda. A randomized evaluation commissioned by the World Bank found that compared to a traditional funding mechanism, RBF led to 23 percent more institutional deliveries and 132 percent more preventive visits for children aged two to five.11 Delivering 23 percent more results by changing the way social programs are funded is highly motivating for any international development practitioner. However, RBF is far from a silver bullet: RBF can only deliver these sorts of benefits where the necessary conditions are in place and where it is designed and implemented in a way suited to the context. This report is an invitation to consider when and how RBF can be effectively used to enhance development results. 2.1 The three key steps of RBF RBF follows a three-step process: • Step 1: Signing the RBF agreement • Step 2: Verifying results12 • Step 3: Paying for achieved results. Basinga et al. 2011. 11 12 It is assumed here that the program has been implemented and that results can be verified. A Guide for Effective Results-Based Financing Strategies 13 In Step 1, the development funder13 signs an agreement with the incentivized agent where all or part of its funding/payments are contingent on the achievement of measurable and predefined results. The funder is the organization that provides funding. It can, for instance, be a government, a bi/multilateral funder, or a foundation. The incentivized agent is the organization whose payments are contingent upon results. It is normally responsible for the delivery of outputs and/or outcomes (except in the case of Impact Bonds) and can include national or local governments and service providers, either NGOs, for-profit organizations, or government agencies.14 When signing the RBF agreement, the parties must agree on at least the following core features of the agreement: • The results that will trigger payment. For instance, in the case of a poverty reduction program, the trigger result could be an increase in beneficiary income. • How payment varies with results achieved. For instance, in the same poverty reduction program, the funder could agree to pay $10,000 for each percentage point increase in beneficiary income. • How results will be verified. Using the same example, the verification method could involve surveying households using a consumption and assets tracking survey at the end and beginning of the program and calculating the increase in beneficiaries’ income as compared to the start of the program. In Step 2, an independent evaluator verifies the extent to which results have been achieved. It is important for this evaluator to be independent to guarantee the credibility of the verification process. Evaluators are considered independent when they have no financial or other interest in the results. In the case of a poverty reduction program, the evaluator would survey households to measure income levels among the beneficiaries at one or several defined points in time. In Step 3, the funder pays the incentivized agent upon verification of the achieved results as per the conditions laid out in the RBF agreement. For instance, following the example above, a 5-percentage point increase in income levels of the beneficiaries compared to the start of the program would trigger a $50,000 payment from the results funder to the incentivized agent. 13 From this point onward, the report will use the terms “development funder” and “results funder” interchangeably. Further, it uses the term “funder” as an abbreviated form of these terms. Public or private utility companies, health facilities, and education providers are some examples of possible incentivized agents. 14 14 A Guide for Effective Results-Based Financing Strategies 2.2 Categories of RBF instruments This section introduces the different types of instruments and categorizes them in an easy-to- understand framework, along with their subtypes in each category. This categorization forms the basis for guiding choices on which RBF instruments are most suitable in different contexts, as outlined in sections 4.1 and 5.2. One useful way to organize RBF instruments is in terms of the first-order question, ‘who is the incentivized agent?’ This question is the foundation of the simplified typology of RBF instruments presented in figure 2.1. 15 Further detail on each type of RBF instrument category follows, along with explanations of the different variations of each instrument. Understanding this terminology and how different instruments differ from one another can help practitioners grapple with the great profusion of RBF terminology commonly used across other organizations, publications, and sources. FIGURE 2.1 Simplified Typology of RBF Instruments This typology is considered ‘simplified’ because it is based on five types of instruments and clarifies the profusion of 15 instrument names used in the literature. It can be challenging, even for experts, to explain the difference between such terms as “Performance-Based Incentives,” “Payment By Results,” “Pay For Performance,” “Performance-Based Disbursements,” “Impact Bonds,” and “Pay For Success.” A Guide for Effective Results-Based Financing Strategies 15 PERFORMANCE-BASED AID If the national government is incentivized, the RBF instrument is called Performance-Based Aid (PBA). The term “aid” indicates that those types of instruments generally involve a transfer of funds between a bi/multilateral donor to a national government. Payments to national governments are at least partly based on performance. Figure 2.2 depicts the structure of a PBA and box 2.1 provides an example. FIGURE 2.1 Performance-Based Aid Structure A PBA instrument can take several forms depending on the type of financial vehicle that the results funder uses to pay the incentivized agent: • Cash on Delivery (COD) is a PBA financed through a grant. CODs generally involve a fixed payment to the government for each unit of progress against an agreed result and provides the government with full responsibility and discretion for using the funds. • A Performance-Based Loan (PBL) is a PBA financed through a loan. The World Bank’s Program- for-Results (PforR) is an example of a PBL (see box 2.1). In this arrangement, the tranches of the loan are disbursed to the national government upon the achievement of predefined results. • Performance Debt Buy-Down (PDBD) is a PBA whereby a results funder pays off a portion of an outstanding loan held by a central government if certain results are achieved. 16 A Guide for Effective Results-Based Financing Strategies BOX 2.1 An Example of Performance-Based Aid (PBA): Local Bridge Construction and Road Asset Management, Vietnam, 2016 While economic growth has been strong in Vietnam over the past two decades, not all regions have benefitted equally from that growth. For instance, in the Northern Mountains region where many ethnic minorities live, poverty reduction has been slower than in other parts of Vietnam. This lag can in part be attributed to poor connections between rural areas and job markets. For that reason, investing in local roads and bridges can help reduce poverty and improve social participation in the country. Responding to these needs, the World Bank has established a Program-for-Results (PforR) agreement with Vietnam’s national government entitled Results-Based Operation for Local Bridge Construction and Road Asset Management Project. The program incentivizes the central government to improve road and bridge connectivity for the rural communities of Vietnam with the World Bank’s offer to disburse a $380.5 million loan conditional upon the achievement of results defined as part of the program. These results are: • Number of kilometers of roads that received improvements or were rehabilitated. • Number of kilometers of roads that received routine maintenance at a minimum level. • Increased amount in budget allocation for maintenance. • Number of bridges built or rebuilt. • Percentage of participating provinces managing a bridge database. The project started in 2016 and is expected to close in 2023. Source: World Bank 2016c. A Guide for Effective Results-Based Financing Strategies 17 PERFORMANCE-BASED TRANSFER If the local government is incentivized, the RBF instrument is a Performance-Based Transfer (PBT). The term “transfer” indicates that this type of instrument generally involves a transfer of funds between a national government or a bi/multilateral donor and a local government. Payments to local governments are at least partly based on performance. Figure 2.3 depicts the structure of a PBT and box 2.2 provides an example. Output-Based Disbursement (OBD) is a specific type of PBT, most commonly used by the World Bank. Local governments receive payments for results such as improvements in the efficiency of a service-related asset, system, or recurrent government activity. Some of these programs link service outputs with associated unit costs, and disbursements reflect the actual cost of service. Figure 2.2 Performance-Based Transfer Structure 18 A Guide for Effective Results-Based Financing Strategies Box 2.2 An Example of a Performance-Based Transfer (PBT): Plan Nacer, Argentina, 2007 The 2001 economic crisis led to an increase in poverty and unemployment in Argentina. As a result, many Argentines lost their private health coverage and turned to the public health system. The increased pressure on the country’s health system led to an increase in infant mortality and motivated the development of Plan Nacer to both reduce infant mortality and improve the efficiency and quality of public health services. As part of Plan Nacer, provincial governments were incentivized to enhance the coverage and quality of health services to poor pregnant women and newborns. A PBT agreement was signed between the central government and provincial governments, under which (i) the national government disbursed 60 percent of the predefined payments depending on beneficiaries’ enrollment rates; and (ii) the remaining 40 percent was disbursed based on performance against 10 indicators tailored to each province. These indicators include health outcomes such as the proportion of newborns with low birth weight and outputs such as the use of priority services in specific periods, like the first 20 weeks of pregnancy. A 2014 impact evaluation of Plan Nacer found that the program increased the use and quality of prenatal care, measured by the number of visits and the probability of receiving a tetanus vaccine. It estimated that the probability of low birth-weight for beneficiaries was reduced by 19 percent. In larger hospital facilities, beneficiaries had a 74 percent lower chance of in-hospital neonatal mortality. Approximately half this reduction was attributed to preventing low birth weight and half to better postnatal care. Sources: World Bank 2013a; Gertler, Giovagnoli, and Martinez 2014. A Guide for Effective Results-Based Financing Strategies 19 PERFORMANCE-BASED CONTRACT If a service provider is incentivized, the RBF instrument is a Performance-Based Contract (PBC). The term “contract” is used for RBF instruments whereby a transfer of funds occurs from a results funder (bi/multilateral donor, government, foundation, or the like) to one or more service providers (including public, private, or nongovernmental organizations). Payments to service providers are at least partially based on performance. Figure 2.4 depicts the structure of a PBC and box 2.3 presents an example. Figure 2.3 Performance-Based Contract Structure A PBC is generally financed through a grant, but other common RBF instruments have emerged under this category due to variations in features such as the financial vehicle, the incentivized agent, or the contracting mechanism: • Output-Based Aid (OBA) is a form of PBC most commonly used by GPOBA, a multi-donor trust fund managed by the World Bank. It is an agreement where service delivery is contracted out to a public or private provider, which receives a subsidy to complement or replace the required user contribution if certain results are achieved. OBA is used to enhance access to and delivery of basic infrastructure and social services for the poor. • Performance-Based Financing (PBF) is a form of PBC typically used to fund public or private health service providers (such as health facilities) with primarily a fee-for service payment mechanism, in which health care providers are paid for each service performed. PBF payments are generally on the basis of fee-for-service-conditional-on-quality, which means that payments are made based on service providers’ performance against targets or for undertaking specific actions. 20 A Guide for Effective Results-Based Financing Strategies • Prize-Based Challenge is a form of a PBC where an open-bid competition awards a financial prize for the best innovation developed within a predefined time frame in response to a prespecified social issue. The open-bid allows many organizations to compete for the prize. The winner is the organization that develops the most cost-effective solution to the specified social issue. Box 2.3 An Example of a Performance-Based Contract (PBC): Morocco Improved Access to Water and Sanitation Services Output-Based Aid Project In Morocco, delivering water supply and sanitation services in urban fringe (peri-urban) areas is a big challenge. Peri-urban households commonly use unimproved sources of water and sanitation (mostly latrines or toilets and cesspits). To address this issue and close the financial gap between households’ ability to pay for connections and service costs, an RBF scheme was launched in 2006 between GPOBA and two water private operators. These companies received OBA grants upon the verification of individual connections to water and sewerage services. Specifically, 60 percent of the pre-established unit subsidy was paid upon certification of a working connection to an eligible household, and the remaining 40 percent upon verification of at least six months of sustained service provision. By 2014, the initiative provided 10,504 low-income households with piped water supply and 9,036 households with improved sanitation services. The results of a beneficiary survey reported high satisfaction with the new services. Source: GPOBA 2014a. A Guide for Effective Results-Based Financing Strategies 21 IMPACT BOND If an investor is incentivized, the RBF instrument is an Impact Bond. The term “bond”16 reflects the involvement of an investor in the RBF instrument (a private investor, foundation, or the like). The investor(s) bear the financial risk, which is why they are considered the incentivized agents, whereas the service providers are responsible for delivering results. Payments to the investor are determined by service provider results. Figure 2.5 depicts the structure of an impact bond and box 2.4 provides an example. There are two types of impact bonds, depending on the nature of the results funder. When a government funds at least part of the results, the agreement is called a Social Impact Bond (SIB). When a donor or any other nongovernment entity is the results funder, the agreement is called a Development Impact Bond (DIB). FIGURE 2.4 Impact Bond Structure Impact Bonds are not bonds in the conventional financial sense of the term. 16 22 A Guide for Effective Results-Based Financing Strategies Box 2.4 An Example of a Development Impact Bond (DIB): Educate Girls, India, 2015 Despite substantial commitments to education, India still has the largest illiterate population in the world. Nationally, an estimated 3.7 million girls are out of school and in Rajasthan, 40 percent of girls drop out before reaching fifth grade. For those who remain, learning quality is low. Uneducated girls in India are also three times more likely to contract HIV, earn 10 percent less income, and marry three years earlier than educated ones. To address this challenge, Educate Girls, a local implementer, identifies champions for girls’ education at the village level who can identify out-of-school girls and work with the community to enroll and teach them in the classroom. As part of the Educate Girls DIB, the UBS Optimus Foundation (the investor) was incentivized to invest in enrollment, retention, and learning and help Educate Girls (the service provider) improve the impact of its program for marginalized girls and boys in Rajasthan. This impact bond also involved the Children’s Investment Fund Foundation (CIFF) as the results funder. As part of the DIB agreement, the UBS Optimus Foundation provided upfront capital to Educate Girls. Upon verification of the achievement of predefined enrollment and learning results, CIFF would pay back the UBS Optimus Foundation their initial investment plus a return, depending on Educate Girls’ performance. UBS also shares a portion of the results payment with Educate Girls, thus providing Educate Girls a financial return dependent on the success of their program. As of June 2017, year two results have been positive: enrollment targets had been exceeded (with 87.7 percent of the three-year enrollment targets achieved) and learning gains continued (with 50.3 percent of the three-year learning target attained). Observers have suggested that the flexibility generated by this DIB had improved results for girls who are hardest to reach. Sources: Instiglio 2017; Instiglio 2015; UBS Optimus Foundation and CIFF 2016. A Guide for Effective Results-Based Financing Strategies 23 Lastly, if households and individuals are incentivized, the RBF instrument is a conditional cash transfer. This scheme involves transferring cash to eligible individuals or households when they complete defined actions (such as enrolling their children in schools, getting regular medical check- ups, or taking nutritional supplements). Conditional cash transfers are not considered in this report, which is primarily focused on instruments that incentivize organizations. While the different instruments outlined here are usually used independently from one another, they can also be combined. Especially when working with governments, combining instruments can provide an opportunity for funders to move a focus on performance along chains of interactions and relationships. For instance, the World Bank could establish a PforR with a national government, which in turn could establish a PBT with a local government, that then outsources delivery to local providers with a PBC. These incentives chains can even extend beyond RBF instruments to a focus on an organization’s internal incentives, such as incentive schemes for front-line workers. For practitioners engaged in designing an RBF approach, the selection of the incentivized agent and corresponding RBF instrument is followed by decisions about key design features. As outlined in section 5.3, these design features include topics such as what results to pay for, how much funding to tie to results, and the approach for determining the level of funding attached to certain results. These design features may vary greatly within each category of RBF instrument. For example, two RBF agreements incentivizing service providers can differ dramatically on nearly all meaningful design features. This is the case of two Impacts Bonds in Rajasthan, India: the Educate Girls Impact Bond and the Maternal and Newborn Health Impact Bond. In the Educate Girls example, payments are based on outcomes, verified through a Randomized Control Trial (RCT), and 100 percent of funding is tied to results. In contrast, in the second example, payments are tied to outputs, the verification is based on observational methods, and less than 100 percent of funding is tied to results. Taking this into account, RBF can be understood as a practice of thinking strategically about financial incentives and how they can drive better results. It is a practice best used flexibly and not as a rigid collection of instruments. Additional real-world examples of the RBF instruments discussed are presented in table A.1 in appendix A. 2.3 RBF trends in low- and middle-income countries This section describes to what extent, where, and by whom RBF instruments have been used, to provide readers with an understanding of key trends before the chapters that follow dive into the diagnostic tool and design considerations. The section draws insights from the RBF database of over 300 projects across the developing world. Practitioners can consult the RBF database to: • Further understand key market trends: how and by whom RBF has been used across dimensions such as sectors, geographic areas, and instruments; and 24 A Guide for Effective Results-Based Financing Strategies • Identify and leverage relevant project benchmarks to inform design decisions on RBF such the types of results that funders usually pay for and the type of verification approaches used. The rest of this section presents insights derived from this database in relation to market trends. Other market trend analysis such as sectoral or regional deep dives can be conducted with this database and its visualization platform. This section also draws on the database for insights on the design benchmarks during the discussion of RBF design decisions in section 5.3. Additional figures based on the database are presented in appendix B. PRESENTATION OF THE RBF DATABASE The RBF database is comprised of: • An Excel document containing raw information about RBF projects in developing countries from 1993 to date, and their key characteristics. • A visualization platform that contains various dashboards enabling an interactive visualization of the information listed in the Excel document. The information in the database for each RBF project includes: program name, country, region, sector, subsectors, start year, status, total project size, amount of funding tied to results, RBF instrument categories and subcategories, type of financial instruments, results funder name and category, incentivized agent name and category, types of results paid for, verification and evaluation approaches, links to the source documents, and details of key contacts. The visualization platforms contain visuals such as: a map showing where RBF projects have been implemented, figures displaying the amount of RBF funding across low- and middle-income countries, deep-dives into geographic areas and sectors, and a comparison of design features used in different contexts. It is important to note that this RBF database may present limitations because of the following elements: • The specific instruments identified in this document do not reflect all RBF instruments. RBF projects using instruments not listed (such as carbon financing, feed-in tariffs, capital subsidies, and CCTs) in the typology presented in section 2.2 may not be included in the database or may be included under a different instrument name. • While a rigorous process was applied to search and include the maximum number of RBF projects in the database, some may not have been registered, especially when the information about certain projects was of poor quality or was difficult to access or unavailable. • Some biases may have been introduced because some projects have been labeled as RBF at a later stage. Quality of documentation in general may have improved over time and may also be skewed toward donor-led projects, thus underestimating the number and value of government-led projects. A Guide for Effective Results-Based Financing Strategies 25 RBF spending has grown dramatically in recent years. As figure 2.6 shows, in the last decade at least $25 billion of development spending has been tied to results, up from just a few billion in the decade before.17 This growth has largely been led by the World Bank with its PforR instrument involving $19 billion tied to results.18 FIGURE 2.6 Cumulative Total of Spending Tied to Results through RBF, 1993–2017 Source: RBF Database The dramatic uptake in the use of RBF has also been associated with its extended use across diverse sectors. Historically, the health sector has been a major focus of RBF spending. This trend in part reflects the relative ease with which results in this sector can be verified based on data from hospitals or health agencies, such as records on the number of vaccinated children. As shown in figure 2.7, our estimates show that by number of projects, the health sector continues to be the most active RBF sector, followed by the education and public administration sectors. All data presented in this report from the database are for projects that were either being implemented or were 17 completed. Projects that are in design, were cancelled, or whose status is unknown were deleted. The estimates presented here represents the total funding tied to results from 182 projects in the GPOBA RBF database. The value of funding tied to results is unknown for 144 additional projects in this database, which is why the actual amount of funding tied to results is likely to be greater. 18 Our estimates show that the Program-for-Results has been also been associated with a further $82 billion of related spending, predominately spent by client governments and recorded in the associated agreements. This funding is not tied to results and is used to fund the delivery of the services required to achieve the desired outcomes. 26 A Guide for Effective Results-Based Financing Strategies FIGURE 2.7 Estimated Number of RBF Projects by Sector, 2018 Source: RBF Database In terms of the regional distribution of RBF projects, RBF has been used throughout most of the developing world, as shown in map 2.1. However, the extent of this use varies by region, with our sample containing the largest concentration of RBF programs in Sub-Saharan Africa (with an estimate of 140 projects), followed by South Asia (with an estimate of 67 projects). MAP 2.1 RBF Projects by Country, Low- and Middle-Income Countries Source: RBF Database Note: Darker shading signifies a greater number of projects. A Guide for Effective Results-Based Financing Strategies 27 The use of RBF is also spread widely across the different instruments defined in section 2.2. In terms of project numbers, as shown in figure 2.8, the most commonly used instruments are PBAs, with the World Bank’s PforR instrument representing the largest share with 76 projects. PBCs are the next most common category, with a large share from the World Bank’s OBA instrument, accounting for 48 projects. Reflecting their relative novelty, impact bonds are the smallest category, with only 7 to date. FIGURE 2.8 Estimated Number of Projects by RBF Instrument Source: RBF Database Note: Numbers are based on projects that are known to be being implemented or have been completed. PBA = performance-based aid; PBC = performance-based contract; PBT = performance-based transfer. In terms of total spending tied to results for each of these instruments, PBAs take an even clearer lead, again driven by the scale of the World Bank’s PforR, accounting for almost $19 billion or nearly 75 percent of all RBF spending tied to results. PBTs are the next largest category in the database, entailing approximately $2.9 billion of development spending tied to results. 28 A Guide for Effective Results-Based Financing Strategies FIGURE 2.9 Estimated Amount Tied to Results by RBF Instrument ($million) Source: RBF Database Note: Percentages are based on projects that are known to have been implemented or completed. PBA = performance- based aid; PBC = performance-based contract; PBT = performance-based transfer. The variety of sectors, regions, and instruments to which RBF has been applied displays RBF’s wide-ranging potential. As the use of RBF continues to grow and spread to new frontiers, a one-size-fits-all approach cannot be applied effectively to these diverse settings. Very different approaches are needed, for instance, when working with utilities to enhance water access in Morocco compared with efforts to improve girl’s education outcomes by working with service providers in India. Responding to these diverse circumstances requires a problem- driven approach, deep engagement with the local context, and careful tailoring of RBF’s use to match this context. The diagnostic tool outlined in the next chapter responds to these requirements. A Guide for Effective Results-Based Financing Strategies 29 The powerful potential of RBF will only be realized if it is tailored to the policy and implementation context. To aid this objective, the rest of this report presents a diagnostic tool that provides a set of structured questions and frameworks to guide decisions by funders regarding whether RBF should be used in a given context and how it should be applied to improve results. This chapter presents an overview of the diagnostic tool, its purpose, structure, and intended use. The chapter provides an illustration of how the diagnostic tool can be used by funders, taking the perspective of an aid agency interested in using RBF to enhance education outcomes. This is followed by practical guidance of an iterative diagnostic approach to use this tool effectively. 30 AN OVERVIEW OF THE DIAGNOSTIC TOOL A Guide for Effective Results-Based Financing Strategies 3 3. OVERVIEW OF THE DIAGNOSTIC TOOL KEY CONCEPTS OF THIS SECTION • RBF value-add: The extent to which RBF enhances development spending effectiveness. • Barriers to results: Various contextual factors that prevent development spending from effec- tively delivering results. Examples include issues such as the absence of the services relevant to the problem at hand, insufficient capacity of service providers to effectively deliver the service, or the rigidity of service delivery. • Conditions for RBF: The contextual factors that need to be place for RBF to achieve its full val- ue-add. These conditions can be sorted into three categories: technical, institutional, and politi- cal. For example, institutional conditions include the capacity of the incentivized agent to pre-fi- nance delivery costs. • RBF design features: The specific parameters of an RBF agreement. A non-exhaustive list of such parameters includes the types of results paid for, verification methods, or the amount of funding tied to results.  • RBF strategy: Decisions on whether to use RBF, and the best way to apply it in a specific context. This entails answering three key questions with the goal of maximizing RBF’s effectiveness: (1) Which conditions should be created? (2) What RBF design features should be used? (3) How can RBF be refined over time? Consider an aid agency exploring the use of RBF to improve education outcomes in a middle- income country. The agency has a portfolio of education projects in several partner nations but progress in measurable results has been slow and budget pressures have been growing. The director of the agency has been impressed by anecdotes of RBF’s potential to deliver value for money but is not sure how her agency can make use of it. How can the agency determine if RBF is appropriate in this context? Which actors should be involved? What capacity is needed by stakeholders to use RBF? What types of results should they pay for? These and many other questions will need to be answered. The diagnostic tool is intended to provide initial guidance in answering preliminary questions faced by this aid agency and results funders in general. Specifically, the diagnostic tool provides: • A structured and sequenced overview of the questions that need to be considered. • Guidance for developing contextually appropriate answers to these questions. The guidance will not be sufficient to fully design an RBF agreement but will inform decisions pertaining to core design features and strategies. The rest of this section illustrates how a potential results funder can use the diagnostic tool (see figure 3.1). A Guide for Effective Results-Based Financing Strategies 31 3.1 Contextual Analysis The first step in assessing RBF’s potential is understanding the challenges that need to be addressed. Developing good policies and programs requires sound problem diagnoses; this is no different when using RBF. The contextual analysis component of the diagnostic tool responds to this requirement, outlining the key contextual factors that should be considered to understand the barriers to results at hand and to assess RBF’s potential to address them. Specifically, the contextual analysis centers on two main dimensions: the RBF value-add and the contextual conditions needed for RBF. To understand if RBF is worth pursuing, the aid agency will need to assess whether RBF is likely to add value in the given context. That is, can RBF play a significant role in addressing any of the barriers to results? To assess RBF’s value-add, the aid agency will need to understand what factors are limiting education results and evaluate how RBF could address these barriers. For instance, perhaps the main barrier is high rates of teacher absenteeism caused by insufficient interest among local governments in ensuring teacher attendance. In this context, tying the local governments funding to teacher attendance could create the incentives needed for them to focus on this issue and find solutions to teacher absenteeism. FIGURE 3.1 The Diagnostic Tool Framework 32 A Guide for Effective Results-Based Financing Strategies RBF should only be used where funders plausibly expect it to deliver enough value to justify the costs it entails. Thus, understanding its value-add allows the aid agency to make an early assessment of whether to pursue RBF. In the case at hand, given that RBF could create the incentives needed to address teacher absenteeism, the aid agency has an early indication of the potential value-add and who might be the incentivized agent. This early indication of which agent is best able to address the barriers to results and provides the first-order definition of the RBF instrument. Funders should then assess if the conditions for RBF’s effective use are in place. Because of the changes RBF entails for business-as-usual development practices, specific conditions must be in place to move forward with an RBF project. In the example, the aid agency should ascertain whether sufficient data is available for setting education outcome targets? Does the local government have the capacity to experiment and find solutions to teacher absenteeism? Is the government willing to achieve relevant improvements using RBF? If these conditions are not in place, the effectiveness of RBF will be impaired, and the magnitude of its value-add will be limited. Funders will therefore have to consider how these conditions can be created or how the use of RBF can be adjusted to minimize any negative consequences of the missing conditions. 3.2 RBF Strategy Next, funders will need to select the specific RBF instrument or instruments most suited to the context. This choice will be a product of factors including who the incentivized agent is, the funders’ objectives and mandate, existing funding arrangements, the sectoral focus, and the program’s maturity in terms of the level of evidence available about the intervention’s potential impact. With these details resolved, funders can move to designing the RBF agreement, deciding which core design features are most appropriate. Core decisions include: what type of results should be paid for? How much funding should be tied to results? How should results be verified? Design decisions concerning these features should be tailored to the context, focusing on maximizing the RBF value-add. Finally, to ensure greater and more sustainable impact in the future, funders should also consider how the use of RBF can be developed over time. Can the use of RBF be strengthened over time to improve its effectiveness? Can the use of RBF be scaled to expand its impact to other demographics, geographic areas, and services? A Guide for Effective Results-Based Financing Strategies 33 3.3 The RBF diagnostic process The diagnostic tool lays out the questions that need to be considered to establish an RBF strategy. While presented sequentially, the diagnostic tool should not be viewed as a linear process: devising an RBF strategy is an iterative process, with key questions revisited multiple times throughout the deliberation process. As new information is uncovered during the diagnostic process, stakeholders’ understanding of the considerations at hand will increase, allowing for better-informed and more definitive decisions on the RBF strategy. For instance, early in the contextual analysis, or even before, funders may have decided which actors should be incentivized, perhaps because of political factors. This could be the case in the example given above in which the aid agency might decide from the start to work with the local government, rather than, for example, a local education service provider. Forming these early views can help to focus the analysis and determine with whom to work on the RBF strategy. However, this should be balanced against the need to stay open to adjusting these early ideas in response to insights from the RBF diagnostic. Likewise, the question of whether RBF should be used in the given context should be reassessed throughout the diagnostic process. As described in the next chapter, stakeholders can form early views on the applicability of RBF to a given context, but these views should be reassessed as the diagnostic process reveals greater insights on the RBF value-add and the costs required to use RBF. 34 A Guide for Effective Results-Based Financing Strategies This chapter covers the diagnostic tool’s Contextual Analysis component, which provides the foundation for determining if and how RBF should be used. First, it outlines how funders can assess the potential of RBF to enhance the impact of their funding in a given context: that is, the potential RBF value- add. The value-add analysis rests on an understanding of RBF’s four drivers of impact, which are outlined in this chapter. The chapter then presents the conditions often needed for RBF to be used effectively. Funders must assess these conditions and often must work to create them to ensure that RBF fulfills its potential to improve development outcomes. 36 Diagnostic Component 1: CONTEXTUAL ANALYSIS A Guide for Effective Results-Based Financing Strategies 4 4. DIAGNOSTIC COMPONENT 1: CONTEXTUAL ANALYSIS KEY CONCEPTS OF THIS SECTION • RBF’s four drivers of impact: The four channels through which RBF can drive greater effec- tiveness in development spending. These distinct channels are: (1) drawing attention to what matters; (2) aligning incentives to improve beneficiaries’ welfare; (3) providing greater flexibility to maximize results; and (4) enhancing accountability of development actors to beneficiaries. • Service delivery value chain: The policies, programs, and related actors involved in delivering results. • Technical conditions: Contextual factors related to the availability of suitable interventions, re- sults, and data required to design the RBF project. • Institutional and legal conditions: Contextual factors related to the capacity of the stakeholders involved, including the ability to pre-finance and regulations on disbursement. • Political conditions: Contextual factors related to the interests and motivations of the stakeholders. • Creating conditions: Taking actions to produce the conditions needed to use RBF effectively, such as investing in data collection if data are insufficient at the time. • Mitigating design choices: Design choices that reduce the negative implications of the absence of conditions needed to use RBF effectively. Thinking strategically about how RBF can drive results in a given policy context requires a sound analysis of the context, centered on two key topics: Assessing RBF’s potential value-add: Funders should assess the extent to which RBF can address barriers to results and how. Understanding how RBF can be used to improve results provides an early indication of whether RBF should be used in this context and informs decisions on which RBF instrument(s) is/are most appropriate. Assessing the conditions for RBF: Funders should identify to what extent the conditions needed for RBF’s success are in place. Understanding the extent to which these conditions exist is critical to a refined understanding of RBF’s overall value-add and informs work to create the needed conditions. 4.1 The RBF value-add When deciding if and how to use RBF, results funders should assess how and to what extent RBF can contribute to improved service delivery and results. That is, funders should seek to understand RBF’s potential value-add for the given context. A useful framework to inform this assessment is to consider RBF’s four drivers of impact: the, four distinct channels for how RBF can drive greater development effectiveness.19 Perakis and Savedoff 2015. 19 A Guide for Effective Results-Based Financing Strategies 37 • RBF draws attention to what matters: Paying for results requires measuring results that stakeholders agree matter. Thus, RBF makes results visible and draws the attention of all actors to what matters. In addition to measuring outputs or outcomes, RBF often requires a data collection and analysis system that generates valuable insights and provides a feedback loop to improve the performance of the program. For the incentivized agent, gaining access to data- driven insight is a way to learn about what works and what does not and provides an opportunity to adapt the program. • RBF aligns incentives to the welfare of program beneficiaries: By putting a portion of the funding at risk, or providing a bonus payment, RBF promotes alignment between the interests of the funder, the incentivized agent, and the welfare of the beneficiaries. It does so by rewarding the incentivized agents financially for delivering results, thus compelling them to implement activities that meet the beneficiaries’ needs. • RBF provides flexibility to maximize results: By paying for results, funders can relax their control over activities and provide the incentivized agent with greater flexibility to adjust their program to improve results. Incentivized agents can use this flexibility to try new approaches, learn and adapt in response to new contexts, and pursue more effective solutions. For instance, an incentivized agent might then be able to invest in new service delivery methods without requiring prior approval from funders. • RBF enhances the accountability of the incentivized agent to beneficiaries: RBF can make incentivized agents accountable for achieving predetermined results for beneficiaries and drive a shift toward treating beneficiaries more like clients, placing them at the center of every decision and action. This driver is likely to be activated when the beneficiaries are involved in the assessment of the incentivized agent’s performance and when beneficiaries have access to information about results, better enabling them to demand changes and holding the incentivized agent accountable.20 These drivers can pertain to the process of enhancing the effectiveness of specific interventions or strengthening delivery systems more broadly. In relation to specific interventions, RBF can play different roles depending on the extent to which evidence is available on the intervention’s impact: that is, on the intervention’s maturity. For instance, as detailed in section 5.1, where very little information is available on the effectiveness of an intervention, RBF can provide the flexibility to experiment and find solutions. In contrast, where the intervention has a proven track-record of impact, RBF can provide the incentives needed to scale it up. Likewise, RBF can also strengthen delivery systems by incentivizing a network of delivery agents to deliver results effectively and sustainably. For instance, by drawing attention to results and sustaining this attention, RBF can help drive long-term reforms and improved capacity despite political economy challenges, crises, and fads.21 20 Instiglio 2017a. World Bank 2017. 21 38 A Guide for Effective Results-Based Financing Strategies 4.1.1 Assessing the RBF value-add The four drivers of impact illustrate RBF’s potential to enhance development effectiveness. However, it cannot be assumed that these drivers will add value in all circumstances. To assess RBF’s potential value-add for a given context, funders should start by identifying the current barriers to results and assessing whether these barriers can be addressed by any of RBF’s four drivers of impact. Funders can use a value-chain analysis to identify barriers to results. A value-chain analysis involves drawing on sector-specific expertise to assess current efforts addressing the social issue of concern (such as a lack of water access or high unemployment) and to identify the barriers to results that exist along this chain. To illustrate, take the problem of high disease rates caused by poor sanitary practices. Analyzing this issue’s value chain could reveal that the problem is caused by a lack of sanitation infrastructure. Further investigation based on engaging with sector experts reveal two specific barriers at play. First, utility monopolies are preventing other providers from working in the sector. Second, beneficiaries favor current sanitation practices because they lack knowledge about the detrimental health implications. With these barriers identified, analysis should be undertaken to determine which are the most severe impediments to results and which must be prioritized to improve results. Funders can then assess if any of RBF’s four drivers of impact could plausibly help address these barriers. In some instances, this analysis may reveal little need for RBF. For instance, RBF may be ineffective in relation to the problems of the utility monopoly’s negative effects, leaving traditional funding the most cost-effective approach. In contrast, RBF may be able to address the issue of a lack of knowledge about sanitation by providing the flexibility to test different approaches to change knowledge and behavior. These considerations are summarized in table 4.1. Box 4.1 provides an example of identifying barriers to results for a maternal and child health care initiative in Nigeria. A Guide for Effective Results-Based Financing Strategies 39 TABLE 4.1 Assessing the RBF Value-Add BARRIERS TO RESULTS Identifying the barriers to results requires analyzing the service delivery value chain. The value chain consists of the policies, programs, and actors currently aimed at achieving the desired results. Funders should assess the following considerations: • What is the social issue (such as lack of access to water) and what are the related results the funder seeks to achieve? • Are relevant services being used? If the relevant services are missing or not being used, why is this? For example: »» Are suitable interventions to address the social issues available? »» Is the necessary funding available? Identify »» Is there sufficient political interest in the social issues? and prioritize »» Are certain populations excluded from access to relevant services (for example, barriers because of cost or geography)? to results • If relevant services are available but not delivering results, why is this the case? »» Is the intervention poorly designed? For example: Is it not sufficiently targeted? Does it fail to account for contextual factors? Does it not respond to beneficiaries’ actual needs or preferences? »» Is the approach poorly implemented? Impediments might include low implementation capacity; program rigidity and/or an excessive institutional burden caused by financial reporting requirements; misaligned incentives; or weak coordination among stakeholders. • Which barriers are most severe and need to be addressed to improve results? ALIGNMENT WITH RBF DRIVERS OF IMPACT • Can the most important barriers to results identified above be addressed by RBF? That is, could improvements to results be driven by: Assess whether »» Greater flexibility and how RBF can »» Greater attention to results address the »» Greater accountability to beneficiaries identified barriers »» Better incentives? • What other changes or complementary reforms are required to address barriers (that RBF cannot solve)? 40 A Guide for Effective Results-Based Financing Strategies Further, properly assessing RBF’s value-add requires comparing RBF’s potential benefits against those of alternative solutions. The alternatives will depend on which barriers to results have been identified. For instance, if the barrier relates to a lack of attention to results, alternatives to RBF would include “management to results” approaches, which involve data-driven collaboration and course corrections. Deciding between these alternatives should be driven by an assessment of which approach offers the most effective means to improve results. This decision will also need to be informed by consideration of which approaches are most suitable given the contextual constraints, such as the conditions considered in the next section. BOX 4.1 Example of Identifying Barriers to Results from in Nigeria In 2012, the Nigerian government introduced the State Health Investment Project involving results payments for local governments and health facilities. It aimed to increase maternal and child health service delivery and its quality. To build capacities for RBF and increase its likelihood of success, a pilot program was undertaken with three local governments. The pilot showed that while RBF was effective in increasing the efficiency and quality of health facilities, this did not translate to the expected increase in service utilization among the most vulnerable and marginalized women and children. Specifically, a qualitative evaluation found that service uptake remained low because of four demand-side factors related to the affordability of services as well as cultural norms that the program did not address. First, transportation costs impeded access to health facilities among the poorest. Second, the user fees meant many households continued to use providers who accepted credit and in-kind payments (such as traditional birth attendants). Third, ethnic ties and political divisions were a barrier to some groups attending the health facilities. Last, the influence of cultural norms and traditions contributed to some favoring traditional practices over the health facilities. This example illustrates how RBF’s impact can be muted by a lack of focus on the most important barriers to results. Identification and targeting of the demand-side barriers revealed in the evaluation may have enabled a program to be developed in which RBF was used more effectively to achieve the desired results. Indeed, the World Bank reports that the evaluation’s findings were subsequently used to inform the design of demand-side interventions to better address the barriers. Source: Mabuchi and McCune 2015. A Guide for Effective Results-Based Financing Strategies 41 4.1.2 Selecting the RBF instrument category As outlined in section 2.2, the decision of which type of agent to incentivize provides the first-order definition of the RBF instrument category. Funders have four broad options corresponding to the four primary RBF instrument categories: national governments (performance-based aid, PBA); local governments (performance-based transfer, PBT); service providers (performance-based contact, PBC); or investors (impact bond). Typically, the primary driver of this decision will be the funder’s mandate and objectives. For example, certain funders may have a mandate or long-established practice to work with client governments, aiming to build their capacity to deliver results sustainably in the future. Where such mandates exist, the funder will have little discretion on what category of RBF instrument they use. In contrast, where funders have discretion to choose the type of entities they fund, this choice should be driven by an understanding of which agent is best placed to address the barriers to results described above. Taking the previous sanitation example, the funder could assess which actor is best placed to drive the types of knowledge and behavior change required to achieve results. Perhaps a community organization already exists with a track record of delivering similar behavior change projects with the same target population and confidence in its capacity to improve sanitation practices. Such circumstances could point to the value of establishing a PBC with this community organization. As described in section 4.2, along with these technical considerations, funders should also consider the absence of certain institutional, legal, and political conditions as an impediment to the RBF value-add. Identifying which actor should be incentivized, and therefore which instrument category is most appropriate to the context, can help focus the analysis and narrow subsequent choices, but should be subject to refinement as new information emerges. Further, the selection of the specific instrument most suitable to the context (such as deciding between different types of PBCs such as output-based aid or performance-based financing) involves additional considerations outlined in section 5.2. 42 A Guide for Effective Results-Based Financing Strategies IMPLICATIONS This section has reviewed the importance of understanding RBF’s value-add in a given context This understanding should be based on an identification of the current barriers to results and an assessment of the extent to which these barriers could be addressed by any of RBF’s drivers of impact. This analysis enables funders to develop preliminary answers to questions including: • RBF desirability »» To what extent could any of RBF’s drivers of impact address existing barriers to results? »» Is RBF likely to add enough value to justify its use in the specific context? • Selection of RBF approach and instrument category – Which type of agent is best placed to deliver the results, and which specific instrument(s) (as detailed in section 5.2) would work best? These early assessments can help focus the subsequent analysis, but should be treated as provisional until the end of the contextual analysis. 4.2 Conditions for RBF Because RBF is relatively novel and involves changes to standard development practices, an enabling environment with specific conditions in place is required for the effective use of RBF. The absence of these conditions can weaken RBF’s value-add. These conditions can be organized in terms of technical, institutional, legal, and political conditions, as summarized in table 4.2.22 Technical conditions must be addressed for the fundamental reason that paying for results requires that results can be delivered, measured, and rewarded. RBF cannot deliver results by itself; it requires suitable interventions that can deliver the desired results. Moreover, RBF can only be applied when suitable results are available that funders can pay for (that is, when appropriate payment metrics are in place). For instance, to activate any of the four drivers of impact, it must be possible to identify results that are aligned with the desired impact. In addition, the design of an RBF agreement requires the availability of certain data to inform decisions, such as identifying the intended beneficiaries, setting performance targets, and measuring and pricing results. The successful design and implementation of RBF will also often require stakeholders to have more advanced institutional capacities and appropriate legal arrangements. For instance, the shift to paying for results requires additional technical expertise such as that needed to define the measurement approach and set up verification systems. Likewise, incentivized agents often need additional capacities to deliver the full benefits of RBF, such as the capacity to use performance management systems to implement data-driven course corrections. 22 Pritchett 2005. A Guide for Effective Results-Based Financing Strategies 43 Finally, perhaps the most critical prerequisite for RBF’s success often is sufficient political will and alignment across actors. Given its technical and institutional demands, developing and implementing RBF is often a time- and resource-intensive process, making sufficient political support critical to any prospect of success. One implication is that, when working with governments, funders should aim to use RBF on topics that are a political priority for the government to ensure the necessary buy-in. To illustrate, consider the example of a funder seeking to expand water access for residents of small rural towns by establishing a PBC with a regional water utility. In these circumstances, the funder might find that certain necessary conditions for the successful use of RBF are not in place. For instance: • Technical conditions: Sufficient data on the costs of providing water access are not available to calculate appropriate prices for results achieved. • Institutional and legal conditions: The incentivized agent does not have access to the pre- financing necessary to cover the investment required to expand water coverage and manage other expenses until results can be measured and payments are received. • Political conditions: There is insufficient buy-in from decision-makers to test a new approach to financing service delivery because of the limited political influence of the affected communities. Table 4.2 summarizes the most important conditions for RBF’s effective use that funders should assess before adopting an RBF strategy. TABLE 4.2 Conditions for RBF CONDITION ASSESSMENT KEY CONSIDERATIONS CATEGORIES PURPOSE TECHNICAL CONDITIONS FOR RBF Are there existing or prior interventions to address the social issue that: To identify whether Suitable suitable interventions • Can be applied to the current context (for example, interventions exist that can deliver interventions dealing with the same social issues and target relevant results population)? • Has some evidence based on results? Are there results that are: To assess whether • Reliably connected with the desired social impact? Measurable measurable results • Not likely to produce perverse incentives? results exist that are suitable as payment metrics in • Objectively measurable at reasonable cost and within an RBF agreement reasonable time frames? • Within the manageable control of the incentivized agent? 44 A Guide for Effective Results-Based Financing Strategies TABLE 4.2 Conditions for RBF (cont.) CONDITION ASSESSMENT KEY CONSIDERATIONS CATEGORIES PURPOSE To understand Are data of sufficient quality available on: whether the available data is adequate • The nature of the social problem (such as the target Data to estimate targets, population size and demographics)? availability appropriate funding • The cost of service delivery? levels, pricing results, and assess payment • Past performance? risks for both parties INSTITUTIONAL AND LEGAL CONDITIONS FOR RBF (BY RELEVANT STAKEHOLDER) Does the funder have sufficient capacity in such areas as: • Human capacity: Sufficient staff and leadership with the right skills, experiences, and expertise to fulfill the often time- intensive requirements of developing RBF. • Verification: Capacity to undertake or contract an effective To use RBF effectively, verification process. Capacity requirements range from funders need certain Capacity of collecting and managing institutional or survey data to capacities, available results undertaking evaluations. either internally or funder(s) accessed through • Disbursements: Capacity to make timely and reliable outsourcing payments against the achievement of results. • Contract management: Capacity to identify the most competitive providers and often actively manage their RBF contracts. This can include monitoring performance, identifying performance issues, and having practices in place in case of underperformance. Does the incentivized agent have capacity in such areas as: • Delivery capacity: Capacity to achieve the desired results. • Technical capacity: Capacity to understand what RBF will require of them, whether the design is aligned with their To take on the mission, and what payment risks they will face. responsibilities • Results management: Appropriate performance Capacity of created by RBF, such management systems in place to help improve the delivery incentivized as managing results, of results (such as capacities in data collection, management, agent(s) incentivized agents and analysis and the ability to devise and implement course- will require certain corrective measures based on data-driven insights). capacities • Pre-financing capacity: Capacity to access the financing necessary to cover the cost of delivery until results payments are made. • Risk management: Incentivized agents need the capacity to manage payment risks. A Guide for Effective Results-Based Financing Strategies 45 TABLE 4.2 Conditions for RBF (cont.) CONDITION ASSESSMENT KEY CONSIDERATIONS CATEGORIES PURPOSE Are key systems elements in place such as: • Appropriate regulatory conditions: Disbursement and procurement regulations and processes must be compatible To operate effectively, with RBF (such as enabling the advanced commitment of System certain systems level funds to pay for future results and providing transparency and readiness elements need to be in flexibility for the incentivized agent). place for RBF • Sufficient suitable providers: In circumstances where funders engage services providers at scale, a strong market made up of numerous providers is preferable because it allows for more efficient delivery of results. POLITICAL CONDITIONS FOR RBF Sufficient stakeholder • Is there sufficient buy-in from the relevant decision makers, buy-in is critical to including the government? Stakeholder RBF’s successful buy-in • Is buy-in likely to be sufficiently stable and sustainable? design and implementation Relevant stakeholders Do stakeholders have: must be sufficiently aligned in terms of • Shared objectives (such as emphasis on results, innovation, Stakeholder and building the evidence base through the chosen their strategies and alignment approach)? interests to enable agreement on an RBF • Shared understanding of the problem? approach The myriad of conditions needed to use RBF effectively have significant implications for RBF’s value-add and the timelines to deploy RBF. Therefore, comprehensively identifying where these conditions are not met, considering likely implications, and developing approaches to address them is critical to a successful RBF strategy. One approach to address the absence of certain necessary conditions is to adjust the RBF design to minimize their negative effects, as discussed in section 5.3. Alternatively, stakeholders interested in proceeding with RBF can work to create these conditions, as outlined in the following subsection. Box 4.2 presents a cautionary example of an RBF scheme in which some technical conditions were not in place. 46 A Guide for Effective Results-Based Financing Strategies BOX 4.2 Example of Technical Conditions: Lack of Appropriate Measurable Results in the Municipal Solid Waste Minimization and Recycling Project in Ningbo, China, 2010 The municipality of Ningbo, China, generated 1.2 million tons of municipal solid waste in 2009. This figure was projected to grow to 1.4 million tons of waste per year, which would result in a shortfall of disposal capacity of 1,000 tons per day. This posed an environmental problem, particularly because a reported 85.2 percent of Ningbo households did not separate waste at home and even waste that was separated was not transported and treated separately because facilities were lacking. Responding to this challenge, the World Bank and the municipal government of Ningbo launched an RBF scheme in 2010. The RBF instrument was a performance-based transfer (PBT) in which the incentivized agent was the Neighborhood-level Residents Committee (NRC), the lowest level of local government. RBF payments totaled $4.54 million and were disbursed based on measures of the quality and quantity of recycling achieved. The PBT’s effectiveness and sustainability was limited by the absence of technical conditions – specifically, timing difficulazties in reliably measuring results achieved by the NRC as the basis for payment. The challenge emerged because the waste delivered to neighborhood collection points was targeted by private waste recyclers that would take the waste for resale before it could be measured as the basis for verifying the NRC’s results. The inability to accurately measure waste separation meant the NRC did not receive the anticipated payments. The payment metrics were affected by factors outside the agent´s control, reducing the incentives for municipalities to find new ways of driving greater waste separation. Source: World Bank 2014a. A Guide for Effective Results-Based Financing Strategies 47 4.2.2 Assessing the conditions for RBF To proceed with RBF, funders and other stakeholders will often have to work toward building an enabling environment by creating the necessary conditions. In deciding which conditions need to be created, it is useful to assess each condition against two dimensions: how critical is it; and whether it can be created at reasonable cost and within a reasonable time frame. First, how critical the condition is relates to the extent to which it would reduce RBF’s value added. Relatively minor shortfalls against some conditions do not need to delay the use of RBF and can instead be addressed over time (such as small gaps in the agent’s capacity to manage results). In contrast, more critical conditions will need to be addressed before an RBF strategy can be developed (such as issues with the incentivized agent to pre-finance the delivery costs before results payments are made; see box 4.3).23 Second, funders must assess whether the necessary conditions can be created within a reasonable time frame and at reasonable cost relative to the anticipated benefits of RBF. For instance, with sufficient political support, creating the necessary regulatory environment through reforms to procurement and disbursement regulations might cost relatively little and be achievable within a reasonable time frame. In contrast, conditions that cannot easily be created include situations in which it is very difficult to identify measurable results appropriate for payment metrics (such as some development projects focused on promoting democracy or building civil society). To inform their understanding of the viability of RBF and how it can be used, funders should assess the context for the necessary conditions against these two dimensions, as summarized in figure 4.1. For instance, where the conditions are partially in place and the remaining issues can be addressed, RBF funders can proceed with the use of RBF and work to create the necessary conditions during the implementation, as depicted in the top right-hand quadrant of figure 4.1. In contrast, where the conditions for RBF are severely lacking and creating them is likely to be expensive relative to RBF’s anticipated benefits, funders may decide not to pursue RBF in this context, placing them in the bottom left hand quadrant of figure 4.1.24 23 In practice, the assessed lack of a condition should take into account the offsetting effects of the mitigating design choices discussed in section 5.3. That is, assessed comprehensively, the lack of a condition is the extent to which it would reduce the RBF value added after accounting for mitigating design choices. This conclusion assumes appropriate mitigating design choices are not possible. See box 5.1 for summary details of 24 mitigating design choices. 48 A Guide for Effective Results-Based Financing Strategies FIGURE 4.1 Assessing the Conditions for RBF Where funders have decided to proceed with RBF by creating the necessary conditions, they will often face several options on how to do so, with implications for the timing of RBF’s use. For example, where suitable interventions to be funded by RBF are not available, funders can start by providing flexible innovation grants to a range of promising providers to develop ideas for the social issue at hand and then transfer to RBF once sufficient evidence has been developed. A Guide for Effective Results-Based Financing Strategies 49 BOX 4.3 Example of Creating the Preconditions for Pre-financing for RBF: Increasing Access to Water in Kenya In Kenya, only around half of the population has access to water. County-owned and community- based water service providers (WSPs) are responsible for providing water and sanitation services in rural and peri-urban areas. They face increasing pressure from rapidly growing populations and related investment needs. Responding to these needs, in 2007 the World Bank and other development partners launched an RBF pilot scheme, using an output-based aid (OBA) subsidy, to expand water coverage. An important condition needed for WSPs to make effective use of RBF was their capacity to pre- finance needed investments before receiving partial results payments. This was because many WSPs did not have sufficient cash on hand to finance service improvements and expansion, and lacked access to other sources of finance. To improve the WSPs’ access to pre-financing, the World Bank and other development partners provided technical assistance and other support. This included toolkits to strengthen financial management, project modeling, and business plan writing. Further, under the subsequent scale- up initiatives, a credit guarantee structure was introduced, which, combined with the OBA subsidy, incentivizes both WSPs and domestic lenders to invest in water and sanitation improvement projects. With this approach, the WSPs contributed 20 percent of the total project cost upfront and borrowed 80 percent through a five-year loan from a domestic bank. Upon the achievement of pre-agreed output targets (new water connections and increased average monthly revenues), the World Bank provided OBA to the WSPs to buy down up to 40 percent of the debt. Together, these efforts effectively created the pre-financing capacity needed to use RBF and enhanced the program’s overall results. The project, which closed in 2013, increased water connections for more than 200,000 low-income consumers. Its successes led to the preparation of two scale-up projects to support additional WSPs in small towns and urban areas to access loan financing from the domestic private sector. Source: Advani 2016. 50 A Guide for Effective Results-Based Financing Strategies IMPLICATIONS This section has summarized the conditions required for RBF’s effective use and discussed how these conditions can be assessed and how they affect the decision to use RBF. Combined with a knowledge of the likely RBF value added, an understanding of the contextual conditions for RBF should underpin funders’ decisions as to whether RBF represents a better option than traditional funding in the given context. If the RBF drivers of impact are not likely to address identified barriers to results or the necessary conditions cannot be created, traditional funding is likely a better option. Alternatively, if the RBF drivers of impact are likely to improve results and the necessary conditions are in place or can be created at reasonable cost, funders can proceed with RBF and may need to consider how the conditions for RBF’s use can be further improved. In sum, this section should support funders in addressing the following questions: • RBF desirability: Given the existing conditions, is RBF likely to add enough value to justify its use? • Building the conditions for RBF: What conditions need to be created and how does doing so effect time frames? A Guide for Effective Results-Based Financing Strategies 51 This section details the four elements of the RBF strategy component of the diagnostic tool. First, the section provides guidance for funders on the choice of RBF instrument, focusing on the different objectives that various RBF instruments advance, different funder mandates, existing funding arrangements, sectoral focus, and the maturity model. Second, it presents several key design choices and offers guidance on these choices, centering on tension between efforts to activate one or more of RBF’s drivers of impact and practical constraints. The section concludes by summarizing issues related to strengthening and scaling the RBF strategy over time. 52 Diagnostic Component 2: RBF STRATEGY A Guide for Effective Results-Based Financing Strategies 5 5. DIAGNOSTIC COMPONENT 2: RBF STRATEGY Drawing on the contextual analysis, the RBF strategy component entails making decisions on how RBF should be used in the given context. These decisions center on three questions: 1. Which RBF instrument should be used? 2. What core RBF design features should be adopted? 3. How can RBF be refined over time to improve impact? These questions are considered in turn in the sections that follow. 5.1 Choosing the RBF instrument Section 4.1 outlined how funders’ mandates and an understanding of barriers to results can guide the choice of the incentivized agent and provide the first-order definition of the RBF instrument. This section builds on that discussion, outlining further considerations that can inform the choice between specific RBF instruments. One way of selecting between different RBF instruments is based on the objectives they are best suited to achieve. This approach is depicted in figure 5.1 and described throughout this section in relation to two overarching objectives that different RBF instruments can advance: (1) improving the effectiveness of the delivery systems, and (2) improving the effectiveness of specific interventions. A Guide for Effective Results-Based Financing Strategies 53 FIGURE 5.1 RBF Instruments Organized by Typical Objectives Delivery Systems are made up of the full range of actors working toward achieving the desired results. For instance, an education system might involve the national government responsible for funding and setting policy; regional governments with responsibility for administering school systems; schools responsible for delivery; and nonprofit service providers playing a range of support roles. Strengthening delivery systems then involves incentivizing a network of agents to deliver results effectively and sustainably. Strengthening delivery systems can center on national governments, local governments, or service providers, cutting across multiple responsibilities and a myriad of interactions. As depicted in figure 5.1 and explained throughout this section, several RBF instruments are particularly suited to strengthen delivery systems. In a delivery system context, different types of RBF instruments can be used in combination to drive enhanced performance along chains of interactions and relationships. This might involve a donor establishing a performance-based loan with a national government, complimented by a performance-based transfer to a local government, and/or outsourcing delivery to local providers with a performance-based contract. For example, in 2016 the World Bank established a Program for Results (PforR) with the national government of the Arab Republic of Egypt to improve the business environment and support enhanced local administration of infrastructure and services. In addition to incentivizing the national government, the PforR also made some of the funding ($270 million) available to two local governments conditional on their achievement of minimum performance targets. These arrangements therefore incentivize a national government and two local governments and effectively create a PBL and PBT.25 World Bank 2016a. 25 54 A Guide for Effective Results-Based Financing Strategies In contrast to a focus on delivery systems, other RBF instruments are used to enhance the effectiveness of specific interventions. In this context, the focus should be on choosing the RBF instruments best able to drive enhanced effectiveness by accounting for the intervention’s maturity in terms of the extent to which evidence is available on the intervention’s impact. This assessment can be informed by the maturity model, presented in figure 5.2, which provides a categorization for interventions based on the different levels of evidence underpinning their effectiveness. With this rubric, interventions can be loosely grouped into three categories entailing different implications for RBF’s potential to drive enhanced effectiveness. FIGURE 5.2 Intervention Maturity Levels and Implications for RBF The rest of this section undertakes a deeper analysis of specific RBF instruments and characterizes their typical use, grouped in terms of the four RBF instrument categories. A Guide for Effective Results-Based Financing Strategies 55 5.1.1 Performance-Based Aid – Working with national governments Performance-based aid (PBA), defined by funders working with national governments as the incentivized agent, is the most common type of RBF, both by number of projects (169 out of 326) and spending tied to results ($21 billion out of $25 billion). As shown in figure 5.1, PBA instruments are particularly useful for strengthening delivery systems and are therefore typically used when bilateral or multilateral donors want to support a government-driven reform process. Within this category, funders have three choices of RBF instruments, corresponding to the three different types of PBA: • Cash on delivery (COD) (entailing performance-based grants) • Performance-based loans (PBL) • Performance debt buy-down (PDBD) The choices between these instruments will normally be driven by the funder’s mandate and objectives. For instance, COD will be particularly well suited to funders that traditionally use grants, such as most bilateral aid agencies. Alternatively, multilateral organizations that focus on loans may choose PBLs, such as the World Bank’s PforR instrument, which was developed to support partner governments in enhancing their capacity to deliver services. In contrast, foundations seeking to increase spending in a priority sector may opt for PDBD. The rest of this subsection illustrates the different rationales for each PBA instrument and characterizes their typical use. The concept of COD was developed in 2010 by the Center for Global Development (CGD) and advocated for with a string of seminal reports.26 In these reports, CGD outlines several rationales for COD, including: • Enhancing accountability among funders and governments to constituents, in part by promoting transparency. • Improving local ownership and strengthening institution building based on greater discretion for governments. • Promoting learning by doing, experimentation and greater investment in data.27 Since the first COD in 2005, it has been applied across at least 71 projects as identified in the database. Of these projects, 92 percent (65) target the health sector, predominantly funded by Gavi, the Vaccine Alliance. Other sectors include education, agriculture, energy, and public administration, with funders including the United Kingdom Department for International Development (DFID) and the Inter-American Development Bank. Across these projects, most (83 percent, or 59) have paid for outputs, compared to 13 percent that paid for a mix of outputs and outcomes and only one that based payment entirely on outcomes. Geographically, 52 percent (37) of identified projects have been implemented in Sub-Saharan Africa. 26 Birdsall et al. 2011; Perakis and Savedoff 2014. Birdsall et al. 2011. 27 56 A Guide for Effective Results-Based Financing Strategies PBL instruments are most commonly used to work with partner governments to enhance their capacity to deliver services. The most common type of PBL is the World Banks’ PforR instrument, launched in 2012. PforR now has an extensive track record in building national capability for service delivery through collaboration between the World Bank and recipients, particularly in education.28 PBLs are the largest RBF instrument by value, accounting for 83 percent ($21 billion) of the total $25 billion of identified RBF spending tied to results. PforRs account for 90 percent ($19 billion) of PBL funding tied to results and for 80 percent (76) of identified PBL projects. The main sectoral focus of PBLs has been in education, with one-quarter (23) of projects in this sector, including 16 PforRs. Public administration and health are also important areas of focus, with 16 and 15 PBLs in each sector, respectively. 31 percent (29) of PBL projects have taken place in South Asia, followed closely by Sub-Saharan Africa, with 28 percent (27) of projects. PDBD instruments have been used by third-party funders to increase spending on priority areas. PDBDs allow third-party funders, usually private foundations, to pay a portion of a loan held between a national government and a lender based on the achievement of certain results. This enables foundations to increase spending in priority areas by reducing the repayments faced by countries and incentivizing them to take extra loans to target priority issues.29 As such, PDBDs help crowd-in more spending for priority areas and enable private funders to play a catalytic role in the pursuit of results. Only three PDBDs with complete information were identified, one of which entailed $50 million of funding tied to results.30 These projects were in the health sector and funded by a group of donors, including the European Commission and Bill and Melinda Gates Foundation, with the underlying loan provided by the World Bank. 5.1.2 Performance-Based Transfers – Working with local governments Performance-based transfers (PBTs) involve local governments receiving grants conditioned on certain results. PBTs are often pursued to strengthen local government service delivery by providing the incentives and flexibility needed to enhance their performance.31 For instance, PBTs can help address the capacity issues commonly faced at lower levels of government by creating the incentives to ensure that capacity building efforts actually deliver improvements.32 Likewise, by requiring the publication and dissemination of results, PBTs create the transparency to strengthen the relationship between tiers of governance and citizens, improving accountability 28 World Bank 2017. 29 Burnett et al. 2013. 30 Some projects commonly referred to as examples of PDBDs are not included in the database because they are not based on results. UNCDF 2017. 31 32 UNCDF 2017. A Guide for Effective Results-Based Financing Strategies 57 in the process.33 Funders choosing to work with local governments by using PBTs can also support effective decentralization, ensuring that decision-making power and responsibility is placed closer to the ground and enhancing accountability to citizens. Some of these benefits are illustrated by Plan Nacer, a program introduced to “enhance the existing health system by providing incentives to provincial governments, which they passed on to healthcare providers,” and that was found to have improved prenatal care results34 (see box 2.2). OBD is a subtype of PBT developed by the World Bank to improve predictability for government budgeting and to enhance the efficiency of government assets. These objectives are achieved by linking service outputs with associated unit costs and ensuring that disbursements reflect the actual cost of the services delivered.35 Only two OBDs were identified, but they entail significant spending tied to results, with a total of almost $758 million across both projects. The database includes 28 PBTs (including OBDs), almost 90 percent of which (25) involve transfers from national governments (sometimes working with external donors), while the remainder involve external funding from organizations such as the World Bank or DFID. These PBTs are estimated to entail at least $2.9 billion tied to results. By numbers, 60 percent (17) of these projects were focused on public administration, with multiple projects also identified in health and transport sectors. The largest geographic focus is in Sub-Sahara Africa, with one-third of identified projects (12). 5.1.3 Performance-Based Contracts – Working with service providers Performance-based contracts (PBCs) are used by funders when working with one or multiple service providers. From the database analysis, 122 PBCs were identified, consisting of at least $1.4 billion tied to results. This spending was spread across a wide range of different types of PBCs. These subtypes include: • Performance-based financing (PBF) • Output-based aid (OBA) • Prize-based challenges • Standard PBCs As illustrated in figure 5.1, PBCs can be used either to strengthen delivery systems or with a focus on enhancing the effectiveness of specific interventions. The choice between these instruments will often rest on which of these objectives is being pursued. Broadly, PBF and OBA are generally focused on strengthening delivery systems, while prize-based challenges and most other standard PBCs are usually focused on enhancing the effectiveness of interventions. The rest of this 33 UNCDF 2017. CGD 2015. 34 World Bank 2007. 35 58 A Guide for Effective Results-Based Financing Strategies subsection categorizes these PBCs in terms of their focus and explores their different rationales and typical sectors of use. STRENGTHENING DELIVERY SYSTEMS For PBCs focused on strengthening service delivery, the choice of PBF, OBA, or other types of PBCs will depend on factors such as the funder’s sector of focus and existing funding arrangements. PBF is a form of PBC used in the health sector that has the potential to strengthen delivery systems by shifting the focus from the quantity of services delivered to the quality of services delivered.36 PBFs build on existing funding arrangements by adapting the fee-for-service approach used to fund health facilities, in which payment is conditioned on certain activities, such as the number of patients seen for a certain medical procedure. PBF changes this approach by paying for fee-for-service-conditional- on-quality, where funding is based on performance against certain targets and/or on the verified improvements against a baseline indicator. For example, a PBF in Rwanda conditioned payment on results including the number of children that successfully completed a vaccine course, contrasting a fee-for-service approach in which course completion may not be considered.37 This shift helps to improve the quality of services and supports the expansion of these programs at scale.38 OBA is a type of PBC used to enhance access to basic infrastructure and social services for the poor through performance-based incentives, rewards, or subsidies. OBA subsidies seek to increase the level of transparency with which development funds are used and incentivize providers to deliver an acceptable level of service to low-income consumers who would otherwise not be able to afford it.39 As such, OBA can contribute to strengthening delivery systems by supporting service providers to effectively expand their coverage to the poor. For instance, as cited in box 2.3, in 2006 an OBA instrument was used to expand water access to low-income households in Morocco. By 2014, the initiative had provided 10,504 households with piped water, 9,036 households with improved sanitation services and found high beneficiary satisfaction with the new services.40 The majority of OBA projects are funded by GPOBA, with 48 projects since 2006 tying $239 million to results. Because OBA involves a subsidy for the upfront user fees required to receive a service, it focuses on sectors traditionally funded in part through user fees. Most OBA projects have been implemented in infrastructure sectors, with 33 percent of projects in water and sanitation and 44 percent in energy. However, it has also been applied more broadly to other basic services, including health, information and telecommunications technologies (ICT), solid waste management, and education. 49 percent of all projects have been implemented in Sub-Saharan Africa, followed by South Asia, with 22 percent of projects.41 36 Bauhoff 2017. 37 Basinga et al. 2011. 38 Bauhoff 2017. 39 GPOBA 2014b. 40 GPOBA 2014a. All figures on OBA spending and project numbers are from the GPOBA 2017 Annual Report. 41 A Guide for Effective Results-Based Financing Strategies 59 ENHANCING PROGRAM IMPACT In contrast to PBF and OBA, where the focus is on enhancing the effectiveness of delivery systems, the decision between the use of prize-based challenges and other standard PBCs should be based on the intervention’s maturity and guided by the maturity model introduced in figure 5.2. Prize-Based Challenges, by tying 100 percent of funding to results and providing complete flexibility to achieve these results, are a powerful instrument to promote the discovery of new solutions in a low maturity environment. That is, prize-based challenges can be used in contexts with almost no evidence on past effectiveness and without the funder even needing a plausible theory of change as the basis for their funding. However, prize-based challenges still require the availability of suitable measurable results and are generally only suitable for sectors with a range of providers that can compete to develop new interventions. The database identified seven prize-based challenges, including two in the education sector, with total funding tied to results of $44 million.42 Finally, standard PBCs include all PBCs not included among the other categories identified in this report. These PBCs can be used in a variety of ways corresponding to the circumstances of different interventions at different levels of maturity. For instance, in a high maturity context, PBCs can focus on providing the incentives to support an expansion of an intervention at scale with impact. The database identified 51 PBCs that do not fall into one of the above categories and that account for approximately $900 million tied to results. The majority (32) of these PBCs are focused on education and more than half (26) have been implemented in Sub-Sahara Africa. 5.1.4 Impact Bonds – Working with investors Finally, impact bonds provide funders with the option of working with investors as the incentivized agent to fund service delivery. Impact bonds allow funders to draw on private financing by making investors, rather than service providers, responsible for financing service delivery and the related payment risks. These features mean that, as depicted in figure 5.1, SIBs and DIBs are generally best suited for enhancing intervention effectiveness. Specifically, as reflected in figure 5.2, impact bonds are often appropriate to enhance the effectiveness of medium maturity interventions since they enable stand-alone projects in which 100 percent of funding is tied to results and leave the payment risk with the investor. This can afford the service provider significant flexibility to iterate the design and delivery of the interventions, allowing for the adaptations necessary to catalyze promising interventions. These benefits are apparent in the Educate Girls development impact bond, described in box 2.4, where the DIB supported Educate Girls to refine its intervention based on increased flexibility and data-driven insights enabled by RBF.43 42 Many prize-based challenges are not based on results and are therefore excluded from the definition of RBF used in this report 43 Instiglio 2017 60 A Guide for Effective Results-Based Financing Strategies Impact bonds with 100 percent of funding tied to results are less appropriate for low maturity environments because high uncertainty about likely results makes them difficult to design and creates excessive risk for all stakeholders. Likewise, impact bonds are less likely to be a cost-effective option in a high maturity context. This is because of their relatively high costs (due to high transaction costs and the required return for investors), which can seldom be justified for a well-understood intervention with a demonstrated impact and which is ready to be implemented at scale. However, there are circumstances where impact bonds can play a useful role in a high maturity context, such as when the intervention is preventative and able to generate savings. In this case, choosing an evidence-based preventative intervention and scaling it up using the upfront capital provided by investors will reduce the treatment costs down the line, creating savings to repay the investor. Impact bonds therefore provide an alternative to standard debt for governments to finance service delivery and can be particularly attractive where the investor is socially minded and able to offer concessionary low-interest rates. Finally, impact bonds may also be preferred to PBCs because they create the necessary financial conditions for some providers to participate in an RBF agreement. Specifically, by providing upfront funding for providers, they offer the bridging capital necessary for some providers to cover cost of delivery before results payments are made. Further, impact bonds shift the financial risk of nonpayment toward investors, thus allowing providers that could not take on this risk to operate using RBF. An analysis of the database indicates that impact bonds are the smallest class of RBF instrument by a substantial margin. Only seven impact bonds in low- middle-income countries were identified, entailing approximately $44 million of funding tied to results. Three of these examples focus on health and two on education. In contrast to the other RBF instruments, impact bonds are much more likely to pay for outcomes, rather than outputs, and to use either experimental or quasi- experimental verification methods rather than observational approaches. These design features are described in detail in section 5.2, along with presentations on their implications. IMPLICATIONS This section has outlined the factors that should inform the choice of RBF instrument. This section provides funders with guidance on selecting an RBF instrument, offering insights on which instru- ment is most appropriate for the funder’s context. A funder’s choice of instrument should usually respond to their objectives for using RBF (either strengthening delivery systems or enhancing inter- vention effectiveness), their mandates, existing funding arrangements, the sector of focus, and the maturity of the intervention at hand. A Guide for Effective Results-Based Financing Strategies 61 5.2 Designing the RBF instrument Following the selection of the RBF instrument, funders will need to design the RBF instrument. This section discusses several of the key design decisions that stakeholders need to resolve. These decisions are presented in table 5.1. The considerations outlined here are far from exhaustive, intended only to identify key issues and trade-offs that apply to many but not all RBF instruments. Readers interested in detailed guidance to complement this diagnostic tool can refer to Instiglio’s 2017 report, A Practitioner’s Guide to Results-Based Financing - Getting to Impact, upon which the content that follows is based.44 The decisions of funders covered in this section should all be driven by the objective of maximizing the potential RBF value added. This means making design choices that maximize the effect of one or more of the four drivers of impact identified in chapter 4. However, these choices will continue to be subject to the contextual conditions that can constrain RBF’s effectiveness identified in chapter 4. Good design decisions require funders to carefully balance the trade-offs between the pursuit of impact against practical feasibility. This tension is highlighted in the following discussion and summarized in table 5.4 at the end of this chapter. TABLE 5.1 Key Design Features DESIGN FEATURE DESCRIPTION Choosing outcomes and outputs to be measured and verified as 1. Payment metrics the basis of payment 2. Portion of funding Setting the level of funding tied to results versus the level attached attached to results to inputs and activities Determining the prices attached to different metrics and across 3. Pricing structure populations 4. Verification and reporting Deciding on the approach for the verification and reporting on approach payment metrics Defining the level of delivery prescription stipulated in the RBF 5. Contractual flexibility agreement Selecting the intervals at which results will be verified and 6. Frequency of payment payments made 5.2.1 Selection of payment metrics Ideally, funders would pay for the achievement of the ultimate intended impact of an intervention, such as increased lifetime earnings and reductions in poverty due to an education intervention. Paying for impact would align incentives and attention toward the desired impact, provide the agent with maximum flexibility to achieve this impact, and promote accountability to the beneficiaries for the things that matter most. Other documents with detailed guidance for funders include NAO 2015; DFID 2014; World Bank 2012; and USAID 2010. 44 62 A Guide for Effective Results-Based Financing Strategies However, in practice, RBF will always use some form of output or outcome proxy for the intended impact because it is difficult to use the intended impact as an effective payment metric. In practice feasible proxies are often only distantly related to the intended impact due to the following practical considerations: • Measurability – It is often easier and less costly to measure outcomes and outputs more distantly related to the desired impact. For example, the increase in well-being (the impact) stemming from early childhood development interventions is much more difficult to measure than early learning gains (an outcome). • Time horizon – The ultimate impact may not be measurable within a reasonable time frame, therefore requiring the use of outcome and output proxies. For instance, the increased lifetime earnings due to an early childhood development intervention will take decades to become apparent and again can be proxied by early learning gains, which materialize much sooner. • Manageable control – Measures closely related to the ultimate impact are often more sensitive to external factors, limiting the incentivized agent’s control over these outcomes. Indicators over which agents have limited control usually make for poor payment metrics. This is because they increase the risks involved in the transaction for all parties, potentially limiting the agents’ willingness to experiment, and raising costs for the funder. Low manageable control can also weaken the agents’ incentives to achieve the results because the reward for their effort is lower. Continuing the early childhood development example, this type of intervention may improve lifetime earnings or increase well-being. However, the multiplicity of other factors that affect these outcomes leave them only tenuously within the control of a provider of early childhood interventions, rendering them unsuitable as payment metrics. Reflecting on these considerations, in practice, output measures are by far the most common type of results paid for across RBF projects. Drawing on the database, it is estimated that two-thirds (207) of the projects identified in the RBF database use only outputs as the payment metrics. A further quarter (81) use a combination of outputs and outcomes, while only 6 percent (18) make exclusive use of outcomes. These statistics highlight the need to balance the benefits of selecting measures closely connected with the intended impact against practical considerations. Further, in choosing the precise outputs and outcomes to be used as payment metrics, funders should consider: • Cost – The cost of data collection and analysis should be proportionate to the RBF value added. • Perverse incentives – Metrics should incentivize the agent to achieve the intended impact rather than focusing on activities that will trigger payment but that are poorly aligned with impact. Finally, as well as assessing payment metrics individually, the overall bundle of payment metrics should also be assessed. Selecting too many indicators risks distracting the incentivized agent from A Guide for Effective Results-Based Financing Strategies 63 the metrics that matter the most. However, having too few can risk neglecting important areas of focus, especially where the selected results are relatively distant from the intended impact, pointing to a need for a careful balance in the number and type of metrics selected. 5.2.2 Portion of funding tied to results Rather than tying all funding to results, most RBF agreements leave a portion of funding based on inputs and activities. The portion of funding tied to results can range from 100 percent, as with most impact bonds and some PBCs, to leaving all existing funding tied to inputs and activities but introducing an additional bonus payment tied to results. The portion of funding tied to results is a significant design choice because it contributes to the strength of the incentives and the level of flexibility provided to the incentivized agent. Where the RBF instrument relies on incentives to improve impact, this driver will generally be stronger when more funding is tied to results. Likewise, a larger portion of funding attached to results will often mean less funding attached to inputs and, therefore, will increase the flexibility provided to the incentivized agent.45 Both these prospects provide a rationale for tying relatively more funding to results. However, this conclusion should be balanced against practical constraints: the more funding attached to results, the higher the risks for all parties. For the funder, tying a large portion of funding to results raises costs because of the related additional risk premium. Further, it increases the risk of non-disbursement if results are not achieved – a risk that is often a major concern for large bilateral and multilateral donors.46 Increased risk also poses a challenge for incentivized agents, especially where they are not able to absorb financial loss. For a relatively small NGO, making a large portion of payments contingent on results could seriously jeopardize their financial sustainability. In these circumstances, excessive risk could limit the incentivized agent’s appetite for taking the risks necessary for experimentation, learning, and adapting. Therefore, tying too much funding to results could actually reduce the incentivized agent’s flexibility as part of an RBF agreement. This can be of particular concern in a context of low maturity, where the certainty of achieving results is lower. The portion of funding tied to results is also constrained by the incentivized agent’s pre-financing capacity. When pre-financing capacity is low, the incentivized agent will need relatively more upfront traditional funding to cover its delivery costs, which leaves relatively little room to tie funding to results. This factor therefore creates a constraint on the portion of funding tied to results; however, RBF can be used to help incentivized agents get access to pre-finance, as described in section 4.2 on the preconditions for RBF. If the alternative to RBF is a flexible unearmarked grant, instead of traditional activity-based funding, then tying funding 45 to results would actually reduce the flexibility of the incentivised agent. 46 Conversely, tying a large portion of funding to results also raises the risk that funders will have to pay more than expected. This risk is dealt with by stipulating caps or maximum payment amounts in RBF agreements. 64 A Guide for Effective Results-Based Financing Strategies Given these factors, the optimal portion of funding tied to results balances the need to provide the incentivized agent with strong incentives and flexibility against the need to limit risk and account for the agent’s pre-financing capacity. 5.2.3 Pricing structure Funders will also need to determine how much they should pay for the achievement of the defined results. For PBCs and impact bonds in particular, prices may vary across different types of results (such as payments for installations of solar home systems or connections of new consumers to a grid) or for the same results achieved across beneficiary groups (for example, with higher prices for connecting consumers living in remote areas). Getting these prices correct is critical to ensure that incentives are aligned with the desired impact (such as improving quality of life through electricity access), thus effectively activating the incentive driver of impact. A range of approaches can be used to price results. These are summarized in table 5.2. TABLE 5.2 Overview of Different Pricing Methods METHOD DESCRIPTION AND APPLICABILITY This approach is based on the known costs of delivering the results and adding Cost-plus a percentage.a It can be useful where reliable data on costs and performance of the specific case at hand are available. Cost-driven With this method the price is derived by comparing the average cost for Comparative delivering the same results with similar interventions. It is useful where cost cost- data are not available for the specific program but are available for similar effectiveness interventions. When incentivizing providers, a competitive procurement process can be Market used to determine the most appropriate price. The viability of this approach is determined contingent on suitable conditions for competitive procurement, including the presence of numerous possible providers. Benefit-driven This approach is based on an estimate of government future savings or revenue Cashable that could be made by achieving the defined results. It is most appropriate benefits where government is the funder. Prices can also be set based on an estimate of the social value attainting Social value the specified results. This approach is useful in ensuring alignment between payments and their social value. See Instiglio 2015 These approaches can also be used in combination: a triangulation approach whereby the resulting prices from each approach can be compared for validation. For example, a social-value approach could be used to test that a cost-plus method does not lead to payment levels that exceed the social value of the results. The rigor with which all these different methods can be used to set appropriate prices will be contingent on the extent and quality of available data. A Guide for Effective Results-Based Financing Strategies 65 5.2.4 Verification and reporting approach To trigger payment as part of any RBF instrument, results must be measured and verified. Determining the verification approach involves two key decisions. Both involve balancing rigor against cost and capacity demands. First, funders should decide on their preferred measurement method, defining who collects the data, when, and how. Data collection by independent third parties and larger samples typically increase the confidence in the results but are more expensive.47 Second, funders need to choose the verification method, deciding whether they want to pay for observed results or results attributable to the program. Choosing the most appropriate verification method requires judgement based on the circumstances of the proposed RBF approach and the trade-offs of the different methods, as outlined in table 5.3. Analysis of the database indicates that observational approaches are used in the majority of cases. Of the projects identified, 99 percent (308) use observational methods, with only 1 percent (three) of projects use causal methods. TABLE 5.3 Observational and Experimental Verification Approaches OBSERVATIONAL CAUSAL METHODS Paying for observational results Causal methods, such as randomized control entails paying for the number trials or quasi-experimental methods, seek of outputs and / or outcomes to estimate results that are attributable to Definition achieved. the intervention. That is, causal methods allow funders to pay only for those results estimated to be additional to those that would have occurred without the program. • Cheaper and easier to understand, • Offers greater rigor and therefore is able administer, and scale up. to better estimate the interventions’ true impact, providing better evidence and lower • Always used where the result is an risk of overpaying or underpaying. Benefits output because there is less doubt on the causal relationship. • Appropriate for complex and exploratory interventions where results are sensitive to external factors. RBF for infrastructure projects use RBF projects testing new approaches in observational methods, such as social sectors, particularly impact bonds, Common counting the number of households are more likely to use causal methods, such application connected for water access. as testing the impact of an intervention on learning outcomes. For more information about independent verification and how verification is built into the RBF design, refer to GPOBA 47 2016. 66 A Guide for Effective Results-Based Financing Strategies 5.2.5 Contractual flexibility Some RBF contracts limit the flexibility provided to the agent. This includes contracts that prescribe certain activities or inputs despite making payment contingent on results. Alternatively, the practical flexibility provided to the agent can also be limited, intentionally or not, by arduous reporting requirements on the types of activities and inputs used, hindering the agent’s institutional and legal capacity to flexibly adjust its approach. Program rigidities introduced by complex legal or practical constraints limit the potential of the agent to adjust its program in the pursuit of results and therefore weaken the role of flexibility as one of RBF’s drivers of impact. This can mean that the agent is held accountable for results while also not being able to flexibly adjust its approach to achieve these results. This is particularly problematic in low and medium stages of maturity because it can deprive agents of the capacity to experiment and adjust their program in the search of better solutions. In practice, funders’ capacity to develop sufficiently flexible contracts can be affected by certain institutional, legal, and political conditions. For instance, government regulation may require the inclusion of substantial reporting requirements in the contract. Likewise, political conditions may prevent a government funder from removing its oversight and control of entities’ funds through RBF. For example, this can be seen with PBTs from a national to a local government where the national government faces political pressure if results are not achieved and therefore may be reluctant to trust the local government to achieve these results without oversight and control. 5.2.6 Frequency of payments RBF payments are generally made periodically as results are verified (on an annual basis, for example) or upon completion of the RBF agreement. High frequency payments can be useful in driving impact by encouraging stronger attention to results. This is because more frequent reporting and disclosure of results can ensure that attention is more sustained and less sporadic. Likewise, higher frequency payments can drive greater accountability by providing beneficiaries with more regular updates. However, the frequency of payments can be affected by practical considerations. For instance, frequent payment requires frequent measurement, often involving substantial cost and placing significant demands on verification systems in terms of data collection and analysis. Alternatively, more frequent payments might be necessary for incentivized agents with limited capacity to cover delivery costs for long periods without payment. Box 5.1 summarizes principles to guide RBF design, while table 5.4 summarizes this section’s review of design features, outlining design recommendations and required conditions to enhance the effectiveness and sustainability of RBF. A Guide for Effective Results-Based Financing Strategies 67 BOX 5.1 Principles to Guide RBF Design In addition to the specific design issues identified in this section, funders should also keep in mind key lessons learned from past design processes. These include: • Prioritize stakeholder engagement – Often the hardest part of an RBF design process is not determining the technical design features, but rather finding an effective way to collaborate with the diverse stakeholders involved in the design process. Successful stakeholder engagement can be underpinned by certain practices, including starting with a detailed understanding of the relevant stakeholders; building alignment among stakeholders by creating shared objectives; and effectively planning the sequencing and engagement approach for each stakeholder. • Consider pre-financing needs – The pre-financing needs of incentivized agents is a key consideration that emerges at many points along the diagnostic tool. The pre-financing capacity of the incentivized agents is recognized as an important condition for RBF in section 4.2 and highlighted as a key factor in selecting the type of RBF instrument, such as the choice between a PBC and impact bond described in section 5.1. Moreover, many design choices also require consideration of the incentivized agent’s pre-financing capacity. As discussed, this includes the portion of funding tied to results and the frequency of payments. This means that the agent’s capacity for pre-financing is an issue that should be revisited throughout the diagnostic process and is one that can be addressed from many different angles – including by creating the appropriate conditions for RBF, selecting the appropriate instrument, or through design decisions. • Select mitigating design decisions – Rather than always pursuing the theoretically best design choices to maximize the impact of RBF, it will often be necessary to make design choices that mitigate the absence of certain conditions needed to use RBF effectively. For instance, the absence of suitable metrics closely related to the intended impact may require funders to select payment metrics, such as outputs, which are more distantly related to the desired impact. As shown throughout this chapter, this principle applies to all the design features considered here. • Use the maturity model to guide key choices – The maturity model is a useful framework for understanding how to use RBF most effectively in different contexts. For instance, as discussed, a low maturity context entails more risk regarding results payments; thus, tying relatively little funding to results often makes sense in this context. Likewise, where maturity is low, it is particularly important that RBF contracts provide sufficient flexibility for experimentation. • Recognize that design features are interrelated – It is impossible to make good choices concerning one design feature in isolation from others. For instance, when deciding on the portion of funding tied to results, funders should asses the risk created by other design features, such as the payment metrics and the verification approach. • Balance ambition with practicality – A good design provides objectives that are both ambitious and achievable. This means balancing the need for ambition in the selection of results, prices, and targets against practical realities. Nobody benefits from an RBF instrument which is set up for failure. • Aim for simplicity – Funders should also focus on ensuring the design is kept as simple as possible. Technically sophisticated designs might be appealing in theory but can add to the difficulty of implementation and communication across stakeholders. • Focus on learning – As outlined in section 5.4, ensuring that the RBF instrument delivers greater impact over time requires the creation of dedicated learning agendas, with questions specified from the outset that can be answered over time and used to inform the evolution of the RBF instrument. Sources: Instiglio 2017; World Bank 2017; NAO 2015; DFID. 2014; USAID 2010; World Bank 2012. 68 A Guide for Effective Results-Based Financing Strategies TABLE 5.4 Summary of Design Features with Design Recommendations and Required Conditions DESIGN RECOMMENDATIONS CONDITIONS DESIGN What design choices would maximize the What practical constraints may limit the design FEATURES effect of the relevant drivers of impact? choices? (Key condiTIONS IN BOLD) (relevant drivers in bold) All RBF drivers of impact will be Paying for outcomes is subject to the stronger when paying for results availability of suitable measurable results: that are more closely linked to that is, outcomes that are: impact because this: • objectively measurable and verifiable at • reduces prescription and increases reasonable cost and time frames flexibility • within the manageable control of the 1. Payment • more closely aligns incentives with incentivized agent. metrics impact, limiting perverse incentives Because of these constraints, it will often be • more closely aligns attention to necessary to pay for outputs, rather than impact outcomes. • enhances accountability to beneficiaries for results they care about. Flexibility can be increased by The amount of funding that should be tying more funding to results, attached to results will be constrained by and therefore having less funding factors including: attached to activities and inputs. However, tying more funding to • the reliability with which results can be results also increases the risk faced estimated – attaching a large portion to 2. Portion of by the agent, likely reducing their highly uncertain results will often create funding appetite for flexible experimentation. excessive risk attached Maximizing flexibility requires to results • the capacity of the agent to bear financial balancing these prospects. risk To strengthen incentives, more • the capacity of the agent to finance funding should generally be attached delivery (for example, through its own to results; the more funding attached resources, loans, or impact bonds). to results, the stronger the incentives are to achieve them. To ensure that incentives are aligned Setting appropriate prices will be subject with impact, payment metrics must to the availability of quality data covering be priced correctly. For instance, the factors such as the nature of the social 3. Pricing costs and risks of delivering different problem (population demographics, for structure results or the same results across example), the cost of service delivery, and heterogeneous populations should past performance. be reflected in the pricing. A Guide for Effective Results-Based Financing Strategies 69 TABLE 5.4 Summary of Design Features with Design Recommendations and Required Conditions (cont.) DESIGN RECOMMENDATIONS CONDITIONS DESIGN What design choices would maximize the What practical constraints may limit the design FEATURES effect of the relevant drivers of impact? choices? (Key condiTIONS IN BOLD) (relevant drivers in bold) The verification approach should be The independence and rigor of the independent and rigorous enough verification approach should be balanced 4. Verification to ensure the agent believes results against the capacity it requires and cost it and reporting will be accurately assessed and imposes on the funder and agent. approach rewarded/penalized and therefore has the incentive to deliver on them. Agreements must minimize Developing appropriate agreements is prescription to enable flexibility. subject to constraints such as: This includes minimizing formal 5. Contractual prescription on inputs and activities • regulatory conditions that prevent flexibility and creating conditions that enable nonprescriptive agreements monitoring and reporting. • political conditions that limit the potential for nonprescriptive agreements. High frequency payments can Flexibility to determine the frequency of encourage stronger: payments can be constrained by factors such as: • attention – More frequent reporting and disclosure of results • the nature of the verification system (the can ensure that attention is data collection approach, for example) 6. Frequency of maintained with fewer gaps. payment • the capacity of the funder to make • accountability – More frequent payments reporting and disclosure of results can provide beneficiaries with • the incentivized agent’s financial capacity, more regular information as the with organizations with less capacity to basis of holding the agent to self-finance in greater need of higher account. frequency payments. IMPLICATIONS This section has outlined several of the key design features that will significantly influence how the selected RBF instrument operates and the impact it will deliver. The discussion has also provided guidance for funders deciding between design features, emphasizing the trade-offs that are often required between strengthening one or more of the RBF drivers of impact against practical constraints, as summarized in table 5.4. 70 A Guide for Effective Results-Based Financing Strategies 5.3 Strengthening and Scaling RBF While designing and implementing an RBF strategy, funders will also need to plan for the RBF instrument’s evolution over time. This involves two dimensions: how to strengthen RBF, and how to scale it up. First, funders should work to strengthen the RBF value added over time to deliver greater impact. This can include enhancements such as greater tailoring of the intervention to the target population, better-suited RBF design features, and better coordination across parties. Improvements in these and other areas can be promoted by three key practices: • Testing and calibrating: Starting RBF implementation with an explicit focus on testing and calibrating key design features can help improve the intervention and add to its sustainability. This can be achieved by piloting an RBF instrument at small scale and/or with a moderate level of incentives, to then learn from its strengths and weaknesses and adjust the design accordingly. An RBF pilot serves as a proof of concept for its future scale up, and can help answers questions such as: »» Is the intervention likely to produce unanticipated (positive or negative) effects? »» What benefits and challenges will the intervention produce for the incentivized agent? »» Has the cost of delivering results been correctly estimated? • Learning and revising over time: Once the RBF design has been tested and finalized, continuous learning should be promoted. This might involve establishing an institutional framework and/ or dispute resolution or course-corrective mechanisms to guide the evolution of the RBF design based on experiences. Systematically assessing and revising the RBF design is needed to improve its value add and sustainability. • Developing sustainability strategies: It is important to identify factors that affect an RBF agreement’s sustainability early on and define preliminary strategies to address them. For instance, a strategy to sustain RBF despite a change in government or to sustain funding beyond the intervention’s duration could prove necessary. The national scale-up of an RBF program in primary health care in Armenia48 is a good illustration of the importance of these three practices for successful scaling. From 2003 to 2005 a primary health care pilot was tested and experienced multiple challenges for effective RBF implementation, such as inadequate infrastructure, limited human resources, and insufficient information systems. The lessons learned from this pilot led development partners (USAID and the World Bank) to work with the Ministry of Health and the State health agency to develop various sustainability strategies. Together they developed the necessary infrastructure and information systems, incorporated RBF into the national regulatory systems, and committed funds from the national budget. Moreover, the continuous learning throughout the program was key to its success. Performance indicators, for example, were constantly assessed and adapted as necessary to ensure improved results. Petrosyan et al. 2017. 48 A Guide for Effective Results-Based Financing Strategies 71 In some contexts, it may also make sense to scale the use of RBF over time. The scaling process can involve growing an RBF pilot into a program and eventually integrating it into a broader system.49 Growth can occur across one or several dimensions, such as increasing the number of beneficiaries served, expanding geographic coverage, or diversifying the types of interventions delivered. For example, in 2010, a GPOBA grant50 for $13.95 million was approved to support a World Bank and Global Environment Facility initiative to improve electricity access for poor households in Bangladesh in rural, off-grid areas. In 2015, an addition of $15 million enabled the scaling of the program. The scale-up aimed to benefit over 1.2 million people and cover the installation of multiple clean energy technologies, and provided more effective targeting of poor households in rural remote areas by offering subsidies for small systems only. In addition, economies of scale and technological advancements allowed for a gradual reduction in the subsidy amount overtime. By leveraging the capacities of microfinance institutions and the private sector, the scale up contributed to strengthening the ongoing development of a commercial market for renewable energy solutions. In general, some of the key considerations for scaling include: • Is the intervention relevant at scale? For instance, will there be a need for it in other geographic areas? Is there a large enough beneficiary population that is not yet receiving a similar intervention? Are there service providers with sufficient capacity to support this growth? • Will there be sufficient funding now and later to support growth? • Is the RBF instrument suited to growth or would another instrument be more appropriate? For instance, as outlined in section 5.2, interventions at medium maturity can often best be tested with impact bonds, but as maturity increases and the focus shifts to delivery at scale, PBCs are often the most appropriate model. While distinct topics, strengthening and scaling RBF are interrelated. In many contexts, strengthening the RBF approach to make it more effective is a perquisite to scaling it up. Likewise, during the scaling up process, all actors should maintain a focus on strengthening the approach to ensure that its impact continues to be achievable at scale. For instance, when expanding RBF to new beneficiaries with different needs, continuous learning and revision ensures that the intervention scales in a way that effectively serves this new target population and achieves the desired results. 49 Meesen 2017. World Bank 2015. 50 72 A Guide for Effective Results-Based Financing Strategies IMPLICATIONS This section has laid out some key considerations to strengthen and scale up RBF over time. This analysis enables funders to understand how RBF can deliver results in the long term. It has implications for: • RBF value add: Will RBF keep adding value over time? • Selecting the RBF instrument: Can the current instrument allow for growth or would another instrument be more appropriate? • Designing the RBF instrument: Should certain design features be adjusted to strengthen or scale up RBF? A Guide for Effective Results-Based Financing Strategies 73 APPENDIX A. RESULTS-BASED FINANCING INSTRUMENTS TABLE A.1 RBF Instruments and Examples RBF INSTRUMENTS EXAMPLES Incentivized agent: National government RBF instrument: Performance-based aid (PBA) Performance debt buy-down: A Botswana HIV Initiative, 2009, Botswana.a This initiative PBA where a results funder pays aimed to increase the coverage, sustainability, and off a portion of an outstanding loan efficiency of HIV/AIDS interventions. For that purpose, a held by a central government if $50 million loan agreement was signed between the World certain results are achieved. Bank and central government. The latter then signed a performance-based aid agreement with the European Commission, which agreed to buy down $20 million of the original loan if targets were achieved. The results were moderately satisfactory: 55% of the predefined targets were achieved or surpassed and 33% were partially achieved.b Performance-based loan: A PBA Enhancing Teacher Education Program, 2016, Vietnam:c financed through a loan. The World This program aims to improve primary and secondary Bank’s Program-for-Results (PforR) education. For that purpose, the World Bank agreed to is an example of a performance- lend $95 million to the central government, which receives based loan. In this arrangement, the periodical disbursements conditional upon the provision tranches of the loan are disbursed of training, retraining, and continuous professional to the national government upon development activities to school teachers and principals. the achievement of predefined results. Cash on delivery: A PBA financed Salud Mesoamerica Initiative, 2012, eight Central through a grant. Cash on delivery American countries:d This initiative supports governments generally involves a fixed payment in the provision of high-quality health services to the most to the government for each unit of vulnerable populations. Cash-on-delivery agreements were progress against an agreed result signed between the Inter-American Development Bank, the and provides the government with Carlos Slim Foundation, the Spanish Agency for International full responsibility and discretion for Development Cooperation, the Bill & Melinda Gates using the funds. Foundation, and the central governments. Payment metrics and associated targets were predefined by the results funders and each government. Examples of these metrics include the number of health units that had the minimum required supplies for newborn and pregnancy emergencies. a World Bank 2008. b World Bank 2015. c World Bank 2016b. d IDB 2015. 74 A Guide for Effective Results-Based Financing Strategies TABLE A.1 RBF Instruments and Examples (cont.) RBF INSTRUMENTS EXAMPLES Incentivized agent: Local government RBF instrument: Performance-based transfer (PBT) Performance-based transfer: A Plan Nacer, 2007, Argentina: This program provides transfer of funds between a national incentives to provincial governments to enhance the government or a bi/multilateral coverage and quality of health services to poor pregnant donor and a local government. women and newborns. A PBT agreement was signed Payments to local governments between the central government and local governments, are at least partly based on whereby the national government disbursed 60% of performance. the predefined payments depending on beneficiaries’ enrollment rates; and the remaining 40% was disbursed based on performance against 10 indicators tailored to Output-based disbursement each province. Results showed that in large hospitals, the is a subtype of PBT to improve program reduced risk of neonatal death by 74% among the predictability for government beneficiaries and by 22% nationwide.e budgeting and to enhance the Mexican Initiative Transforming the Water Sector in efficiency of government assets. Guanajuato (output-based disbursement), 2005, Mexico: These objectives are achieved To augment connections to safe and reliable water services by linking service outputs with and improve waste water treatments, the World Bank associated unit costs and ensuring financed this program through an OBD scheme. It granted that disbursements reflect the $38 million to the local government upon the completion actual cost of the services. and verification of results. Results showed that by April 2008 the initiative had provided full access to water to more than 90.640 inhabitants, most of them living in poor areas. Similarly, more than 60 municipal water utilities had shown an increase in overall efficiency of at least 10%.f Incentivized agent: Service provider RBF instrument: Performance-based contract (PBC) Output-based aid: A form of Rural Electricity Access with Small-Scale Providers PBC mostly commonly used by Initiative, 2007, Bolivia:g This initiative aimed to increase GPOBA, a multi-donor trust fund affordable access to electricity in remote areas of managed by the World Bank. It is an Bolivia. GPOBA granted $4.9 million to electricity service agreement where service delivery is providers (Enersol and Energetica) through an output- contracted out to a public or private based aid scheme. Service providers took on most of the provider, which receives a subsidy pre-financing risks and received the disbursements from to complement or replace the GPOBA after having delivered results. As a result, 7,700 required user contribution if certain solar home systems were installed for rural households, results are achieved. OBA is used to schools, clinics, and micro and small enterprises. enhance access to and delivery of Additionally, 5,705 Pico-PV systems for lighting and basic infrastructure and social services for ICT services for the poorest households were distributed. 7 8poor. the 56 Both numbers exceeded the original targets.h e CGD 2015. GPOBA 2008. f g World Bank 2013c. h World Bank 2014c. A Guide for Effective Results-Based Financing Strategies 75 TABLE A.1 RBF Instruments and Examples (cont.) RBF INSTRUMENTS EXAMPLES Performance-based financing: A form Maternal and Child Health and Nutrition Improvement of PBC typically used to fund public or Project, 2014, Ghana: i This project aimed to improve private health service providers (such the utilization of community-based health and nutrition as health facilities) with primarily a services, especially among pregnant women and children fee-for-service payment mechanism, under the age of two. A performance-based financing in which health care providers are agreement was signed between the World Bank and paid for each service performed. community health teams in which the World Bank disbursed Performance-based financing a total of $10 million when predefined outcomes and payments are generally for fee-for- outputs were delivered. service-conditional-on-quality, which means that they are funded on the basis of their performance to meet targets or undertake specific actions, and/or on the verified improvements against a baseline indicator. Prize-based challenge: A form of Global Learning XPRIZE, 2014, global:j This program sought PBC where an open-bid competition to find innovative ways to enable children in developing awards a financial prize for the countries to teach themselves basic reading, writing, best innovation developed within a and arithmetic within 18 months. The program allowed predefined time frame in response teams from all over the world to develop and propose to a prespecified social issue. The software solutions. In stage 1, of the 135 global teams open bid allows many organizations that applied, five finalists received $1 million each to field- to compete for the prize; the winner test their solutions with 3,000 to 4,000 children. Finalists is the organization that develops the were selected based on the likelihood of the technology to most cost-effective solution to the produce significant learning gains as well as various design prespecified social issue. features such as usability and customizability. In stage two, a unique winning team will be selected and awarded $10 million to scale up its solution and enter the market. The winning team will be the one that generates the greatest improvement in literacy and numeracy among 7–10-year- olds. Performance-based contract: Any Skills Development Project, 2013, Nepal:k This project other type of RBF agreement in aimed to improve the employability of the Nepalese which payments to service providers workforce through basic and mid-level market-oriented are based on results. training opportunities. A PBC was signed between the Asian Development Bank (ADB), the government of Nepal, and private and public training providers. The ADB contributed $20 million and the central government $5 million. As part of this RBF arrangement, grants were disbursed once 123 predefined targets, such as providing additional places for market-oriented and basic skills training, were achieved. World Bank 2014b. i Instiglio 2017. j k ADB 2013. 76 A Guide for Effective Results-Based Financing Strategies TABLE A.1 RBF Instruments and Examples (cont.) RBF INSTRUMENTS EXAMPLES Incentivized agent: An investor RBF instrument: Impact bond Social impact bond: An agreement Workforce Development Social Impact Bond, 2017, in which an investor funds a service Colombia:l This Impact Bond aims to improve the provider and is paid back (at least in employability of vulnerable beneficiaries. It is considered part) by the government based on a SIB because the results funder is the government results. of Colombia. The investors are SECO, IDB-Fundación Bolivar Davivienda, Fundación Mario Santo Domingo, and Fundación Corona. The service providers include: Fundación Carvajal, Volver a la Gente, Gente Estrategica, and Kuepa. The investors provide upfront capital to the services providers and the government will pay the investors back their initial investment plus a return depending on performance. Development impact bond: An Educate Girls (EG), 2015, India: This Impact Bond agreement in which an investor aimed to improve Educate Girls’ impact on enrollment, funds a service provider and is retention, and learning for marginalized girls and boys paid back by a donor or any other in Rajasthan. It is considered a DIB because the results nongovernment funder based on funder is the Children’s Investment Fund Foundation results. (CIFF). Educate Girls is the service provider and the UBS Optimus Foundation is the investor. The investor provided upfront capital to Educate Girls and upon verification of the achievement of predefined results CIFF will pay back the UBS Optimus Foundation their initial investment plus a return. Year two results show that Educate Girls has achieved 87.7% of the three-year enrolment target and 50.3% of the three-year learning target. Instiglio internal knowledge. l A Guide for Effective Results-Based Financing Strategies 77 APPENDIX B. RBF PROJECTS BY INSTRUMENT TYPE AND SECTOR FIGURE B.1 Number of RBF Projects by Instrument Type and Sector, 1993-2018 80 70 80 60 SECTOR 70 Agriculture, fishing and forestry 50 Education 60 Energy and extractives SECTOR 40 Financial sector Agriculture, fishing and forestry 50 Health Education Industry, trade and services 30 Energy and extractives Information and communications technologies 40 Financial sector Multisector 20 Health Public administration Industry, trade and services 30 Social support Information and communications technologies 10 Solid waste management Multisector 20 Transportation Public administration 0 Water and sanitation Social support nk ry ct id fe r n nd g nd ge wn en t 10 d Ba ve tra da ns loa bo cin bo en do rl d eli con se tra s ed ct in an ct all y- rse m Solid waste management a pa pa h u o on ed t-b ed ba df dc tb bu –W s h as pu as c e- t im a se l im s e e b d is Transportation ult s Ca e- b ut e- b an en -b - cia ba ed ed s c O ac rm m ce So ize an c as Water and sanitation rR e 0 an rm rfo lop an Pr t-b fo rm fo e ve r m m rd u r on fa k ry e ra ct r p id D ef e r f o a n n d n g r f on t pg e n en t ra m eB v e Ptr ed a ns r lo o c i ebo u O en ow og rpld eli co n th se tra Peed tb an Pt all -d se m Pr t heor nd d Oa b d as p ac fin p ac ch b uy ur OW o e t - e - b d d t is b s– sh ba s tp u ba s ce t im se l im se de b dd ult Ca e- e- an en ba cia ba nc Ou ac m e- So e- ce se R es a m f o rm l op a nc riz an - ba r r e P t Source fo r : RBF rfo Database rm rfo rp e De v rm rfo r m tp u m pe Pe he rfo Pe Ou ra Pe r og h er Ot P Ot 80 FIGURE B.2 Number of RBF Projects by Instrument Type and Region, 1993-2018 70 80 60 70 50 60 REGION 40 East Asia & Pacific 50 Europe & Central Asia 30 Global REGION 40 Latin America & the Caribbean East Asia & Pacific 20 Middle East & North Africa Europe & Central Asia South Asia 30 Global 10 Sub-Saharan Africa Latin America & the Caribbean 20 Middle East & North Africa 0 South Asia nk er y ct id fe r n nd ing nd ng e wn en t 10 ld Ba ra a s oa o c o o Sub-Saharan Africa eli v nt ed an dl tb an tb all e -d em or d co as tr as e pa c in pa c h uy rs W h on s ed t-b s ed -b im e df im e dc tb is bu u e as b s– as ba tp ba nc -b nt ial as de dd ult C ce - Ou ac e- a e ce So c -b ce as e es 0 an rm pm ize an rR rm rm ot e rfo v elo m an Pr m rd u t-b fo k y f e r r n g o p e t m er o f n er r erac p r ai d D fe rflo oa n d in rfon Ou tg wn en ra Ba liv Pt th ed ns ed bo nc Pteb en do m r og rrp eo ld de c on Oa se t ra Pe s a ct f i na a c h all u y- r se t h n -b ed a p p c b u P o ed ut -b d d bt isb –W O sh as as ce t im se ial im se lts Ca -b tp -b an en ba ba de dd su n ce Ou a ce r m p m c e- S oc z e- n ce a se e o n i a b rR rm a rm rfo ve l a Pr rm ut - fo rfo rfo pe De fo rm rfo tp ra m pe Pe er er Pe Ou g r O th P Pr o he Ot Source: RBF Database 78 A Guide for Effective Results-Based Financing Strategies REFERENCES ADB (Asian Development Bank). 2013. Project Administration Manual-Project No. 38176. Nepal: Skills Development Project. https://www.adb.org/sites/default/files/project- document/77341/38176-015-nep-pam.pdf. Advani, Rajesh. 2016. Scaling Up Blended Financing of Water and Sanitation Investments in Kenya. Washington, DC: World Bank Group. Banerjee, A. E. Duflo, N. Goldberg, D. Karlan, R. Osei, W. Pariente, J. Shapiro, B. Thuysbaert, and C. Udry. 2015. “A Multifaceted Program Causes Lasting Progress for the Very Poor: Evidence from Six Countries.” Science 348 (6236): 1260799. Basinga, P., P. Gertler, A. Binagwaho, A. L. B. Soucat, J. Sturdy, and C. M. J. Vermeersch. 2011. “Effect on Maternal and Child Health Services in Rwanda of Payment to Primary Health-Care Providers for Performance: An Impact Evaluation.” The Lancet 377 (9775): 1421–28. Bauhoff, S., and A. Glassman. 2017. “Health Results Innovation Trust Fund at 10: What Have We Learned So Far?” Global Health Policy Blog (blog), January 30, https://www.cgdev.org/blog/ health-results-innovation-trust-fund-10-what-have-we-learned-so-far. Birdsall, N., W. D. Savedoff, A. Mahgoub, and K. Vyborny. 2011. Cash on Delivery. Center for Global Development. http://cgdev.org.488elwb02.blackmesh.com/sites/default/files/1423949_file_ CODAid_SECOND_web.pdf. Burnett, N., N. Elias, R. Hecht, P. Isenman, J. Schweitzer, and M. Thomas. 2013. Final Report on Buying Down Loans for Education. Washington, DC: Results for Development. https://www. globalpartnership.org/download/file/fid/4684. CGD (Center for Global Development). 2015. Argentina´s Plan Nacer. http://millionssaved.cgdev.org/ case-studies/argentinas-plan-nacer. Clist, P. 2016. “Payment by Results in Development Aid: All that Glitters Is Not Gold.” World Bank Research Observer 31(2): 290–313. DFID (Department for International Development) (United Kingdom). 2014. “DFID Operational Guidance for Payment by Results.” DFID, London. Economist. 2015. “Graduation for Destitution.” http://www.economist.com/node/21660133. A Guide for Effective Results-Based Financing Strategies 79 Gelb, A., A. Diofasi, and H. Postel. 2016. “Program for Results: The First 35 Operations.” Working Paper 430, Center for Global Development, Washington, DC. Gertler, P., P. Giovagnoli, and S. Martinez. 2014. “Rewarding Provider Performance to Enable a Healthy Start to Life – Evidence from Argentina’s Plan Nacer.” Policy Research Working Paper 6884, World Bank, Washington, DC. GPOBA (Global Partnership on Output-Based Aid). 2008. “Output-Based Disbursements in Mexico: Transforming the Water Sector in Guanajuato.” GPOBA, Washington, DC. ---------. 2014a. “Morocco Improved Access to Water and Sanitation Services Output-based Aid Project: Lessons Learned.” https://www.gpoba.org/LL04_MoroccoWater. ---------. 2014b. “The Use of Output-Based Aid in Small Scale Irrigation Schemes in Developing Countries.” https://www.gpoba.org/sites/gpoba/files/OBA_Irrigation_Study.pdf. ---------. 2016. “OBA Approaches— Getting Results: Independent Verification in Output-Based Aid. https://www.gpoba.org/sites/gpoba/files/OBA50_IVA.pdf. ---------. 2017. GPOBA 2017 Annual Report. https://www.gpoba.org/sites/gpoba/files/GPOBA_ AnnualReport_FY17_0.pdf. IDB (Inter-American Development Bank). 2015. Iniciativa salud Mesoamerica. http://www. saludmesoamerica2015.org/es/iniciativa-salud-mesoamerica/iniciativa-salud-mesoamerica- inicio,1904.html. Instiglio. 2015. “Educate girls development Impact Bond: Improving education for 18,000 children in Rajasthan.” http://www.instiglio.org/wp-content/uploads/2016/05/EG-DIB-Design-2.pdf ---------. 2017a. Results-Based Financing in Agriculture and Land Administration: Potential and Key Design Considerations for RBF to Drive Greater Results in the Sector. Washington, DC: Instiglio. ---------. 2017b. A Practitioner’s Guide to Results-Based Financing: Getting to Impact. Washington, DC: Instiglio; and Mississauga, Ontario: World Vision Canada. Mabuchi, S., and S. McCune. 2015.  “If You Build It, They Will Come? Lessons on Barriers to Service Use at PBF Facilities in Nigeria.” http://www.rbfhealth.org/resource/if-you-build-it-they-will-come- lessons-barriers-service-use-pbf-facilities-nigeria. Meesen, S. 2017. “Scaling Up and Integrating Your Results-Based Financing Scheme: A Progression in Four Phases.” RBF Health. https://www.rbfhealth.org/blog/scaling-and-integrating-your-results- based-financing-scheme-progression-four-phases. 80 A Guide for Effective Results-Based Financing Strategies NAO (National Audit Office) (United Kingdom). 2015. “Payment by Results: Analytical Framework for Decision Makers.” NAO, London. Perakis, R., and W. D. Savedoff. 2015. “Does Results-Based Aid Change Anything? Pecuniary Interests, Attention, Accountability and Discretion in Four Case Studies.” Policy Paper 52, Center for Global Development. https://www.cgdev.org/sites/default/files/CGD-Policy-Paper-52-Perakis- Savedoff-Does-Results-Based-Aid-Change-Anything.pdf. Perakis, R., and W. D. Savedoff. 2014. “An Introduction to Cash on Delivery Aid for Funders.” Center for Global Development. https://www.oecd.org/dac/peer-reviews/Introduction-cash-on-delivery- aid-funders.pdf. Petrosyan, V., D.M. Melkomian, A. Zoidze, and Z.C. Shroff. 2017. “National Scale-Up of Results-Based Financing in Primary Health Care: The Case of Armenia”. https://www.tandfonline.com/doi/full/10.1 080/23288604.2017.1291394 Pritchett, L. 2005. “A Lecture on the Political Economy of Targeted Safety Nets.”  Social Protection Discussion Paper 501, World Bank. http://citeseerx.ist.psu.edu/viewdoc/ download?doi=10.1.1.557.4874&rep=rep1&type=pdf. UBS Optimus Foundation and CIFF (Children’s Investment Fund Foundation). 2016. “World’s First Development Impact Bond on Track to Deliver Social Impact and Financial Returns.” Press release, July 5. http://instiglio.org/educategirlsdib/wp-content/uploads/2016/07/DIB-results-press- release_05072016.pdf. UNCDF (United Nations Capital Development Fund). 2017. Performance-Based Grant Systems. http://www.uncdf.org/ield/performance-based-grant-systems. USAID (U.S. Agency for International Development). 2010. Performance-Based Incentives - Primer for USAID Missions. https://www.k4health.org/sites/default/files/USAID%20Performance%20 Based%20Incentives%20July%202010.pdf. World Bank. 2007. Implementing Output-Based Disbursement Mechanisms for Investment Operations. ---------. 2008. Project Appraisal Document on a Proposed Loan in the Amount of US$50 Million to the Republic of Botswana for a National HIV/AIDS Prevention Support Project. Report No: 43112- BW. World Bank. ---------. 2012. “Interim Guidance Note to Staff: DLIs and Disbursement Arrangements.” https:// policies.worldbank.org/sites/ppf3/PPFDocuments/090224b08234146d.pdf. A Guide for Effective Results-Based Financing Strategies 81 ---------. 2013a. “Brief: Argentina’s Plan Nacer: Delivering Results for Mothers and Their Children.” http://www.worldbank.org/en/topic/health/brief/argentinas-plan-nacer-delivering-results-for- mothers-and-their-children. ---------. 2013b. “Global Fund, Partner to Expand Results-Based Financing for Maternal and Child Health.” Press release, December 11. http://www.worldbank.org/en/news/press- release/2013/12/11/world-bank-global-fund-results-based-financing-maternal-child-health. ---------. 2013c. Implementation, Completion and Results (ICR) Report - Report No. ICR00002995 (GPOBA Grant to Bolivia for a Decentralized Electricity for Universal Access Project). documents.worldbank.org/curated/en/856561468006309431/pdf/ ICR29950ICR0P10al0Access000P102479.pdf. ---------. 2014a. Results Based-Financing for Municipal Solid Waste. Urban Development Series Knowledge Paper No. 20. Washington, DC: World Bank. ---------. 2014b. Project Appraisal Document-Report No. 80618-GH. (Maternal and Child Health and Nutrition Improvement Project, Ghana) http://documents.worldbank.org/curated/ en/710561468281080517/pdf/806180PAD0P145010Box385199B00OUO090.pdf. ---------. 2014c. Implementation Completion Report (ICR) Review - Bolivia Decentralized Electricity for Universal Access. World Bank, Independent Evaluation Group (IEG). ---------. 2015. Implementation Completion Report (ICR) Review Report Number ICRR0020276. BW-HIV/AIDS Project SIL (FY09) (P102299). World Bank, Independent Evaluation Group. http://documents.worldbank.org/curated/en/453591468015668497/pdf/ICRR-Disclosable -P102299-06-13-2016-1465865167777.pdf. ---------. 2015b. Bangladesh rural electrification and renewable energy development - SHS Project. ---------. 2016a. Project Appraisal Document - Report, Upper Egypt Local Development Program. http://documents.worldbank.org/curated/en/292181475507142347/pdf/108094-PAD-REVISED- OUO-9-Egypt-UELDP-PAD-25Aug16-Clean-final-2-08292016.pdf. ---------. 2016b. Project Appraisal Document Report No: PAD1138. (Vietnam Enhancing Teacher Education Program). http://documents.worldbank.org/curated/en/528391518533702770/pdf/ ISR-Disclosable-P150060-02-13-2018-1518533692766.pdf. 82 A Guide for Effective Results-Based Financing Strategies ---------. 2016c. Vietnam - Results-Based Operation for Local Bridge Construction and Road Asset Management Project. http://documents.worldbank.org/curated/en/370451468186857408/ pdf/104789-PAD-P155086-IDA-R2016-0061-1-Box394885B-OUO-9.pdf. ---------. 2017. Results-Based Financing in Education: Financing Results to Strengthen Systems. http://documents.worldbank.org/curated/en/715791489054110215/pdf/113265-REVISED-RBF- Approach-Final-Digital-Apr21.pdf. ---------.2018. The World Bank Country and Lending Groups. https://datahelpdesk.worldbank.org/ knowledgebase/articles/906519-world-bank-country-and-lending-groups. A Guide for Effective Results-Based Financing Strategies 83 86 A Guide for Effective Results-Based Financing Strategies