77080 TECHNICAL REPORT 004/13 Results-Based Financing in the Energy Sector An Analytical Guide Chapter 1 1 ESMAP MISSION The Energy Sector Management Assistance Program (ESMAP) is a global knowledge and technical assistance program administered by the World Bank. It provides analytical and advisory services to low- and middle- income countries to increase their know-how and institutional capac- ity to achieve environmentally sustainable energy solutions for poverty reduction and economic growth. ESMAP is funded by Australia, Austria, Denmark, Finland, France, Germany, Iceland, Lithuania, the Netherlands, Norway, Sweden, and the United Kingdom, as well as the World Bank. Copyright © January 2013 The International Bank for Reconstruction And Development / THE WORLD BANK GROUP 1818 H Street, NW | Washington DC 20433 | USA Energy Sector Management Assistance Program (ESMAP) reports are published to communicate the results of ESMAP’s work to the develop- ment community. Some sources cited in this report may be informal documents not readily available. The findings, interpretations, and conclusions expressed in this report are entirely those of the author(s) and should not be attributed in any manner to the World Bank, or its affiliated organizations, or to members of its board of executive directors for the countries they represent, or to ESMAP. The World Bank and ESMAP do not guarantee the accuracy of the data included in this publication and accept no responsibil- ity whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement of acceptance of such boundaries. The text of this publication may be reproduced in whole or in part and in any form for educational or nonprofit uses, without special permis- sion provided acknowledgement of the source is made. Requests for permission to reproduce portions for resale or commercial purposes should be sent to the ESMAP Manager at the address below. ESMAP encourages dissemination of its work and normally gives permission promptly. The ESMAP Manager would appreciate receiving a copy of the publication that uses this publication for its source sent in care of the address above. All images remain the sole property of their source and may not be used for any purpose without written permission from the source. Written by | Vivid Economics for Oliver Knight Energy Sector Management Assistance Program | The World Bank Chapter 1 3 T a b le o f C ontents Acronyms and Abbreviations ii Executive Summary 1 1 | INTRODUCTION 9 2 | WHEN MIGHT RBF BE CONSIDERED? 13 3 | THE CHOICE BETWEEN RBF AND CONVENTIONAL FINANCING APPROACHES 17 RBF Approaches and the Allocation of Risk 17 Implications for Choosing Between RBF and Conventional Approaches 18  Preconditions 19   Factors which Affect the Cost/Risk Trade-Off 23   Risks and Incentives | Bringing it Altogether 26 Non-Risk Based Factors 28 4 | DESIGNING RBF MECHANISMS 31 Six Questions to Inform the Detailed Design of RBF Instruments 31   Who should be Eligible to Receive an RBF Payment? 31   What should be the Conditionality of an RBF Instrument? 32   What should be the Structure of an RBF Payment? 34   How should the Size of an RBF Instrument be Determined? 38   What is the Role of the Principal? 40   What should be the Duration of the RBF Intervention, and What is the Appropriate Exit Strategy? 42 ANNEX | LITERATURE REVIEW ON RESULTS-BASED APPROACHES 45 Description of the Existing Literature 45 Key Findings 49   Theoretical Findings 49   Empirical Findings 55   Findings on Prizes 62 ENDNOTES 66 REFERENCES 68 The Case Studies for this report are available at http://www.esmap.org/esmap/node/2726 List of Boxes and Figures Box 3.1 An Example of How to Apply Some of the Preconditions 21 Box 3.2 Trustworthy Monitoring and Verification Procedures Creates a Trade-Off between 25 Capacity Building and Fast Implementation Box 3.3 Considering Allocation of Risk: A Pure RBF and Pure Upfront Support—Two Ends of a Spectrum 27 Box 3.4 Using Quantitative Modelling Analysis: RBF Approaches vs. Conventional Approaches 29 Figure 1 A Three-Stage Approach for Considering RBF 2 Figure 1.1 The Analysis is Divided into Three Key Nodes 10 Figure 2.1 RBF Might be Appropriate in Cases Where the Principal Wishes to Provide a Subsidy to a 16 Private Market or in Cases of Procurement Figure 3.1 RBF Provides Stronger Incentives for Agents to Perform but Also Places Greater Risk on Agents 18 Figure 4.1 Questions to Help Identify the Appropriate Conditionality for an RBF Intervention 34 Figure 4.2 An RBF Intervention Can Achieve the Same Market Impact up to a Certain Quantity Threshold 37 Figure 4.3 The Cost Difference to the Principal of Direct Procurement and Subsidies 41 i A cron y ms an d A b b re v iations AMC advance market commitment OECD Organisation for Economic Co-operation and Development CCT conditional cash transfers PBF performance-based financing COD cash on delivery PBR payment by results CO2 carbon dioxide PERZA Off-Grid Rural Electrification Program (Nicaragua) DFID Department for International Development (UK) PforR Program-for-Results EBRD European Bank for Reconstruction and Development QP quantity-performance instrument ESCO energy service company R&D research and development GBP British Pound Sterling (currency) RBA result-based approach GPOBA Global Partnership on Output-Based Aid RBF results-based financing J-PAL Abdul Latif Jameel Poverty Action Lab RoI return on investment kWh kilowatt hour SHW solar hot water M&E monitoring and evaluation UK United Kingdom NGO nongovernmental organization US$ United States dollar (currency) OBA output-based aid ii Executive Summary This report identifies the circumstances when results-based financing (RBF) approaches might be an appropriate energy sector intervention in developing countries, especially to promote energy access and energy efficiency. It was commissioned by the Energy Sector Management Assistance Program (ESMAP) as part of a broader initiative looking at the potential for greater use of results-based approaches in the energy sector in developing countries. The fundamental idea of RBF approaches is that payments that would otherwise be made automatically are made contingent on delivery of a preagreed (set of) result(s), with achievement of the result(s) being subject to independent verification. There has been increasing interest in whether and how they could be used within the energy sector to deliver more, or more cost-effective, results but, to date, relatively little work has been done on the circumstances in which different versions may be best employed. This report aims to help fill the gap. Its novelty rests on a conceptual examination of RBF mechanisms to derive insights on when and under which circumstances they are desirable, and the application of these insights to the energy access and energy efficiency contexts. A strong understanding of the circumstances in which different versions of RBF are best deployed is important; it may prevent RBF from being used in adverse circumstances, where its disadvantages may outweigh its advantages; it may also encourage a wider deployment of RBF in favorable circumstances, so that fewer good opportunities are missed. The report is focused on individuals and organizations who are considering whether an RBF approach would be a sensible way to deliver some specific goods or services to help meet a defined public policy goal. These individuals or organizations might then go on to implement an RBF scheme—in the language of the report, they would be the RBF scheme’s focus on the energy sector in developing countries principal.1 Although there are a range of potential principals, those that are most likely to act as an RBF principal given ESMAP’s focus on the energy sector in developing countries might be either national or subnational governments or development agencies. The decision about whether and what form of RBF instrument to use can be broken down into three decision points or nodes. These three nodes are: 1 | What are the circumstances in which RBF approaches might be a possible solution? 2 | When would an RBF approach be preferred to a conventional2 intervention approach? 3 | How might an RBF instrument be designed in terms, for instance, of who should be entitled to receive an RBF; what should be the trigger for receiving an RBF payment; how should any payment be structured and its size determined; and what is the appropriate role of the principal? This framework is illustrated in Figure 1. When Should RBF be Considered? RBF can be used when a principal wishes to subsidize output in a market. RBF can be used as a means to disburse subsidies in a market. In turn, subsidies should primarily be used to address the Results-Based Financing in the Energy Sector | An Analytical Guide 1 Figure 1 A Three-Stage Approach for Considering RBF 3 2 Design RBF 1 Choose RBF Who should be eligible? Consider RBF Choose Policy What is the condition or trigger Objective Conventional for paying out the RBF? Disregard Approaches RBF What is the structure of the pay-out? What is the size of the payment(s)? What is the role of the principal? What is the RBF’s duration and exit strategy? Source | Vivid Economics. problem of positive externalities: when there are benefits to society as a whole from greater production or consumption of a good or service but this is not taken into account by those making the production or consumption decision. In these circumstances, subsidies can align private and social interests, and boost output in a market. Other forms of intervention, such as taxes or technology mandates, may be able to perform a similar role to subsidies in terms of adjusting price signals in favor of particular goods or services, for example, taxing high carbon activities will provide a stimulus to low carbon activities, but they will not provide the same overall market expansion impact, that is, the overall supply of high and low carbon goods will likely fall under a tax. This makes subsidies a particularly attractive intervention when social/distributional impacts are taken into account. RBF might also be considered by a principal who wishes to increase the efficiency of its procurement. As well as intervening in private markets, principals may consider RBF when contemplating how to procure certain goods or services, either for its own consumption or as a result of a policy choice that such products should be bought by the principal for subsequent disbursement to final consumers. When RBF is used in this context, there may be less opportunity for the principal’s intervention to be withdrawn at a later stage to leave a self-sustaining market. When Might RBF be Preferred to Conventional Approaches? At heart, the choice between an RBF approach and a conventional approach relates to the allocation of project risks between principal and agent. In this report, the term ‘agent’ refers to those firms or providers who are expected to deliver the results. Under a conventional approach, 2 the financial risks of project failure are substantially borne by the principal: in the event that the project fails to deliver the anticipated results, then the principal’s resources will have been committed already and will be (substantially) unrecoverable. The agent faces relatively little risk. By contrast, under an RBF approach, the risks borne by the principal are reduced: if the project fails to deliver the expected results, then the financial resources of the principal are not expended. As a corollary, since, under an RBF approach, the agent will only receive additional resources in the event that the desired results are provided, the risks it faces are much greater than under a conventional approach. Placing greater risks on an agent has both an advantage and a disadvantage that will determine whether an RBF approach is to be preferred. The advantage of placing greater risk on the agent is that the stronger incentives mean that they will be more likely to provide the results desired by the principal. This may increase the quantity or quality of output that the principal can achieve with a given amount of resources. The disadvantage is that placing additional risk on the agent will mean that the agent will require (the prospect of) higher reward as compensation, which may reduce the amount that the principal can achieve with a given amount of funding. Consequently, the principal needs to balance the possibility that RBF approaches increase the probability of achieving a set of results against the likelihood that the delivery of these results will be more expensive. This trade-off determines, to a large extent, whether a principal can achieve more or better results with an RBF, or whether a more conventional approach may be more effective. There are at least three fundamental preconditions for introducing an RBF approach. 1 | It needs to be possible to monitor and verify the results against which payments might be made and these results need to be closely linked to the overall outcomes desired. An RBF instrument will be impractical in the absence of a measure against which to disburse it. 2 | If payments are only made by a principal after the delivery of the results, the agent will need to have access to sufficient finance to cover any upfront costs prior to delivery of results. This largely relates to access to capital markets, although it is possible that certain agents could finance any upfront costs from their own resources.3 Sufficient access to prefinance may be a more acute challenge for capital intensive and large projects and/or in countries where financial sectors are less mature.4 A further critical determinant of access to finance will be the credibility of the RBF scheme. It will be easier to raise finance against an RBF scheme when the principal responsible for paying against delivered results is financially robust (and, more critically, has financial resources that extend beyond the horizon of the RBF scheme) and when determination of whether the results have been delivered resides in a trustworthy institution. 3 | Both principals and agents need to have sufficient institutional capacity to set up and respond to an RBF incentive, respectively. For the principal, this relates to the ability of constructing and monitoring the instrument and ensuring quick disbursement of resources when agents deliver agreed results. For agents, technical capacity relates to the planning tools and managerial ability that allow them to map out the expected cash flows from an RBF instrument, and to react to the new incentive structure. Results-Based Financing in the Energy Sector | An Analytical Guide 3 There are likely to be cases where it is ambiguous whether these preconditions can be met. Ascertaining whether these necessary preconditions are met will require considerable judgement and experience. Measures to overcome such problems, for example, the provision of technical support to principals, may need to accompany an RBF scheme. Such measures can either be implemented from the outset of an RBF scheme or be available on a contingent basis if it becomes apparent during the implementation of the RBF scheme that the precondition will not otherwise be satisfied. Due to these preconditions, RBF schemes are more likely to be used in circumstances where results are easily measurable and can be delivered in a short space of time. While this does not speak against the use of RBF in general, it suggests that principals need to be careful not to neglect longer term and harder to measure targets. This can be achieved by testing whether the results that are targeted by the RBF instrument in question are consistent with the long-term and less easily measured results that the principal is also interested in. There are a series of factors that will determine whether or not the stronger incentives of RBF will offset the greater compensation required for bearing more risk. Three factors appear particularly critical as, when they are in place, they imply both that the additional risk imposed on the agent will be relatively low and also that the power of the RBF incentive will be high. Specifically, RBF will be favored when: • the risks borne by the agent are substantially within its control; • there is a clear ‘line of sight,’ including that the monitoring and verification arrangements are trustworthy, so that all parties are able to easily observe the relevant results, and can trust the reported results (and know this in advance); and • the length of time before the agent receives any benefit from the RBF is relatively short. In addition, in cases where the extent to which the investment needed to deliver the results entails only a small proportion of the agent’s (potential) resources and/or when the cost base of the agent varies according to the number of results delivered, the additional risk imposed on an agent from an RBF instrument will be lower. RBF may also be favored to conventional instruments in cases where it is hoped that the project will provide a demonstration impact outside of the scope of the project itself. Detailed Design Issues The presumption should be against restricting the list of potential agents for an RBF instrument. Prior exclusion of agents introduce the possibility of barring agents who could effectively deliver the results desired and may encourage gaming to meet the eligibility criteria. However, restrictions may be appropriate in order to support the objectives of the intervention when the results trigger alone cannot ensure that these objectives are met. A particularly salient example is where there is uncertainty over product quality and linking the RBF payment to product quality is unattractive or infeasible. There are three broad options as to what the trigger for disbursing an RBF payment might be: it can be linked to an investment in an asset or technology; or to the investment followed by subsequent Executive Summmary 4 operation/production/sales; or simply to operation/production/sales (with any investment supported separately, as necessary). The preference for each of these three options will need to take into account two factors. On the one hand, the principal will wish to place the trigger on a result that is linked as closely as possible to the final outcome/impact it desires. On the other hand, the principal will need to be wary about placing risks on an agent that they are not best placed to bear. We suggest how these three options and two factors may be combined to assess the optimal location of any trigger. For successfully reaching a trigger, there are a number of options available for the principal regarding the structure of any pay-out. This might be broken down into three issues. 1 | The proportion of overall support that should be linked to meeting the results. It is important to note that RBF and conventional support are just either ends of a spectrum where either all or none of the support is linked to the achievement of the given results. It is perfectly possible and, in many cases may be desirable, to adopt an intermediate or hybrid approach combining some upfront support with some support provided on a results basis. For instance, in cases where the agent faces capital constraints, the benefit of the stronger incentives from RBF can be achieved without making the entirety of the principal’s support contingent on the delivery of results. 2 | Whether the pay-out should be linked to market variables. A market-linked RBF intervention changes demand for a good or service—for example, by providing a subsidy so as to deliver a minimum price guarantee up to a certain quantity threshold. Where a market exists—and hence a market link is feasible—this becomes closely linked with the one above regarding whether the trigger should be linked to production/operation/sales (implying a market link) or just to investment. 3 | In cases where a link is made to market variables, how this link is made (e.g., a per unit subsidy, a price commitment, a quantity commitment). Previous analysis has shown that price commitments may be valuable when there is significant demand uncertainty and quantity commitments where there is significant cost uncertainty. Practical considerations will also be important. The principal also needs to determine the value of any RBF instrument. The basic options are between an administrative approach and an auction. In some circumstances, auctions may be a desirable tool. They can help overcome the problem that the principal will have less information than the agents on various factors that will determine the appropriate value, for example, production costs. Without this information, if a principal proceeds with an administrative approach, then there is a risk of creating excess returns that will limit the cost effectiveness of the RBF instrument. However, there are also a number of detractions from using an auction. First, they can only be used when the principal has rights that it can allocate through an auction (e.g., a principal cannot auction the rights of private consumers to purchase a particular good) and where the principal has the institutional capacity to organize the auction. Second, auctions transfer the decision as to who is entitled to an RBF payment from the market to the principal; this is only likely to be desirable when there would be ineffective competition in the market. Auctions also run the risk that the resulting value of the RBF instrument will end up being too low and that the agent is not able to profitably operate, or that the low value Results-Based Financing in the Energy Sector | An Analytical Guide 5 will discourage dynamic competition and innovation. A balanced assessment based on individual circumstances of each case will be required. The principal needs to decide whether its intervention should be structured so as to change the incentives for private buyers and sellers or to directly procure the output. Four factors will be relevant and will need to be assessed on a case-by-case basis. First, when the principal requires the good for its own consumption or otherwise gets benefits from ownership, then direct procurement should be preferred although this consideration will only apply in certain circumstances. Second, direct procurement may make quantity commitments easier to deliver which, as discussed above, may be particularly valuable when agents face cost uncertainty (but less so when there is price uncertainty). Third, there may be differences in the transaction costs associated with each of the two routes depending on, for instance, the efficiency of the principal’s procurement processes. Finally, although both approaches can lead to the same market expansion impact, the fiscal costs will likely differ. Procurement should be favored when the principal can discriminate by providing different payment terms for different units of output and/or when demand is insensitive to price. Changing incentives and subsidies will be preferable when the subsidy need only be applied to incremental output and/or when demand is more sensitive to price. The principal also needs to determine the duration of an RBF instrument and, closely related, the exit strategy for winding down the RBF scheme. This decision will partly turn on whether the principal is primarily interested in procuring goods and services for itself or for society, or whether the principal is primarily interested in creating a self-sustaining market. Where the principal wants to procure goods or services, there is a natural limit to the RBF instrument. A gradual exit strategy, with purchasing quantities phased down rather than abruptly ended, may maximize the lasting benefits of such an RBF scheme, allowing suppliers to readjust and to potentially survive as firms by transitioning their product range and customer base away from the principal. Where the principal is interested in creating a self-sustaining market, the duration of the RBF scheme will depend on how far away from self-sustainability the market is, and on how fast it can be expected to move towards this goal. An optimal exit strategy in this scenario phases support out as the market approaches viability. To ensure that firms do not rely on RBF support in perpetuity, a cap on the total amount of funding could be declared in advance. A good exit strategy includes contingency planning for scenarios in which market development does not proceed as expected. In particular, principals may wish to include a number of checkpoints into their plan, at which they check progress against preagreed targets. These checkpoints should be communicated to agents well in advance of them being undertaken (it may be productive to involve agents in the process of drawing up these checkpoints), allowing them to make appropriate preparations. For each checkpoint, the principal should chart what the target is (and how precisely it will be measured); what the next step will be in case the target is met; and what the range of options are in case the target is missed, including the broad principles by which the principal will choose one of these options. This type of comprehensive contingency planning not only helps in dealing with unexpected scenarios, but also builds trust between agents and principals (when adhered to), and, Executive Summmary 6 by increasing confidence between the parties, it may strengthen the overall effectiveness of an RBF scheme. Key Findings from the Literature Review To complement and support the operational guide, the report also provides a detailed review of the existing literature on RBF approaches. This is provided in the Literature Review at the end of this report. As well as identifying the challenges associated with the nomenclature around RBF approaches and the relative lack of analysis on what conditions support their application (as discussed above), the review focuses on the positive, cautionary, and negative findings in the literature on RBF approaches, from both a theoretical and empirical perspective. Theoretically, there are a wide range of findings on RBF: for instance, on the positive side, many authors have stressed the importance of aligning motivations, incentives, and objectives of principals and agents, but others have expressed concerns about the additional costs that RBF tends to imply. It is difficult to draw unequivocal conclusions from this theoretical literature. Empirically, the (somewhat limited) research to date suggests that RBF approaches can lead to improved outcomes but that appropriate design is essential to such success. This provides further justification for the main focus of this report. Results-Based Financing in the Energy Sector | An Analytical Guide 7 1 Introduction Results-based financing (RBF) approaches are becoming an increasingly popular way to support development objectives and wider public policy goals. The fundamental idea of RBF approaches is that payments that would otherwise be made automatically are made contingent on delivery of (a) preagreed result(s), with achievement of the result(s) being subject to independent verification. RBF approaches have been pioneered in the health sector5 but there has been increasing interest in whether and how they could be used within the energy sector, and especially on how they may promote private sector investment in low carbon energy sector opportunities (ESMAP 2012). The work has been commissioned by the Energy Sector Management Assistance Program (ESMAP), as part of a broader initiative looking at the potential for greater use of results-based approaches (RBAs) in the energy sector in developing countries.6 The report assesses the circumstances in which different forms of RBF approaches, including hybrid approaches which combine some element of RBF approaches with more conventional instruments, may be appropriate, with a particular focus on how they may be used to promote energy access and energy efficiency. A reasonable amount of previous analysis focuses on the advantages and disadvantages of RBF approaches; while there have also been (somewhat confusing) efforts at developing a taxonomy of different forms of RBF. This material is discussed in the literature review at the end of this report. However, as this review notes, there has been little attention given to identifying the circumstances in which various forms of RBF might be applied. This is important for policymakers: if RBF approaches are applied when they are not appropriate, then there is a risk that the disadvantages of RBF approaches will be over-emphasized; likewise, if RBF is not applied when it might be appropriate, then some of the possible advantages of the approach may not be properly appreciated. This report aims to address this gap by helping to identify, based on first principles, when and where different forms of RBF may be best deployed, as well as where RBF approaches may not be suitable. To our understanding, this is one of the first occasions in which the applicability of a wide range of different RBF instruments has been considered through an assessment of the underlying economic characteristics of such instruments7 and these insights then applied to the specific challenges of energy efficiency and energy access. The report is focused on individuals and organizations who are considering whether an RBF approach would be a sensible way to deliver some specific goods or services to help meet a defined public policy goal. These individuals or organizations might then go on to implement any RBF scheme— in the language of the report, they would be the scheme’s principal. Although there is a range of potential principals, those that are most likely to act as an RBF principal given ESMAP’s sectoral and geographic focus might be either national or subnational governments or in-country development agencies. Results-Based Financing in the Energy Sector | An Analytical Guide 9 The report is structured into three chapters. Each section corresponds to a ‘node’ on a decision tree. The three chapters/nodes are as follows: • Chapter 2 identifies the public policy problems and issues for which RBF is a candidate solution. • For those issues where RBF is at least a candidate solution, Chapter 3 considers when RBF may be a preferred approach to conventional8 forms of support. • In circumstances where RBF approaches might be preferred, Chapter 4 addresses the detailed design issues of an RBF scheme that policymakers will need to address, including who should be eligible; what are the conditions/triggers for making the RBF payment; how any RBF pay-out may be structured and its level determined; and the appropriate role for the RBF principal. • Finally, there is a review of the existing literature on RBF approaches. This structure is illustrated in Figure 1.1. A separate series of case studies accompany this report.9 Five case studies have been developed to illustrate the practical relevance of this framework. These case studies cover: increasing the use of fuel-efficient institutional cookstoves in Uganda and Kenya by schools; increasing the penetration of high-quality lighting products in challenging markets by the Lighting Africa program; an assessment of ways to use RBF to increase the penetration of solar water heaters in India; an analysis of the possible role for RBF in increasing innovation in low carbon energy access technologies and business Figure 1.1 The Analysis is Divided into Three Key Nodes 3 2 Design RBF 1 Choose RBF Who should be eligible? Consider RBF Choose Policy What is the condition or trigger objective conventional for paying out the RBF? Disregard approaches RBF What is the structure of the pay-out? What is the size of the payment(s)? What is the role of the principal? What is the RBF’s duration and exit strategy? Source | Vivid Economics. Chapter 1 10 models in India; and a review of where and when RBF approaches could be used to promote energy efficiency, drawing on examples from a range of non-OECD countries. This report, and the accompanying case studies, largely refrain from using the nomenclature typically used to describe different types of RBF. As discussed in the literature review, many of the terms that have been used to discuss RBF approaches (e.g., output-based aid, advance market incentives, etc.) are confusing. Sometimes, the same term is used to refer to different interventions and on other occasions different terms are used to refer to very similar interventions. As such, this report tries to avoid using these labels and instead describes the underlying structure of the proposed intervention.10 There are, however, two key pieces of nomenclature that are used throughout the report: • First, following the definition put forward by the UK’s Department for International Development (DFID 2012), the report distinguishes RBF, where payments are provided to service providers (who may be in the public or private sector) upon achievement of results, from RBA, which is taken to refer to government-to-government transfers of resources upon delivery of particular results, such as, for example, proposed under the Energy+ initiative (Energy+ 2011). This report, and the associated case studies, focus on RBF11 although, in passing, it is noted that many of the factors that determine whether or not RBF should be preferred to conventional approaches to supporting service providers is also germane to whether RBA might be preferred to conventional delivery of official development assistance. • Second, throughout, we use the term principal to refer to the body that designs and administers the RBF scheme and agent(s) to those who are expected to deliver the results. Depending on the context, the principal could either be a local or national government, development partner, or international financial institution. The agents will depend on the context but could, for example, be private firms, including banks, utility companies, or civil society organizations. The report’s focus on RBF, as opposed to RBA, has implications for the read-across from the report to the World Bank’s Program-for-Results (PforR) initiative. PforR is an innovative new lending instrument where disbursements to client country governments are linked to their achievement of particular milestones or disbursement-linked indicators (The World Bank 2011). PforR is best thought of as an RBA scheme as it relates to the transfer of resources between the Bank and governments, rather than to service providers. As such, it is not directly covered by this report. However, many of the underlying arguments of this report about when RBF approaches might be best used, especially those that relate to risk transfer, will also apply to when RBA approaches, such as PforR, might be considered. This report notes these overlaps. Results-Based Financing in the Energy Sector | An Analytical Guide 11 2 When Might RBF Be Considered? RBF may be used to administer subsidies and increase the efficiency of procurement In private markets, justification for market intervention ordinarily rests on the identification of a market failure. The broad presumption is that markets are an efficient means for allocating scarce resources so intervention should be limited to cases where there is clear evidence that markets are failing to deliver desirable outcomes, especially for consumers. A list of the possible market failures that most commonly justify intervention includes the following: • Externalities | when an individual’s actions have an impact on others (either positive or negative), which is not reflected in the price/cost of that good or service. Greenhouse gas emissions are an example of a negative externality, while the additional knowledge generated from using a new technology that is not appropriated by the user of that technology is an example of a positive externality. • Information Asymmetry | when one party has better information than another, or acquiring information is costly. For example, ignorance about energy efficiency opportunities often holds back activity that would be profitable to undertake while capital providers’ lack of information about the likely success of a project or program may make them reluctant to provide finance. • Market Power | when firms are in a dominant position in a market and face insufficient competitive constraint, they may restrict output/raise prices, offer poor quality, or use their position to restrict the ability of others to compete; for example, vertically integrated utilities may obstruct independent power producers access to electricity transmission networks. • Coordination Failures | where the consumption of one good requires the consumption of another or where benefits do not accrue to those incurring costs. For instance, energy efficiency opportunities in tenanted residential premises are often held back as the costs of the investment would be incurred by a landlord while the benefits, in terms of lower energy bills, would be realized by the tenant. • Public Goods | goods that it are not possible to exclude others from benefitting from once provided and for which one person’s consumption does not reduce the amount available for another, for example, street lighting. Two further categories of justification for intervention are also important. First, while markets may often be expected to deliver efficient outcomes, there may be concerns over their distributional implications. Interventions, therefore, may be justified in terms of promoting equity, such as promoting universal energy access. Second, the ‘textbook’ analysis above generally assumes a broader investment climate that supports (long-term) investment in terms of rule-of-law, clear property rights, and so on, as well as sufficient capacity on behalf of individuals to respond to market signals. When these are partially absent, then intervention may also be appropriate. Results-Based Financing in the Energy Sector | An Analytical Guide 13 There are a range of different policy instruments that are available to principals to correct these market failures. These include the setting of minimum performance/quality standards, mandating certain forms of activities, taxing activities, establishing patents, as well as providing subsidies. When intervening in private markets, the policy instrument that is most relevant to RBF is a subsidy. In cases where a principal decides to introduce a subsidy, this may be structured through an RBF mechanism. Subsidies allow the overall level of output in a market to expand, making them particularly relevant where there are positive externalities. Subsidies represent an injection of additional financial resources into a market, which can lead to additional output. This is most desirable in cases where there are positive externalities, that is, benefits from the production or consumption of a good or service that are not taken into account by those making the decision. In this situation, subsidies can bring the private benefit from production or consumption into line with broader social considerations. For instance, the economic rationale for subsidizing renewable energy technologies12 is that the greater output that these subsidies induce can increase understanding and learning about a technology, reducing future deployment costs, which is beneficial to society as a whole (Philibert 2011). This rationale can apply to technologies across the energy value chain where scale benefits can be expected to lead to cost reductions; for example, either to large-scale electricity generation technologies, such as grid-connected solar photovoltaics (PV) or wind power, where such subsidies often take the form of feed-in tariffs, or to technologies for off-grid energy consumption/access, such as biomass stoves or off-grid solar lighting appliances. Similarly, subsidizing the costs of acquiring information can often be justified by the wider social benefits delivered by the ensuing market expansion. In extreme cases, subsidies may lead to entirely new markets being created, that is, that without the subsidy it would be unprofitable for any supplier to produce any output. This was, for instance, the initial justification for the creation of an advance market commitment (a form of RBF) for vaccines for diseases prevalent in the developing world (Center for Global Development 2005). Often there is a decision to be made as to whether to subsidize ‘good’ activity or penalize ‘bad’ activity: the fact that subsidies promote output will often be crucial in choosing between the two. In many of the areas of ESMAP’s focus, there is, in principle, a choice either between subsidizing desirable activity or increasing the costs of undertaking a less desirable activity. For instance, increasing the penetration of low carbon energy access technologies can either be achieved by subsidizing these products or taxing high carbon fuels such as kerosene. Either approach can provide the same relative incentive between the two activities; however, taxing kerosene will lead to an overall decline in energy consumption (and likely decrease consumer utility or welfare) while subsidizing lanterns will lead to an overall increase in energy consumption13 (and likely enhance consumer utility or welfare). The latter is likely to be desirable for social/equity reasons. An alternative to subsidizing technologies would be to mandate their use. For instance, some countries mandate that a certain proportion of energy generation or supply should come from renewable energy sources, or that certain energy-consuming appliances (e.g., boilers, light bulbs) meet a specified Chapter 2 14 efficiency threshold. However, if this is introduced without providing any additional financial resources then, in the event that complying with the mandate raises production costs, these higher costs will be borne by a combination of producers and consumers.14 Once again, this may raise social/equity concerns. It also requires that the principal has a good understanding of the technologies such that they can calibrate the mandate accordingly. RBF may also be used when principals wish to procure goods and services. In addition to intervening in private markets, principals often need to purchase goods and services in their own right, for example, materials, labor, buildings. The contracts for the procurement of these goods and services can also be turned into an RBF instrument by introducing a degree of contingency into the payment terms. For example, a government may choose to switch from incandescent to compact fluorescent light bulbs in some or all of its buildings; it may choose to offer a contract where part of the contract payment is linked to preagreed results, such as the speed at which the switch takes place. There are two particular areas where principals may procure goods and services on an RBF basis that are particularly interesting. Although principals might use RBF approaches to purchase any good or service, there are two that will be particularly relevant. 1 | Where the public sector is purchasing goods that have some public good characteristics, such as infrastructure, policing and security, health services, which it has been decided that the private sector should not, or will not, supply in adequate amounts. For instance, one of the accompanying case studies explores the idea that the public sector might procure cookstoves for schools due to concerns that the interaction of private buyers (schools) and sellers (manufacturers) may lead to only a modest scale-up of activity. 2 | Where the principal is contracting with an external agency or parastatal for the delivery of certain goods and services and the principal wishes to improve the efficiency with which that body provides its services. For instance, in Bangladesh, a public-private body has been created to scale-up the use of solar home systems. Bodies such as this could receive part of their remuneration according to success in meeting objectives. This idea is explored in relation to scale-up of improved biomass and biogas cookstoves in East Africa. However, in cases where RBF is used to promote procurement as opposed to disburse subsidy, then it is less likely that the RBF instrument can be withdrawn at a later point with a self-sustaining market of buyers and sellers is established. Figure 2.1 summarizes the analysis on where it is appropriate to consider RBF when designing policy. Results-Based Financing in the Energy Sector | An Analytical Guide 15 Figure 2.1 RBF Might Be Appropriate in Cases Where the Principal Wishes to Provide a Subsidy to a Private Market or in Cases of Procurement Focus of this Study Public Sector Interventions in Procurement Private Markets Technology Mandates Subsidies Performance Public Goods Other Standards (e.g., roads) Goods/Services Purchased by Taxes and Services the Public Sector Emissions Procured from Trading Agencies/ Parastatals Patents RBF Approaches (also possible) Source | Vivid Economics. Chapter 2 16 3 The Choice Between RBF and Conventional Financing Approaches RBF Approaches and the Allocation of Risk A fundamental difference between RBF and conventional15 approaches is how the risks that a project might fail are allocated between principal and agent. Under a conventional approach, the financial risks of project failure are substantially borne by the principal: in the event that the project fails to deliver the anticipated results, then the principal’s resources will already have been committed and will be (substantially) unrecoverable. By contrast, under an RBF approach, the risks borne by the principal are reduced: if the project fails to deliver the expected results, then the financial resources of the principal are not expended. By corollary, the opposite is true for the agent: under an RBF approach, it faces substantially greater risk and if the project fails, it will receive no contribution from the principal; these risks are lessened for the agent under a conventional approach.16 The question of who can (or should) bear different project risks will be a crucial determinant of whether different forms of RBF can outperform conventional approaches and deliver more results for the same resources (or the same results for fewer resources). As suggested in Figure 3.1, placing greater risks on an agent will have both an advantage and a disadvantage. The advantage is that the agent will face a stronger incentive to provide the results that are desired by the principal. If the agent is in a position to respond to that incentive, and mitigate the risks that would otherwise threaten the delivery of the project, then this will increase the probability of delivery of the desired results. This increases the amount of results that a principal can achieve with a given amount of resources. The disadvantage is that placing additional risk on the agent will mean that the agent will require (the prospect of) higher reward as compensation, which directly reduces the amount of results that can be delivered with given resources. The less that they are able to control the risks that threaten project success, the greater the compensation they will require. In other words, the principal needs to balance the possibility that RBF approaches increase the probability of achieving a set of results against the likelihood that RBF approaches make delivery of these results more expensive. The less/ more able agents are to control the additional risks that they face under an RBF scheme, the less/ more attractive an RBF approach will be for a principal looking to maximize results achieved with a given budget. RBF will not transfer all risks away from the principal. RBF reduces or removes the financial risks that principals face as a result of project non-delivery. However, there are more fundamental risks around the ultimate delivery of outcomes that principals cannot transfer. For instance, if the desired results from a project are not obtained then, although no payment will be made, principals will still face reputational risks regarding the non-delivery of the results (Ghosh, Müller, Pizer, & Wagner 2012a). Results-Based Financing in the Energy Sector | An Analytical Guide 17 Figure 3.1 RBF Provides Stronger Incentives for Agents to Perform But Also Places Greater Risk on Agents Stronger incentives for Greater risk performance Source | Vivid Economics. They will often be accountable to external bodies, for example, parliaments and shareholders, for this non-performance. This further emphasizes the importance of careful consideration of whether to use an RBF instrument and how to tailor it to the specific circumstance. The same logic and arguments also apply when donors/development partners are considering whether to provide conventional aid (or lending support) or RBA (lending). The latter will transfer more risk to the recipient government than the former. This should increase the probability of program success from an aid or lending program but the additional risk placed on recipient governments will carry a price that the donor will likely have to bear. Implications for Choosing Between RBF and Conventional Approaches The recognition that RBF reallocates project risk between principal and agent yields important insights on when it may be an appropriate policy instrument. In particular, it suggests that RBF should be preferred in circumstances when the agent is able to manage the risks that they have been asked to bear better than the principal: this will increase the probability of project success without making it unduly costly for the principal to compensate the agent for the additional risk. This feature of a results- based lending scheme has been noted before (Kumar & Mumssen 2010) but this section aims to expand on this knowledge: first by identifying a series of preconditions that need to be in place for an RBF scheme; and second by providing a clear checklist of factors that will influence whether it is desirable to ask agents to bear the risks of RBF. Chapter 3 18 Preconditions There are at least three conditions that need to be in place before an RBF approach can be adopted: 1 | it must be possible to monitor/verify the results against which RBF payments might be made; 2 | the agent must have access to capital consistent with the capital costs of the project which will partly depend on the maturity of the financial market in the country and partly on the credibility of the RBF scheme and its principal; and 3 | both principal and agent must possess sufficient institutional capacity. First, it needs to be possible to monitor and verify the results against which payments might be made and these results need to be closely linked to the overall outcomes desired. An RBF instrument will be impracticable in the absence of a measure against which to disburse it. It will also be unlikely that agents will be willing to accept risks associated with delivery of results if there are significant difficulties in verifying such results, as this increases the probability of (costly) disputes. From the principal’s perspective, the existence of these problems would also make the administrative costs of the RBF scheme more costly.17 Many of the results that are typically sought from energy sector interventions tend to be relatively easy to monitor and measure (e.g., number of households reached, (renewable) electricity supplied) although, as experience in the carbon market shows, determining the baseline—what would have happened without the intervention—can be difficult. These problems may be particularly challenging in relation to energy efficiency results where a variety of factors over and above the existence of an energy efficiency investment, for example, business cycle fluctuations, can affect the difference between energy consumption at two points in time. A closely related point is that the results need to be linked to the overall outcomes desired and (close to) ‘uni-dimensional,’ that is, there should only be a small number of aspects of performance about which the principal cares. If this is not the case, then there is potential for perverse incentives with those aspects of performance subject to financial incentive to be prioritized over others. For example, reports that when AT&T employees were compensated in relation to the lines of code that they wrote in their program, they responded by writing longer programs than was necessary (Prendergast 1999). In the energy context, this may suggest that an RBF scheme may be easier to implement where the product is homogenous, such as renewable power generation (where one kWh is the same as the next kWh), and that more careful design is needed where aspects of product quality are just as important as product quantity. The associated case study on how RBF instruments could be used to scale up use of off-grid solar lanterns in Africa discusses these challenges in more detail and suggests that payment should only be made available to products that meet preagreed product specifications. Second, if payments are only made by a principal after the delivery of the results, the agent will need to have access to sufficient financing to cover any upfront costs prior to delivery of results. This largely relates to access to capital markets, although it is possible that agents could finance any upfront costs from their own resources. The importance of this barrier will obviously depend on the scale of the upfront capital costs that need to be incurred, so that smaller and less capital intensive projects Results-Based Financing in the Energy Sector | An Analytical Guide 19 may face fewer challenges as will RBF instruments that are aimed at larger organizations with greater financial resources. This can often be a very important barrier to the implementation of RBF approaches, especially those that require substantial fixed investment costs to be incurred before there is hope of recouping the RBF payment. As such, it may be a particular challenge in using RBF instruments to stimulate investment in generation and transmission technologies, including mini-grids, in some contexts. Likewise, the case studies examining the possibility of using an RBF instrument to stimulate research and development (R&D) in energy storage technologies in India, most notably batteries, concludes that the scale of the financial resources required to effect a substantial improvement in technology is likely to be too great to make an RBF instrument for most small-scale entrepreneurs plausible. However, there will also be many situations in the energy access and energy efficiency contexts where access to capital is less of a constraint. For example, the case studies that examine the use of RBF instruments to scale up activities under the Lighting Africa program and the penetration of solar hot water (SHW) systems in India both conclude that this need not be a constraining factor for an RBF scheme in these contexts, as the capital investment requirements are relatively small. Furthermore, while access to finance can be a barrier to using RBF approaches in isolation, there may still be scope to introduce RBF approaches in combination with other instruments that aim to increase the supply of capital. The case study on using an RBF instrument to stimulate the innovation needed to develop a business model that allows for off-grid products to be paid for on a variable use basis in India notes that the emerging Climate Innovation Centre (CIC) in that country would be an obvious complementary initiative that could provide access to finance in support of the RBF. Interestingly, as explored in more detail in both the cross-country energy efficiency case study and that looking at scale-up of cookstoves in schools in East Africa, these complementary initiatives may also take the form of an RBF incentive for financial institutions. A further key point about access to finance is that, in part, it will depend on the design of the RBF mechanism itself. For instance, the more credible the principal’s commitment to make an RBF payment the easier it will be to raise finance; in particular the principal will need to have financial credibility to deliver any payments throughout the duration of the RBF scheme. Additionally, the more credible is the process for determining whether results have been delivered—in other words, the monitoring, reporting, and verification (MRV) of the RBF scheme—the easier it will be for the agent to raise financing. Third, both principals and agents need to have sufficient institutional capacity to, respectively, set up and respond to an RBF incentive. For the principal, this relates to the ability of implementing and monitoring the instrument and ensuring quick disbursement of resources when agents deliver agreed results. It has sometimes been argued that RBF requires lower technical capacity on behalf of principals than conventional instruments as, for instance, it does not require appraisal of different grant applications. However, a more appropriate assessment may be that administering RBF requires different technical skills and that, especially in cases where monitoring and verification is not trivial, these may still be substantive.18 The case studies concerned with using RBF instruments to promote R&D and greater penetration of SHW systems in India highlight concerns about the institutional Chapter 3 20 capacity of some potential principals in these contexts. For agents, technical capacity relates to the planning tools and managerial ability that allow them to map out the expected cash flows from an RBF instrument, and to react to the new incentive structure. Box 3.1 discusses how these preconditions might, at a high level, have an impact on the countries where RBF approaches could be most suitable. Two points of caution are raised by this analysis of preconditions for RBF: 1 | There are likely to be cases where it is unclear whether these preconditions are definitely met or are definitely not met; often performance is best conceptualized as being on a spectrum (as undertaken in the analysis in Box 3.1). As such, there may be a challenge in discerning whether these preconditions are met prior to implementing an RBF scheme, and so this will require judgement and experience, in addition to careful evaluation of all available evidence. Where it is truly impossible to assess these preconditions beforehand, a pilot program may be useful for testing them without committing major resources. Alternatively, or in addition, provisions may be put in place to overcome failings against these preconditions, either from the outset of a program or on a contingent basis. Box 3.1 An Example of How to Apply Some of the Preconditions The second precondition, the availability of sufficient prefinance, and the third precondition, the existence of sufficient (principal) institutional capacity, can be measured on a high level. • The World Bank’s Doing Business survey measures the ease of getting credit in 185 countries, which can be used as a broad proxy for an agent’s ability to secure prefinance, that is, the second precondition. • Barr, Fankhauser, and Hamilton (2010), compile an index of institutional capacity for 72 countries, sorting them into 4 quartiles. The index is based on World Bank portfolio data, and Barr, Fankhauser, and Hamilton use the same formula to aggregate the different components as the World Bank uses for IDA allocation. This index can be taken as a proxy for the institutional capacity of the principal where country governments are the principal in the RBF programs or projects.* These two measures are mapped against each other in the table below. This shows that an RBF instrument where the national government is the principal is more likely to succeed in Ghana than it is in Burundi, all else being equal. Other conclusions may be that in countries in the bottom row, such as Gambia, Lesotho, and Yemen, RBF mechanisms may need to be accompanied with associated interventions to augment the availability of capital; and that in countries in the right-most column, such as Nigeria, Cambodia, and Chad, national governments may not be the most suitable principal to design and implement an RBF instrument, or that they may require capacity building before they are able to do so. *It can also be thought of as a proxy for the institutional capacity of the agent in the case of an RBA or results-based lending scheme, such as PfroR, and as such the table also has relevance in identifying which countries may be most appropriate partners for these schemes. Results-Based Financing in the Energy Sector | An Analytical Guide 21 B OX 3 . 1 ( c o n t i n u e d ) Countries are Highest Institutional Least Institutional grouped into quartiles Capacity Capacity along both axes Best Access – Armenia – Albania – Kyrgyz Republic – Nigeria to Credit – Georgia – Kenya – Zambia – Ghana – Rwanda – Honduras – Uganda – India – Sri Lanka – Azerbaijan – Bangladesh – Cambodia – Bosnia and – Nepal Herzegovina – Papua New Guinea – Dominica – Sierra Leone – Mongolia – Solomon Islands – Pakistan – Tonga – Vanuatu – Bhutan – Benin – Cameroon – Angola – Burkina Faso – Bolivia – Niger – Central African – Cape Verda – Ethiopia Republic – Grenada – Indonesia – Chad – Nicaragua – Malawi – Congo, Rep. – Samoa – Mali – Côte d’Ivoire – Senegal – Mozambique – Guinea-Bissau – St. Lucia – Togo – St. Vincent and the – Zimbabwe Grenadines – Tanzania – Maldives – Guyana – Djibouti – Burundi – Lesotho – Gambia – Comoros – Madagascar – Kiribati – Congo, Dem. Rep. – Mauritania – Eritrea – São Tomé and – Guinea Principe – Haiti Worst Access – Tajikistan – Lao PDR to Credit – Yemen – Uzbekistan Note | ‘Access to Credit’ ranking is based on country ranking in the ‘Getting Credit’ section of the World Bank’s Doing Business survey; ‘Institutional capacity’ ranking is based on Barr et al. (2010), Table 5. Source | Barr et al. 2010; Vivid Economics; World Bank 2012. 2 | Those preconditions may bias RBF towards short-term interventions that produce timely, easily measured results, and away from long-term interventions that produce results which may be harder to measure. In particular, the precondition of sufficient prefinance discourages a focus on results that take a long time to deliver, and encourages a focus on results that can be delivered quickly. The precondition of measurable and verifiable results discourages a focus on hard-to-measure qualitative improvements, and encourages a focus on easily measurable and verifiable (often quantifiable) results. Given this natural aptitude of RBF for certain types of projects, principals need to be aware of potential trade-offs between short-term and long-term Chapter 3 22 goals, and between easy and more difficult goals. While, for example, short-run energy efficiency investments are consistent with the overall long-term goal of better energy efficiency, it may be that a short-term RBF intervention in energy access (e.g., improving off-grid access to solar energy) clashes with a long-term energy access goal (e.g., expanding grid access). The same preconditions will also apply when considering an RBA scheme. In an RBA scheme donors or multilateral agencies are the principals and partner governments are the agents. In this case, it will still be necessary for the results being targeted by the RBA scheme to be simple and easily monitored and verified, for both the donor/multilateral institution and partner government to have sufficient institutional capacity to set up and respond to the scheme and for the partner government (as agent) to be able to access sufficient finance to implement a program prior to the provision of resources through the RBA scheme. Factors which Affect the Cost/Risk Trade-Off In cases where these preconditions are met, there are a series of factors which will determine (i) how great the additional risk placed on agents from an RBF instrument might be, as well as (ii) the scale of the potential gains from agents facing stronger incentives. Costs There are at least five key factors that will influence the additional risks faced by agents as a result of RBF. The significance of these will determine how much higher the cost of capital of the agent will be under an RBF compared to under a conventional approach. 1 | The extent to which the agent can control the results being incentivized. 2 | The extent to which there is a clear ‘line of sight’ to the results and the extent to which all parties have confidence in monitoring and verification arrangements. 3 | The extent to which the investment needed to deliver the results will entail a significant proportion of the agent’s (potential) resources. 4 | The extent to which the cost base of the agent varies according to the quantum of results delivered. 5 | The time horizon of the RBF scheme. Risk-driven cost increases are lower when results are largely under the control of the agent and not subject to many external factors. In cases where the factors that will influence whether or not the results are delivered by the agent are largely under the control of the agent; the premium an agent would need to bear the additional risk from the RBF scheme will be lower. For example, placing an RBF instrument on physical units of energy saved would be less costly than placing it on the monetary value of energy savings, as the former is largely under the control of project developers while the latter is driven substantially by broader macroeconomic and international factors. Indeed, in the Results-Based Financing in the Energy Sector | An Analytical Guide 23 accompanying case study on energy efficiency, it is suggested that placing an RBF payment on the delivery of the energy efficiency investment, rather than the energy savings themselves, may be more appropriate as this is more readily under the control of the agent. As a further example, an RBF instrument on a staple product, for which demand does not substantially increase or decrease with the business cycle, will place less risk on project developers than an RBF instrument for more luxury products where demand may be much more volatile depending on the economic climate. Risk-driven cost increases are lower when there is a clear line of sight for the principal and agent to the result against which results-based payments are made, and both parties can trust the monitoring and verification arrangements. This links closely to the precondition discussed above. If all parties are able to easily observe the relevant results (and know this in advance) and there is no contractual ambiguity, then the additional risk borne by an agent from RBF will be lower. In addition, as the disbursement of funds depends on the monitoring and verification arrangements, insufficient trust in monitoring and verification arrangements will increase the amount of risk that the agent perceives—absent good monitoring and verification procedures, an agent may deliver the desired result, but then fail to get paid for it. This drives up the amount of compensation that the agent requires to shoulder that risk. On the principal’s side, trust in monitoring and verification procedures is essential to avoid the risk of overpaying and/or the possibility of gaming by the agent. Given the importance of robust monitoring and verification, and recongizing that setting up these arrangements may be costly, this may lead to a preference for using RBF mechanisms for relatively large programs, perhaps at the sectoral level, which would allow some of the fixed costs of establishing such procedures to be spread over a larger number of agents/results. In most of the case studies accompanying this report, there is a clear line of sight to the desired results (Box 3.2). The Lighting Africa case study illustrates the challenges that may be present in some instances, however, as the portable nature of solar lanterns makes sales verification a challenge. Partly because of this, the authors suggest that an RBF instrument linked to the creation of a distribution network might be easier to observe. Risk-driven cost increases are lower when the additional investment required to deliver the results does not entail a significant proportion of the agent’s (potential) resources (the balance sheet of the agent). In cases where the activities associated with the RBF instrument represent a small proportion of the overall firm’s resources, then the success or failure of the project will have little or no impact on the viability of the agent and on the willingness of capital providers to provide capital to the agent. Correspondingly, there should be little or no change in the terms on which that firm can access financing. By contrast, if the RBF instrument is linked to a substantial proportion of the agent’s activities then this makes success in delivering the results critical to the agent’s financial viability. The RBF instrument is more likely to alter the extent to, or terms on, which the agent can access financing. In this context, it is noteworthy that one of the most high profile examples of RBF to date has been the Advance Market Commitment for pneumococcal vaccines that was placed on very well capitalized, multi-national pharmaceutical companies. As noted in a number of case studies, including the cross- national energy efficiency and East African institutional cookstoves and Lighting Africa examples, this factor suggests that an RBF instrument may well be suited to financial institutions for whom lending towards the activities supported by the RBF intervention is likely to represent only a relatively small Chapter 3 24 Box 3.2 Trustworthy Monitoring and Verification Procedures Creates a Trade-Off Between Capacity Building and Fast Implementation RBF requires credible monitoring and verification procedures. Where these are absent, agents are unsure whether they will get paid for the results that they deliver (which may greatly increase the difficulties they face in obtaining prefinancing); at the same time principals are unsure whether the results that they are paying for are really being produced. This need for trustworthy monitoring and verification procedures implies that RBF cannot be deployed where these are completely absent. This leads to a trade-off between importing skilled professionals to perform these functions, which would allow RBF to go ahead but would fail to build up local expertise; or building up in-country expertise in these areas, and delaying the introduction of RBF until sufficient expertise has been developed. This choice also affects ownership perceptions: importing monitoring and verification expertise is likely to directly decrease perceived ownership; training local monitoring and verification experts may increase ownership, but the associated delays may prove to be unpopular and may themselves decrease perceived ownership. The less acute this trade-off is the easier it is to measure the results against which payment is made but this also raises the issue (discussed above) that RBF interventions will be biased in favor of situations where results are easiest to measure. Source | Vivid Economics. proportion of the overall lending activity of the bank or other financial institution. By contrast, the energy efficiency case study proposes that, because the resources that a housing association (or equivalent) would require to finance an energy efficiency retrofit of a multi-family dwelling would be excessive, a hybrid RBF approach should be used. This hybrid would pay the first installment at the time a decision is made on whether to proceed with a retrofit or a contract is signed with a contractor. Risk-driven cost increases are lower when the cost base of the agent varies according to the amount of results delivered. In cases where the cost base of an agent varies as the amount of results increases or decreases, then the agent has a natural ‘hedge’ against poor performance; although its revenues are lower, so are its costs, muting the impact on its overall profitability. By contrast, an agent that has very substantial fixed costs (high operational gearing) will be much more affected by poor performance: its revenues will fall but its costs will be unaffected. This factor will be particularly important when not all of the factors influencing delivery of the results are within the control of the company. In an energy access context, this might suggest that an RBF instrument might be more appropriate for importers/dealers/ wholesalers of off-grid appliances whose costs are largely variable, depending on levels of throughput than small-scale power generators/mini-grid operators who would need to incur substantial fixed costs. This is reflected in the case studies where, for example, RBF schemes are recommended for the manufacture/retail of SHW systems or for the installation of off-grid energy products that customers pay for according to use, but rejected for the delivery of a radical change in the way of supplying school meals in Kenya and Uganda (which would require substantial fixed investment). Results-Based Financing in the Energy Sector | An Analytical Guide 25 Risk-driven cost increases are lower the shorter the time horizon for RBF payments. The further out into the future an RBF payment is promised, the more risk an agent faces before it receives the payment. Consequently, RBF might be preferable for assets with shorter operating lives, and a short-duration RBF instrument may be preferable over a longer duration RBF instrument. Again, it is partly for this reason that the energy efficiency case study suggests that an RBF payment might be best targeted at the delivery of a capital investment and/or the signing of a loan agreement and/or the signing of a contract with an energy service company (ESCO) which, in each case, would happen prior to the delivery of the actual energy savings. Benefits The benefits from an RBF instrument will be influenced by the ‘power’ of the incentive offered. A powerful RBF instrument will have a greater impact on the probability of project success, and therefore be more desirable, than a weaker RBF instrument. At least three of the factors mentioned above in the discussion of costs and risks are also important in this regard. Specifically, the power of the incentive from an RBF instrument will be greater when: • agents can control the risks that they face as a result of the RBF intervention, as it will then encourage the agent to take actions that they might otherwise not have taken; • when there is a clear line of sight to the results that are being incentivized so that the agent can be more certain that the results they achieve are observed by the principal; and • the length of time until the RBF payment is received is relatively short so that the value to the agent from any prospective results payment is higher. This makes these factors particularly important as they have a simultaneous (positive) impact on the likelihood that an RBF instrument will increase the probability of project success, as well as diminishing the cost/risk impact borne by an agent from the RBF intervention. In addition, the importance of increasing the probability of project success will be an important determinant of whether to introduce RBF. In some situations, there may be external ‘spillover’ benefits from the success of an initial project. For instance, a pilot project demonstrating the benefits of a particular technology may help overcome suspicion and inertia among those not involved in the pilot. In these situations, the benefit that RBF brings in increasing probability of project success is compounded as it also increases the probability that these further benefits can be obtained. This is part of the rationale/attraction, for example, behind the proposal in the case study to encourage prepay energy access business models (which could have spillover benefits with the same business model being used to supply other goods/services to those who are unable to access such goods and services at present). Risks and Incentives | Bringing it Altogether The discussion above highlights a process for considering whether to adopt an RBF approach, drawing on the observation that the choice between an RBF and a conventional approach is largely concerned with the allocation of project risks. It first identifies three necessary preconditions for RBF to be Chapter 3 26 considered: the ability to monitor/verify results; the agent’s access to capital and the institutional capacity of the principal and agent. In cases where these preconditions are met, there are at least five key factors that will determine the extent of the additional risk—for which a premium will be demanded—that agents face as a result of RBF. Three of these five factors also influence the power of the incentive provided by RBF, that is, the change in the probability of project success brought about by RBF. Finally, where initial project success may have longer term positive repercussions, the value of the increase in probability of initial project success brought about by RBF may be particularly high (Box 3.3). These factors will broadly apply whether the agent is in the public sector or private sector. The fact that RBF involves the transfer of risk and that those who bear risks face additional costs applies regardless of whether the agent to whom risk might be transferred is in the public or private sector. Therefore, the Box 3.3 Considering Allocation of Risk | A Pure RBF and Pure Upfront Support— Two Ends of a Spectrum In practice, hybrid approaches that share risk between principal and agent may well be desirable in some circumstances. This is particularly likely to be the case in two scenarios: first, where access to prefinance is an issue but RBF is otherwise both feasible and desirable; second, where absolute risks are too much for agents to bear, but relative risks, in other words, performance relative to peers, can be borne by the agents. Drawing on the literature on remuneration contracts for employees, two particularly attractive hybrid structures that deal with the two cases mentioned above are: • To provide some upfront payment but to withhold some upon satisfactory delivery of results. This is analogous to an employee who receives both a base salary and a performance-related bonus. Some of the accompanying case studies make this proposal due to concerns about the large fixed costs associated with the investment under consideration. For instance, Kumar and Mumssen (2010) note the case of Nicaragua’s off-grid rural electrification project (PERZA) which disburses 70 to 80 percent of the subsidies against construction milestones and the remaining 20 to 30 percent against final results. • To (partially) link any performance related element to the relative performance of different agents, rather than purely on their absolute performance, for example, to provide a bonus to dealers who install relatively more units of a product than other dealers. This will retain a performance-related element while removing from any assessment factors that are outside of the control of all agents. However, it will only be appropriate in a situation where the background conditions for all agents are similar. Again, many employee contracts provide performance-related payments assessed on a relative basis. On the other hand, hybrid approaches will have less power than ‘pure’ RBF approaches so are unlikely to be appropriate where the context suggests that the incentive strength of a pure RBF will deliver significantly more results (or those additional results are particularly valuable) at limited extra cost compared with a conventional approach. At the same time, if there is a complete absence of capital available to an agent or the risks transferred to the agent by a hybrid RBF approach cannot be well controlled by that agent then a conventional upfront approach will likely remain preferable. Source | Vivid Economics. Results-Based Financing in the Energy Sector | An Analytical Guide 27 same factors should broadly apply regardless of the ownership of the agent. That said, the scoring against these factors may differ significantly, for example, a small-scale entrepreneur in the private sector may have much less capacity to absorb the risk of an RBF approach and still access capital markets on reasonable terms than a national government agency or a state-owned enterprise. One particularly important consideration in this regard is that the ‘power’ of any RBF incentive may be lower for a public sector body, if it has a less strong incentive to maximize revenues/profits, than a private sector entity. In situations where the desired result is very clearly and unambiguously under the control of the public sector entity this may point to placing a very strong RBF mechanism on the agent to overcome this problem. This might be an RBF instrument without any upfront support and that could cause financial difficulty for the agent if it is not met. In other cases where it is less obvious whether the result is under the control of the public sector agent, then this consideration may make RBF instruments less compelling. Often, an assessment of these factors will need to be undertaken qualitatively, but quantitative modelling may be possible on occasion. The framework identified above will often be populated using a qualitative assessment. This is the approach that is taken in the case studies presented alongside this report. However, where data is easily available, it may be desirable to supplement this with a quantitative modelling analysis. This could proceed through identifying the cost of capital/risk increase that could be brought about by an RBF intervention while still making it more attractive than a conventional approach. An example of this modelling framework is provided in the box below. The same factors can also be used to identify when an RBA scheme, or a results-based lending scheme, may be more appropriate than conventional donor to government transactions. RBA schemes transfer greater risk to the recipient government than conventional lending or support and so are more likely to be appropriate when the costs of that risk transfer to the recipient government are low and/ or the benefits from the stronger incentives placed on the recipient government are high. This makes the same factors as listed above salient for this context; in particular that RBA schemes may be more appropriate when the results are clearly in the control of the recipient government, there is a clear line of sight to the results so that verification of achievement of results is easy and uncontroversial, and when there is a reasonably small lag for the recipient government between undertaking any activity and receiving any conditional aid/lending (Box 3.4). Non-Risk Based Factors There are at least two other factors that may determine whether or not a principal wishes to pursue RBF. The first relates to the funding preferences of the principal. As identified above, the choice between RBF and conventional approaches is strongly driven by the impact that RBF has on the agent’s cost of capital. However, the principal’s cost of capital (discount rate) also matters: RBF approaches, by their definition, postpone payment until verified delivery of the results. For some principals, such as national governments, this may be attractive as these principals may prefer to defer liabilities into the future. By contrast, other principals, such as bilateral aid agencies, may prefer to disburse any funds more quickly, as uncertainty over their budgets means that they are not in a position to make multi-year financial commitments. For these principals, RBF may be less attractive. So, in general, the more willing a principal is to defer payments into the future, the more attractive RBF is as an option/solution. Chapter 3 28 Strictly speaking, it is possible to separate the timing of payments from whether or not they introduce contingency, that is, they are based on results. Although RBF approaches imply postponing some payments into the future, it would also be possible for a principal to make a multi-year funding commitment without linking this funding to delivery of particular results. In other words, although desire to make disbursements in the near future will make it more difficult to adopt RBF, a willingness or desire to defer payments further into the future need not necessarily imply preference for RBF. Box 3.4 Using Quantitative Modelling Analysis: RBF Approaches vs. Conventional Approaches Principals are interested in achieving maximum results with a constrained budget. With this in mind, the choice between RBF and conventional approaches rests to a significant extent on whether the increased probability of project success from an RBF intervention offsets the additional risks borne by agents, which principals will need to compensate them for. In practice, it will be very difficult to assess what is the increase in the risk/cost of capital faced by an agent. Therefore, the most attractive approach to modelling is to address the issue from the other perspective and ask what the increase in the cost of capital from an RBF intervention would have to be before it no longer became attractive to use this approach. This could be computed from data about a potential project. Information is needed about the costs and revenues; the probability distribution of the variables; the asset lifetime; the years of RBF assistance and the discount rate of the principal. A hypothetical example is given below. The agent is assumed to receive sufficient subsidy to make them willing to undertake a project ex ante: this is either provided upfront as a grant or as an RBF instrument over time. The discounted value of the RBF payment to the principal is higher than the upfront grant as the principal needs to compensate the agent for the additional risk that it will face (reflected in a higher cost of capital, or discount rate, for the agent). Over time, actual costs and revenues diverge (randomly) from those expected at the time that the investment is made. In cases where the operating costs of the asset exceed the revenues/consumer benefit, the agent decides to stop using the asset for that year. This is less likely to happen when any utility/revenue is supplemented by an RBF payment. The principal aims to maximize the number of years of operation per dollar of subsidy. The model shows the line at which RBF and conventional upfront grants are precisely equally effective (achieve the same results per dollar of subsidy). At points below the lines RBF is more effective, while at points above the lines conventional upfront grants are more effective. This specific example shows that the likelihood that RBF is preferred to a conventional grant is greater, the shorter the lifetime of the RBF payment. The asset with the shorter lifetime (10 rather than 20 years) is shown with the red line. As the red line is considerably above the blue line, RBF for this asset can increase the agent’s cost of capital by more while still remaining preferable to a conventional grant. It also shows that assets that are less profitable (generate less utility) are more likely to be appropriate for RBF as the RBF intervention will have a greater impact on the probability that the asset continues to be used. Results-Based Financing in the Energy Sector | An Analytical Guide 29 B OX 3 . 4 ( c o n t i n u e d ) Shorter Asset Lifetimes Make RBF Instruments More Attractive (that is, they allow an RBF instrument to lead to a large increase in the agent’s cost of capital, and still be more effective than upfront finance) 50.00% Asset lifetime–10 years RBF duration–5 years In Cost of Capital from RBF) 40.00% Risk premium (Increase 30.00% Asset lifetime–20 years RBF duration–10 years 20.00% Constellations at which RBF is preferable (for 20 year asset) 10.00% 0.00% –10% –5% 0% 5% 10% 15% Annual profit (loss) of asset Note | The modelled asset has the following characteristics: capital cost—$500, operating costs—$50 per year; basic cost of capital—12%; principal discount rate—10%; agent discount rate: 10% + risk premium; no learning effects; the blue curve shows an asset with 20 years’ operating life and a 10 year RBF instrument, the red line shows an asset with 10 years’ operating life and a 5 year RBF instrument. RBF payments/upfront finance are adjusted for each level of operating costs to render the asset break-even. Source | Vivid Economics. A second factor which affects the attractiveness of RBF to a principal is whether they gain value from a variety of different processes being tried or from not having to choose or prescribe a certain number of approaches. By placing risk for delivery onto agents, principals do not need to become involved in choosing between different processes for delivering a particular result. As well as this risk transfer increasing the probability of project success (as those who better understand the market and are in a better position to choose which approach is most likely to lead to success), principals may also prefer this approach as it may facilitate a wider variety of different approaches being attempted than would be possible if funding decisions were made upfront.19 It may also be attractive to principals as it avoids them having to go through a process of picking between alternative options. Chapter 3 30 4 Designing RBF Mechanisms Six Questions to Inform the Detailed Design of RBF Instruments This chapter considers the appropriate design of RBF instruments in different contexts. In cases where RBF is deemed to be an appropriate policy intervention, considerable care and attention is required to design an RBF instrument that is appropriate to a particular context. This section aims to assist in this design work. The work is structured around six key questions. These are: • • • Who should be eligible to receive an RBF payment? What should be the trigger or condition for payment of an RBF instrument? What should the structure of any RBF payment be? • How should the size of the RBF payment be determined? • What should the role of the principal be in any RBF mechanism? • What should the duration of the RBF intervention be, and what is the appropriate exit strategy? A significant element of this analysis draws on the same thinking as used in Chapter 3. The choice adopted in relation to many of these issues will, ultimately, affect the way in which risk is shared between principal and agent. As such, much of the approach discussed in the previous section about risks and incentives is also germane to this discussion, and, indeed, on occasion, there is a degree of overlap in the discussion. Who should be Eligible to Receive an RBF Payment? The presumption should be against restricting the list of potential agents for an RBF instrument. One of the key features of RBF is that excluding agents that appear underqualified may not be necessary with a well-structured RBF instrument: if the agent turns out to be unable to deliver the result, then no funds will be disbursed. At the same time, excluding agents could do considerable harm. In the case of an RBF instrument aimed at suppliers, limiting the set of potential suppliers may lead to the exclusion of some low cost/high quality suppliers; any results delivered thus may be more expensive or of a lower quality than necessary. It also may create the possibility of competitive distortion and complex eligibility requirements may encourage gaming by agents to suggest that they should be entitled to receive the RBF payment. However, the objectives of the scheme may make it desirable to restrict eligibility. On occasion, it may be desirable to establish restrictions to support the overall objectives of the intervention when these objectives cannot be achieved by the results payment alone. For instance, it may be easier to support the penetration of clean energy access options through restricting the technologies that can Results-Based Financing in the Energy Sector | An Analytical Guide 31 receive the results-based payment to renewable technologies rather than by linking the results based payment to CO2 reductions themselves (because, for instance, there would be too much risk placed on agents by linking payment to CO2 reductions, as discussed further below). Alternatively, the scheme’s objectives may make it appropriate to restrict eligibility to agents in certain geographic areas or according to their wealth or income. A particularly important example of this exception is that restrictions may be necessary in cases where there is uncertainty about product quality and linking the RBF payment to product quality is unattractive or infeasible. The case study accompanying this report on using RBF to promote Lighting Africa’s objectives with respect to solar lanterns notes that market growth has been hindered by poor quality products that have led to customer disillusionment. Although the ideal approach to this problem may be to only provide an RBF payment in the event that it can be demonstrably proven that the solar lantern continues to provide a useful service to consumers after a number of years, this may be difficult as the product will be distributed across a geographically wide (and often rural/isolated) area. In this case, restricting the eligibility of the RBF payment to those agents that, at the point of sale, can demonstrate compliance with certain preagreed quality specifications, such as those already agreed under the Lighting Africa program, may be easier to implement. What should be the Conditionality of an RBF Instrument? This subsection addresses the question of ‘who needs to do what?’ in order for an RBF payment to be made. Two key factors determine the appropriate conditionality of an RBF payment: 1 | The ‘trigger’ should correspond as closely as possible to the desired outcome or impact;20 this minimizes perverse incentives—for example, the possibility that an agent might deliver an investment21 required for a desired outcome/impact in a low cost and low quality way as they know that this low quality would only become apparent (long) after receipt of payment—and also maximizes the strength of the desired incentive. 2 | The resulting risk allocation should maximize the trade-off between costs/risk and incentive discussed in Chapter 3 (between RBF and conventional finance); in other words, agents should bear those risks that incentivize them well and that they are able to control, while principals bear the remaining risks. These two criteria can be used to assess which broad type of ‘trigger’ may be most appropriate. • Targeting the availability of / investment in an asset (a pay-out upon successful provision of capacity). In the language of a results chain, the will often be an ‘output’. This will be the preferred approach if, either, (a) availability or capacity itself is the desired outcome (for example, in some energy contexts additional energy generation capacity may be valuable as it will increase the peak load capacity of the system, and hence allow the system to deal with demand peaks, but the actual operation of that capacity will be rare) or (b) if the agent is unable to bear the risks associated with operating the asset. This, for instance, is the approach that was taken Chapter 4 32 by the European Bank for Reconstruction and Development (EBRD) in relation to the capital investment it supported through its Bulgarian Energy Efficiency and Renewable Energy Credit Line (BEERECL).22 • Targeting availability and operation of the asset. This is preferred if operation of the asset is required to reach the desired outcome but it is only viable if the agent is able to bear both operational and investment risks. It is likely that the trigger under this approach will be closer to the desired outcomes/impact from the intervention. Offset credit markets, such as the Clean Development Mechanism, may be thought of as an example of this approach: agents only receive Certified Emission Reductions (CERs) if they both invest in the project and that project is subsequently operated and delivers emission reductions. • Targeting only the operation of the asset. This is preferred if operation of a pre-existing asset is required to reach the desired outcome or if the agent is unable to bear the investment risk associated with making production available. To achieve this conditionality, pay-out must be tied to asset operation, but with investment costs and risks, if necessary, dealt with outside of the RBF intervention. Performance-based management contracts for infrastructure operation are an example where only the operation of the asset is rewarded; efficient use of the asset is rewarded with a payment but the franchisee does not invest in the assets. This analysis can create a decision tree to determine the appropriate conditionality. This decision tree consists of three elements. First, it is necessary to determine which result is most closely aligned with the overall outcome that is desired. The second stage determines whether this result is better defined as the provision of capacity or the provision of a good or service. Finally, the third stage factors in the risks that it is appropriate to ask agents to bear. This is illustrated in Figure 4.1. This framework can be illustrated by reference to a specific example in relation to energy access. In this case, the desired outcome might be increased, sustainable, and reliable energy use among the rural poor. In terms of Stage 1, the result which most closely aligns with this might be the amount of renewable power consumption measured in terms of kilowatt hours of consumption. In terms of Stage 2, renewable power consumption requires the delivery of a good/service rather than just the provision of capacity. Stage 3 then considers what risks an agent might be best able to bear in relation to the provision of this good. It is plausible that in many markets agents may be unwilling to bear operational risk, for example, risks on the amount of renewable power that a household chooses to consume on the basis that they are not able to control the macroeconomic factors that may be a strong determinant of these consumption choices.23 Given this, the RBF intervention might be best applied on an availability trigger, such as the number of live connections created.24 A similar chain of logic also explains why the energy efficiency case study proposes that the RBF payment may be better linked to the delivery of a capital investment (or the signing of a loan agreement or contract with an ESCO) rather than the delivery of energy savings or the value of those savings. Results-Based Financing in the Energy Sector | An Analytical Guide 33 Figure 4.1 Questions to Help Identify the Appropriate Conditionality for an RBF Intervention First, which particular result Second, is this result Third, which Conclusion: corresponds most closely to the provision of risks can the which trigger or the desired outcome? capacity, or the agent bear? conditionality is provision of a good or most suitable? service? Investment risk Availability Result 1 Provision of capacity Cannot bear RBF is not Result 2 investment risk suitable Desired Outcome Result 3 Investment and Operation/ operational risk delivery Result 4 Investment risk Availability only Provision of good or service Operation, and Operational risk only agent is to be shielded against investment risk Cannot bear investment nor RBF is not operational risk suitable Source | Vivid Economics. What should be the Structure of an RBF Payment? There are three key questions that will help to assess what the appropriate structure of an RBF payment should be: • What proportion of overall support/payment should be results based? • Should the results-based element be linked to market variables or independent of them? • In the event that there is a link made to market variables, should the link be made to market prices, quantities, or in some other way? Each of these three elements is discussed further below. What proportion of payment/support should be linked to results? The choice between pure conventional support/payment and RBF represents either end of a spectrum. As discussed in Chapter 3, although it is conceptually helpful to distinguish between these two Chapter 4 34 alternatives, from a practitioner perspective, the presentation of the two as binary, mutually exclusive options may be unhelpful. In many circumstances, the best solution may be to provide some upfront support (so as to help overcome financing challenges the agent might otherwise face) and some payment linked to results. This approach may help to provide an allocation of risk between principal and agent that is mutually acceptable. However, it would mean that some of the non-risk related advantages of adopting an RBF measure, that is, to eliminate the need to choose beforehand between recipients, would likely be lost. The same factors that help to determine whether or not an RBF approach should be used at all, can also help to determine what proportion of payment/support might be results based. There would appear to be two ‘bookends’ that can be used to determine the proportion of overall support that might be subject to an RBF intervention: • At the one end, if upfront support is being provided due to concerns that the agent will not be able to access financing under a pure RBF intervention, then the amount of upfront support needs to be sufficient to overcome this barrier. This can be informed by the size of any capital investment needed for a project, taking into account the capital availability (internal and external) that the agent may have. • At the other end, any RBF payment needs to be sufficiently strong to continue to provide an additional incentive impact. This can be informed by the (ongoing) costs that the agent will face in delivering the results, net of any other revenues or benefits the agent would get from delivering the results. Between these two end points, the same factors that influence the appropriate risk allocation between principal and agent when determining whether to use RBF at all—controllability, costs, time horizon— can also be factored into the assessment of what proportion of payment/support is made results based. As an example of this logic, the energy efficiency case study proposes that part of any payment to support housing associations undertaking an energy efficiency retrofit should be paid immediately upon signing a contract to proceed with that retrofit. This is in order to ensure that the housing association does not face insurmountable problems in accessing the finance needed for the investment. At the other end of the spectrum, the case study on expanding SWH penetration in India, investments which require a relatively smaller capital outlay, suggests that all of the payment can be linked to sales, so long as they are made in target areas or to target households and meet technical standards. Should the results-based element be linked to market variables? An RBF payment can either be independent of market variables, or linked to them. An example of an RBF payment that is not linked to market variables would, for instance, be the payment of a subsidy on successful completion of an energy efficiency investment project. By contrast, an RBF payment linked to market variables would be set by reference to the price or quantity of a good or service within a particular market, for example, a guarantee to purchase a certain number of units of output or to use a subsidy to increases the price received by a supplier of a good or service. Two factors will influence whether a link to market variables should be adopted. These relate, in turn, to its feasibility and its desirability. Results-Based Financing in the Energy Sector | An Analytical Guide 35 First, there will be some instances where there is no well-functioning market to which results might be linked. For instance, when RBF is used to encourage a firm to undertake a series of energy efficiency improvements—which may include behavioral change—there may be no obvious market outcomes to which an RBF intervention may be linked. Likewise, there are unlikely to be markets for many prototype/innovation products. Second, linking an RBF intervention to market variables implies that the agent is being asked to bear market-related risks; this may or may not be desirable according to the factors identified above. This discussion is closely linked with the earlier discussion about circumstances in which asking agents to bear market related risks requires them to bear risks that they are not in a position to control and for which they would require a considerable risk premium. For instance, in some energy access contexts, agents may be unwilling to take the risk regarding how much power consumers choose to purchase if this is heavily influenced by external macroeconomic factors. On the other hand, in some situations, it will be crucial for agents to bear market-related risks if they are to face strong enough incentives to deliver results close to the final outcome desired. For instance, in many energy access contexts, consumer acceptability remains a key barrier to adoption of new technologies. In these cases, establishing an RBF intervention that did not require a product to demonstrate that it had overcome this barrier would not be desirable. How should RBF instruments be linked to markets? An RBF intervention linked to market variables is often best thought of as changing the demand for a good or service. An RBF instrument linked to market variables alters the revenues that an agent delivering (supplying) a particular result can expect to achieve, either by changing the price that the agent receives for each unit of output (result) or by changing the amount of outputs (results) that might be purchased. In economic terms, it is normally easiest to consider a market-linked RBF intervention as changing the demand for a good.25 Some of the most common ways in which a market-linked RBF intervention might alter demand include: • by providing a commitment to purchase a certain quantity of output, for example, a principal might commit to undertake energy efficiency retrofits on a certain number of buildings; • by providing a fixed subsidy for all units purchased (often the most common form of RBF intervention); • by providing a subsidy so as to deliver a minimum price guarantee; and/or • by providing a subsidy so as to deliver a minimum price guarantee up to a certain quantity threshold, for example, the Advance Market Commitment for pneumococcal vaccines. In principle, all of these interventions could have the same market impact if the principal had sufficient knowledge about market conditions. Figure 4.2 diagrams each of the four interventions identified above. In each case, the impact of the intervention on the size of the market (quantity of output sold at the market clearing price) is the same. In other words, with sufficient information, a principal could calibrate any intervention to result in the same market impact. Chapter 4 36 Figure 4.2 An RBF Intervention Can Achieve the Same Market Impact up to a Certain Quantity Threshold Minimum Quantity Guarantee A Per-Unit Subsidy Dquantity guarantee 1 Price 2 Price S Dsubsidy S Dno intervention Dno intervention P2 P2 Market with Market with RBF RBF P1 P1 Market without RBF Market without RBF Psubsidy Psubsidy Q1 Q2 Quantity Q1 Q2 Quantity A Minimum Price Guarantee A Minimum Price Guarantee 3 Price 4 Price S S Dno intervention Dprice commitment to quantity threshold P2 P2 Market with Dquantity guarantee Market with Dno intervention RBF RBF P1 P1 Market without RBF Market without RBF Psubsidy Q1 Q2 Quantity Q1 Q2 Quantity Source | Vivid Economics. In practice, the principal will not have sufficient information about demand and supply conditions in a market: previous analysis (Vivid Economics 2010) has demonstrated that using RBF instruments that provide greater certainty over prices will be particularly important when there is demand uncertainty; while providing greater certainty over quantities will be more valuable when there is cost uncertainty. Intuitively, in cases where there is substantial uncertainty over costs, a price commitment may be unhelpful if it results in a principal committing to a price that ends up being lower than the costs of production. By contrast, knowing that there will continue to be demand for a certain amount of product regardless of the changes in costs will help to reduce risk. By contrast, where costs are relatively well understood and do not fluctuate very much, but demand is uncertain or fluctuates significantly, a price commitment will be beneficial in increasing certainty over profit levels. This logic may help to explain why fixed price feed-in tariffs are often found to be effective policies for supporting grid-connected renewables in developed countries (International Energy Agency 2008), where, otherwise, fluctuating costs of fossil fuel generation could create significant demand uncertainty. By contrast, quantity-based Results-Based Financing in the Energy Sector | An Analytical Guide 37 procurement schemes may be particularly effective at encouraging the development of newer energy access and end-user energy efficient technologies where cost uncertainty may predominate. Practical considerations may also be important and these may lead to a preference for per unit subsidies. In many situations, more sophisticated market-based RBF instruments may be difficult to implement and/or exposed to the risk of fraud. For instance, unless the principal adopts a procurement role, making commitments to a fixed quantity may be difficult to achieve. Similarly, when there are a large number of diffuse transactions between buyers and sellers, the administration costs associated with providing a subsidy that guarantees a particular price may be prohibitive and potentially more easily gamed by agents (who could claim sales at a lower price in order to receive more RBF payment). In these circumstances, a per-unit subsidy may be easier to implement. Practical considerations should also dictate whether any subsidies are administered through the demand side or the supply side of the market. For a number of RBF instruments, there is a choice between administering the subsidy on the demand side of the market or on the supply side of the market. For instance, a per-unit subsidy could either be achieved by payments to a supplier for each verified unit of sale or on the demand side by providing vouchers to consumers that could only be used to supplement the purchase of a particular good (and which had the same value as the supply- side subsidy). In well-functioning markets, these two approaches will have an equivalent impact on participants in the market and should lead to the same increase in overall market size. The choice between the two therefore should be primarily determined by other factors. The relative administrative cost is likely to be the most important factor. How should the Size of an RBF Instrument be Determined? The calibration of an RBF instrument runs the risk of creating windfall profits. If an administrator has imperfect information about market conditions, then an RBF intervention could lead to the creation of windfall profits, that is, prices that exceed economic costs (including a normal rate of return). Agents will have an incentive to game the mechanism by taking advantage of their greater knowledge about the costs of providing the result and inflate these cost estimates to the principal. This risk will be compounded if different suppliers have different cost structures such that the same RBF intervention will have very different implications for profitability. Windfall profits will imply that principal resources are not delivering as many results as might be possible and could also threaten the legitimacy of the use of the RBF instrument. In some situations, the risk of windfall profits might be overcome by the use of auctions to calibrate the RBF instrument. In cases where the principal is concerned about windfall profits, they may use a reverse auction to allocate the RBF instrument. The mechanics of such an auction are clearest in the case where the RBF is a subsidy in a private market: different agents would offer bids as to the magnitude of the subsidy that they would require to deliver a certain (quantum of) result(s) and the principal would choose the agent(s) that required the lowest subsidy, accounting for the quality of the bids.26 In some situations, rather than ask agents to bid for a particular subsidy amount, the principal could set the size of the subsidy up front and use the auction to see which agent would be willing to Chapter 4 38 provide the greatest amount of additional output for that subsidy amount. Auctions, or beauty contests, can also be used when the principal is using an RBF to improve the efficiency of its procurement. A voluminous literature exists on auction design that will be relevant to this topic (Klemperer 2002). Although attractive in principle, there are a number of factors or constraints that need to be considered before using an auction. • Auctions are a way of introducing competition ‘for’ a market. As such, they are only practical in market contexts where the principal initially possesses ‘rights’ (such as entitlement to make an RBF payment) that it can auction; and they are only desirable when the principal is confident that it will be a better judge of who should own those rights than might result from competition ‘in’ the market. For instance, auctions may be used to allocate subsidies associated with grid-connected renewable energy capacity as the principal can control whether capacity is grid connected and the market power associated with incumbency may mean that simply relying on competition ‘in’ the market may not reveal who the most efficient generators are (for instead, the ‘winners’ may simply be those who connected first). It is also attractive in this context as the small number of potential agents makes it practical to provide different amounts of support to different generators, according to their costs. By contrast, in some situations the principal may not possess rights that can be auctioned. For example, a principal would not be able to auction the rights of private consumers to choose how much of a particular good or service they buy from which supplier. This is why a quantity-based RBF instrument is difficult to deliver outside of a situation in which the principal is directly procuring a good (see the next section). In other situations, using auctions to allocate an RBF-based subsidy where there is strong competition ‘in’ the market (for example, to determine which lighting manufacturers should receive an RBF payment) risks prejudging the outcome of competition between those manufacturers for consumers. In these cases an administratively determined RBF intervention is likely to be preferable. This, for instance, is the approach suggested in the Lighting Africa and SHW case studies where, in each case, significant competition ‘in’ the market already exists. • There is a certain level of technical expertise needed in administering an auction. This will vary depending on the complexity of the context with auctions being most challenging where principals care about multiple results.27 That said, it also needs to be acknowledged that where auctions are not used, there will also be a level of technical expertise required to administratively determine the appropriate RBF quantum in order to reduce the risk of windfall profits. • A particular concern that principals need to be aware of is the risk of winners’ curse: an agent may win an auction by optimistically bidding too aggressively and find itself unable to deliver at the promised cost, causing delays and reputational risk for the principal. This, for instance, may help to explain why an auction-based approach for supporting early-stage renewables in the UK was not particularly successful (Butler & Neuhoff 2008). This may be particularly likely when markets and agents are relatively immature. In cases where this is likely to be a concern, extra resources may be required at the time of bid evaluation, as well as performance bonds or other forms of down payment that cannot be recovered in the event on non-delivery. Results-Based Financing in the Energy Sector | An Analytical Guide 39 • Unless there are a reasonable number of bidders competing in the auction, there is a risk that the resulting auction will not deliver value for money for the principal. Bilateral administrative negotiations may be more effective. • There is a trade-off between paying the minimum amount necessary in the short run, and the prospect of lower costs in the future; short-term windfall profits may encourage stronger competition and innovation, driving down costs in the long run. RBF instruments can also lead to windfall profits if they are not adjusted over time and costs change (fall) substantially. To recoup windfall profits in case of falling costs, an RBF intervention should be readjusted over time; this possibility, and the thresholds needed to trigger it, should be announced in advance in order not to harm investor confidence (see below). What is the Role of the Principal? There are two main roles available for a principal in an RBF intervention. The principal can either (a) directly procure the result or (b) it can provide incentives/subsidies to private market participants without securing ownership rights. This choice depends on four factors: 1 | Property rights 2 | Transaction costs 3 | Risk mitigation 4 | Resource implications First, and most obviously, procurement should be preferred when the principal has a good reason to own the underlying result. For instance, a government ministry (e.g., education) might procure energy efficient lighting for all the buildings that are under its control (schools and universities). Second, transaction cost arguments might work either in favor of or against procurement. In cases where the principal wishes to incentivize scale up of a product or service that will be consumed by a large number of discrete individual consumers (who may be diffused across a broad geographic area) and who might also suffer from information barriers and/or capital constraints, then rapid deployment might favor procurement by the principal, rather than trying to incentivize these consumers to purchase the product themselves. The principal could then distribute the procured product to the ultimate consumer at low or zero cost. The accompanying case study to this report highlights this as an approach that may be favored if there is a desire to rapidly scale-up institutional cookstoves in East Africa. Outside of the energy field, these arguments are often (implicitly) invoked when decisions are made by principals to procure medical interventions, such as vaccines. On the other hand, if the procurement process is corrupt, slow, or otherwise wasteful, then incentivization of private sector agents might be preferred. Further, procurement of this type may make it more difficult to guarantee the long-term sustainability of a market as customers may be willing to pay a low price for a good initially procured by government but less willing to pay market prices once the initial procurement phase is complete. Chapter 4 40 The desire to use quantity commitments to reduce cost uncertainties may favor procurement. As discussed in the previous section, RBF instruments that involve the principal making commitments around the quantity of output that will be demanded may be particularly valuable in reducing risks faced by agents in cases where there is substantial cost uncertainty that may be particularly prevalent for immature technologies. In such a situation, procurement will often be a much easier tool to deliver a quantity commitment, for example, as the principal can make good on its commitment by direct purchase. By contrast, using incentives/subsidies to ensure that consumers definitely purchase a certain amount of output may be very difficult in many situations. Finally, when the RBF instrument is linked to market conditions, the costs to the principal from direct procurement may be higher or lower than providing incentives and subsidies. Following the same logic as outlined in the previous section, both approaches can expand the size of the market by the same amount (if the principal has sufficient knowledge). However, the cost implications may be very different. This is best shown by comparing the two diagrams in Figure 4.3. The left hand side diagram illustrates the case of direct procurement. In this case, the principal commits to buy extra output; this is shown by an outward shift of the demand curve. This additional demand increases the amount of output in the market from Q1 to Q2. However, it also increases the market price from P1 to P2 which has the unfortunate impact of reducing the amount of private (non-principal) demand, that is, some private demand is ‘crowded-out’. The result is that the principal has to pay the price P2 for all units of output28 between that which private demand will purchase at P2 and the desired final level of output. This is shown as the grey box. The diagram on the right hand side shows a situation in which the increase in Figure 4.3 The Cost Differences to the Principal of Direct Procurement and Subsidies Will Be Different Direct Procurement Subsidies Price Price Procurement Quantity S Ddemand-subsidy S Dno intervention Dno intervention per-unit subsidy P2 P2 Dprocurement P1 P1 Fiscal Costs of Procurement PS Ssupply-subsidy Fiscal Costs of Fiscal Costs of Subsidy (if only Subsidy (if paid paid on additional units) on all units) Q1 Q2 Quantity Q1 Q2 Quantity Source | Vivid Economics. Results-Based Financing in the Energy Sector | An Analytical Guide 41 output is delivered through a subsidy (which can either be provided through the demand side or the supply side). If consumers are to be persuaded to purchase Q2, then the price that they have to pay must be PS. At the same time, if producers are to be persuaded to supply Q2, then they must receive a price P2. The subsidy that the principal has to pay is the difference between these two prices. Depending on the market context, this may need to be paid on either all output in the market (the larger box) or just the newly incentivized output (the smaller grey box). The crucial point is that there is nothing that would suggest that the size of the subsidy (grey box) in the diagram on the left hand side will be the same size as the size of the subsidy (grey box) in the right hand side diagram. Various market factors will determine whether the costs of procurement will be higher or lower than the costs of providing subsidies/incentives. Procurement will be favored in cases where the principal can price discriminate, that is, rather than have to pay the same market price for all units of output, it can pay different prices for different units of output. It will also be favored when demand is relatively unresponsive to prices as there will be limited crowding-out of private demand. By contrast, subsidies will be favored when the market structure allows for subsidies to be paid on only the ‘new’ output, for example, subsidies can be paid only on new electricity connections, which will be more likely when the output being incentivized is a durable good (wires and meters). Subsidies will also be favored when demand is relatively responsive to prices and supply relatively unresponsive to prices. What should be the Duration of the RBF Intervention, and What is the Appropriate Exit Strategy? Both duration and exit strategy depend on the particular goal that a principal is pursuing. In particular, RBF duration and exit strategy will vary depending on whether the principal is aiming to create a self-sustaining market that can survive and flourish in absence of the RBF intervention; or whether the principal is aiming to procure goods or services for itself or for society in a cost- effective manner. If the principal is primarily interested in securing goods and services for itself or for society, then duration and exit strategy are clear: once the desired quantity has been achieved, the RBF intervention is completed and should be wound down. However, even in this scenario there are important sustainability considerations to bear in mind. Where possible, the principal should reduce its commitment to the RBF intervention gradually, and give ample notice that it intends to do so. In this way, unnecessary disruptions can be avoided and it may be possible for suppliers to plan ahead, to search for replacement customers, and to change their product line-up. If the goal is a self-sustaining market, then the duration of the RBF intervention and its exit strategy depend on how far away the relevant market is from self-sustainability. A self-sustaining market occurs if suppliers are willing and able to supply the relevant good at such prices as customers are willing and able to buy it. This implies that self-sustainability can be achieved through two main mechanisms: first, by lowering production costs, so that suppliers are able and willing to supply at lower prices; Chapter 4 42 second, by increasing customers’ willingness to pay for a product, such as a result of rising incomes or because initial use increases the value consumers attach to the product or service. The appropriate duration of an RBF intervention intended to create a self-sustaining market is determined by two considerations: first, how far away from self-sustainability is the relevant market, as measured by the gap between the price at which customers are willing to buy and the price at which suppliers are willing to sell? Second, what rate is the market expected to move towards self- sustainability once the RBF is effective? Based on these two pieces of information, the principal can make a prediction for the minimum period for which he needs to commit to the RBF intervention. It may be the case, that one or both of these data points is unavailable; in that case the principal may wish to gather the relevant information before designing a potentially overly long-lived (overly generous) or overly short-lived (insufficient to achieve self-sustainability) RBF intervention. As discussed in the preconditions analysis (in Chapter 3) if the minimum required period for self-sustainability is longer than the principal’s ability to commit its resources, or requires more resources than are available, then an RBF intervention alone is unlikely to succeed. The corollary of this is that for some contexts an RBF may be most suitable for principals with deep pockets and long time-horizons (consider, for example, feed-in tariffs in developed countries); when the market is close to self-sustainability (and it is primarily inertia and other non-cost/income factors that needs overcoming), such that the RBF intervention can be relatively short; or where the market is comparatively small and the technology (or new business model) is comparatively inexpensive, so that the RBF intervention requires relatively few resources. Finally, for markets where the gap between production cost and purchasing power is particularly large, rather than incentivizing sales or purchases in general, the RBF intervention could specifically target development of innovative approaches to sales and distribution, or cost reductions in the production of the good or service. Prizes in particular, may be an option here. Where an RBF intervention is intended to create a self-sustaining market, a phased exit strategy is likely to be most effective. After a minimum period during which the RBF payment is guaranteed, the principal may transparently begin to reduce RBF payments, linked to the extent to which the market has become self-sufficient. As above, self-sufficiency can be measured by the gap between the price at which firms are willing to offer the good and the price at which consumers are willing to buy the good. In order to create an incentive for firms to cut costs, rather than rely on the RBF payment to subsidize them permanently, the principal may wish to set a fixed maximum time period or maximum total funding after which the RBF intervention is wound down. There may be situations in which RBF-induced market development does not work out as planned; the principal should be prepared for such cases and also allow agents to prepare. As detailed above, in setting up the RBF intervention, the principal will form a view as to how long it expects to take for the market to reach self-sustainability. However, unexpected developments, either positive or negative, may change conditions such that the original assessment proves to be mistaken. When this happens, the principal needs to respond—particularly if market development is slower than expected, and resources are reaching their limit. While any particular response will depend on the case at hand, there Results-Based Financing in the Energy Sector | An Analytical Guide 43 are some principles that may be relevant. These principles ensure that both principals and agents can plan ahead, giving each party confidence in the commitments made by the other party: • Build in intermediate checkpoints and publicize them; the principal should build in intermediate checkpoints to detect any deviation from the expected development early on; these can be very basic, such as payments made at certain intervals, as percentage of the payments expected to be made at these points if everything proceeds according to plan. These should be communicated to agents, so that they are aware of when the next decision is made, and on what grounds it will be taken. The timing for these checkpoints should be announced well in advance and, ideally, at the time that the RBF intervention is first established to avoid creating market uncertainty. • For each intermediate checkpoint, outline what the next steps are: on the one hand, the principal should declare what will happen if the target for the checkpoint is met; this may be as simple as saying ‘if the target for this checkpoint is met, the program continues as before,’ or it may be ‘if the target is met, the RBF payment is reduced by 10 percent,’ or it may be something different altogether. Making the next step known explicitly and in advance allows agents to plan ahead, and provides for predictability in the process of drawing down the RBF resources. On the other hand, the principal should lay out broad principles for what will happen in case the target for the checkpoint is missed; while the particular action in this case is likely to depend on the amount by which the checkpoint is missed, the principal may still be able to outline broadly the options that it will consider in this scenario. For example, the principal may say in advance that it will consider transferring resources to alternative projects if the target is missed by more than a certain percentage, or that it will reduce payments in proportion to the extent by which the target is missed. Alternatively, the principal could commit in advance to increase the strength of the RBF payment although this would need to be managed carefully to avoid perverse incentives for gaming whereby agents deliberately underperform in the initial stages in order to benefit from stronger incentives at a later point. Chapter 4 44 A n n e x | L i t e r at u r e R e v i e w o n R e s u lt s - B a s e d A p p r o a c h e s This literature review assesses previous research on results-based approaches especially those used in the field of international development.29 The scope of the review is deliberately broad—covering both research related to energy access, low carbon investment, as well as other sectors, most notably the health sector—in order to obtain the lessons and insights from the design of these instruments from a wide range of contexts. An immediate challenge in this area relates to the question of definitions. At a very broad level, the concept of interest is that a payment is only made upon the verified delivery of predefined results. In this text, we follow the approach of UK’s Department for International Development (DFID; and ESMAP) in using the term results-based financing (RBF) to refer to situations where these arrangements are in place between funders (which can be donor agencies, developing country governments, or others) and service providers, such as firms, civil society organizations, and results-based aid (RBA), as a term referring to similar arrangements between governments/multilateral institutions. Collectively, these approaches may either be referred to as payment by results (PBR) or more simply, RBAs. The latter term is used, where relevant, in this literature review. A further discussion of nomenclature is provided in Box A.1. The review provides two kinds of findings: a description of the literature and findings from the literature. The description of the literature explains where the existing academic and grey literature with relevance to RBAs is found and what it covers. Findings from the literature, that is, the key substantive research results found within the literature reviewed, are described in the remainder of the appendix. Description of the Existing Literature The existing literature largely falls into three broad categories: (i) specific project evaluations and analysis papers from sectors other than energy, in particular health; (ii) practical reports considering the implementation and monitoring challenges of RBF approaches, drawing on a range of evidence from multiple studies; and (iii) meta-evaluations and higher level theoretical analyses of particular results-based instruments or the RBAs in general. A total of 41 relevant sources have been identified and included in this literature review. This list of sources has a broad sectoral coverage, including literature on the use of results-based instruments in health, education, and infrastructure. However, there is relatively little literature on the use of RBAs in the energy sector, reflecting their limited application in this sector to date. Two areas that are comprehensively dealt with in this literature are the following: first, reviews of RBAs in healthcare, and second, four specific types: • Advance Market Commitments (AMCs) are agreements to subsidize the future purchase30 of products—so as to guarantee a particular price—which meet a set of criteria. AMCs are primarily being developed to incentivize the production of vaccines in the developing world by providing a guarantee of a future market for vaccines (Monitoring and Evaluation Subgroup of the AMC Donor Committee 2008). Results-Based Financing in the Energy Sector | An Analytical Guide 45 Box A.1 RBA Nomenclature | The Need for Clarification and Agreement There is no universally agreed nomenclature in international development for instruments that involve payment upon the verification of certain results. Indeed, there is no single definition for the entire category: it is variously known as ‘performance-based financing’ (Canavan, Toonen, & Elovaino 2008), ‘performance-based funding’ (The Global Fund 2012), ‘payment by results’ (DFID 2012), ‘results-based approaches’ (Birdsall and Savedoff 2011; Neal 2012;), ‘results-based financing’ (RBF; Brenzel 2009), or ‘performance incentives’ (Eichler & Levine 2009). In addition to this plethora of terms and abbreviations, the three abbreviations RBA, RBF, and PBF can each also refer to a more specific subclass of instruments. The UK’s DFID uses the two terms RBA and RBF to refer to specific subsets of results-based approaches (Pearson 2011); this is explored further below. GAVI also uses the abbreviation PBF, but unlike the Dutch Royal Tropical Institute, GAVI takes it to mean specifically financing at below-national level (Hansen 2011). In addition to the lack of clarity about the term for the category as a whole, there is some confusion about the terms used to denote particular instruments. A range of specific terms exist which denote particular types of instruments or particular subcategories, for example ‘output-based aid’ (OBA), ‘output-based disbursement’ (OBD), ‘advance market commitments’ (AMCs), ‘conditional cash transfers’ (CCTs), ‘cash on delivery aid’ (COD aid), ‘performance- based financing for health’ (PBFH), ‘payment for environmental services’ (PES), and ‘carbon finance’ are all distinguished in Output-Based Aid in the Results-Based Financing Universe (GPOBA 2011a). Savedoff (2010) on the other hand distinguishes between OBA, ‘performance-based incentives’, ‘performance-based contracting’, ‘performance-based financing’, ‘performance-based aid’, and COD aid. COD aid and OBA are both established definitions, yet they have conceptual overlap (see the figure). Definitions in the Broader RBF Space Overlap, Leading to a Lack of Clarity Performance Bonuses OBA COD Results-Based Financing PBC CCT CCT = Conditional Cash Transfers; COD = Cash on Delivery; OBA = Output-Based Aid; PBC = Performance-Based Contracting (for example, for roads) Source | Johannes, Mimmi, & Mumssen 2010. (continued) Annex | Literature Review on Results-Based Approaches 46 B OX A . 1 ( c o n t i n u e d ) Ghosh et al. (2012), focusing on instruments used to reduce emissions, use the term quantity-performance instruments (QP) to refer to instruments that tie payments to quantifiable performance indicators, such as tonnes of CO2, kWhs, or hectares. They distinguish between three types of QP instruments, namely ‘direct purchase’ (the direct purchase of verified emission reductions), ‘top-up’ (the topping up of revenue from selling verified emission reductions to achieve a preagreed minimum price), and ‘tradable put option’ (a tradable option that gives the holder of the option the right to sell verified emission reductions at a specified price before a specified date). Thus, while there are names for specific results-based instruments, there seems to be no taxonomy that structures the space of results-based instruments. The specific instrument names hence have a certain ad-hoc nature and there is some overlap between different instrument names. A first attempt to clarify some of this confusion has been undertaken by the UK’s DFID (2012), and was adopted by ESMAP in the terms of reference for this study: DFID defines the category as a whole as ‘payments by results’ (PBR). This category is then subdivided into ‘results-based aid’ (RBA) and ‘results-based financing’ (RBF). RBA refers to funding arrangements between donors and recipient governments, while RBF refers to funding arrangements between funders (which can be development agencies, developing country governments, or others) and service providers (such as firms or civil society organizations). In addition, DFID (2012) provides a clear definition for PBR-type instruments; the three key elements that delineate PBR-type instruments from other aid instruments are: 1. (Part of) payments based on (previously agreed) results 2. Recipient discretion as to how results are achieved 3. Verification of results by an independent third party as the trigger for disbursement • Cash on Delivery (COD) Aid is an RBA approach that is donor-to-government funding, proposed by Nancy Birdsall and the Center for Global Development. It refers to a specific form of RBA in which a donor commits to paying the recipient a fixed and agreed sum for each increment of progress against an agreed outcome, for example, US$ 100 for each additional child entering secondary school (Birdsall & Savedoff 2011). • Output-Based Aid (OBA) is primarily an RBF instrument that is donor-to-service provider funding that ties disbursement to outputs that are as closely connected to the desired outcome as possible. It focuses particularly on the provision of basic services, such as access to water, energy, and health, and different modalities, such as performance-based contracting and voucher schemes (Johannes, Mimmi, & Mumssen 2010). • Prizes31 are a results-based instrument in which the achievement of a predefined goal is rewarded with an incentive (usually, though not always, a lump-sum cash payment). Examples of such prizes include the Longitude Prize offered by the British government in 1714, or the Ansari X prizes for space flight (Everett 2011). An in-depth coverage of RBAs in healthcare is provided on the World Bank Group’s Results-Based Financing for Health portal,32 which hosts a large number of studies encompassing most aspects of Results-Based Financing in the Energy Sector | An Analytical Guide 47 health-related results-based projects. The literature on health-related results-based projects covers economic aspects (Savedoff 2010), practical implementation issues in general (Brenzel & Naimoli 2009; Naimoli 2009; Toonen, Canavan, Vergeer, & Elovaino 2009), as well as specific issues such as monitoring (Naimoli, Brenzel, & Sturdy 2009), contracting (Loevinsohn 2008), and individual (Cashin & Chi 2011; Morgan 2010), and comparative (van den Looij 2009) project reviews. A number of extensive reports give detailed descriptions and theoretical analyses of particular RBAs. These include, in particular, the extensive literature on cash on delivery aid (COD aid), centred on the work of Birdsall and Savedoff (2011), who give a description of their proposal and make a first principles argument in its favor. Additional literature on COD aid includes Birdsall (2012), who considers an extension to energy access; Birdsall and Perakis (2012), who describe the design of a COD aid pilot in Ethiopia; and Savedoff and Martel (2011), who ask which indicators might be best suited for using COD aid instruments. Other RBAs that are well documented in the literature include output-based aid (OBA) and the use of prizes. OBA is championed by the Global Partnership on Output-Based Aid (GPOBA), a multi-donor trust fund administered by the World Bank, which actively funds and supports OBA approaches and disseminates documents on topics related to OBA. Besides an annual report written by GPOBA (2011b), GPOBA published Evaluation Notes, Lessons Learned (which are working papers), and resources for practitioners (GPOBA 2012a, 2012b; Mimmi 2012). Tremonti (2005) provided a theoretical analysis of the potential of advance market commitments (AMCs) to enable development of public health measures, primarily vaccines, for the G7 Finance Ministers. This examines and endorses the role of AMCs as a means of incentivizing the development and deployment of vaccines that are primarily required in the developing world. A large scale, US$ 1.5 billion, pilot project was instituted in 2008 for the delivery of pneumococcal vaccines. The use and suitability of prizes for development has been investigated by both DEW Point and Vivid Economics. Though the number of publications on the subject of prizes is small compared to the literature on COD aid and OBA, contained within the work of Vivid Economics (2007) and Everett (2011), there are a number of detailed studies that combine theoretic analysis with case studies. Three main deficiencies can be seen in the current literature: 1 | No exhaustive categorization or listing of the different results-based instruments is available. Instead, varying abbreviations are used to denote the same specific instrument or various groupings of instruments. 2 | Comparatively little work has been done on the conditions under which different types of results-based instruments may be desirable; the work that has been undertaken in this area is summarized in the Theoretical Findings section below. Whilst empirical evaluations of particular programs and instruments have been carried out, there is little systematic review of the appropriate microeconomic conditions for RBF. 3 | Even less work has been done on ascertaining where, geographically and sectorally, to obtain such optimal conditions. Annex | Literature Review on Results-Based Approaches 48 Key Findings A number of substantive results have emerged from the literature review. These can be structured into the following categories: (i) findings can be divided depending on whether they are empirical or theoretical findings; and (ii) within each of these categories, findings can be divided into (approximately) positive, cautionary, or negative in regards to the effectiveness of the instrument in achieving its aims. These three categories represent a spectrum rather than fixed and distinct categories; the same finding could conceivably be read as either cautionary or negative, or a finding might seem positive to one reader, but cautionary to another. Nevertheless, these three categories may be helpful in structuring an otherwise largely disorganized set of empirical and theoretical findings. In addition, two sets of findings are presented outside of this structure: first, findings on when the use of RBF is appropriate; given their particular relevance, these are presented under a separate heading at the beginning of the Theoretical Findings section. Second, findings on the use of prizes; since prizes differ in important economic features from other results-based instruments, these are shown as a separate category towards the end. Theoretical Findings When to use RBF approaches Three sources, GPOBA (2011a), Vivid Economics (2010), and Oxman and Fretheim (2008), present analysis that is relevant to answering the question of when RBF instruments can and should be deployed. Their findings, presented below, provide a useful starting point for answering this question and, on which, the findings in the main report have drawn. In particular, the three sources highlight the need to consider both wider circumstances beyond the features of the instrument (e.g., alignment with recipient policy priorities, sufficient technical skills, the presence of sufficient prefinancing capacity, and public sector credibility), interactions between demand and supply, and more specific features of the intervention and the instrument, such as the obstacles that the instrument is meant to overcome, and the expected ability of the specific conditional incentives in question to overcome these obstacles. In its 2011 Annual Report, GPOBA provided a set of broad guidance criteria that could be used to assess the sustainability of OBA. The six criteria identified are structured into two areas, ‘institutional capacity and arrangements’ on the one hand and ‘financial mechanics’ on the other. The three criteria included under institutional capacity and arrangements are: 1 | alignment with recipient government policy and general recipient government ownership; 2 | the presence of effective monitoring and evaluation (M&E) procedures; and 3 | the presence of sufficient technical, managerial, and financial capacity required for implementing the incentivized activity. The three criteria included under financial mechanics are: 1 | capacity for consumers to pay for a reasonable share of operating and maintenance costs; Results-Based Financing in the Energy Sector | An Analytical Guide 49 2 | practical, feasible, performance-based, and explicit nature of the subsidies provided through the instrument; and 3 | sufficient prefinancing capacity. The second source, Vivid Economics (2010), considering the choice between price or quantity AMCs33 and upfront capital grants, finds six situations in which AMCs are preferable over upfront instruments. These six situations are: 1 | when private sector firms have a comparative advantage over the public sector in managing delivery risk; 2 | when there is a specific component of output that is desirable and to be encouraged (e.g., output rather than capacity); 3 | when public sector credibility is high enough to provide and abide by an advance commitment; 4 | when there is an identifiable market for a product that can, in principle, be sustained by private demand; 5 | when there are a diversity of products satisfying the same demand, and it is unclear which might be superior; and 6 | when there are few supply-side barriers, or when these have already been addressed by other policy instruments. The third source, Oxman and Fretheim (2008), does not provide explicit criteria for deciding when to use an RBF-type instrument. However, they do present four guiding questions that can be used to structure the decision making process. These questions are: 1 | Is the activity that the instrument in question incentivizes worth encouraging and good value for the money? 2 | If so and if already incentivized, is there currently low compliance? 3 | If there is low compliance, why? 4 | In the light of the reason for low compliance, are financial incentives likely to be helpful? However, while this literature provides useful insights on when to use such instruments, there are a number of deficiencies. First, the optimal relationship between RBF and more conventional instruments can be better identified. Second, there is relatively little focus on different forms of RBF and criteria for deciding between a range of different RBF approaches. Third, there is relatively little attention to issues specific to the energy sector. The analysis in the main body of the report, complemented with the case studies, and taking into account more of the literature findings discussed below, attempts to address some of these deficiencies. Positive theoretical benefits of results-based instruments There are a number of positive theoretical results concerning results-based instruments. They can be grouped loosely into direct effects of the RBA and into indirect effects (or spillovers/externalities). Annex | Literature Review on Results-Based Approaches 50 First, there are a number of directly beneficial effects that various theoretical assessments identify: results-based instruments can align the motivations, incentives, and objectives of donors and aid recipients (Savedoff 2010). They also give recipients independence with respect to the means by which they choose to deliver the goods, which in turn encourages innovation: “It gives recipients discretion, permitting experimentation and innovation” (Birdsall & Savedoff 2011). They can also be cost effective: “All QP instruments have the potential to be cost-effective” (Ghosh, Müller, Pizer, & Wagner 2012b). Looking specifically at AMCs, these have been identified as instruments that can overcome various market failures in relation to the development of vaccines. Most notably, they have been praised as providing a stable policy platform, to allow private capital investment (if sufficient private capital markets exist) while allowing latitude in the technical means of achieving the public health outcome (Center for Global Development 2005). Further, Tremonti (2005) identified that while private investment in health R&D responded to the private returns, it failed to take into account the subject of sufficient public benefits. AMCs allow policymakers to internalize the public good benefits to firms’ R&D calculations by using subsidies to increase the size of the potential market revenue in line with these public benefits. The gap between public benefits and private returns are particularly stark in relation to vaccines in the developing world given the time taken to bring vaccines to market. Assessments also identify positive indirect effects. Results-based instruments are capable of leveraging and directing considerably larger funds than just the anticipated payments that are expected: “The RBA payment will also be an incentive to use existing donor funds and the government’s own resources more efficiently” (Birdsall & Perakis 2012). Due to its focus on monitoring and verification, RBAs can improve the accountability of both providers and recipients (Birdsall & Savedoff 2011; Eichler & Levine 2009). An indirect effect of the increased autonomy resulting from RBAs may be to encourage capacity building, particularly in M&E. This in turn may lead to better data quality and availability (Birdsall & Perakis 2012; Eichler & Levine 2009). In addition, due to reduced monitoring and micro-management by donors, RBAs may help to change the donor-recipient relationship into a more equal one (Birdsall & Savedoff 2011; Holmqvist 2012). Another indirect impact on the donor-recipient relationship may be that RBAs can lead to greater clarity concerning the objective that both donors and recipients aim for: “The decision to pay for results focuses attention on what results are really desired” (Savedoff 2010). In sum, the literature identifies considerable theoretic advantages for results-based instruments, both directly resulting from the changed incentive structure, and indirectly resulting from the activities needed to respond well to this new incentive structure. Cautionary findings in relation to the use of RBAs Besides straightforwardly positive findings, the literature reveals a number of more cautionary results. While the positive findings often focus on generic features of RBAs in general, the more cautionary results point out specific circumstances that are needed in order for RBAs to succeed. There are 12 separate cautionary findings that have been identified. In the interest of clarity and accessibility, these Results-Based Financing in the Energy Sector | An Analytical Guide 51 are provided in Table A.1. They can be grouped into three classes: necessary preconditions for RBAs to be effective; design challenges that need to be overcome; and inherent tensions that, although possible to mitigate, are unlikely to be overcome completely although they need not undermine the approach. A separate cautionary finding that is worth pointing out separately is the risk of non-performance. Ghosh et al. (2012) point out that, on the one hand, results-based instruments do protect the public sector from facing the financial risk for failure; if an asset fails to perform, the public sector or donor does not have to pay for it. On the other hand, non-performance has an impact on the public sector or donor: first, the funds that were planned to be disbursed have been earmarked, and so could not be used for other purposes. If they are not disbursed, the public sector or donor incurs an opportunity cost. Second, the outcome which the funder desired did not occur. In other words, “non-performance will still [even with a conditional instrument] imply that public money has been tied up for some period of time, and no reductions have occurred” (Ghosh, Müller, Pizer, & Wagner 2012b). Negative findings in relation to the use of RBAs Following on from positive and cautionary findings, there have been a number of straightforwardly negative points made in the literature. These may be grouped around four key points: unnecessary cost increases; perverse incentives; adverse equity impacts; and a misunderstanding of data requirements. First, there are two main ways in which RBAs may lead to avoidable cost increases. Due to the need for prefinancing, the overall financing costs of a project may increase. This is likely to be the case if the donor can procure financing at lower costs than the recipient (Mumssen et al. 2010). In addition, results-based instruments may lead to considerably higher monitoring and verification costs: “Participants expressed concerns about high costs for verification relative to program costs” (Center for Global Development & Deutsches Institut für Entwicklungshilfe 2012). Second, RBAs, under certain circumstances, may lead to perverse incentives. Tracking only a subset of activities may lead to reduced effort on non-monitored but important activities, or may undermine the intrinsic motivation of service providers (Savedoff 2010). Alternatively, even if all relevant activities are incentivized, results-based approaches may lead to an increase in total quantity at the expense of overall quality (Birdsall & Savedoff 2011). Third, results-based instruments may have adverse equity impacts: “There is a risk that countries, provinces, districts, facilities or individuals (depending on the type of scheme) with stronger capacities might be able to secure most of the rewards” (Pearson 2011). However, Pearson qualifies this finding: “There is little evidence, to date, of this happening,” and so it remains for empirical assessments to verify the extent to which adverse equity impacts occur. Fourth, where the results-based instrument sets a particular standard as the benchmark of success, this may create an incentive to achieve but not exceed this benchmark. In the design of AMC for vaccine development, Farlow et al. (2005) express the concern that benchmarks may be set too low Annex | Literature Review on Results-Based Approaches 52 53 Tabl e A . 1 12 Cautionary Findings When Using RBAs Category Finding Description Source Necessary Availability of RBAs, to the extent that they only pay out funding after the verification of results, Birdsall & Savedoff 2011; Vivid Preconditions prefinance require the recipient to obtain prefinancing in order to deliver the results. This Economics 2010 limits the range of circumstances under which RBAs can be used to those where prefinancing can be obtained at non-prohibitive costs. However, RBF can also be used to lower financing costs by providing an incentive to lenders. Need for supply-side RBF approaches are only effective if the supply side of the particular market is Pearson 2011; Vivid Economics 2010 capacity able to respond to the new incentive. If the general enabling environment is lack- ing, a single results-based intervention may fail to deliver its intended outcome. However, RBF approaches can also be targeted at the supply side to overcome this challenge. Sufficient monitoring RBAs presuppose the ability to effectively monitor the outcome on which payouts Birdsall & Savedoff 2011 capability depend. Where such monitoring capacity does not exist, RBAs cannot be used. Results-Based Financing in the Energy Sector | An Analytical Guide Need for simple and RBAs are only feasible if the desired outcome can be measured with simple and Logan Brenzel 2009; Center for Global appropriate indicators verifiable indicators that are affordable to measure, difficult to forge, and closely Development & Deutsches Institut für track the desired outcome. While these may exist in some areas (e.g., vaccination), Entwicklungshilfe 2012 they may not exist in others, (e.g., preventive care or public sanitation). Design Challenges Need for simple design Overly complex results-based instruments may be misunderstood, or may Neal 2012; Eichler & Levine 2009 ­demotivate recipients. Corruption still Analogous to risk shifting, the possibility of corruption is moved rather than Pearson 2011; Farlow et al. 2005 possible removed under RBAs. ‘Capture’ of monitoring and verification staff would allow for corrupt practices to take place. Possibility of rents RBF approaches can create rents (e.g., through excessive per-unit payments or Vivid Economics 2010; Birdsall & through efficiency improvements that reduce costs faster than the fall in per-unit Savedoff 2011 payments). While this is a valid point of caution, not every instance of rents is to be regarded critically. In cases where rents come from efficiency improvements, they represent a particular way of splitting up the gains from innovation in which the innovator reaps most/all of them. This is broadly similar to the way in which patents allocate the gains from innovation, and partly compensates for the risk that the aid recipient bears. Nevertheless, the possibility of ‘excessive’ rents may undermine the political acceptability of RBF approaches. Trade-off between RBF approaches, in targeting outcomes and results, inherently favor the most Ghosh et al. 2012 equity and efficiency competitive and those most able to deliver the targeted good or service. Where an agency, a firm, or a region has a strong competitive advantage, it may lead, at the limit, to all funds flowing towards one agency, firm, or region. This can be prevented by adding further conditions; but this will reduce the efficiency with which the originally desired goal will be achieved. In other words, “any approach will necessitate decisions on balancing overall cost-effectiveness with concerns about equity” (Ghosh, Müller, Pizer, & Wagner 2012b). Challenges of long- RBF instruments often require long-term commitments in order to create a stable Hecht, Palriwala, & Rao 2010; Farlow term commitments investment environment for firms to enable investment in R&D. However, careful et al. 2005 consideration must be given to the balance of risk between the funders of the instrument and the recipients. Problems can arise due to inherent uncertainties in inflation, currency fluctuations, medium- and long-term demand, or the possibil- ity of technology shocks. There is a risk that contracts that are too tightly defined may discourage innovation but that allowing too much flexibility may discourage investment. Further, tax, regulatory, and budgeting restrictions may make the future binding commitment of funds required by some RBF instruments difficult. Inherent Tensions Inherent challenge of Finding the right baseline and incentive modality is inherently difficult: if a Savedoff 2010; Cernuschi et al. 2011 within RBAs baseline and incentive baseline is established through negotiations, then the recipient has a strong setting incentive to negotiate a low baseline and high per unit payment; if historical performance is used for baseline setting, service providers may be discouraged from making large performance improvements as they know that it will raise future expectations (ratchet effect). Unilaterally setting a baseline is likely to reduce recipient buy-in. Providing incentives in a single RBF intervention where multiple recipients are desirable to facilitate competition can be difficult when recipients have different costs and strategic objectives. Danger of vicious and RBF instruments can be both discouraging and encouraging: strong initial Pearson 2011 virtuous circle achievement breeds a virtuous circle of confidence, investment by the recipient, and successful disbursing of funds; initial failure breeds a vicious circle of small or no payouts, leading to stranded costs and a lack of confidence. Risk shifting, not risk RBF approaches primarily shift risk from one party to another, rather than reducing Vivid Economics 2010; Savedoff reduction it. However, as Savedoff (2010) points out, risk shifting can align incentives more 2010; Mumssen, Johannes, & closely, thus leading to better outcomes. As Vivid Economics (2010) shows, the Kumar 2010; costs of risk can also be reduced by risk shifting, if one party is more capable of dealing with risk. This suggests that RBF approaches should be limited to contexts where aid recipients are able to take on risks. Source | Vivid Economics. Annex | Literature Review on Results-Based Approaches 54 to encourage participation and, thus, result in an incentive to be the first to achieve the benchmarks but would not drive technological improvements and “. . . would actively discourage the development of highly effective and safe vaccines.” Lastly, those who count improved data collection in favor of RBAs are said to misunderstand the data requirements for accurate impact assessments (Pearson 2011). An accurate measurement of the impacts of a program is a necessary, but not a sufficient, requirement for assessing the impacts of a program: a counterfactual scenario, ideally via a control group, is needed to distinguish program impacts from background noise (Pearson 2011). However, this last argument seems misdirected: it may be true that monitoring results alone (as is a side-effect of RBAs) is not sufficient, but there is nothing inherent to results-based instruments that prevents the use of control groups or other more advanced assessment methods. This point is relevant both to baseline setting and finding the right incentive modality. Summary of theoretical results In sum, the theoretic literature provides a wide range of positive, cautionary, and negative arguments concerning RBAs. It also provides a starting point for thinking about the circumstances under which RBAs can and should be implemented. However, no clear conclusion is possible from theory alone; “the approach may be better suited to some settings than others” (Pearson 2011). Evidence from empirical studies is therefore provided next. Empirical Findings Similar to the theoretic findings about results-based instruments, though there are some highly pertinent results, the sum of empirical findings is somewhat unstructured. The results from the literature review are presented in the same framework as the theoretic findings—namely grouped into positive, cautionary, and negative findings. In addition, reflecting their more rigorous and scientific methodology, the relevant findings from the Abdul Latif Jameel Povery Action Lab (J-PAL) are discussed in Box A.2. Positive empirical findings There have been a number of positive findings from large-scale evaluations of past results-based projects. Mumssen et al. (2010), covering 197 World Bank and non-World Bank OBA projects, find that “the piloting phase of OBA has generally been a success.” OBA projects, for example, significantly outperformed traditional projects on cost overruns and benefit shortfalls: in a sample of 50 projects, all with quantifiable outputs as their objectives, 85 percent of all OBA projects met or exceeded their objectives, while only 49 percent of traditional projects succeeded in doing so (Mumssen et al. 2010). Eichler and Levine (2009) similarly find that “such incentives [performance incentives in healthcare] can work in a variety of settings,” and more specifically that “large-scale performance incentive programs in Haiti and Rwanda, for example, have improved health outputs” (Eichler & Levine 2009). In Results-Based Financing in the Energy Sector | An Analytical Guide 55 Box A.2 J-PAL Results are Encouraging, but Monitoring and Accountability Structures are Needed The Abdul Latif Jameel Povery Action Lab (J-PAL) conducts, coordinates, and collects evaluations that use randomized controlled trials. This method is considerably more robust than single-project evaluations, because it rigorously constructs (and then monitors) a counterfactual scenario, in which the instrument was not deployed. In more sophisticated trials, multiple versions of an instrument are compared with a control group and amongst each other, in order to assess which particular design best achieves a certain goal. These studies, thus, isolate the particular impact of the instrument under investigation, and filter out incidental or circumstantial effects that would otherwise be falsely attributed to the instrument. The findings from the limited number of studies so far collected under J-PAL’s methods point towards the importance of sensitive yet sound monitoring and accountability structures. However, when these features are well designed, there is clear evidence of the effectiveness of results-based financing (RBF). One case in point is the use of different monitoring regimes for incentives to reduce teacher absenteeism. These were shown to work well when administered impersonally, and perceived to be monitored in that way (e.g., through time-stamped pictures of teachers and their class that are submitted to an independent monitoring team; this leads to a reduction of teacher absenteeism by 21 percent (Duflo, Hanna, & Ryan 2012)). “However, in settings when supervisors were given discretion over administering incentives, these programs became entirely ineffective” (J-PAL 2012). Thus, similar incentives, monitored in different fashions, can have radically different impacts—showing the importance of well-designed monitoring structures. Secondly, and underlining the first point, financial structure can make a big difference: environmental auditors in India were found to systematically understate pollution when hired directly by the plant they were investigating (in fact, in comparison plants, auditors systematically reported many pollution readings to be just below the regulatory standard (Duflo et al. 2012)); two changes to these arrangements were each found to have a significant impact. The payment structure and commissioning was changed (without changing the amount paid to auditors) such that auditors are paid from a central pool and assigned to plants on a random basis. In addition, the pay of a subgroup of auditors was made variable, depending on their performance (as measured by comparing their findings with findings obtained by independent auditors paid by the experimenter). As a result, “the treatment group improved the accuracy of their reports by 0.21 standard deviations, on average, enough that by the end of the experiment they were statistically equal to the truth. [. . .] Both financial incentives and backchecks appear to have played independent roles in improving auditors’ accuracy” (Duflo et al. 2012). Source | Vivid Economics. the Ugandan case, performance incentives were tested alongside traditional input-based payments to support an extension of health coverage. In provinces with RBF-type funding, the rate of curative care increased from 0.22 to 0.55 visits per person, compared to an increase from 0.2 to 0.3 in provinces with traditional funding. Similarly, the rate of women giving birth under professional medical supervision increased from 12 to 23 percent in provinces with RBF-type instruments, but only from 7 to 10 percent in provinces with traditional funding (Eichler & Levine 2009). Annex | Literature Review on Results-Based Approaches 56 Drilling down into the nature of these positive outcomes, Mumssen, Johannes, and Kumar (2010) point out that the process of verification improves overall transparency; the World Bank’s guidance on Program-for-Results Financing (World Bank Operations Policy and Country Services 2011) in turn states that greater transparency “will greatly contribute to improved accountability,” which itself is a successful outcome. Eichler and Levine (2009) show that RBAs can be successful both in improving demand-side uptake (drawing on examples of conditional cash transfers (CCTs) in Latin America, and the provision of incentive packages for treatment continuation in Russia), and in increasing supply-side provision (pointing out increased healthcare provision in Haiti and Rwanda). However, these positive results are nuanced by other studies. Oxman and Fretheim (2008) point out that “financial incentives [. . .] appear to be effective in the short run for simple and distinct, well-defined behavioural goals,” but “there is less evidence that financial incentives can sustain long-term changes.” Booth (2011), speaking broadly about the idea of results-based disbursement, is even more critical: “We lack evaluation of previous experiments on these lines [the idea of results- based disbursement] or even robust evidence on how the relevant in-country incentives currently work.” However, it appears from the rest of Booth’s paper that he is mostly concerned with the role of institutions and government-to-government RBA, not RBF. Hence, this criticism is less relevant, especially given that J-PAL and others are in the process of conducting rigorous scientific studies on the impacts of RBF incentives. It should also be noted that part of the lack of evidence on the long- term success of RBF instruments is explained by their relative novelty. Cautionary empirical findings The cautionary empirical findings can be grouped into five broad messages: 1 | Sound monitoring, evaluation, and verification is both challenging and highly important for the success of result-based approaches; moreover, it may be particularly challenging for projects aiming to stimulate long-term capital investment. 2 | There are some reasons to be sceptical about the positive results that have been reported from pilot RBF programs to date. 3 | Where there are obstacles both on the demand and the supply side, a single RBF instrument addressing either only demand or only supply is unlikely to be successful. 4 | Recipient buy-in at both the local and national level is vital for success. 5 | The additional costs associated with RBAs may or may not be significant; this is an on-going debate which is not yet settled. Across these five messages, which are described in more detail below, one overarching conclusion emerges: though results-based can be effective in the right circumstances, it is no silver bullet. First, considering M&E, World Bank Operations Policy and Country Services (2011), points out that high data quality is of high importance for the success of results-based programs. The same report also points out that within GAVI Alliance projects, “monitoring and reporting processes were Results-Based Financing in the Energy Sector | An Analytical Guide 57 considered burdensome and the collection of data difficult in the developing country context.” This issue is further highlighted in EnDev’s 2010 impact report: “Studies indicated that there is a strong need to verify reported figures of partner organizations” (Op de Coul et al. 2010). Balancing this was the finding that “the Data Quality Audit process, conducted in Year 2 [. . .], was reported to have a catalytic effect on country reporting systems” (World Bank Operations Policy and Country Services 2011). Besides these generic findings that apply to all RBF instruments, some cross-sectoral differences in M&E may exist. Thus, some outcomes may be easier to monitor and evaluate than others. For instance, number of children vaccinated, school attendance rates, or kilometres of road constructed are all immediately observable. However, where intended outcomes include the creation of a self-sustaining market, M&E becomes more challenging as the outcome only becomes observable over time. While good M&E practice can take this into account, the delay with which indirect outcomes often emerge may lead to an underestimation of benefits for certain RBF instruments. Thus, monitoring, evaluation, and verification is both important and difficult, but can be improved through appropriate support procedures. In addition, achieving good M&E can be more challenging in some sectors than others, particularly where indirect or longer term outcomes are targeted. A second cautionary finding, connected to the challenges of monitoring and verification, is that it is often difficult to “disentangle the effects of RBF,” since they are “commonly [. . .] part of a package that may include increased funding, technical support, training, changes in management, and new information systems” (Oxman & Fretheim 2008). Resulting from this, Oxman and Fretheim find that “there is very limited quantitative evidence of RBF per se having an effect.” This cautionary finding is supported by Toonen et al. (2009), who point out that “the attribution of improved results to PBF [performance-based financing] is not undisputed.” They mention three reasons why, for the evaluations that they investigate in their review, the improved outcomes that were observed may not be directly due to results-based instruments: 1 | Similar improvements were sometimes found in similar areas that were not covered by the PBF instrument in question. 2 | Results-based instruments were often introduced in combination with other changes, such as the abolition or reduction of user fees (for health care services) or increased local autonomy, which may have driven (part of) the improvements. 3 | Projects supported with results-based instruments may suffer from a piloting bias: it may be the case that pilot projects receive extra resources and attention, independent of the particular instrument being piloted; thus, it is not clear if results-based pilot projects result in improved results because of their PBF modality or because they are pilot projects. However, this cautionary finding is at least partly superseded by the rigorous studies conducted by J-PAL in this area. Third, evidence from project evaluations have shown that demand stimulation alone may not be successful where supply is insufficient (in quantity or quality). Fiszbein (2010) finds that conditional cash transfers34 have been highly effective in encouraging demand for school attendance, and that Annex | Literature Review on Results-Based Approaches 58 school attendance did increase significantly. However, actual learning outcomes did not improve: this was the case both in Mexico and Cambodia. In both countries, children who stayed at school for an additional year due to a CCT program did not outperform children who left school a year earlier. Fiszbein (2010) interprets this as “evidence that weaknesses in the supply of services are a limiting factor to the effectiveness of CCT programs.” Brenzel (2009) supports this finding, citing a voucher scheme in India that led “to a massive increase in utilization of institutions for delivery, but in some areas, this demand may outstrip the ability of the health system to cope.” Fourth, there is strong evidence that recipient buy-in is highly important for the success of any project supported through an RBF intervention. Toonen et al. (2009), in a multi-country review of RBF-type instruments find that the “buy-in from national governments is one of the major determinants that will influence the progress and scale up of performance-based financing.” This is supported both by positive results where strong buy-in was present, and by negative results where it was absent: Brenzel and Naimoli (2009) find that “success in Rwanda with Health Results-Based Financing was linked to the strong political commitment of the President,” but also that “lack of ownership by local government authorities [. . .] hampered project implementation in Indonesia.” Fifth and last, there is cautionary evidence concerning the additional costs associated with RBF instruments in general, and with M&E and verification in particular. Thus, Naimoli, Brenzel, and Sturdy (2009) point out that high quality monitoring is important, and that “an adequate budget should be set aside to support the costs associated with this critical task.” Oxman and Fretheim (2008), discussing the magnitude of costs, estimate that “the flows of money required for RBF interventions may be substantial, including [. . .] administrative costs.” Savedoff and Martel (2011) agree, explicitly asking whether the costs of measurement and verification are manageable. Toonen et al. (2009) supplement these critical statements with quantitative evidence. In healthcare projects where administrative costs could be separated out, they turned out to be considerable: between 15 and 30 percent of total costs of the project, with one project in Democratic Republic of Congo reporting that administrative costs amounted to 40 percent of total costs (Toonen et al. 2009). However, these figures include all administrative costs, not just the additional costs associated with improved M&E and independent verification. Thus, they do not allow for straightforward cost comparison between RBF and other instruments, nor do they sustain the conclusion that RBF instruments are necessarily more costly than traditional input-based approaches. An evaluation of an OBA scheme for improving access to clean water in India allows for a more instructive comparison between these two types of aid, finding that the additional costs of hiring an independent verification agent amounted to 6.25 percent of total project costs (GPOBA 2012b). Furthermore, while it may be the case that RBF instruments have slightly higher administrative costs, they may still be more cost-effective overall. As Toonen et al. (2009) point out, the relevant comparison is between total costs per unit of desired outcome for different instruments: Loevinsohn (2008) supports the view that RBF may be more cost effective overall, despite potentially higher M&E costs. In a randomized controlled trial in Cambodia, a minimum package of health care—immunization, family planning, antenatal care, provision of micronutrients, and basic curative services—was provided Results-Based Financing in the Energy Sector | An Analytical Guide 59 either through traditional government provision, or competitively tendered out to nongovernmental organizations (NGOs), with contracts featuring financial bonuses for exceeding baselines and financial penalties for underperformance. The results-based contracted provision of health care in Cambodia came in at US$ 19.12 per head per annum, while non-incentivized government provided health care cost US$ 21.69 per head per annum. In addition, the uptake and quality of care was reported to have been considerably higher for results-based contracted-out health care (Loevinsohn 2008). However, this study does not give conclusive evidence either. It may be the case that cost savings occurred not because of the incentive structure that was part of the contracting out, but due to prior cost advantages of the contracting NGOs. Summing up this fifth cautionary finding, RBF may or may not be more costly than traditional input-based aid. However, the evidence on this matter is currently too weak to reach a decisive judgement either way, and further research is required. Similarly, limited empirical assessment of AMC instruments is available due to their relative novelty. The largest AMC in public health, for the development of the pneumococcal vaccine, is considered a pilot project for AMC use more generally and, as such, frameworks for assessing the impact of this AMC are an essential part of its implementation (Monitoring and Evaluation Subgroup of the AMC Donor Committee 2008). The aim of the AMC is primarily to incentivize the supply of vaccines for the target countries and it has resulted in two agreements to supply 30 million doses annually for 10 years starting in 2012/13. The preliminary findings from evaluation reports suggest that it is difficult to generalize from the use of AMC in the pneumococcal vaccine development to AMCs more generally and that AMC application and effectiveness is heavily dependent on circumstance (Cernuschi et al. 2011). The pilot has also illustrated some of the challenges of RBF design, including the design of a single AMC incentive mechanism for vaccine suppliers with different cost functions and in different stages of vaccine development (Monitoring and Evaluation Subgroup of the AMC Donor Committee 2008). A recurring theme in the empirical literature about RBAs is that “performance incentives are by no means a panacea” (Eichler & Levine 2009), and that they work well only under certain circumstances. However, work on what those circumstances might be remains largely lacking. Negative empirical findings In addition to positive and cautionary findings, the empirical literature has revealed two negative results that seem ingrained within the structure of RBAs. First, the need for prefinancing can be a serious challenge, and to the extent that results-based instruments genuinely make disbursement conditional upon the verification of results, it seems unavoidable. Mumssen, Johannes, and Kumar (2010), for example, point out that “dealers are reluctant to extend credit to rural customers with little credit history, and credit administration and collections may be costly.” While conventional upfront aid can avoid this problem by eliminating the need to extend credit, RBF approaches cannot do this, since aid is only disbursed upon verification of the result. Thus, prefinancing both drives up total costs by increasing financing costs, and Annex | Literature Review on Results-Based Approaches 60 reduces the reach of the RBF instrument by potentially excluding those households that are most in need. However, while this is true of any one RBF instrument seen in isolation, it may not be true when multiple RBF approaches are suitably combined: a second RBF approach could be used to incentivize lenders to provide upfront capital, in order to mitigate or completely overcome the problem of prefinance. On the upside, this facilitates the emergence of financial markets, more so than grants do, as local lenders are not competing with donor grants but are explicitly targeted with an additional incentive. On the downside, this will add to the overall costs of the program. A further consideration is that although the availability of credit may be a challenge at the time of designing the RBF, the RBF itself may help to stimulate the growth of credit markets. Second, avoiding perverse incentives may be challenging in practice. Whenever the desired outcome cannot be measured directly (or only at prohibitive costs), this may be an unavoidable problem. A good example of this is DFID’s Ethiopia COD aid pilot. This was a scheme where the UK agreed to pay the government of Ethiopia for improvements in learning. Initially, the number of students sitting for a particular exam was considered as the disbursement-driving indicator, because the main alternative (number of people passing the exam) was seen to “create public pressure for improved test scores over time” (Birdsall & Perakis 2012), which in turn may lead to a slow degradation of exam and learning standards. However, it is unclear how good a proxy sitting for the exam is for “student learning, the real outcome of interest” (Birdsall & Perakis 2012). Furthermore, paying for exam attendance may incentivize increasing the quantity of students while neglecting to maintain quality standards. Ultimately, DFID opted for a dual incentive structure, where GBP 50 to 100 (depending on gender and location) are paid for each student who sits the exam, and an additional GBP 50 to 100 (same scale) are paid for each student who passes the exam. Even with this structure, worries remained: independent of how DFID chooses to structure its payments, pressure to cheat or reduce test quality may result from the way in which the Ethiopian government decides to distribute the additional funds. Distribution by performance “would increase the stakes and thus the incentive for cheating—and the more so the lower the level at which performance is rewarded” (Birdsall & Perakis 2012). This problem occurs because COD aid is a funding relationship between a donor and a recipient government—which has discretion as to how to distribute the additional funds that it receives for the achieved results. While this second-layer incentive problem primarily applies to RBA, it may still be present for RBF instruments in certain situations. In sum, avoiding perverse incentives is challenging where the desired outcome cannot be measured directly. While this does not apply to all types of RBAs or to all relevant circumstances, it may be a serious problem for a subset of these. Summary of empirical results In sum, the empirical evidence suggests four things. First, the evidence suggests that RBF approaches can indeed lead to improved outcomes. Second, J-PAL’s studies in particular show that the success of RBF instrument depends both on wider circumstances and on specific design features; within design Results-Based Financing in the Energy Sector | An Analytical Guide 61 structures, monitoring and accountability structures are found to be of particular importance. Third, a range of studies produced cautionary evidence that suggests that RBAs are no panacea, for the following five reasons: 1 | Some of the most important elements of RBF (M&E, and verification) are both difficult and potentially expensive. 2 | There are reasons for which the initial evidence on RBF may overstate the benefits of RBF. 3 | RBF interventions that target only supply (or only demand) may fail if demand (or supply) is not also addressed. 4 | RBF interventions depend crucially on recipient buy-in, both at local and national levels. 5 | It is not yet clear if the additional costs of RBF are a serious concern or acceptable. Fourth and lastly, various studies suggest that two of the most critical, though not insurmountable, challenges facing RBF are prefinancing and avoiding perverse incentives. Findings on Prizes Three major publications on the effectiveness of prizes as policy instruments were identified for this literature review (Everett 2011; McKinsey 2009; Vivid Economics 2007). The findings are structured into the following four categories: (i) the economic characteristics and functions of prizes, (ii) and (iii) the advantages and disadvantages of prizes, and (iv) a comparison between prizes and alternative public policy instruments, including an account of when to use prizes. Economic characteristics of prizes First, the economic characteristics and functions of prizes, as identified by Vivid Economics (2007) are the following: they provide a financial incentive, which serves to lever investment towards specific purposes; that in the process of competing for a prize, investors, entrepreneurs, inventors, and potential future customers are brought together; they encourage commercialization of innovations, both through the matching discussed above, but also through signalling the most promising innovations. Finally, prizes also have broader sociological impacts, such as the popularization of a particular challenge, which may further strengthen the financial, matching, and commercialization effects of prizes. Everett (2011) points out that prizes can be classified in various ways, the simplest distinction being between recognition and inducement prizes. Recognition prizes exist in order to honor or recognize an already achieved success; inducement prizes aim to induce a certain innovation, feat, or other outcome. The category of prizes that is of most relevance to RBF is that of inducement prizes. Advantages of prizes Prizes have at least four specific advantages: first, they are capable of delivering very specific results, thus, filling particular gaps in technology or provisions. This is due to two factors: on the one hand, it is due Annex | Literature Review on Results-Based Approaches 62 to the targeted nature of the incentive, which is only disbursed upon fulfilling the prize-winning conditions; but on the other hand, to the extent that the prize is well publicized, it is due to engaging the public at large in the search for a way of winning the prize, potentially tapping a larger knowledge base than traditional R&D (McKinsey 2009). One of the clearest examples of this is the Longitude Prize: as pointed out in Everett (2011), conventional wisdom at the time thought that the solution to determining longitude would be found in astronomy. However, the eventual prize winning solution used time differences to solve the problem. The prize may have considerably sped up the discovery of this solution, since a more hands- on approach to publicly funded research would in all likelihood have focused on the wrong area. Second, prizes exhibit a particularly attractive risk profile for public sector. They come with a fixed cost risk, and place delivery risk firmly on the private sector (Vivid Economics 2007). To the extent that the private sector is better able to bear delivery risk, this is not just an attractive risk profile from the point of view of the public sector, but also an advantageous social allocation of risk. Third, and related to the shift in risk allocation, prizes shift the level at which governments and other funding institutions (for example, universities or large corporations) have to ‘pick winners.’ In classic input-funded R&D, the funding institution has to pick at the level of a particular technology or research team; with a prize, this act of picking winners is shifted up to the level of picking a problem. While there is still a significant risk of government picking the wrong problem to solve, this risk is likely lower than the compounded risk of government picking both a problem to solve and favoring one particular solution or team with research grants. Fourth, prizes also perform well in leveraging funds from the private sector: a prize of US$ 25,000 was shown to trigger US$ 400,000 in private sector investment (Vivid Economics 2007). Taking the costs of running the prize (include the prize itself) as the investment, and the spending undertaken by competitors in order to compete for the prize as the return on investment (RoI), the Shell Springboard prize achieves a RoI of between 200 and 900 percent; the RoI is 900 percent if all prize-relevant investment is attributed to the prize competition. Recognizing that part of this investment would have occurred in the absence of the prize, a lower attribution rate may be correct. Nevertheless, even with a low attribution estimate of 20 percent of prize-relevant spending, the Shell Springboard prize achieves an RoI of 200 percent (Everett 2011). RoI for the Global Security Challenge is even more impressive, ranging from a low of 6.5 (assuming 20 percent attribution) to a high of more than 33 (assuming 100 percent attribution). Everett presents similarly impressive RoIs and leverage ratios for a number of further prizes and competitions. Disadvantages of prizes The positive features identified above are counterbalanced by a number of critical features: in particular; prizes were only shown to be effective if they are both of the right financial size and designed well (Vivid Economics 2007). Designing a prize well can be challenging: calibrating the prize reward, setting the target and defining the prize-winning conditions, setting the entry conditions, choosing judges, establishing the necessary administrative infrastructure, and advertising the prize effectively all need to be addressed. Results-Based Financing in the Energy Sector | An Analytical Guide 63 In addition to the challenges of setting up a prize, it will only be effective if those who wish to compete for the prize can mobilize funding (Everett 2011). Thus, only a limited number of innovators may be able to secure the R&D funding to enable them to compete for the prize. It may be possible to design prizes to overcome these challenges by utilizing staged milestone-based, pay-for-success approaches. An example is the Global Health Innovation Quotient Point-of-Care Diagnostics Prize (2011). The use of multiple prizes based on milestones in the development of the diagnostic tool may encourage the participation of a larger group of participants than a single prize. However, these milestone-based approaches are somewhat deterministic in their approach in regards to how the goal of the project should be achieved and may potentially reduce innovation. The success of such approaches depends on whether the overall goal of the prize can easily and efficiently be broken down into milestones. However, if the problem can easily and efficiently be broken into milestones, marketplaces, such as Innocentive,35 exist to facilitate the use of these prize strategies. Furthermore, even if a prize has succeeded in stimulating a desired innovation, a widespread adoption of this innovation may be hindered by further obstacles. The market failures that reduce innovation activity may be different from those that hold back widespread deployment. Solving the first type of market failures therefore does not guarantee the solution of the second kind of market failures. This is particularly likely for innovations with public good character where private benefits are below average cost of provision: even when shown to be technologically viable, such an innovation will not be adopted without further policy support as individual benefits are below costs (Everett 2011). One particularly relevant example mentioned by Everett is this: “The development of off-grid illumination devices through a prize process has had a limited take-up in the developing world (other than through donation schemes), despite considerable social benefits.” Comparison between prizes and alternative policy instruments Combining these advantages and disadvantages, prizes compare favorably with alternative instruments (e.g., regulation or subsidies) in the following ways: • they have lower monitoring costs than subsidies (Vivid Economics 2007); • they have a more attractive risk profile than subsidies (for the public sector, though not, of course, for the private sector); this will be to the benefit of society as a whole whenever the private sector is better able to handle risks than the public sector (Everett 2011; Vivid Economics 2007); • they provide a more direct and frontloaded financial incentive than the patent system (Vivid Economics 2007); • they tap a wider knowledge base than subsidies and the patent system (Everett 2011); and • they are better at raising publicity than either subsidies, regulation, or the patent system (Everett 2011; Vivid Economics 2007). However, prizes are inferior to alternative policy instruments in achieving certain purposes: once a technological breakthrough has been achieved, regulation may be more effective at encouraging wide Annex | Literature Review on Results-Based Approaches 64 roll-out and technology diffusion. Regulation and prizes may hence complement each other when deployed in succession. Regarding the choice between prizes and other policy instruments then, it seems clear that prizes are best used for enticing innovation, be it technological, operational, or otherwise, while other policy instruments may be more appropriate for encouraging scaling up and technology diffusion. With regards to eliciting innovation, McKinsey (2009) points out that prizes are best used under three conditions: (i) if the policymaker can identify a clear goal or challenge; (ii) if this challenge can attract a large number of potential competitors for the prize; and (iii) if potential competitors are willing and able to absorb appropriate amounts of risk. Conclusion on prizes In sum, Vivid Economics (2007) finds prizes well suited to achieving certain public policy goals, in particular triggering key bottleneck innovations that may be held back by a variety of market failures, at low costs. McKinsey (2009) agrees with this, and points out that prizes are particularly well suited whenever the problem is clearly defined, a large number of competitors can be attracted, and competitors are willing and able to bear risk. Everett (2011) points out that, while the optimal level of resources that should be devoted to prizes may not be clear, it is “surely much larger than at present.” Finally, both point out that prizes are not an alternative to other public policy interventions, and that combinations of instruments may be needed to address certain challenges. Results-Based Financing in the Energy Sector | An Analytical Guide 65 ENDNOTES 1 Throughout, we use the language of ‘principal’ to refer to whoever is responsible for designing and implementing an RBF scheme and agent to refer to those who respond to the RBF scheme. 2 ‘Conventional’ refers to an intervention in which payment is not made ex post contingent on delivery of a particular set of results. 3 However, it may be possible to use an RBF placed on financial institutions, that is, ‘upstream,’ to increase the availability of capital for other RBF-incentivized projects. 4 Although this will not be a problem if the agent is able to secure financing from outside the country. 5 See, for instance, http://www.rbfhealth.org/rbfhealth/ 6 Further details available from the ESMAP website at http://www.esmap.org/Results_Based_Approaches 7 As opposed to an ‘inductive’ approach in which the applicability of (certain forms of) RBF mechanisms is assessed through analysis of the success and failure of previous interventions. The two approaches are a complement to one another. 8 Throughout, we refer to conventional forms of support as being transfers that are made that are not linked to the delivery of a certain set of results. Some of the most common forms of conventional support may be grants, concessional loans, or payments on contracts that are not linked to a specific set of results. 9 The Case Studies for this report are available at http://www.esmap.org/esmap/node/2726 The main exception to this is where a program has been officially labelled using an RBF term, most notably 10 GAVI’s Advance Market Commitment (AMC) for the pneumococcal vaccine. 11 We recognize that the term ‘finance’ in Results Based Finance fails to reflect the difference between a financial transaction and a commodity purchase. However, since RBF is an already established term, we decided to use it as the umbrella for the wide variety of results-conditional interventions that we consider in this report, rather than add yet another term to the clustered terminology. 12 Even in countries where CO2 emissions are priced. 13 That is, energy consumed from solar lanterns, kerosene lamps, and other forms of domestic lighting. By contrast, in situations where the costs of the mandated technology are lower than that of the original 14 technology—but where other barriers hold back penetration of the more energy efficient technology—mandates may be a more appropriate instrument than subsidies. 15 The definition of ‘conventional’ will depend to a certain extent on the context. In the case of subsidies, a conventional approach might be the provision of an upfront grant or concessional financing (with a certain grant- equivalent amount). In the case of principal procurement of goods and services, a conventional approach would involve payment based on inputs used. In both cases, the key aspect is that there is very little or no attempt for transfers to be contingent on the results provided by the agent. The same logic also applies when donors/development partners are considering whether to provide conventional 16 aid (or lending support) or results-based aid (lending). The latter will transfer more risk to the recipient than the former. 17 Although these are genuine challenges to the introduction of RBF, it should be noted that recent advances in information technology have significantly improved the potential for measuring and verification, expanding the set of results that can easily be measured. For instance, one of the challenges that many Least Developed Countries have identified as holding back access to 18 carbon credit markets (which are a form of RBF) is the relative scarcity of Designated Operating Entities (DOEs) who can monitor and verify purported emission reductions (United Nations Environment Program Finance Initiative 2011). Endnotes 66 Although, in theory, a number of different approaches might be supported by conventional approaches through 19 consciously adopting a portfolio approach to funding decisions (akin to a venture capital firm that deliberately provides capital to a range of, sometimes competing, technologies), which may be politically difficult and choices will still be restricted to those available and put forward at the time of the initial funding decision. 20 In the language of an energy results chain, as far ‘up’ that value chain as possible. 21 In the language of a results chain, an ‘output’. 22 http://beerecl.com/cms/?q=en 23 While household electricity consumption in developed countries may be relatively insensitive to macroeconomic or other external factors, demand for power may be much more sensitive to external factors in communities where it is perceived as a luxury good. Although this may create a perverse incentive where the agent could build connections that do not reflect the 24 actual energy needs of the area. This could be mitigated by identifying in advance the areas in which the RBF would apply. Below we discuss how an RBF intervention could be administered either on the supply or demand side of the 25 market. However, in both cases, the same quantum of RBF payment should lead to the same market outcome. Auctions could also be particularly useful when different agents have different cost structures with the auction 26 used to discriminate the amount of subsidy required by different agents. 27 A characteristic that makes RBF instruments more difficult, regardless of how they are implemented. 28 Except in the case where the principal can price discriminate. This is discussed further below. 29 Results-based instruments are increasingly being deployed by developed country governments in domestic policy, for example, by the UK in a range of areas, including the distribution of social benefits and the rehabilitation of offenders. 30 Or possibly direct procurement. 31 Prizes are a results-based instrument in which the achievement of a predefined goal is rewarded with an incentive (usually, though not always, a lump-sum cash payment). Examples of such prizes include the Longitude Prize offered by the British government in 1714, or the Ansari X prizes for space flight. 32 http://www.rbfhealth.org/rbfhealth/ AMCs are temporary interventions to make revenues from markets more lucrative and more certain in order to 33 accelerate investment (Vivid Economics 2010). 34 CCT programs ‘provide cash payments to poor households that meet certain behavioural requirements, generally related to children’s health care and education’ (GPOBA 2011a). 35 http://www.innocentive.com/ Results-Based Financing in the Energy Sector | An Analytical Guide 67 References Barr, R., Fankhauser, S., & Hamilton, K. (2010). Adaptation Investments: A Resource Allocation Framework. Mitigation and Adaptation Strategies for Global Change, 15(8), 843–858. doi:10.1007/s11027-010-9242-1 ­ Birdsall, N. (2012). Cash on Delivery Aid: For Energy Access? What is COD Aid? Presentation given in Washington DC. Birdsall, N., & Perakis, R. (2012). Cash on Delivery Aid: Implementation of a Pilot in Ethiopia, (pp. 1–8). Birdsall, N., & Savedoff, W. D. (2011). Cash On Delivery—A New Approach to Foreign Aid. Washington DC: Center for Global Development. Booth, D. (2011). Aid, Institutions and Governance: What Have We Learned? Development Policy Review, 29, 5–26. Brenzel, L. (2009). Taking Stock: World Bank Experience with Results-Based Financing (RBF) for Health. ­ Washington DC: World Bank Group. Brenzel, L., & Naimoli, J. (2009). Health Results-Based Financing (RBF) in Practice: Ten Institutional Challenges (pp. 1–3). Washington DC: RBF Health. Butler, L., & Neuhoff, K. (2008). Comparison of Feed-in Tariff, Quota and Auction Mechanisms to Support Wind Power Development. Renewable Energy, 33(8), 1854–1867. Canavan, A., Toonen, J., & Elovaino, R. (2008). Performance-Based Financing: An International Review of the Literature. Amsterdam: Royal Dutch Tropical Institute. Cashin, C., & Chi, Y.-L. (2011). Australia: The Practice Incentives Program (PIP). Washington DC: RBF Health. Center for Global Development. (2005). Making Markets for Vaccines—Report of the Working Group. Washington DC. Center for Global Development, & Deutsches Institut für Entwicklungshilfe. (2012). Summary of “Results-Based Aid: Workshop on implementing Cash on Delivery Aid and other outcome-oriented approaches.” Workshop held in Bonn. Cernuschi, B. T., Furrer, E., Mcadams, S., & Schwalbe, N. (2011). The GAVI Alliance Pneumococcal Advance Market Commitment: Lessons Learnt on Disease and Design Choices and Processes. DFID (Department for International Development). (2012). DFID Pilots on Payment by Results. London: Department for International Development. Duflo, E., Greenstone, M., Pande, R., & Ryan, N. (2012). Evaluations—Improving Third-Party Audits and Regulatory Compliance in India. Cambridge, MA: Abdul Latif Jameel Poverty Action Lab. Duflo, E., Hanna, R., & Ryan, S. P. (2012). Incentives Work: Getting Teachers to Come to School. American Economic Review, 102(4), 1241–1278. ESMAP. (2012). Results-Based Approach in the Energy Sector: Learning Event. Washington, DC: The World Bank. Eichler, R., & Levine, R. (2009). Performance Incentives for Global Health: Potential and Pitfalls. Washington DC: Center for Global Development. Energy+. (2011). An International Initiative to Accelerate Efforts to Promote Universal Access to Energy, Energy Efficiency and Low carbon Development (pp. 1–2). Everett, B. (2011). Evidence Review—Environmental Innovation Prizes for Development. London: DEW Point. Farlow, A., Light, D., Wahoney, R., & Widdus, R. (2005). Concerns Regarding the Center for Global Development Report “Making Markers for Vaccines.” Fiszbein, A. (2010). Using Financial Incentives to Achieve Education Results: A Review of Evidence. Washington DC: World Bank Group Working Paper. 68 Ghosh, A., Müller, B., Pizer, W., & Wagner, G. (2012a). Mobilizing the Private Sector: Quantity-Performance Instruments for Public Climate Funds. Oxford Energy and Environment Brief. Ghosh, A., Müller, B., Pizer, W., & Wagner, G. (2012b). Oxford Energy and Environment Brief Mobilizing the Private Sector. Oxford Energy and Environment Brief (August). Global Fund, The. (2012). Performance-Based Funding. Retrieved June 29, 2012, from http://www.theglobalfund.org/en/performancebasedfunding/ GPOBA (Global Partnership on Output-Based Aid). (2011a). Output-Based Aid in the Results-Based Financing Universe. Washington DC: World Bank Group. GPOBA. (2011b). 2011 Annual Report. Washington DC: World Bank Group. GPOBA. (2012a). Lessons Learned: Uganda Reproductive Health Voucher Scheme. Washington DC: World Bank Group. GPOBA. (2012b). Lessons Learned: Improved Rural Community Water in Andhra Pradesh. Washington DC: World Bank Group. Hansen, P. (2011). Perspectives on Performance Based Financing within the GAVI Alliance. Presentation given at Clermont-Ferrand, 11 May 2011. Hecht, R., Palriwala, A., & Rao, A. (2010). Innovative Financing for Global Health: A Moment for Expanded US Engagement? Holmqvist, G. (2012). External Financing of Social Protection: Opportunities and Risks. Development Policy Review, 30(1), 5–27. International Energy Agency. (2008). Deploying Renewables: Principles for Effective Policies. Paris: OECD/IEA. J-PAL. (2012). Policy Lessons—Teacher Attendance. Cambridge, MA: Abdul Latif Jameel Poverty Action Lab. Johannes, L., Mimmi, L., & Mumssen, Y. (2010). A Snapshot of the OBA Universe. OBA Approaches, (33). Klemperer, P. (2002). What Really Matters in Auction Design. Journal of Economic Perspectives, 16(1), 169–189. doi:10.1257/0895330027166 Kumar, G., & Mumssen, Y. (2010). Output-Based Aid and Energy: What Have We Learned So Far? OBA Approaches, (39). Loevinsohn, B. (2008). Performance-Based Contracting for Health Services in Developing Countries. Washington DC: World Bank Group. McKinsey. (2009). “ And the winner is . . . .” Mimmi, L. (2012). Should We Re-evaluate What to Evaluate (and How)? Washington DC: Global Partnership on Output-Based Aid. Monitoring and Evaluation Subgroup of the AMC Donor Committee. (2008). Advance Market Commitment for Pneumococcal Vaccines Report of the Monitoring. Morgan, L. (2010). Some Days Are Better Than Others: Lessons Learned from Uganda’s First Results-Based Financing Pilot. Washington DC: World Bank Group. Mumssen, Y., Johannes, L., & Kumar, G. (2010). Output-Based Aid: Lessons Learned and Best Practices. Washington DC: World Bank Group. Naimoli, J. (2009). Contributing to Global Learning About Results-Based Financing (RBF): The Health Results Innovation Trust Fund (pp. 1–3). Washington DC: RBF Health. Naimoli, J., Brenzel, L., & Sturdy, J. (2009). Thinking Strategically About Monitoring Health Results-Based Financing (RBF) Schemes: Core Questions and Other Practical Considerations. Washington DC: RBF Health. Neal, C. (2012). “Simplicity” Key to Results-Based Aid & Financing. Retrieved June 29, 2012, from http://blogs.worldbank.org/voices/simplicity-key-to-results-based-aid-financing Results-Based Financing in the Energy Sector | An Analytical Guide 69 Op de Coul, M., Huntjens, E., Brinkmann, V., Feldmann, L., & Heine, R. (2010). Energising Development—Report on Impacts. Eschborn: GTZ and NL Agency. Oxman, A., & Fretheim, A. (2008). An Overview of Research on the Effects of Results-Based Financing. Oslo: Nasjonalt Kunnskapssenter for Helsetjenesten. Pearson, M. (2011). Results Based Aid and Results Based Financing: What Are They? Have They Delivered Results? London: HLSP Institute. Philibert, C. (2011). Interactions of Policies for Renewable Energy and Climate. International Energy Agency Working Paper. Prendergast, C. (1999). The Provision of Incentives in Firms. Journal of Economic Literature, 37(1), 7–63. Savedoff, W. D. (2010). Basic Economics of Results-Based Financing in Health (pp. 1–18). Bath, ME: RBF Health. Savedoff, W. D., & Martel, K. D. (2011). Cash On Delivery Aid for Health: What Indicators Would Work Best? CGD Working Paper 275. Toonen, J., Canavan, A., Vergeer, P., & Elovaino, R. (2009). Learning Lessons on Implementing Performance Based Financing, From a Multi-County Evaluation. Amsterdam: KIT (Dutch Royal Tropical Institute). Tremonti, G. (2005). Background Papers to Advanced Market Commitments for Vaccines: A New Tool in the Fight Against Disease and Poverty. United Nations Environment Programme Finance Initiative. (2011). And Yet it Moves. Success Stories and Drivers of CDM Project Development in Sub-Saharan Africa. van den Looij, F. (2009). Performance-Based Financing (PBF) within the Catholic Organisation for Relief and Development Aid (CORDAID). Washington DC: RBF Health. Vivid Economics. (2007). The Economics of Climate Change Prizes. London: Vivid Economics. Vivid Economics. (2010). Advance Market Commitments for Low-Carbon Development: An Economic Assessment. Vivid Economics. World Bank. (2011). Program-for-Results: An Overview. Retrieved from http://siteresources.worldbank.org/EXTRESLENDING/Resources/7514725-1313522321940/PforR_ Overview_12.2011.pdf World Bank. (2012). Doing Business, “Getting Credit” Topic. Washington DC. Retrieved from http://www.doingbusiness.org/data/exploretopics/getting-credit World Bank Operations Policy and Country Services. (2011). A New Instrument to Advance Development Effectiveness: Program-for-Results Financing. Washington DC: World Bank Group. References 70 ENERGY SECTOR MANAGEMENT ASSISTANCE PROGRAM THE WORLD BANK 1818 H STREET, NW WASHINGTON, DC 20433 USA EMAIL: ESMAP@WORLDBANK.ORG WEB: WWW.ESMAP.ORG Photography Credits Cover, Inside Front Cover | Pepukaye Bardouille / © International Finance Corporation Table of Contents | Arne Hoel / © The World Bank Page iv | © iStockphoto Page 8 | © Arunabha Ghosh / Council on Energy, Environment and Water Page 12 | Barefoot Power (© Ashen International Award Winner 2012) PRODUCTION Credits Production Editor | Heather Austin DESIGN | circle graphics, inc. REPRODUCTION | automated graphic system, inc. Results-Based Financing in the Energy Sector | An Analytical Guide 71