June 2016 Note Number 50 Getting Results: Independent Verification in Output-Based Aid A s interest has increased in results-based financing In an OBA project, service delivery is contracted out to a (RBF), an approach to development work that shifts third party, either a government or private sector en- the focus from inputs to outputs through use of tity, who provides pre-financing and receives a subsidy performance-based incentives or subsidies, the need for reli- to complement or replace user fees once outputs have able verification systems has become all the more relevant. been verified by an independent verification agent (IVA). Independent verification is a key component of output-based Outputs could be the installation of solar home systems, aid (OBA), a form of RBF that links the disbursement of provision of solid waste management collection, the con- funds to the achievement of specified results. The verification nection of households to water supply systems, delivery of process is built into OBA project design, helping to integrate specific health interventions, or other services. monitoring into the project cycle. This note discusses aspects Because verification is linked with subsidy disburse- of the verification process, and offers case studies of projects ment, the IVA can improve service quality and account- in Indonesia and Kenya that are making innovative use of ability, providing assurance that funds have been used for the independent verification mechanism. the intended purpose. Although verification is ultimately focused on outputs, it can be ongoing throughout the Supporting the delivery of basic services in developing countries Box 1. Supporting Independent Verification in Indonesia One of GPOBA’s largest technical assistance activities to date—the support for the IVA in the World Bank’s Local Govern- ment and Decentralization Project (LGDP) in Indonesia—is a pilot approach in which government auditors are converted to IVAs, supported and monitored by civil society. The LGDP includes an output-based disbursement approach: a portion of the central government’s grants for basic infrastructure in the transport, irrigation, water and sanitation sectors in selected local governments is reimbursed to the Ministry of Finance following output verification. Criteria includes physical output delivery, technical quality, and compliance with a range of regulations. The IVA in this case is the Indonesian National Government Internal Auditor (BPKP). Prior to the project, which started as a pilot in 2011, the government had no mechanism for verifying outputs of the infrastructure grants. The GPOBA grant provides (i) capacity building and technical assistance to the IVA to strengthen monitoring and evalu- ation of outputs using GPOBA methodological materials, and (ii) technical assistance to strengthen municipal management and raise citizen awareness of government accountability. So far, 213 BPKP staff have been trained, and the project now covers 209 local governments; it will be scaled up nationwide to cover around 460 local governments by 2018. The training methodology will be institutionalized within BPKP’s training center curriculum, enabling its replication for wider audiences. Two verification handbooks for BPKP and local government have been developed that make clear the quality required in infrastructure in order to pass IVA inspection. This new verification system reinforces accountability, enabling the benefits of infrastructure and service provision to be enjoyed by local citizens. All project implementation units and functions are already embedded within local government institutions. This strength- ening of local capacity supports the government’s ability in the future to verify outputs from other investment projects. project cycle, so that shortcomings or obstacles can be developed the Multi-tier Framework (MTF) to monitor identified and corrective actions put in place before the and evaluate energy access. The MTF redefines energy point of final verification. access from a simple yes/no count to a multi-dimensional The IVA can be an audit firm, an NGO or civil soci- assessment that includes duration, reliability and quality ety representative, a qualified individual consultant, or a of supply; affordability; legality; safety; and other factors. government agency. The IVA ideally possesses technical Energy access is then graded on a tiered-spectrum from sector expertise and financial audit expertise, as well as zero to five. experience working in poor communities, and—especially for rural projects—an understanding of local culture and Scale and Frequency: Seeking 100 percent verification language. Most OBA projects have used independent con- of outputs is not feasible, and statistical sampling is often sultants, either local or international, to enhance transpar- the most economical and effective approach, with care ency. Generally speaking, a long-term goal of strengthen- taken to ensure that the sample adequately represents the ing and capacity-building of country systems to perform universe. Many projects in the infrastructure sector have a verification is desirable. [see Box 1] two-stage verification and disbursement process—the first being the service provision (eg, a working electricity con- Key Considerations nection), and the second taking place after several months of verified service, involving a review of billing and col- Indicators: Setting indicators should be an inclusive lection records. In other sectors, such as health, educa- process, involving a realistic assessment of the reporting tion, or SWM, verification and disbursement are ongoing. capacity and internal control systems of service providers Flexibility in the timing and frequency of verification may (SPs) and implementing agencies. Effective verification be required. In an OBA education project in Vietnam, relies on available data sources and baseline data, such for example, a more frequent verification cycle instigated as existing access to services and development indica- during project implementation allowed for more regular tors. [See Box 2] Results and indicators must be spe- disbursement of funds, helpful to SPs who had found pre- cific, measurable, and under the control of the SP. They financing difficult. should be directly linked to incentives, and aligned with stakeholder objectives and priorities. While the verifica- Balancing Simplicity and Effectiveness: While verifica- tion process should be kept simple where possible, more tion mechanisms should be kept simple, particularly where complex systems are sometimes needed to accurately local capacity is weak or has been decimated by conflict or measure access to services. For example, the Energy natural disaster, it may be advisable to complement the use Sector Management Assistance Program (ESMAP), un- of IVAs with multi-layered verification systems involv- der the Sustainable Energy for All (SE4All) initiative, has ing different stakeholders. Where risk of corruption is June 2016 Note Number 50 Box 2. IVAs Working to Prepare OBA Sanitation Projects in Nairobi Two OBA projects supporting improved access to water and sanitation services in Kenya expanded the role of IVAs to include critical baseline data collection that informed project design. The gathering of data prior to project commencement is often done through an independent consultant other than the IVA; here, IVAs performed the task, leaving them well positioned to track progress toward indicators and outcomes during project implementation. The Kayole Soweto Water Sewer Project and the Mukuru Settlement-Kenya Railways Reallocation Project are part of the Nairobi Sanitation Output-Based Aid Program for Low Income Areas, a new social services provision arrangement being implemented by Nairobi City Water and Sewerage Company (NCWSC), in collaboration with National Treasury and the World Bank Water and Sanitation Program for Africa (WSP-Af). In the Kayole Soweto village, the project is connecting 16,000 households to NCWSC’s water and sewerage network. NCWSC has obtained a commercial loan to finance the project, and an OBA grant will reimburse the company a percent- age of the cost of sewerage and water connections. NCWSC will recuperate the balance through upfront connection fees and monthly billing surcharges for up to five years. Prior to the start of the project, the IVA visited sample households and residential blocks in Kayole Soweto’s nine settlements to determine the baseline situation related to the proposed project’s eligibility for funding under the OBA program and to specific project outputs. The IVA assessed: the numbers of existing connections and households to be considered for connection; output targets and the appropriate OBA subsidy level; project costs and cost per capita; and willingness/ability of residents to pay for connection fees and supply services. In Mukuru Kwa Njenga and Mukuru Reuben settlements, home to over 100,000 people, residents living close to the railway line fall under the remit of the Kenyan government’s Railway Re-allocation Action Plan. The plan aims to provide new housing units, and partners with other government bodies to provide basic services for affected residents. As part of the OBA project to subsidize water and sewer connections for the re-allocated population, the IVA conducted a baseline survey, gath- ering data on those affected by Re-allocation Plan; the status of the construction of the re-allocation units and the readiness of Kenya Railways to move people to new premises; the number of units to be connected to water supply and sewer services; output targets; and residents’ willingness/ability to pay for connection fees and water supply services. high and accountability is low, such systems are especially inherent risk of capture. The funding entity should be relevant. protected against potential manipulation or inflation of results, which may occur when a government entity (such Avoiding Conflicts of Interest: When selecting an IVA, as the project’s implementing agency) is responsible both the aim is to avoid conflicts of interest and reduce the for overseeing SPs and for output verification. Likewise, Box 3. Using Technology in Results Verification Going forward, technology will play a greater role in the verification process, as it can reduce costs while increasing efficien- cy, rigor, and credibility. A number of results-based approaches are already making innovative use of technology in verifica- tion. OBA projects supporting installation of public phones in rural locations have used automated network management terminals, which provide alerts when a phone is out of service and collect traffic and maintenance statistics. In Maputo, a World Bank project funded by an Innovation Grant developed a participatory, results-oriented information system for urban services. The system collected information from citizens about SWM services through a web portal, mobile apps, SMS, and voice calls, and made this information publicly available. The World Bank’s Program for Results (PforR) is also piloting the use of technology in verification. In PforR’s Red River Delta Rural Water Supply and Sanitation Project in Vietnam, the task team partnered with the ICT Unit to introduce mobile data collection in the first round of verification. The benefits were immediately apparent: access to data in real time through an online database system (including mapping functionality), and the capacity to detect errors before they became system- atic. With each output geo-coded and photographed, any questions about compliance could refer back to these data points as evidence. PforR task teams are also partnering with the World Bank’s Innovation Labs to pilot other new technology-based approaches to verification in the coming year. These pilots include: low-cost sensors, which provide continuous data on func- tionality while reducing or eliminating the need to physically visit sites; verification using images, where images are combined with site visits and GPS coordinates, then analyzed for compliance by desk-based agents; and dynamic sampling, in which data collection software provides continuous feedback alerting field teams when they have reached a statistically precise level of accuracy and can stop sampling from a particular cluster. the agency that hires the IVA should not have a vested financial interest in the IVA’s performance. If the same Sources SP who hires the IVA is the direct funds recipient, there Baseline Survey Reports on Assessment and Determina- exists scope for collusion or lax application of verification tion of the Project Status for Mukuru Settlement-Kenya protocols. Railways Re-Allocation and Kayole Soweto Low-Income Risks can be mitigated by hiring either a reputable au- Settlement Projects. WSP-Af IVA, May/June, 2015. diting firm or the SP’s existing external auditors. Auditors Chase, Claire, and Vazquez, Victor. ‘Using Technology to are subject to established professional standards and un- Improve the Cost-Efficiency of Results Verification in likely to risk their credibility or license. The main solution PforR Projects: How the Water GP and Innovation Labs adopted by GPOBA has been the use of external firms or are partnering to get practical operational solutions’. consultants. In projects where the World Bank funds the The Water Blog. The World Bank, May 2016. http:// IVA’s contract, conflicts of interest are mitigated by requir- blogs.worldbank.org/water/using-technology-improve- ing the hiring entity to follow World Bank guidelines on cost-efficiency-results-verification-pforr-projects selection of consultants. ESMAP: Multi-Tier Framework for Measuring Energy Ac- cess. (https://www.esmap.org/node/55526) Note: All monetary amounts are in US$ unless stated otherwise. About OBApproaches OBApproaches is a forum for discussing and disseminating have been chosen and presented by the authors in agreement recent experiences and innovations in supporting the delivery with the GPOBA management team and are not to be attribut- of basic services to the poor. The series focuses on the provi- ed to GPOBA’s donors, the World Bank, or any other affiliated sion of water, energy, telecommunications, transport, health, organizations. Nor do any of the conclusions represent official and education in developing countries, in particular through policy of GPOBA, the World Bank, or the countries they output- or performance-based approaches. The case studies represent. To learn more, visit www.gpoba.org e Global e Partnership on Global Partnership Output-Based Aid on Output-Based Aid Supporting the delivery of basic services in developing countries