IDA19 The IDA19 Results Measurement System May 31, 2019 ACRONYMS AND ABBREVIATIONS FISCAL YEAR (FY) = JULY 1 TO JUNE 30 ASA Advisory Services and ICT Information and Analytics Communication Technologies BPS Budget, Performance Review IDA International Development and Strategic Planning Association CC Climate Change IEG Independent Evaluation Group CEN Country Engagement Note IFF Illicit Financial Flows COP Country Opinion Surveys IMF International Monetary Fund CPF Country Partnership Framework IPF Investment Project Financing CRI Corporate Results Indicator ISR Implementation Status and CRV Civil registration and vital Results Report Statistics JET Jobs and Economic CSC Corporate Scorecard Transformation ESF Environment and Social MDG Millennium Development Goals Framework M&E Monitoring and Evaluation FAO Food and Agriculture MJ Megajoules Organization OPCS Operations Policy and Country FCS Fragile and Conflict-Affected Services Situations PEFA Public Expenditure and FCV Fragility, conflict, and violence Financial Accountability GAVI Global Alliance for Vaccines PM Particulate Matter and Immunizations PPP Public-Private Partnership GBV Gender-based Violence RMS Results Measurement System GD Gender SCD Systematic Country Diagnostic GED Gender SDG Sustainable Development Goal GDP Gross Domestic Product SMEs Small and medium-sized GHG Greenhouse Gas Enterprises GI Governance and Institutions TCO2eq Total Carbon Dioxide Global Findex Global Financial Inclusion Equivalent Database UNFCCC United Nations Framework GP Global Practices Convention on Climate Change GW Gigawatt UNICEF United Nations Children's Fund HA Hectare WBG World Bank Group HCI Human Capital Index WDI World Development Indicators HIV Human Immunodeficiency WFA World Bank Group Finance and Virus Accounting HNP Health Nutrition and Population WHO World Health Organization ICR Implementation Completion Report TABLE OF CONTENTS EXECUTIVE SUMMARY ..................................................................................................................... i I. THE IDA RESULTS MEASUREMENT SYSTEM (RMS) .................................................... 1 A. Objective and Strategic Context ........................................................................................... 1 B. Structure and Components of the IDA Results Measurement System ................................. 2 II. GUIDING PRINCIPLES FOR CHANGES TO THE IDA RMS ........................................... 4 III. PROPOSED INDICATORS FOR THE IDA19 RMS ............................................................. 4 A. Tier 1 – IDA Countries’ Progress ......................................................................................... 5 B. Tier 2 – IDA-supported Results ........................................................................................... 9 C. Tier 3 – IDA Operational and Organizational Effectiveness.............................................. 17 IV. ALIGNMENT WITH THE SUSTAINABLE DEVELOPMENT GOALS ......................... 22 V. REPORTING AND COMMUNICATING IDA RESULTS ................................................. 22 VI. ISSUES FOR DISCUSSION .................................................................................................... 22 LIST OF ANNEXES Annex 1: IDA19 Results Measurement System ..................................................................................... 23 Annex 2: New and Dropped Indicators .................................................................................................. 31 Annex 3: Alignment of the SDGs with IDA19 Results Measurement System Indicators...................... 34 Annex 4: Brief background on the evolution of the IDA RMS .............................................................. 35 Annex 5: Overview of the data aggregation process for the IDA RMS ................................................. 37 LIST OF BOXES, FIGURES AND TABLES BOX: Box 1: Learning What Works and What Doesn’t Work in Tier 2 ............................................................. 16 FIGURE: Figure 1: Aggregation Methodology for Tier 2 Indicators (IDA-Support Results) ...................................... 3 TABLES: Table 1: Number of New and Retained Indicators by Tier ........................................................................... 5 Table 2: World Bank Group Corporate Goals Indicators ............................................................................. 6 Table 3: Tier 1 Indicators – Sustainable and Inclusive Growth .................................................................... 6 Table 4: Tier 1 Indicators – Human Capital ................................................................................................. 7 Table 5: Tier 1 Indicators – Resilience ......................................................................................................... 8 Table 6: Tier 1 Indicators – Institutional Capacity ....................................................................................... 8 Table 7: Tier 2 Indicators – Sustainable and Inclusive Growth .................................................................. 11 Table 8: Tier 2 Indicators – Human Capital ............................................................................................... 13 Table 9: Tier 2 Indicators – Resilience ....................................................................................................... 14 Table 10: Tier 2 Indicators – Institutional Capacity ................................................................................... 14 Table 11: Tier 3 Indicators – Development Outcomes and Effectiveness .................................................. 17 Table 12: Tier 3 Indicators – Performance and Quality.............................................................................. 18 Table 13: Tier 3 Indicators – Operational Efficiency and Responsiveness ................................................ 19 Table 14: Tier 3 Indicators – Financial Sustainability and Budget Efficiency ........................................... 19 Table 15: Tier 3 Indicators – Implementation of Special Themes .............................................................. 21 EXECUTIVE SUMMARY i. IDA countries currently face a myriad of complex and inter-related challenges, from the risk of a global growth slowdown to forced displacement, climate change, limited access to the growing digital economy and a dearth in human capital. Despite these challenges, progress made by many IDA countries demonstrates that achieving meaningful development results is possible. ii. IDA19 will build on efforts initiated during IDA17 and IDA18 as well as reflect new priorities such as the cross-cutting issues of debt, disability, technology, and human capital to accelerate progress. The commitments under the special themes in the IDA19 package are tailored towards focusing on development impact. iii. Results are at the core of the World Bank’s development work. The IDA RMS was introduced in 2002 to accompany IDA13 and was the first attempt by an international financial institution to develop a quantitative results framework to monitor results. Since its introduction, it has been periodically revised to reflect the changing nature of IDA’s business and also to incorporate lessons learned from the experience of implementing the framework. iv. The IDA RMS reports on IDA’s activities and impact in IDA countries at an aggregated level, in a three-tier format. Tier 1 indicators report on the long-term development outcomes achieved by IDA countries, and on the broader context in which IDA operates. Tier 2 indicators report the aggregated development outputs and outcomes supported by some of the IDA- financed operations. Tier 3 indicators report on measures tracking IDA’s operational and organizational effectiveness. v. In addition to the IDA RMS, other reports are useful to fully reflect IDA’s development impact and results. Since Tier 2 indicators report IDA outputs and outcomes that can be counted and aggregated, they leave out activities and operations that produce results that cannot be easily aggregated. This includes projects that do not produce quantifiable outputs but build capacity or support policy, institutional, or regulatory reform; and non-projectized work such as advisory work, analytics and policy dialogue. This work is reported in other ways, including through the results frameworks that exist for every IDA project and every Country Partnership Framework (CPF), IEG and other reviews of IDA at work, and the many stories about IDA@work published from time to time and on IDA’s website, ida.worldbank.org/results. vi. The IDA19 RMS expands and deepens the framework developed in the IDA18 RMS. Modifications in scope, format, and choice of indicators are driven by IDA’s track record, shifting client demand, data quality and availability, and fit for purpose. In proposing revisions to the RMS, three guiding principles have been observed: continuity with IDA18 to enable long-term monitoring, while also adjusting indicators to address weaknesses; relevance to IDA19 special themes and cross-cutting issues (debt, disability, technology and human capital); and alignment with global and corporate priorities. vii. The IDA19 RMS will include 74 indicators, 64 of which have been retained from the IDA18 RMS. Tier 1 will include 30 indicators, 28 of which are retained from the IDA18 RMS. Tier 2 will retain 15 indicators from the IDA18 RMS and introduce four new ones. Targets for - ii - Tier 2 indicators will be based on the expected aggregated outputs of IDA-supported operations underway during the IDA19 cycle, whether or not the underlying project was approved during the cycle. Tier 3 will include 25 indicators, 21 of which are retained from the IDA18 RMS and four of which are new. viii. The IDA19 RMS is starting to reflect the gradual shift in the IDA portfolio from projects supporting financing of outputs to more complex knowledge engagement integrated in operations that support development outcomes. For example, while IDA continues to support critical inputs and outputs in the education sector such as hiring and training teachers, a new Tier 2 indicator on IDA support for large-scale learning assessments, with important linkages to skills and jobs, reflects a corresponding shift in emphasis taking place in IDA countries. Similarly, a new transportation services indicator reflects that fact that, in recent years, IDA countries have shifted their transportation portfolios away from road construction and towards maintenance, safety, gender integration, and climate resilience, and have also diversified into other sub-sectors, such as urban transport, railways, inland waterways, and logistics. ix. The IDA19 RMS will seek to disaggregate indicators for sex and for Fragile and Conflict-affected Situations (FCS). For FCS, to ensure data comparability and consistency throughout the IDA19 cycle, disaggregation will use the list of FCS agreed at the beginning of the cycle. While further disaggregation by disability status is constrained by lack of systematic results data on disability, under the World Bank’s new Environment and Social Framework (ESF), all investment projects being prepared will be screened through the environmental and social assessment to determine if the project could have a potential impact on, or risk to, people with disability. x. The IDA19 RMS will be updated and publicly disclosed once a year. Management will report on the status of the RMS indicators annually and also provide a detailed update on progress across all tiers at the IDA19 Mid-Term Review. The IDA19 RMS will be shared and publicly disclosed on IDA’s website and the World Bank’s results page. I. THE IDA RESULTS MEASUREMENT SYSTEM (RMS) A. OBJECTIVE AND STRATEGIC CONTEXT 1. Results are at the core of the World Bank’s development work. The IDA RMS was introduced in 2002 to accompany IDA13 and was the first attempt by an international financial institution to develop a quantitative results framework to monitor results. Since its introduction, it has been periodically revised to reflect the changing nature of IDA’s business and to incorporate lessons learned from the experience of implementing the framework. Key changes included alignment with the Millennium (and later the Sustainable) Development Goals, the introduction of an indicator tier that aims to capture results in IDA countries attributable to IDA activities and another tier tracking IDA’s operational and organizational effectiveness, and the introduction of indicators that capture progress under the special themes articulated in IDA17. 2. IDA19 will build on efforts initiated during IDA17 and IDA18; capture new priorities such as the cross-cutting issues of debt, disability, technology, and human capital; and focus more strongly on raising the quality of monitoring and evaluation (M&E). IEG’s retrospective project reviews have shown that M&E quality is positively correlated with project success. IEG’s rating of M&E quality at project completion has been steadily improving over the last four (fiscal) years. To this end, the World Bank has made considerable efforts to strengthen results measurement at the country and project levels – raising operational standards (e.g., by introducing the Corporate Results Indicators (CRIs) in FY17); analyzing and disseminating lessons from IEG, impact and other evaluations; providing staff with general guidance, training, and peer learning to strengthen M&E in various operational instruments; and providing teams with project-specific knowledge and support during project preparation and implementation. That said, with IEG finding that in FY17 only about 47 percent of World Bank operations had satisfactory (or higher) quality of M&E, there is room for improvement. Hence, the World Bank will significantly enhance its efforts to help project teams articulate a sound intervention logic (causally linking project inputs, outputs and outcomes to the problems to be addressed), results framework (articulating indicators that enable progress to be monitored), and data monitoring system (providing for each indicator a baseline value and a stated plan for collecting data in a timely manner). It will also seek to obtain better information about M&E quality throughout the project cycle to shorten the feedback loop, enable better monitoring and mid-course corrections, and improve overall project success. 3. The IDA RMS reports on IDA’s activities and impact in IDA countries, at an aggregated level. Specifically, starting with IDA18, it reports results and performance indicators at three levels: • Tier 1: IDA countries’ progress: Tier 1 indicators report on the long-term development outcomes achieved by IDA countries, and on the broader context in which IDA operates. Progress against Tier 1 indicators is not directly attributed to IDA’s interventions, but to the outcome of a collective effort by IDA countries and their development partners. Tier 1 indicators are selected to align with the WBG’s corporate strategy, IDA’s strategic and priority areas, and the 2030 Sustainable Development -2- Agenda. They report data based on the list of eligible IDA borrowers at the beginning of the reporting fiscal year. • Tier 2: IDA-supported development results: Tier 2 indicators report the aggregated development outputs and outcomes that are supported through some of the IDA financed operations. These indicators mostly reflect IDA-supported results being produced by a subset of IDA projects that are already under implementation when the IDA cycle begins; that is, the results actually reported in the IDA19 RMS will mostly reflect results from projects under implementation during IDA19 but initiated under IDA17 or IDA18. • Tier 3: IDA’s operational and organizational effectiveness: Tier 3 indicators report on measures tracking the performance of the IDA’s portfolio, the quality and timeliness of projects delivered to clients, IDA’s operational efficiency, the results orientation of operations, client and beneficiary feedback, financial sustainability, and the implementation of special themes. 4. In addition to the IDA RMS, other reports are useful to obtain a more complete picture of IDA’s results and development impact. Since Tier 2 indicators report IDA outputs and outcomes that can be counted and aggregated, they leave out: activities and operations that produce quantified outputs that are unique or uncommon due to specific country context and so not suitable for use in aggregated reporting; projects that do not produce quantifiable outputs but build capacity or support policy, institutional, or regulatory reform; and non-projectized work such as advisory, analytics and policy dialogue. This work is reported in other ways, including through the results frameworks that exist for every IDA project and every Country Partnership Framework (CPF), IEG and other reviews of IDA at work, and the many stories about IDA@work published from time to time and also collected on IDA’s website, ida.worldbank.org/results. B. STRUCTURE AND COMPONENTS OF THE IDA RESULTS MEASUREMENT SYSTEM Benchmarks and performance standard/targets 5. The IDA19 RMS will include 74 indicators, 64 of which have been retained from the IDA18 RMS. Tier 1 will include 30 indicators, 28 of which are retained from the IDA18 RMS. Tier 2 will retain 15 indicators from the IDA18 RMS and introduce four new ones. Targets for Tier 2 indicators will be based on the expected aggregated outputs of IDA-supported operations underway during the IDA19 cycle, whether or not the underlying project was approved during the cycle. Tier 3 will include 25 indicators, 21 of which will be retained from the IDA18 RMS and four of which will be new. Benchmarks will reflect the latest reported value at the end of the previous fiscal year, as most such indicators are updated annually, and targets will be established based on available information. Data Disaggregation 6. The IDA19 RMS will seek to disaggregate indicators for sex and for Fragile and Conflict-affected Situations (FCS). Results will be disaggregated by sex where applicable and available. For FCS, to ensure data comparability and consistency throughout the IDA19 cycle, disaggregation will use the harmonized list of FCS at the beginning of the IDA19 cycle. -3- Data sources and aggregation 7. Sources of data vary across tiers of the IDA RMS. Data for Tier 1 indicators are sourced from large standard databases; these include (inter alia) the World Development Indicators database (WDI), UNFCCC framework, Global FINDEX, PovcalNet, and the Women, Development and the Law database, which are managed by the World Bank, external partners or multilateral development agencies (e.g., WHO, IMF, UNICEF, and FAO). The RMS relies entirely on the validation processes undertaken by the owners of these databases. For tiers 2 and 3, the annual update is based on World Bank Group data recorded by June 30, which is the end of the WBG’s fiscal year. For each indicator there is a definition, an agreed data collection and reporting methodology, and a quality assurance process agreed with the data owners responsible for collecting, aggregating, and validating final values (see Annex 5 for more details). 8. Data for most Tier 2 indicators are tracked and reported through the World Bank’s Corporate Results Indicators (CRIs). The CRIs were introduced in 2017 to facilitate the aggregation of IDA results data across projects in key development areas. CRIs are included in the results frameworks of relevant projects, and progress is periodically updated and reported through Implementation Status and Results Reports (ISRs) and Implementation Completion Reports (ICRs). The results reported in Tier 2 include outputs from active or closed IDA operations during a given fiscal year and may also include results from projects in countries that have recently graduated from IDA, provided that the results were achieved with IDA financing. Consistent with the methodology of aggregation introduced in the IDA18 RMS (see Figure 1), Tier 2 results for any specific indicator will be reported annually, on a cumulative basis 1 (that is, each report will provide running totals, starting with the results achieved in FY21 during the first year, FY21 plus FY22 data in the second year, and FY21 through FY23 data in the third and final year). Figure 1: Aggregation Methodology for Tier 2 Indicators (IDA-Supported Results) 1 In the IDA17 RMS reported outputs in tier 2 were based on a three-year rolling basis. -4- 9. Data for Tier 3 indicators is extracted from the World Bank’s information systems. These include the Operations Portal, the IEG database, Country Opinion Surveys (COPs), ISRs and ICRs. Data will be reviewed and validated in coordination with focal points in Operations Policy and Country Services (OPCS), the Independent Evaluation Group (IEG), Human Resources, the Business, Performance and Strategic Planning department (BPS), World Bank Group Finance and Accounting (WFA), External Communications, and focal points for IDA’s special themes. II. GUIDING PRINCIPLES FOR CHANGES TO THE IDA RMS 10. The IDA19 RMS expands and deepens the framework developed in the IDA18 RMS. Modifications in scope, format, and choice of indicators are driven by IDA’s track record, shifting client demand, data quality and availability, and fit for purpose. In proposing revisions to the RMS, three guiding principles have been observed: • Continuity with IDA18 to enable long-term monitoring, while also adjusting indicators to address weaknesses. Continuity is critical for monitoring and learning from evidence. Repeated revisions to indicators would disrupt data trends, make difficult the analysis of progress, and increase the calculation and reporting load, though adjustments will be made to some indicators to address experienced weaknesses. • Relevance to IDA19 Special Themes and Cross-Cutting Issues: The IDA19 RMS will reflect the five special themes (Gender and Development, FCV, Climate Change, Governance and Institutions, and Jobs and Economic Transformation) and incorporate several key IDA19 policy commitments. In addition, where relevant and possible, new indicators have been added and existing indicators adjusted to capture the four new cross-cutting issues (debt, disability, technology, and human capital). • Alignment with global and corporate priorities. The IDA19 RMS will also track existing and emerging priorities (e.g., the SDGs, the Human Capital Project, Maximizing Finance for Development through the “cascade” approach). Experience with the use, strengths, limitations, and relevance of IDA18 RMS indicators has been taken into consideration for retaining, adjusting, and introducing new indicators for the IDA19 RMS. III. PROPOSED INDICATORS FOR THE IDA19 RMS 11. The IDA19 RMS will include 74 indicators (30 in Tier 1, 19 in Tier 2, and 25 in Tier 3). Of these, 64 have been retained from the IDA18 RMS (28 in Tier 1, 15 in Tier 2, and 21 in Tier 3) and twenty have been dropped (five in Tier 1, six in Tier 2, and nine in Tier 3). Ten new ones are being introduced (two in Tier 1, four in Tier 2, and four in Tier 3). See Annex 1 for a complete list of IDA19 RMS indicators. -5- Table 1: Number of New, Retained and Dropped Indicators by Tier Tier 1 Tier 2 Tier 3 Total Indicators in IDA18 RMS 33 21 30 84 • of which have been retained 28 15 21 64 • of which have been dropped [5] [6] [9] [20] New indicators proposed 2 4 4 10 Total indicators IDA19 RMS 30 19 25 74 12. Twenty indicators from the IDA18 RMS are proposed to be dropped. These include five indicators from Tier 1, six from Tier 2 and nine from Tier 3. Indicators were dropped to accommodate shifting priorities in the IDA portfolio; improved measures or methodologies for calculating a (replacement) indicator; or emerging issues and new priorities under the IDA19 special themes. In some cases, indicators were dropped simply because experience showed them to have limited strategic value. The rationale for dropping each dropped indicator is explained in detail in Annex 2. A. TIER 1 – IDA COUNTRIES’ PROGRESS 13. Tier 1 will include 30 indicators covering key development areas. They are grouped into five categories – World Bank Group Corporate Goals, Sustainable and Inclusive Growth, Human Capital, Resilience, and Institutional Capacity – reflecting key principles established in the Forward Look and harmonizing with the World Bank Corporate Scorecard (CSC). Of the 30 indicators, 28 are retained from the IDA18 RMS and two new ones have been introduced to track domestic resource mobilization and the percentage of children who cannot read by the age at which primary school typically ends. Five indicators from the IDA18 RMS have been dropped (countries with growth concentrated in the bottom 40 percent, the Trade Logistics Performance Index, annual freshwater withdrawals, number of IDA countries that have raised the tax-to-GDP ratio above 15 percent, and population exposed to harmful air pollution (PM 2.5)). World Bank Group Corporate Goals 14. The two IDA18 RMS indicators tracking progress against the WBG’s corporate goals will be retained in the IDA19 RMS. Data for both indicators are reported each fiscal year for all IDA eligible countries, including blend countries, and will be disaggregated for FCS. -6- Table 2: World Bank Group Corporate Goals Indicators Included Disaggregation No. Indicator in IDA18 New SDG 2 Special RMS FCS Sex Theme World Bank Group Corporate Goals 1 Population living on less than US$ 1.90 (%)   1.1.1 Growth rate of real per capita income of the 2   10.1 bottom 40% of population (%) Sustainable and Inclusive Growth 15. This category includes eight indicators tracking macroeconomic measures, employment, access to electricity and to financial services, and gender parity. All indicators are in the IDA18 RMS, will be retained without modifications, and will be disaggregated for FCS. Indicators 8 and 9 will be disaggregated by sex. Table 3: Tier 1 Indicators – Sustainable and Inclusive Growth Included Disaggregation Special No. Indicator in IDA18 New SDG FCS Sex Theme RMS Sustainable and Inclusive Growth GDP per person employed (constant 2011 PPP 3   JET US$) Non-agriculture sectors, value added (as % of 4   JET GDP) Legal changes that support gender equality 5 over the past two years (number of legal gender   5.1 GD changes) Proportion of population with access to 6   7.1.1 JET electricity (% of population) 7 Annual growth rate of real GDP per capita (%)   8.1.1 Ratio of female to male labor force 8   8.5 JET participation rate (%) Youth employment to population ratio (age 15-    8.5 JET 24) (%) - Youth employment to population ratio JET, 9    8.5 (age 15-24), women (%) GD - Youth employment to population ratio    8.5 (age 15-24), men (%) Proportion of adults (15 years and older) with an account at a bank or other financial 10    8.10.2 JET institution or with a mobile money service provided (%) 2 Indicator alignment with the SDGs is denoted depending on whether alignment is at broad goal level (e.g., SDG 7); at the target level (e.g., 7.1); or at the indicator level (e.g., 7.1.1). -7- Human Capital 16. This category includes twelve indicators tracking priority areas for human capital development, such as health, education, access to safe water and improved sanitation. Two indicators reflect the Human Capital Index (HCI), namely, under-five morality rate and stunting among children under five years of age, and are currently reported in the IDA18 RMS. All indicators will be disaggregated by FCS; indicators 19 and 20 will be disaggregated by sex. Table 4: Tier 1 Indicators – Human Capital Included Disaggregation in Special No. Indicator New SDG IDA18 FCS Sex Theme RMS Human Capital Prevalence of stunting among children under 5 11   2.2.1 years of age (%) Maternal mortality ratio (number of maternal 12   3.1.1 GD deaths per 100,000 live births) Proportion of births attended by skilled health 13   3.1.2 personnel (%) Under-5 mortality rate (number of under-five 14   3.2.1 deaths per 1,000 live births) Incidence of HIV (% of uninfected population 15   3.3.1 ages 15-49) Contraceptive prevalence by modern methods 16   3.7.1 GD (% of married women ages 15-49) Adolescent fertility rate (number of births per 17   3.7.2 GD 1,000 women ages 15-19) Population of children who cannot read by end- 18   4.1 of-primary-school age (%) Lower secondary gross completion rate (%)    4.1 19 - Ratio of girls’ to boys’ completion rate    4.1 GD Lower secondary enrollment rate (%)    4.1 20 - Ratio of girls’ to boys’ enrollment rate    4.1 GD People using basic drinking water services (% 21   6.1 of population) People using basic sanitation services (% of 22   6.2 population) Resilience 17. This category includes three indicators tracking climate change and environment and two indicators introduced in the IDA18 RMS to reflect the pressing challenges presented by the growing number of refugees and internally displaced persons around the world. All five are currently reported in the IDA18 RMS and will be retained and will be disaggregated by FCS. -8- Table 5: Tier 1 Indicators – Resilience Included Disaggregation in Special No. Indicator New SDG IDA18 FCS Sex Theme RMS Resilience 23 CO2 emissions (metric tons per capita)   9.4.1 CC 24 Countries without wealth depletion (%)   12 CC 25 Average annual deforestation change (%)   15.1 CC Number of refugees by country or territory of 26   16 FCV asylum (million) Internally displaced persons, total displaced by 27   16 FCV conflict and violence (million - high estimate) Institutional Capacity 18. This category will include indicators tracking IDA countries’ institutional capacity to create and use data, mobilize domestic resources, and manage public expenditures. One new indicator will be introduced (29) to report the number of countries that have demonstrated an increase in their tax-to-GDP ratio (the indicator replaces the IDA18 RMS indicator reporting the number of IDA countries that have raised the tax-to-GDP ratio above 15 percent, in light of the fact that some countries made significant progress raising revenues while remaining below the 15 percent target, e.g., Afghanistan). All indicators will be disaggregated for FCS. Table 6: Tier 1 Indicators – Institutional Capacity Included Disaggregation in Special No. Indicator New SDG IDA18 FCS Sex Theme RMS Institutional Capacity No. of IDA countries that have an improved composite PEFA score in dimensions across the pillars of budget reliability, transparency of public finances, and control in budget 28   16.6 GI execution: (1.1) Aggregate expenditure outturn (9.1) Public access to fiscal information (24.2) Competitive procurement methods Number of countries that have demonstrated 29 an increase in tax-to-GDP ratio by [X] percent   17.1 GI (%) Level of statistical capacity (Scale from 0 to  30  17.19 GI 100) -9- B. TIER 2 – IDA-SUPPORTED RESULTS 19. Most Tier 2 indicators use the Corporate Results Indicators (CRIs) to report on development outputs and outcomes in IDA countries supported by IDA-financed operations. They are grouped into the categories of Sustainable and Inclusive Growth, Human Capital, Resilience, and Institutional Capacity, covering IDA operations in the areas of health, education, agriculture, infrastructure, jobs and private sector development, social safety nets, governance, and institutional capacity development. Of the 19 indicators proposed under this tier, 15 have been retained from the IDA18 RMS and four new ones have been introduced to track transportation services, learning assessments, debt reporting, and the use of technology to enable access to public services. 20. Since IDA achieves its targets through a country-based approach, meeting a specific target in Tier 2 fundamentally depends on country-level decisions on IDA’s assistance. As IDA’s portfolio shifts, the specific results that IDA achieves under Tier 2 indicators also shifts, with some targets overachieved and others underachieved. The fact that IDA shifts its portfolio in response to evolving country demand is a strength, not a weakness, as it reflects IDA’s focus on results. IDA is country-based; it works closely with developing countries to identify their development problems and articulate tailored solutions, and so when it shifts its financing in response to changing country contexts and priorities, IDA is doing exactly what it should do to deliver results. Working with its client countries, IDA also appropriately modifies its approach when it sees that old modes of engagement are yielding lower-value results than new modes would do. For example, in several sectors over the last few years, the World Bank has recognized that IDA’s value-addition no longer lies primarily in road construction and rehabilitation or in teacher training and recruitment, but in the policy, regulatory and institutional reforms that need to accompany these activities to ensure their efficiency, effectiveness, and sustainability. Sustainable and Inclusive Growth 21. This category includes seven indicators tracking beneficiaries of agricultural technology, financial services, and jobs-focused interventions, as well as energy and transportation outputs. 3 Six indicators are included in the current IDA18 RMS and will be retained. All indicators will be disaggregated for FCS; and three of them will be disaggregated by sex. 22. IDA19 will continue to focus on agriculture, the economic sector where most of the poor are employed, through two indicators. These include indicator 1, farmers adopting improved agricultural technology (i.e., new practices or technologies for seed preparation, planting, feeding, feeding ingredients, and postharvest storage and processing), and indicator 2, area with new/improved irrigation or drainage services (i.e., new or better delivery of water to, and drainage of water from, arable land, including better timing, quantity, quality, and cost- effectiveness for the water users). 23. There will be two indicators tracking results for the energy sector. Indicator 3, number of people provided with new or improved electricity service, will measure direct beneficiaries (those that benefited from new grid or off-grid household or community electricity services), 3 The possibility of including an indicator reflecting IDA’s support for digital development/ICT is currently being explored. - 10 - inferred beneficiaries (those that benefited from increased generation capacity, a proportion of whose output is reasonably estimated to be powering new household connections), and improved access (additional electricity available for households with an existing connection to the grid). In addition, indicator 4 will report on renewable energy generation capacity, 4 enabled through policy and investment lending operations, technical assistance, leveraged finance of partners, enabling infrastructure development (transmission and distribution, and grid integration), and guarantees to scale up private sector investment. As in the IDA18 RMS, this indicator will remain aligned – in terms of definition, calculation and target – with the IDA19 policy commitment under the climate change theme. 24. Indicator 5 will focus on beneficiaries reached with financial services, where “beneficiaries” include people as well as micro-, small- and medium-sized enterprises; and “financial services” includes transaction accounts, deposit accounts, mobile money accounts, loans, lines of credit, housing finance, micro-insurance, agri-insurance, and so forth. This indicator will be reported separately for people and businesses. 25. Indicator 6 will track the number of beneficiaries who will benefit from job-focused IDA interventions. This indicator was introduced in IDA18 to help provide a more robust evidentiary base for IDA to focus on the (then) newly-introduced special theme of jobs and economic transformation. The indicator captures the cross-cutting nature of the jobs agenda, as beneficiaries (disaggregated by sex) may be individuals, workers, households, farmers, microenterprises, small and medium-sized enterprises (SMEs), and other target groups that benefit from the WBG-supported operations in different sectors and types of operations. The jobs indicator is a composite indicator and in addition to being used as a standalone indicator, it aggregates values from different Corporate Results Indicators (CRIs) estimated by relevant World Bank Global Practices (GPs). 26. Finally, for the transport sector, a new indicator is being introduced (7) to track the number of people with enhanced access to transportation services. In recent years, IDA countries have shifted their transportation portfolios, and (collectively) they are focusing less on road construction and more on maintenance, safety, gender integration, and climate resilience, and also diversifying into other sub-sectors, such as urban transport, railways, inland waterways, and logistics. As a result, road construction and rehabilitation are a decreasing part of the portfolio and the corresponding CRI is less informative about IDA’s work than the new transportation services indicator is expected to be; hence the former indicator is being replaced with this new one. All indicators will be disaggregated for FCS; indicators 1, 5, and 6 will be disaggregated by sex. 4 Renewable power generation refers to the following: biomass co-generation; wind; geothermal; solar; wave and tidal; and hydro power generation of any capacity. - 11 - Table 7: Tier 2 Indicators – Sustainable and Inclusive Growth Included Disaggregation in Special No. Indicator New FCS SDG IDA18 Sex Theme RMS Sustainable and Inclusive Growth Farmers adopting improved agricultural 1    2.4 JET technology (million) Area provided with new/improved irrigation 2   2.4 JET or drainage services (Ha) People provided with new or improved  3  7.1.1 JET electricity service (million) Generation capacity of renewable energy 4   7.2 JET (GW) Beneficiaries reached with financial services 5 supported by WB operations: 5 • People (individuals)    8.10 JET • Businesses Beneficiaries in IDA countries of job-focused  6   8.5 JET interventions (million) Number of people with enhanced access to 7   9.1 JET transportation services (million) Human Capital 27. This category includes IDA-supported results in health, education, social safety nets, access to water sources and sanitation services, and improved urban conditions. All results under these sectors will continue to be tracked through existing CRIs. All indicators will be disaggregated for FCS; and indicators 8 and 9 will be disaggregated by sex. 28. The IDA19 RMS will retain an indicator on beneficiaries of social safety nets. This indicator (8) reports the number of individuals benefiting from IDA-supported programs designed to provide social assistance to poor and vulnerable individuals or families by including them in cash transfers, public works or welfare programs, fee waivers for services, and in-kind assistance to address critical needs. 29. As in the IDA18 RMS, the number of beneficiaries receiving essential health, nutrition and population (HNP) services will be reported (indicator 9 below), by aggregating the results of three sub-indicators (all of which will also be reported separately). Indicator 9a will report the number of children immunized (referring to the number of children five years of age and younger receiving vaccines purchased through an IDA-financed project, as well as the number of children immunized with vaccines purchased with other resources (i.e., GAVI or government funds) that are delivered through an IDA-supported program). Indicator 9b will report the number of deliveries attended by skilled health personnel (referring to the number of women who delivered with the assistance of a skilled health provider – a specialist or non-specialist doctor, midwife, nurse, or other health personnel with midwifery skills – whether in health facilities or 5 Reporting Methodology currently being revised; target pending revision. - 12 - women’s homes). Indicator 9c will report the number of women and children who have received basic nutrition services (referring to direct feeding programs, nutrition programs for adolescent girls, programs monitoring nutritional status, and nutrition and food hygiene education). 30. Indicator 10 will report the number of new country engagements supporting large- scale learning assessments. 6 The indicator will help shed light on a central aspect of the learning crisis in many developing countries, namely, that schooling is not translating into learning. Developing countries that undertake a nationally-representative learning assessment will at least be able to begin to understand and address where learning gaps exist. Moreover, the availability of a nationally-representative learning assessment is a pre-requisite to calculating quality-adjusted years of schooling and therefore to calculating the HCI. This new indicator will replace the current IDA18 RMS indicator, tracking the total number of teachers trained or recruited. Through experience, the World Bank has learned that simply counting the number of teachers trained or recruited is not very informative: how they are trained (e.g., through frequent on-the-job coaching rather than a multilayered cascading model with training of trainers with diluted impacts) and how they are recruited (e.g., through merit rather than political favoritism) are actually much more important, but difficult to measure. 31. The IDA19 RMS will also retain indicators for beneficiaries of improved water supply sources 7 and improved sanitation services. 8 Improved water supply sources (indicator 11) include piped household connections and community water points, and improved sanitation services (indicator 12) include flush or pour-flush to a piped sewer system, septic tank or latrine. 32. The IDA19 RMS will also retain an indicator for people provided with improved urban living conditions. Indicator 13 will measure the number of people living in urban areas provided with access to improved services, housing, tenure, neighborhoods, public spaces, parks, resilience and/or urban environmental conditions, through the direct interventions of operations financed by IDA. 6 Details on definition and potential rewording is pending. 7 The definition of “improved water sources” follows that of the UNICEF-WHO Joint Monitoring Program. 8 The definition of “improved sanitation services” follows that of the UNICEF-WHO Joint Monitoring Program. - 13 - Table 8: Tier 2 Indicators – Human Capital Included Disaggregation in Special No. Indicator New FCS Sex SDG IDA18 Theme RMS Human Capital 8 Beneficiaries of social safety net programs    1.3 People who have received essential health,   nutrition and population services 9: a. Women and children who have received    2.2 GD 9 basic nutrition services b. Children immunized    3.8 c. Number of deliveries attended by skilled    3.8 GD health personnel Number of new country engagements supporting 10   4.1.2 GD large-scale learning assessments People provided with access to improved water 11   6.1.1 sources (million) People provided with access to improved 12   6.2.1 sanitation services (million) People provided with improved urban living 13   11.1 conditions (million) Resilience 33. This category will retain three indicators currently reported in the IDA18 RMS – on energy efficiency, disaster risk reduction, and greenhouse gas emissions. Indicator 14, projected energy or fuel savings, is a measure reporting energy savings and lifetime fuel savings, achieved through energy efficiency measures directly attributed to IDA operations. Indicator 15, net greenhouse gas (GHG) emissions, is a measure of the impact of the World Bank’s lending portfolio on GHG emissions that is applied to projects. Indicator 16, countries supported by IDA in institutionalizing disaster risk reduction as a national priority, refers to direct IDA support toward national policy and legal frameworks, dedicated and adequate resources, community participation, and national multi-sectoral platforms for disaster risk reduction. All indicators will be disaggregated for FCS. 9 Revising methodology of calculation; alternate indicator also currently being explored. - 14 - Table 9: Tier 2 Indicators – Resilience Included in Special No. Indicator New FCS SDG IDA18 Theme RMS Resilience 14 Projected energy or fuel savings (MJ)   7.3 CC  15 Net GHG emissions (tCO2eq/year)  9.4 CC Countries supported towards institutionalizing disaster  16 risk reduction as a national priority with IDA support  13.2 CC (number) Institutional Capacity 34. This final category of Tier 2 indicators includes three measuring initiatives aimed at strengthening governance and institutional development – one retained from the IDA18 RMS and two new ones, on debt and technology. A new indicator (17), will report on the number of IDA countries publishing annual and timely public debt reports. Indicator 18 will be retained from the IDA18 RMS and will report the number of IDA countries provided statistical capacity-building support by the WBG for the implementation of household surveys: it will track WBG efforts to develop IDA country capacity to collect, report and use high-quality data through technical assistance provided to national statistical agencies in IDA countries (e.g., sampling, survey logistics, estimation of poverty-lines). Finally, a new indicator (19) will report on IDA FCS countries supported in building client capacity to use field-appropriate digital tools for collection and analysis of geo-tagged data; and apply this technology to enhance project implementation and coordination. All indicators will be disaggregated for FCS. Table 10: Tier 2 Indicators – Institutional Capacity Included in Special No. Indicator New FCS SDG IDA18 Theme RMS Institutional Capacity Number of IDA countries publishing annual and timely 17   17.1 GI public debt reports Number of IDA countries provided statistical capacity  18 building support by the WBG for the implementation  17.19 GI of household surveys 19 Number of IDA FCS countries supported in building client capacity to use field-appropriate digital tools for collection and analysis of geo-tagged data; and apply   17.8 FCV this technology to enhance project implementation and coordination. - 15 - Incorporating Disability Concerns in IDA19 35. Under the World Bank’s new ESF, all investment projects being prepared will be screened through the environmental and social assessment to determine if the project could have a potential impact on, or risk to, people with disability. If such concerns exist, there will be a further assessment on how this risk will be avoided, managed and/or mitigated, including by how people with disability can share in project benefits (including opportunities for people with disability to be employed, given the World Bank’s commitment to non-discrimination, including in the workplace). Thus, where technically and financially feasible, investment projects will consider reasonable measures to adapt the workplace in relation to project workers with disabilities and will apply the concept of universal access to the design and construction of new buildings and structures. - 16 - Box 1: Learning What Works and What Doesn’t Work in Tier 2 The IDA RMS aims to highlight IDA’s value addition and contributions to development outcomes in IDA countries, but this is difficult to do. The RMS seeks quantifiable outcomes, because it is easier to report, understand and aggregate things that are counted. It seeks aggregation because it wants to provide a summary picture that is more easily understood than the highly-granular picture that comes from listing every result of IDA’s 700+ ongoing projects. Finally, it seeks to identify attributable results because it wants to show IDA’s unique achievement in the countries where it is active. But difficulties quickly arise in each of these three areas. Quantification is difficult because many IDA activities do not produce quantifiable results. Many IDA projects support policy or institutional reform or build institutional capacity: the substantive content of the project lies in the reform or the capacity built, and that is often sui generis and describable only in qualitative ways. The way that IDA implements a project may also generate results that are difficult to quantify: for example, the World Bank’s social and environmental safeguards generate many positive externalities that will not be captured in the RMS. In addition, much of IDA’s most important work consists in advocacy, analytics and policy dialogue, amplified through its convening power and partnerships: these activities may have powerful impacts, but are also difficult to quantify – especially when the result occurs with a long lag or is essentially a decision not to do something harmful (where we need to speculate about a counterfactual to explain the benefit gained). Aggregation is also difficult, even when indicators can be quantified. Many IDA projects have quantifiable results, but the projects themselves are relatively unique and so there is nothing to aggregate. For example, we know from implementation reports exactly how many Nepalis IDA has helped to build seismically-resilient housing since the 2015 earthquake, but there are no other IDA projects currently doing the same thing – and so there is no aggregation to be done. Moreover, even when a number of projects do essentially the same things, aggregation may still be too narrow to be meaningful: for example, at the IDA18 MTR two countries accounted for 60 percent of the total number of teachers that IDA support helped to train or recruit. This problem is likely to worsen as countries face increasingly unique issues and increasing choice in sources of financing, making IDA’s activities increasingly diversified. Difficulties of attribution tend to push the choice of results monitored towards outputs. Strictly speaking, establishing attribution requires a rigorous impact evaluation, contrasting the progress of a treatment group against a non-treated comparison group. Few projects have the resources or the capacity to undertake such an evaluation. Moreover, while it is relatively easy to identify an activity’s outputs, because they follow relatively immediately from the activity, outcomes are farther away on the results chain and also dependent on other factors not under the project’s control. Increasing collaboration with other development partners adds more complexity in differentiating contributions of each financier to overall outcomes. As a consequence, a demand for attributable results tends to focus attention on results that are outputs or follow immediately from outputs, whereas a demand to identify outcomes – and particularly intermediate or higher-level outcomes, tends to change the conversation to one about contribution. While the Tier 2 indicators in the IDA RMS help to illuminate IDA’s results, these constraints imply that they will never provide a complete picture. For this, it is necessary to supplement the RMS with other information. A fuller picture emerges when it is read alongside the results frameworks that exist for every IDA project, the Country Partnership Framework (CPF) or Country Engagement Note (CEN) that exists for every IDA country (with its own results framework), the multiple reviews of IDA conducted over time by the Independent Evaluation Group, IDA’s own reporting on its policy commitments, and the many stories about IDA@work published from time to time and also collected on IDA’s website, ida.worldbank.org/results. - 17 - C. TIER 3 – IDA OPERATIONAL AND ORGANIZATIONAL EFFECTIVENESS 36. Tier 3 of the IDA19 RMS will include 25 indicators tracking the performance of IDA’s operational and organizational effectiveness, of which 21 are retained from the IDA18 RMS and four are new. Tier 3 indicators are grouped into five categories – development outcomes and effectiveness, performance and quality, operational efficiency and responsiveness, financial sustainability and budget efficiency; and implementation of IDA special themes. The new indicators cover the performance and quality of IDA’s Advisory Services and Analytics (ASA); IDA’s budget efficiency; and new priorities under the special themes of climate change and governance and institutions. All Tier 3 indicators, with the exception of three of them, will be disaggregated for FCS. Development Outcomes and Effectiveness 37. This first category includes two indicators on IEG’s quality ratings for the outcomes of IDA operations and IDA CPFs and two on client feedback regarding WBG effectiveness and knowledge products. Indicator 1 will report the percentage of IDA CPFs that IEG deems to have achieved a moderately-satisfactory, satisfactory, or highly-satisfactory outcome, with reported data based on the moving average of IEG ratings for CPFs exited during the last four fiscal years. Indicator 2 will report the percentage of IDA operations that IEG deems to have achieved a moderately-satisfactory, satisfactory, or highly-satisfactory outcome, by number of operations and volume of commitments, with reported data summing IEG ratings for operations which closed in the three previous fiscal years, provided that IEG has evaluated at least 60 percent of the projects in the fiscal year in question. Indicators 3 and 4 will provide feedback from IDA clients on two dimensions: indicator 3 will report client survey data with respect to the WBG’s effectiveness and the extent to which the WBG’s work helps to achieve development results in- country; and indicator 4 will report client survey data with respect to the significance of the WBG’s knowledge work. Client data will be obtained from answers provided by WBG clients in IDA countries from annual WBG COPs, reflecting the reporting ratings in a given fiscal year (i.e., for one-third of all WBG client countries surveyed during each three-year cycle). Table 11: Tier 3 Indicators – Development Outcomes and Effectiveness Included in Special No. Indicator New FCS SDG IDA18 Theme RMS Development Outcomes and Effectiveness Satisfactory outcomes of IDA Country Partnership 1   Frameworks (%, IEG Rating, 4-year rolling) Satisfactory outcomes of IDA operations:  as a share of commitments 2   (%, IEG ratings, 3-year rolling)  as share of operations   (%, IEG ratings, 3-year rolling) Client feedback in IDA countries on WBG effectiveness 3   and impact on results (average rating scale: 1-10) 4 Client feedback in IDA countries on WBG knowledge   - 18 - Performance and quality 38. This category includes measures on the performance of the IDA portfolio, including from IEG reviews and from client surveys. Indicator 5 will report IEG ratings of World Bank performance overall, at entry, and during supervision, as determined in IEG’s assessment of the World Bank’s Implementation Completion Reviews of all closed projects. Indicator 6, quality of M&E in IDA operations, will report the share of the net-commitment amount of closed IDA projects reviewed by IEG that are rated substantial or high for quality of M&E, against the total net-commitment amount of closed IDA projects reviewed by IEG on a three-year rolling basis (the rating is based on IEG ratings for investment projects that closed at least twelve months before the reporting period.) A new indicator, 7, will provide a view on ASA objectives accomplished, reporting client ratings on whether such activities achieved their intended development outcomes. Finally, indicator 8 will monitor the percentage of IDA investment projects with a beneficiary feedback indicator at design. Table 12: Tier 3 Indicators – Performance and Quality Included in Special No. Indicator New FCS SDG IDA18 Theme RMS Performance and Quality Satisfactory World Bank performance in IDA-financed operations (%, IEG Rating) 5 • Overall   • At entry   • During supervision   Quality of M&E in IDA operations (%, IEG 6   ratings (3-year rolling)) Advisory Services and Analytics (ASA) 7   objectives accomplished (client rating, %) Projects with beneficiary feedback indicator at 8   design (%) Operational efficiency and responsiveness 39. Five indicators will be used to track IDA’s efficiency and responsiveness. The first one, indicator 9, will aim to monitor implementation (this indicator currently remains under development). The other four indicators were included in the IDA18 RMS. They are indicator 10, the IDA disbursement ratio, calculated as the ratio of disbursements during a specific fiscal year to the undisbursed balance at the beginning of that fiscal year for all IDA investment project financing (IPF) projects; indicator 11, the proactivity index, a key measure introduced with the IDA18 RMS for tracking actions taken to improve the performance of “problem projects” (i.e., projects that have an unsatisfactory rating for development objective or implementation progress), by upgrading, restructuring, or closing the project; indicator 12, client feedback on WBG responsiveness and staff accessibility; and indicator 13, client feedback on WBG collaboration with other donors. - 19 - Table 13: Tier 3 Indicators – Operational Efficiency and Responsiveness Included Special No. Indicator in IDA18 New FCS SDG Theme RMS Operational efficiency and responsiveness [placeholder indicator monitoring implementation 9   under discussion/development] 10 Disbursement ratio (%)   11 Proactivity Index (%)   Client feedback on WBG on responsiveness and 12   staff accessibility Client feedback on WBG on collaboration with other 13   donors Financial Sustainability and Budget Efficiency 40. The IDA19 RMS will retain three indicators, introduced in the IDA18 RMS to track IDA’s financial sustainability and budget efficiency, which are consistent with the WBG’s budget and performance review process (also known as the “W” process). These are indicator 14, the IDA budget anchor (calculated as the ratio of IDA expenses over IDA net revenue); indicator 15, the World Bank budget to Portfolio Volume Ratio (this reflects the total administrative budget spent for every US$1 billion of portfolio under supervision); and indicator 16, the average cost of IDA project supervision (calculated based on costs posted directly to operational projects in the IDA portfolio, divided by the monthly average number of projects in the IDA portfolio). Table 14: Tier 3 Indicators – Financial Sustainability and Budget Efficiency Included Special No. Indicator in IDA18 New FCS SDG Theme RMS Financial sustainability and budget efficiency 14 IDA Budget Anchor (%)  World Bank budget to Portfolio Volume Ratio (per 15  US$ billion portfolio under supervision) Average cost of IDA supervision projects 16   (implementation support) (US$ ’000) Implementation of IDA Special Themes 41. The nine remaining indicators in Tier 3 will monitor implementation of specific priorities under the IDA special themes; of these, seven were included in the IDA18 RMS and two are new. These indicators complement the proposed IDA19 policy commitments but are not intended to provide a full accounting of those commitments. - 20 - 42. Jobs and economic transformation (JET). Reflecting the continued central importance of the JET agenda, the IDA19 RMS will retain two JET-related indicators: indicator 17, reporting the share of IDA19 Country Partnership Frameworks (CPFs) reflecting key principles underpinning economic transformation; and indicator 18, tracking total private mobilization of WBG-supported operations/transactions in IDA countries; the indicator will also include a breakdown of the private capital directly and indirectly mobilized. 43. Gender and development. Two indicators have been retained from the IDA18 RMS as they remain relevant and consistent with the World Bank Gender Strategy: indicator 19, which will report the percentage of IDA-supported projects that demonstrate a gender results chain by linking gender gaps identified in analysis to specific actions that are tracked in the results framework; and indicator 20, which will report the number of IDA-supported operations that address and respond to gender-based violence (GBV). One indicator has been dropped, namely, the percentage of IDA-supported operations reporting gender results at completion, because indicator 19 is more demanding. 44. Climate change. Two indicators from the IDA18 RMS will continue in the IDA19 RMS. These include indicator 21, share of climate change co-benefits over total commitments in IDA- supported operations (using percentage as a unit of measure, rather than number of projects or volume of commitments in US$ million as currently reported in the IDA18 RMS), and indicator 23, IDA commitments with disaster risk management co-benefits (volume of commitments in US$ million). In addition, a new indicator, 22, will report the percentage share of adaptation co-benefits over total climate co-benefits in IDA-supported operations to reflect IDA19’s enhanced focus on adaptation, in alignment with the policy commitment proposed under this special theme. 45. Fragility, conflict and violence. Under this theme, the Facetime Index will be retained as indicator 24. The Facetime index shows professional (GE+) World Bank staff presence on the ground in FCS by fiscal year, measured as a percent increase from a pre-established baseline (FY17). Calculation is based on the total number of days and then converted to an index where the baseline is 100. The index is calculated once each fiscal year, on a full-year basis, and it will be based on the new list of IDA FCS that will emerge from the FCV strategy. 46. Governance and institutions. Under this theme, the IDA19 RMS will introduce a new indicator (25) to track support in IDA countries with the lowest Human Capital Index ratings to strengthen human capital financing through improving the efficiency of public expenditures and more effectively aligning expenditures with domestic financing and external resources in a sustainable manner. The indicator is consistent with the policy commitment under Pillar B (Maximizing Impact of Public Service Delivery) of the Government and Institutions special theme. The indicator also emphasizes the World Bank’s support for IDA countries’ anti-corruption efforts and the importance of oversight systems to ensure that IDA funds are used for the intended purposes. - 21 - Table 15: Tier 3 Indicators – Implementation of Special Themes Included in Special No. Indicator New FCS SDG IDA18 Theme RMS Jobs and Economic Transformation Share of IDA19 CPFs which reflect at least one of the following four key principles underpinning economic transformation: • Sectoral productivity • Value chain expansion 17  JET • Increased productive capital stock or investment in energy, transport, manufacturing or services • Export sector output/value added; trade facilitation Total private mobilization of WBG-supported operations/transactions in IDA countries. 18 Of which:   17.3 JET • Direct (US$ billion) • Indirect (US$ billion) Gender and Development Percentage of IDA-supported projects that 19 demonstrate a results chain by linking gender gaps   GD identified in analysis to specific actions that are tracked in the results framework (%) 20 Number of IDA-supported operations that address   5.1 GD and respond GBV Climate Change Share of climate change co-benefits over total 21   13.2 CC commitments in IDA-supported operations (%) Share of adaptation co-benefits over total climate co- 22   13.2 CC benefits in IDA-supported operations (%) IDA financing commitments with disaster risk 23   13.2 CC management co-benefits (US$ million) Fragility, Conflict, and Violence (FCV) 24 Facetime index in FCS 10   FCV Governance and Institutions Number of IDA countries among the 30 with the lowest Human Capital Index supported to strengthen human capital financing through improving the 25 efficiency of public expenditures, and more   17.1 GI effectively aligning expenditures with domestic financing and external resources in a sustainable manner 10 The Facetime Index will be reset according to the new FCV list, as the FCV strategy gets finalized. - 22 - IV. ALIGNMENT WITH THE SUSTAINABLE DEVELOPMENT GOALS 47. The IDA19 RMS fully reflects the development priorities of the 17 SDGs adopted during the United Nations General Assembly in September 2015. It is also consistent with the guiding principle of maintaining continuity with the IDA18 agenda to enable longer-term monitoring while capturing new or emerging issues. The exercise to align the IDA18 RMS with the SDGs reflected three key considerations: (i) harmonizing the SDGs with the WBG goals and strategy; (ii) balancing the SDG agenda with the World Bank’s country-led engagement model and varied client capacity to provide or report quality data; and (iii) ensuring that the RMS incorporated critical areas of the SDGs in which IDA has substantial engagements or comparative advantage. These criteria remain relevant for the IDA19 RMS. The alignment of the SDGs with the IDA19 RMS is reflected at the level of a goals’ strategic objective, a target or a specific indicator, as applicable. Of the proposed IDA19 RMS indicators, 55 are aligned with the SDGs (28 in Tier 1, 21 in Tier 2, and six in Tier 3; see Annex 3 for more details). V. REPORTING AND COMMUNICATING IDA RESULTS 48. The IDA19 RMS will be updated and publicly disclosed once a year at the time of the World Bank Annual Meetings. Management will report on the status of the RMS indicators annually and also provide a detailed update on progress across all tiers at the IDA19 Mid-Term Review. The IDA RMS will be shared and publicly disclosed on IDA’s website and the World Bank’s results page. 49. IDA Results Stories will complement results reported in the IDA19 RMS. To complement aggregated quantitative data of the annual IDA RMS updates, results narratives of IDA contributions will be shared with the IDA Participants in the form of results stories. These stories will include a description of the challenges faced by IDA countries, approaches used to address them, and the results achieved. Financial and technical contributions of IDA, the role of local and international partners, and an overview of what happens following IDA’s interventions and subsequent efforts would be included. Efforts will be made to include results associated with IDA19’s special themes. These stories will enable IDA to supplement the RMS by capturing some outcomes of IDA support, including scale-up and replication effects as well as strengthened partnerships, which are difficult to show through quantification and aggregation. The stories will also capture the diverse range and depth of IDA’s activities and results, highlighting its impact to beneficiaries, and presenting a human face to IDA support. Results from lending or ASA operations already completed or still under implementation will also be featured. The results stories are produced on a rolling basis year-round and published on the World Bank’s external website (http://www.worldbank.org/en/results). VI. ISSUES FOR DISCUSSION • Do IDA Participants agree with the guiding principles that have been followed for defining indicators for IDA19 RMS? • Do IDA Participants agree with the changes proposed for the IDA19 RMS indicators? - 23 - Annex 1: IDA19 Results Measurement System TIER 1 – IDA COUNTRIES’ PROGRESS Included Disaggregation Special No. Indicator in IDA18 New SDG FCS Sex Theme RMS World Bank Group Goals 1 Population living on less than US$1.90 a day (%)   1.1.1 2 Growth rate of real per capita income of the bottom 40 percent of population (%)   10.1 Sustainable and Inclusive Growth 3 GDP per person employed (constant 2011 PPP $)   JET 4 Non-agriculture sectors, value added (as % of GDP)   JET Legal changes that support gender equality over the past two years (number of legal 5   5.1 GED gender changes) 6 Proportion of population with access to electricity (% of population)   7.1.1 JET 7 Annual growth rate of real GDP per capita (%)   8.1.1 8 Ratio of female to male labor force participation rate (%)   8.5 JET Youth employment to population ratio (age 15-24) (%)    8.5 JET 9 - Youth employment to population ratio (age 15-24), women (%)    8.5 JET - Youth employment to population ratio (age 15-24), men (%)    8.5 JET Proportion of adults (15 years and older) with an account at a bank or other financial 10    8.10.2 institution or with a mobile money service provided (%) - 24 - Human Capital 11 Prevalence of stunting among children under 5 years of age (%)   2.2.1 12 Maternal mortality ratio (number of maternal deaths per 100,000 live births)   3.1.1 GD 13 Proportion of births attended by skilled health personnel (%)   3.1.2 GD 14 Under-5 mortality rate (number of under-five deaths per 1,000 live births)   3.2.1 15 Incidence of HIV (% of uninfected population ages 15-49)   3.3.1 16 Contraceptive prevalence by modern methods (% of married women ages 15-49)   3.7.1 GD 17 Adolescent fertility rate (number of births per 1,000 women ages 15-19)   3.7.2 GD 18 Population of children who cannot read by end-of-primary-school age (%)   4.1 Lower secondary gross completion rate (%)    4.1 19 - Ratio of girls’ to boys’ completion rate    4.1 GD Lower secondary enrollment rate (%)    4.1 20 - Ratio of girls’ to boys’ enrollment rate    4.1 GD 21 People using basic drinking water services (% of population)   6.1 22 People using basic sanitation services (% of population)   6.2 Resilience 23 CO2 emissions (metric tons per capita)   9.4.1 CC - 25 - 24 Countries without wealth depletion (%)   12 CC 25 Average annual deforestation change (%)   15.1 CC 26 Number of refugees by country or territory of asylum (million)   FCV Internally displaced persons, total displaced by conflict and violence (million - high 27   16 FCV estimate) Institutional Capacity No. of IDA countries that have an improved composite PEFA score in dimensions across the pillars of budget reliability, transparency of public finances, and control in budget execution: 28   16.6 GI (1.1) Aggregate expenditure outturn (9.1) Public access to fiscal information (24.2) Competitive procurement methods Number of countries that have demonstrated an increase in Tax-to-GDP ratio by [X] 29   17.1 GI percent (%) 11 30 Level of statistical capacity (scale from 0 to 100)   17.19 GI 11 This indicator is a modification from the IDA18 RMS Tier 1 indicator Number of IDA countries that have raised taxes/GDP above 15 percent. - 26 - TIER 2 – IDA-SUPPORTED DEVELOPMENT RESULTS Included Disaggregation in Special No. Indicator New FCS SDG IDA18 Sex Theme RMS Sustainable and Inclusive Growth 1 Farmers adopting improved agricultural technology (million)    2.4 JET 2 Area provided with new/improved irrigation or drainage services (Ha)   2.4 JET 3 People provided with new or improved electricity service (million)   7.1.1 JET 4 Generation capacity of renewable energy (GW)   7.2 JET Beneficiaries reached with financial services supported by WB operations (million) 5 • People    8.10 JET • Businesses 6 Beneficiaries in IDA countries of job-focused interventions (million)    8.5 7 Number of people with enhanced access to transportation services (million)   9.1 Human Capital 8 Beneficiaries of social safety net programs (million)    1.3 People who have received essential health, nutrition and population services:   9 a. Women and children who have received basic nutrition services   3.8 b. Children immunized   2.2 c. Number of deliveries attended by skilled health personnel   3.8 10 Number of new country engagements supporting large-scale learning assessments   4.c 11 People provided with access to improved water sources (million)   6.1.1 - 27 - Included Disaggregation in Special No. Indicator New FCS SDG IDA18 Sex Theme RMS 12 People provided with access to improved sanitation services (million)   6.2.1 13 People provided with improved urban living conditions (million)    11.1 Resilience 14 Projected energy or fuel savings (MJ)   7.3 15 Net GHG emissions (tCO2eq/year)   9.4 Countries supported towards institutionalizing disaster risk reduction as a national 16   13.2 priority with IDA support Institutional Capacity 17 Number of IDA countries publishing annual and timely public debt reports   17.1 GI Number of IDA countries provided statistical capacity building support by the WBG for 18   17.19 GI the implementation of household surveys Number of IDA FCS countries supported in building client capacity to use field- FCV, 19 appropriate digital tools for collection and analysis of geo-tagged data; and apply this   17.1 GI technology to enhance project implementation and coordination. - 28 - TIER 3 – IDA ORGANIZATIONAL AND OPERATIONAL EFFECTIVENESS Included Disaggregation Special No. Indicator in IDA18 New SDG FCS Sex Theme RMS Development Outcome Ratings Satisfactory outcomes of IDA Country Partnership Frameworks (%, IEG Rating, 4- 1   year rolling) Satisfactory outcomes of IDA operations: • as a share of commitments   2 (%, IEG ratings, 3-year rolling) • as share of operations   (%, IEG ratings, 3-year rolling) Client feedback in IDA countries on WBG effectiveness and impact on results   3 (average rating scale: 1-10)   4 Client feedback in IDA countries on WBG knowledge (average rating scale: 1-10) Performance and Quality Satisfactory World Bank performance in IDA-financed operations (%, IEG Rating) • Overall   5 • At entry   • During supervision   6 Quality of M&E in IDA-financed operations (%, IEG ratings, 3-year rolling)   7 Advisory Services and Analytics (ASA) objectives accomplished (client rating, %)   8 Projects with beneficiary feedback at design (%)   - 29 - Included Disaggregation Special No. Indicator in IDA18 New SDG FCS Sex Theme RMS Operational Efficiency and Responsiveness 9 [placeholder indicator monitoring implementation under discussion/development]   10 Disbursement ratio (%)   11 Proactivity Index (%)   12 Client feedback on WBG on responsiveness and staff accessibility   13 Client feedback on WBG on collaboration with other donors   Financial Sustainability and Budget Efficiency 14 IDA Budget Anchor (%)  15 Bank budget to Portfolio Volume Ratio (per US$ billion under supervision)  16 Average cost of IDA supervision projects (implementation support) (US$ thousand)   Implementation of IDA Themes Share of IDA19 CPFs which reflect at least one of the following four key principles underpinning economic transformation: • Sectoral productivity 17 • Value chain expansion  JET • Increased productive capital stock or investment in energy, transport, manufacturing or services • Export sector output/value added; trade facilitation Total private mobilization of WBG-supported operations/transactions in IDA 17.3 18 countries.   JET • Direct mobilization (US$ billion) - 30 - Included Disaggregation Special No. Indicator in IDA18 New SDG FCS Sex Theme RMS • Indirect mobilization (US$ billion) Gender and Development Percentage of IDA-supported projects that demonstrate a results chain by linking 19 gender gaps identified in analysis to specific actions that are tracked in the results   GD framework (%)  5.1 20 Number of IDA-supported operations that address and respond to GBV  GD Climate Change 13.2 21 Share of climate co-benefits over total commitments in IDA-supported operations (%)   CC Share of adaptation co-benefits over total climate co-benefits ion IDA-supported 13.2 22   CC operations (%) 13.2 23 IDA financing commitments with disaster risk management co-benefits (US$ million)   CC Fragility, Conflict, and Violence (FCV) 24 Facetime index in FCS   FCV Governance and Institutions Number of IDA countries among the 30 with the lowest Human Capital Index 17.1 supported to strengthen human capital financing through improving the efficiency of 25   public expenditures, and more effectively aligning expenditures with domestic financing and external resources in a sustainable manner - 31 - Annex 2: New and Dropped Indicators 1. New indicators No. Indicator Tier 1 1 Population of children who cannot read by end-of-primary-school age (%) 2 Number of countries that have demonstrated an increase in Tax-to-GDP ratio by [X] percent (%) Tier 2 3 Number of people with enhanced access to transportation services (million) 4 Number of new country engagements supporting large-scale learning assessments 5 Number of IDA countries publishing annual and timely public debt reports Number of IDA FCS countries supported in building client capacity to use field-appropriate digital tools 6 for collection and analysis of geo-tagged data; and apply this technology to enhance project implementation and coordination. Tier 3 7 Advisory Services and Analytics (ASA) objectives accomplished (%) 8 [placeholder indicator monitoring implementation being developed/under discussion] 9 Share of adaptation co-benefits over total climate co-benefits in IDA-supported operations. (%) Number of IDA countries among the 30 with the lowest Human Capital Index to strengthen human capital 10 financing through improving the efficiency of public expenditures, and more effectively aligning expenditures with domestic financing and external resources in a sustainable manner 2. Indicators dropped from IDA18 RMS No. Indicator Rationale for dropping Tier 1 Other indicators in Tiers 1, 2 and 3 are better 1 Trade Logistics Performance Index aligned with the IDA19 special theme on Jobs and Economic Transformation. This indicator value essentially does not change 2 Population exposed to harmful air pollution over the three-year IDA cycle. Indicator 2 (growth rate of real per capita income Countries with growth concentrated in the bottom 3 of the bottom 40% of population) is better aligned 40% to WBG’s second overall goal. - 32 - No. Indicator Rationale for dropping There are no significant year-to-year variations of 4 Annual freshwater withdrawals, total this indicator and so it has been dropped. Number of IDA countries that have raised the tax- An updated version of this indicator has been 5 included, so the original one has been dropped. to-GDP ratio above 15% Tier 2 The IDA portfolio is gradually shifting away from merely constructing or rehabilitating roads to other more complex engagements with clients on a range 6 Roads constructed or rehabilitated of transportation and connectivity activities where IDA can add more value. Accordingly, a new indicator (number of people with enhanced access to transportation services) has replaced this one. While still continuing to support critical education inputs/outputs in the education sector, a new Tier 2 indicator on learning assessment replaces this one, reflecting a shift that even in challenging IDA 7 Teachers trained or recruited country settings, the World Bank is engaging with clients in amplifying the focus on quality of learning – with important linkages to skills and jobs. There is a lack of core IDA project financing to support CRV activities (primarily funded through Number of lending operations with civil 8 Trust Funds), but IDA engagement on critical registration and vital statistics (CRV) issues related to CRV continue under the broader HCI dialogue. This indicator has been dropped to incorporate an alternate measure reflecting priorities for the JET agenda in IDA19. An additional measure tracking private capital mobilized (direct and indirect) to IDA countries is included in tier 3. Furthermore, Private investment catalyzed by World Bank in 9 there is a Multilateral Development Bank (MDB) IDA countries Working Group which is focusing on improving the methodology to estimate private capital catalyzed. Depending on progress on the MDB Working Group, this indicator might be subsequently brought back. This indicator has been dropped to incorporate Number of IDA countries that operationalize OGP 10 alternate measure reflecting priorities for the JET agenda commitments agenda in IDA19. Number of countries with an increase in the This indicator has been dropped to incorporate 11 number of registered tax payers among IDA alternate measure reflecting priorities for the JET countries with substantial Bank tax engagement agenda in IDA19. Tier 3 This indicator has been dropped as the Share of CPFs in IDA countries that have at least 12 performance target in IDA18 RMS has already one joint objective in the results matrix been achieved. The first indicator component “Stock of country 13 Alignment with the WB Strategy strategies underpinned by SCDs” has been dropped because it is achieved; the second indicator - 33 - No. Indicator Rationale for dropping • Stock of country strategies underpinned by component “Qualitative assessment of alignment Strategic Country Diagnostics (SCDs) of country engagement with corporate goals” has been dropped because no monitoring mechanism exists to obtain an aggregated value for all IDA • Qualitative assessment of alignment of CPFs (when the IDA18 RMS was conceived, there country engagement with corporate goals were IEG plans to introduce such a mechanism, but those plans were subsequently dropped). Operations drawing lessons from evaluative This indicator has been dropped due to its limited 14 strategic value. approaches Time from Project Concept Note to the first This indicator has been dropped as it does not disbursement project financing properly reflect the timeliness and quality of delivery of World Bank investment project • Time from Concept Note approval to Board financing. Project teams ensure that essential approval 15 standards (related to environmental, social, • Time from Board approval to project fiduciary, etc.) are in place during the design phase effectiveness – as well as throughout implementation. An • Time from project effectiveness to first alternate indicator is in the process of being disbursement identified. This indicator has been dropped because the new Gender Tag methodology, introduced in FY17, provides a more comprehensive approach towards Percentage of IDA-supported operations reporting 16 ensuring that all the key pillars of the WBG gender results at completion Gender Strategy are ingrained in the design (and implementation) of World Bank operations to meaningfully reduce gender gaps. This indicator has been dropped to include a new Completed ASA products that address climate 17 measure on overall quality and timeliness of change issues delivery of all ASA products in IDA countries. Number of Illicit Financial Flows (IFFs) Other indicators are better aligned with the IDA19 18 Governance special theme. assessments performed in IDA countries The World Bank evaluates all operations in all Number of impact evaluations supported by the 19 IDA countries, which are independently validated World Bank in IDA countries by IEG. The indicator has been combined under one indicator: “total private mobilization of WBG- Total private mobilization of WBG-supported 20 supported operations/transactions in IDA operations/transactions in IDA countries countries”; it will report/disaggregate direct and indirect mobilization. - 34 - Annex 3: Alignment of the SDGs with IDA19 Results Measurement System Indicators Sustainable Development Goal Tier 1 Tier 2 Tier 3 Total 1. End poverty in all its forms everywhere 1 1 2 2. End hunger achieve food security and improved nutrition and promote 1 3 4 sustainable agriculture 3. Ensure healthy lives and promote well-being for all at all ages 6 2 8 4. Ensure inclusive and equitable quality education and promote lifelong 3 1 4 learning opportunities for all 5. Achieve gender equality and empower all women and girls 1 1 2 6. Ensure availability and sustainable management of water and 2 2 4 sanitation for all 7. Ensure access to affordable, reliable, sustainable and modern energy 1 3 4 for all 8. Promote sustained, inclusive and sustainable economic growth, full 4 2 6 and productive employment and decent work for all 9. Build resilient infrastructure, promote inclusive and sustainable 1 2 3 industrialization and foster innovation 10. Reduce inequality within and among countries 1 1 11. Make cities and human settlements inclusive, safe, resilient and 1 1 sustainable 12. Ensure sustainable consumption and production patterns 1 1 13. Take urgent action to combat climate change and its impacts 1 3 4 14. Conserve and sustainably use the oceans, seas and marine resources 0 for sustainable development 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and 1 1 reverse land degradation and halt biodiversity loss 16. Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and 3 3 inclusive institutions at all levels 17. Strengthen the means of implementation and revitalize the Global 2 3 2 7 Partnership for Sustainable Development Total 28 21 6 55 - 35 - Annex 4: Brief background on the evolution of the IDA RMS 1. The development of the IDA RMS was anchored in the World Bank’s broader effort to enhance its results orientation. This effort was launched in 2002 as part of the follow-up to the Monterrey Consensus, drawing on the International Roundtable on Better Measuring, Monitoring, and Managing for Development Results that was held in June 2002. The introduction of a framework for measuring IDA results and the creation of an interim system to monitor results was an innovation of the IDA13 replenishment arrangement in 2002 with the purpose to strengthen the focus on development outcomes of IDA’s activities and to inform Participants’ discussions about IDA’s effectiveness. With the introduction of this new system, IDA became the first among multilateral development institutions to develop and implement a framework with quantitative indicators to monitor its impact. Since it was introduced in 2002, the RMS has evolved over the years and been constantly refined to strengthen IDA’s focus on results monitoring across IDA countries to better capture results. 2. The interim system for results developed in 2002 was followed by the IDA14 RMS, designed to improve consistency with priorities articulated by countries through their Poverty Reduction Strategy Papers (PRSPs), alignment with the Millennium Development Goals (MDGs) agenda and other international monitoring efforts, and relevance to IDA’s activities in borrowing countries. Indicators were developed and introduced for Tier 2 to monitor IDA’s contribution to development effectiveness across three levels of the World Bank’s work – program, project, and aggregate outputs. During the IDA15 replenishment, IDA ramped up its ability to monitor and measure development results and improved the quality of its operational work. The IDA15 RMS kept two tiers of indicators, focused on poverty, human development, infrastructure and public finance management, and introduced measures to monitor improvements in country statistical capacity. 3. For the IDA16 cycle, the RMS was expanded from a two-tier system to a four-tier system to show a clearer distinction between development results achieved by IDA countries; sectoral outputs and outcomes; operational effectiveness; and organizational effectiveness of IDA. 4. IDA17 RMS introduced revisions with a more complete framework to support the achievement of development results in client countries, help to strengthen IDA’s effectiveness, efficiency and accountability, and further clarify IDA’s contribution to key results and outcomes. A key feature for the IDA17 RMS was the introduction of indicators to capture progress on the IDA17 overarching and IDA17’s Special Themes. 5. IDA18 RMS responded to demands to simplify the corporate results measurement and reporting system by reducing the number of tiers and maintaining the number of indicators and harmonizing with the World Bank CSC. Changes were also made on the aggregation methodology for Tier 2 indicators and to align with the Sustainable Development Goals (SDGs). - 36 - EVOLUTION OF THE IDA RESULTS MEASUREMENT SYSTEM - 37 - Annex 5: Overview of the data aggregation process for the IDA RMS The foundation is data provided by clients, the production of which is heavily dependent on local statistical capacity. The overall process consists of four steps and takes about three months to complete. Data collection. Besides using data provided by clients, the World Bank produces its own data, including data to help classify and describe the nature and extent of the World Bank’s engagement in a client country. Data sources for Tier 1 typically comes from the World Bank’s WDI database, external databases from agencies and other MDBs. For Tier 2, data are collected from standard project reports (ISRs/ICRs) and entered into the World Bank’s information systems by projects’ task team leaders and project teams. Data aggregation. This phase entails aggregating the values (results) in a database for the time periods under consideration. Quality assurance. Data are subjected to quality control before being aggregated across clients and operations. This step entails strict, process-oriented, built-in quality assurance measures to prevent errors. Construction and production of final numbers. The final numbers are used to populate the IDA RMS.