51487 IDA15 MID-TERM REVIEW The IDA15 Results Measurement System: Mid-Term Review International Development Association Operations Policy and Country Services (OPCS) October 2009 ABBREVIATIONS AND ACRONYMS AAA analytic and advisory activities CAP-Scan Capacity Scan CAS country assistance strategy CASCR country assistance strategy completion report COMPAS Common Performance Assessment System CPIA Country Policy and Institutional Assessment DPO development policy operation ECA Europe and Central Asia Region ERO emergency recovery operation ESW economic and sector work GDP gross domestic product GFRP Global Food Crisis Response Program GNI gross national income HIPC Heavily-Indebted Poor Country HIV/AIDS Human Immunodeficiency Virus/ Acquired Immune Deficiency Syndrome IAD Internal Auditing Department IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation ISR implementation status and results report MAPS Marrakech Action Plan for Statistics MDB multilateral development bank MDG millennium development goal NSDS National Strategy for the Development of Statistics OPCS Operations Policy and Country Services PAD project appraisal document PARIS21 Partnership in Statistics for Development in the 21st Century PD program document PDO project development objective PEFA Public Expenditure and Financial Accountability PFM public financial management PPP public-private partnership PSD private sector development QAG Quality Assurance Group QALP Quality Assessment of Lending Portfolio RBCAS results-based country assistance strategy RMS Results Measurement System ROC rate of change SSA Sub-Saharan Africa STATCAP World Bank Statistical Capacity Financing Program TA technical assistance TFSCB Trust Fund for Statistical Capacity Building UNICEF United Nations Children's Fund WDI World Development Indicator WHO World Health Organization THE IDA15 RESULTS MEASUREMENT SYSTEM: MID TERM REVIEW CONTENTS Executive Summary The IDA15 Results Measurement System: Mid-Term Review..................................... 1 I. Introduction ............................................................................................................ 1 II. Development Progress in IDA Countries ­ Tier 1 ............................................... 2 A. Global Results ................................................................................................2 B. The Impact of Global Crises ..........................................................................7 III. Increasing Focus on Results ­ Tier 2 .................................................................... 9 A. Improving Agency Effectiveness ..................................................................9 B. Improving Government Statistical Capacity ...............................................24 C. Improving MDB Collaboration in Results Measurement ............................27 IV. Next Steps .............................................................................................................. 28 Annexes: Annex 1: Changes in IDA Results Measurement System Tier 1 Outcome Indicators ..... 29 Annex 2: Data Availability for IDA RMS Tier 1 Indicators ............................................ 30 Annex 3: Data on Core Sector Indicators for IDA 14-15 Operations: August 21, 2009 .. 31 Annex 4: Outputs Table .................................................................................................... 32 Boxes: Box 1: Global Food Crisis Response Program ................................................................... 9 Box 2: Selected Results from IDA Support in Key Sectors ............................................. 10 Box 3: Tier 2 Monitorable Actions on Agency Effectiveness .......................................... 12 Box 4: Core Sector Indicator Data .................................................................................... 18 Box 5: Outputs from IDA Operations that Closed in FY09 ............................................. 19 Box 6: Selected Results from IDA AAA Products ........................................................... 23 Box 7: Improving Government Statistical Capacity ......................................................... 24 Figures Figure 1: Poverty is Falling in IDA Countries as Growth is Accelerating ......................... 4 Figure 2: Starting a Business in IDA Countries has Become Easier .................................. 5 Figure 3: Access to Clean Water and Electricity Has Increased ........................................ 6 Figure 4: Progress on Health and Education Outcomes Has Been Slow ............................ 7 Tables: Table 1: Progress in Tier 1 Outcome Indicators for IDA-Eligible Countries (July 2009) . 3 Table 2: CASCRs Based on RBCASs (FY08-09) ............................................................ 13 Table 3: Quality at Entry for IDA Operations .................................................................. 14 Table 4: First ISRs with Adequate Baselines for PDO Indicators .................................... 17 Table 5: Satisfactory Outcomes at Exit Based on IEG Evaluations ................................. 19 Table 6: IDA AAA Results in FY08 and FY09 ................................................................ 22 THE IDA15 RESULTS MEASUREMENT SYSTEM: MID TERM REVIEW EXECUTIVE SUMMARY i. The Results Measurement System. The International Development Association (IDA) Results Measurement System (RMS) was launched in 2002 during the IDA13 replenishment and enhanced in IDA14 and IDA15 to strengthen IDA's focus on results. The RMS tracks two forms of effectiveness: development or country-level effectiveness which refers to the achievement of sustainable improvements in outcomes on the ground in IDA countries, and agency effectiveness which refers to the work that organizations in the donor community do to ensure that their business practices conform to the principles of good aid management, have enabling effects on country capacity and institutions, and thus contribute to country-level effectiveness. ii. Two-Tiered RMS. Based on this framework, the IDA RMS is a two-tiered system. Tier 1 monitors aggregate progress on 14 country outcome indicators grouped into four categories: growth and poverty reduction; public financial management (PFM) and investment climate; infrastructure; and human development. Tier 2 monitors IDA's contribution to development outcomes using indicators of the quality and effectiveness of IDA programs as well as examples of the outputs that IDA produces in four sectors. It also monitors improvements in country statistical capacity. A. Tier 1 iii. To monitor overall development progress in IDA countries, and to address the broader question of whether IDA-eligible countries are making progress, 14 broad development outcome indicators were selected for high level monitoring of IDA14 and were extended to monitor IDA15. Grouped into four categories, these indicators were selected to focus only on those that are most relevant and measurable in the majority of the IDA countries. iv. Progress. For the most recent period for which data are available, these data show that IDA countries made progress on every indicator, but that there are areas that require additional attention. While extreme poverty has fallen, the pace of poverty reduction would need to be increased in order to meet the goal of halving the poverty rate by 2015. The current rate of progress on meeting the Millennium Development Goals (MDGs) that are reflected in the RMS is adequate for access to an improved water source, but is slower than necessary to meet the goals related to under-five mortality and gender parity in education. In Sub-Saharan Africa (SSA), the data shows lagging Gross Domestic Product (GDP) growth rates, the highest rate of extreme poverty among IDA countries, no improvement in the rate of attended births, and the lowest primary completion rates. Furthermore, because of the varied sources of data, and the limitations of data availability across IDA countries, the most recently available data does not fully capture the impact of the recent crises in IDA countries. IDA has assisted countries in responding to these crises, and the impacts of the crises will continue to be monitored, in order to respond effectively and rapidly. ii B. Tier 2 v. Tier 2 of the RMS monitors IDA's contribution to development outcomes using 13 indicators of the quality and effectiveness of IDA programs as well as examples of the outputs that IDA produces in four sectors (education, health, roads, and water supply). In addition, it includes 4 indicators of improving Government statistical capacity. vi. Expanded Reporting on Tier 2. Six new indicators were added to Tier 2 at the time of the IDA15 Replenishment. This report significantly expands on previous reporting on Tier 2 by: establishing the baseline on these six new indicators; presenting results data on standardized core sector indicators and including selected results from IDA operations; and for the first time presenting the data on outcomes and quality at entry by IDA period. vii. Strengthened Results Reporting and Monitoring. Since the last report to the Deputies, IDA has undertaken a number of key actions to strengthen results reporting and monitoring. On July 1, 2009, IDA introduced standardized core sector indicators ­ well- defined and easily measureable - for the education, health, roads, and water supply sectors. For the first time, through changes introduced in the information capture and reporting systems, IDA has the capability to aggregate and report on project level results from ongoing operations on a regular basis. These quantitative reports have been supplemented by updated IDA at Work stories and briefs, which are now being collected at the country, sector, thematic and project levels. viii. Improved Results Orientation. This report shows that IDA has made progress on Tier 2 indicators, and has increased the results-orientation of its operations. The use of results frameworks for country strategy documents has become mainstreamed. At the project level, the development and use of results frameworks has become embedded throughout the project cycle. The quality of results frameworks at entry is improving, increasing the usefulness of these frameworks for in-country monitoring during implementation and in ex-post evaluation. This improvement is paralleled by improvements in the quality of IDA's operational work - at entry, during supervision, and most importantly on completion as reflected in satisfactory development outcomes. ix. Improved Statistical Capacity. IDA countries have made progress in improving statistical capacity. Donors continue to provide assistance to countries in this effort, and a new web-based system is being launched to better monitor these improvements. Despite this progress, the availability of robust statistics in IDA countries remains a challenge and requires continued attention and support. x. Collaboration. IDA continues to collaborate with partners on sharing experiences and working on comparable approaches to results monitoring and measurement, including the use of standardized sector indicators. iii C. Next Steps xi. While significant progress has been made in reporting on the IDA RMS indicators, the following next steps will be undertaken prior to the IDA16 Replenishment: a. Monitorable Actions. All the monitorable actions have been completed, with the exception of the following, which are in progress: To review the Tier 1 PSD indicators in view of the increased emphasis on private sector in IDA15 and closer collaboration between IFC and IDA. To report on the process for capturing AAA results, and possible implications for Tier 2, on the basis of the work that is under way. To consider incentives for implementing the results agenda as part of any reform of the Strategic Performance Contract. b. Management Reviews. Management will review the following: The short and medium-term data availability for the `Access to an all-season road (% of rural population)' Tier 1 indicator. Different approaches to the assessment of the quality at entry of IDA operations so as to be able to monitor and report on this Tier 2 indicator on a timely basis. c. Proposed Change. Management has reviewed the PFM indicator and proposes to incorporate the following change into the RMS: To change Tier 1, Indicator 3 from "Public Financial Management (Number of HIPC Benchmarks Met)" to "Quality of Budgetary and Financial Management (CPIA13)" so that more recent and relevant data can be presented as part of the IDA RMS. THE IDA15 RESULTS MEASUREMENT SYSTEM: MID-TERM REVIEW I. INTRODUCTION 1. The IDA RMS was launched in 2002 during the IDA13 replenishment and enhanced in IDA14 and IDA15 to strengthen IDA's focus on results. The RMS tracks two forms of effectiveness: development or country-level effectiveness, which refers to the achievement of sustainable improvements in outcomes on the ground in IDA countries; and agency effectiveness, which refers to the work that organizations in the donor community do to ensure that their business practices conform to the principles of good aid management, have enabling effects on country capacity and institutions, and thus contribute to country-level effectiveness. 2. Two-tiered RMS. The IDA RMS is a two-tiered system. Tier 1 monitors aggregate progress on 14 country outcome indicators grouped into four categories: growth and poverty reduction; public financial management (PFM) and investment climate; infrastructure; and human development. Tier 2 monitors IDA's contribution to development outcomes using indicators of the quality and effectiveness of IDA programs as well as examples of the outputs that IDA produces in four sectors. It also monitors improvements in country statistical capacity. 3. IDA Results Orientation. Since the introduction of the IDA RMS, IDA has made concerted efforts to improve the results orientation of its operations, and to improve the reporting and systems to support those efforts. Since the last report to the Deputies,1 IDA has introduced standardized core sector indicators, improved the systematic collection and reporting of qualitative IDA at Work stories and briefs to complement the quantitative data, and worked on improving the controls environment for IDA. It is broadly recognized that it is difficult to measure development outcomes, and even more difficult to attribute specific development outcomes to particular IDA operations. Even so, the Deputies noted at the time of the IDA15 Replenishment that the IDA RMS contributes significantly to strengthening the results orientation of IDA programs and projects. 4. Purpose and Structure of this Report. The purpose of this report is to provide a midterm update for the IDA15 replenishment period on the progress made on each Tier 1 and Tier 2 indicator. The report has 4 sections. Section II presents data on the progress made in Tier 1 indicators, Section III reports on the Tier 2 indicators, and Section IV summarizes recommendations and ongoing Management actions that need to be completed. 1 IDA, "Additions to IDA Resources: Fifteenth Replenishment," February 28, 2008,Washington, D.C. 2 II. DEVELOPMENT PROGRESS IN IDA COUNTRIES ­ TIER 1 5. Since the creation of the IDA RMS, IDA has been tracking indicators of development progress at the global level. For the most recent period for which data are available, these data show that IDA countries 2 have made progress on every indicator; however, given data availability issues discussed subsequently, the data do not yet capture the most recent developments particularly the impact of the recent crises. IDA has been supporting countries in the achievement of these indicators through financing and analytic and advisory activities. Despite this progress, the rate of progress fell short of that needed to achieve most of the targets of the Millennium Development Goals (MDGs). Progress has been uneven across IDA countries; some showed strong performance, while others lagged behind. A. Global Results 6. To monitor overall development progress in IDA countries, and to address the broader question of whether IDA-eligible countries are making progress, 14 broad development outcome indicators were selected for high-level monitoring of IDA14 and were extended to monitor IDA15. These indicators focus on outcomes that are most relevant and measurable in the majority of the IDA countries. 7. At the time of the IDA15 replenishment, Management was asked to take two specific actions related to Tier 1 indicators. The first was to review the Tier 1 private sector development (PSD) indicators in view of the increased emphasis on the private sector in IDA15 and closer collaboration between the International Finance Corporation (IFC) and IDA. This review is in progress. The second was to review the feasibility of developing a Public Expenditure and Financial Accountability (PEFA)­based aggregate indicator to measure the quality of IDA countries' PFM and to replace the current indicator, "number of Heavily Indebted Poor Country (HIPC) benchmarks met."3 This review was undertaken, and is reported on below. 8. Data Availability. Table 1 summarizes the progress of IDA countries measured by the Tier 1 indicators. The data used for Tier 1 indicators come from a number of different sources, including the World Bank's World Development Indicators (WDI) and the Doing Business Report, the World Health Organization (WHO), the United Nations Educational, Scientific and Cultural Organization, and the United Nations Children's Fund (UNICEF). Each of these institutions updates data on a different cycle, and the most recent available data are presented in Table 1. In addition, some data are normalized and extrapolated from country surveys that are carried out on three- to five-year cycles--for example, for the percentage of the population living on less than US$1.25 per day. Because of the limitations of data availability across IDA countries, the baseline values and most recent values refer to different years and slightly different time periods for each of the indicators. This section elaborates on the information in the table, and Annexes 1 and 2 provide additional information on the IDA Tier 1 indicators. 2 Defined as both IDA-only and IDA/IBRD blend 3 IDA, "Additions to IDA Resources: Fifteenth Replenishment," February 28, 2008,Washington, D.C., p. 22. 3 Table 1: Progress in Tier 1 Outcome Indicators for IDA-Eligible Countries (July 2009) Aggregate outcomes Country-level outcomes % of Year of Year of countries baseline most recent Baseline Most recent % of countries making making Indicator value value value value substantial progress a progress Growth and Poverty Reduction 1. GDP per capita (constant 2000 US$) 2005 2008 511 607 89 23 2. Population living on less than US$1.25 2002 2005 46.4 42.4 83 20 b a day (%) Public Financial Management and Investment Climate 3. Public financial management (number 2001/02 2004 6 6.5 41 .. of HIPC benchmarks met) 4. Cost required for business start-up (% 2006 2009 90 68 96 26 of GNI per capita) 5. Time required for business start-up 2006 2009 43 35 58 22 (days) Infrastructure for Development 6. Access to an improved water source 2000 2006 73.1 78.4 75 38 c (% of population) 7. Mobile and fixed line phone 2004 2007 8.0 25.4 97 42 subscribers (per 100 population) 8. Access to an all-season road (% of 2000 .. 61 .. .. .. d rural population) 9. Household electrification rate (% of 2004 2007 55.1 61.9 85 .. households) Human Development 10. Under 5 mortality rate (per 1,000 live 2005 2007 107 103 91 17 births) 11. Prevalence of HIV/AIDS (% adult 2004 2007 1.5 1.38 34 34 population aged 15-49) 12. Births attended by skilled health staff 2003 2006 42.7 44.4 63 32 (% of total births) 13. Primary completion rate, total (% of 2004 2007 73.7 77.8 70 37 relevant age group) 14. Ratio of girls to boys in primary and 2004 2007 88.6 89.8 73 35 secondary education (%) Source: WDI 2009 and World Bank staff estimates. Annexes 1 and Annex 2 provide more detailed information. a IDA countries making substantial progress are those exceeding the 75th percentile of progress observed in all developing countries. b. The US$1.25 a day poverty line (measured in 2005 prices) replaces the previous US$1.08 a day poverty line (measured in 1993 prices). Poverty estimates are revised to reflect this change, using the 2005 purchasing power parity data. c This indicator measures the actual number of people who benefited from "Improved Water Sources" which include piped household connections (house or yard), public standpipes, boreholes, protected dug wells, protected spring or rainwater collection. "Improved Water Sources" do not include water provided through tanker truck or vendor, unprotected wells, unprotected springs, surface water (river, pond, dam, lake stream, irrigation channel), or bottled water. d The most recent value is not available because only a few observations have been added since the last major data compilation effort in 2006. Most countries still have only one data point available, so it is not yet possible to assess progress. (..) Data not available. 1. Growth and Poverty Reduction through 2008 9. In IDA countries, real gross domestic product (GDP) per capita grew by 5.8 percent per year between 2005 and 2008, more than double the average rate of growth of the same countries between 1990 and 2005. Almost 9 of every 10 IDA countries have enjoyed positive growth since 2005 (Figure 1). Overall, SSA IDA countries experienced little growth in real GDP per capita between 1990 and 2005, but between 2005 and 2008 growth increased at an average rate of 4 percent annually. Still, there is a gap between the growth of African economies and that of other developing countries. 4 10. Extreme Poverty. The proportion of people in IDA countries living on less than US$1.25/day4 dropped by 4 percentage points to 42.4 percent between 2002 and 2005, the last year for which aggregate estimates of poverty rates are available. However, the pace of poverty reduction in IDA countries since 1990 still falls short of the progress needed to halve the extreme poverty rate by 2015. Progress at the country level varies greatly. Extreme poverty rates dropped in four out of every five IDA countries. The largest improvement occurred in East Asia, Eastern Europe and Central Asia (ECA), followed by SSA and South Asia. On the other hand, there was little change in the small group of IDA countries in Latin America and the Caribbean countries and in the Middle East and North Africa. However, the recent slowdown in growth will likely have an adverse impact on poverty reduction in IDA countries. Figure 1: Poverty is Falling in IDA Countries as Growth is Accelerating a) Growth has been strong, though at slower pace in b) Extreme poverty has fallen in IDA countries, but not IDA countries since 2006 fast enough 10% Population living below $1.25/day (%) Blend countries 80 All IDA countries 8% 1990 2002 2005 IDA only countries 60 6% 4% 40 2% 20 0% 1990 1995 2000 2005 2010 0 2% All IDA Blend IDA only IDA countries Non-IDA countries countries countries in Sub- developing Saharan countries Africa 4% Source: World Development Indicator 2009 Source: PovcalNet and staff estimates 2. Public Financial Management and Investment Climate 11. The PEFA Steering Committee has discussed the question of the feasibility of developing a PEFA-based aggregate indicator to measure the quality of IDA countries' PFM and to replace the current "number of HIPC benchmarks met" indicator. The general conclusion is that such an aggregate indicator should not be adopted, because combining the 28 PEFA indicators will not lead to any better understanding of the PFM performance in a country and it could, in fact, lead to distorted results as countries have not made uniform progress in all the dimensions of PFM. As an alternative, Management proposes that the Country Policy and Institutional Assessment (CPIA) Question 13 continue to be used as a proxy for the aggregate PFM performance, as it was at the time of the IDA15 replenishment. Going forward, the recommendation is that this indicator be changed from "Public Financial Management (Number of HIPC Benchmarks Met)" to "Quality of Budgetary and Financial Management (CPIA13)" so that more recent and relevant data can be presented as part of the IDA RMS. 4 The extreme poverty line was updated following the availability of the 2005 benchmark Purchasing Power Parity (PPP) estimates in 2005, as were the poverty headcount estimates at country and global level. 5 12. PFM Progress. The CPIA component for quality of budgetary and financial management--CPIA Question 13--has stayed constant for IDA countries, with the average score for 75 countries increasing slightly from 3.21 to 3.24. In the ECA Region, the average score for eight IDA countries increased from 3.50 to 3.69 between 2006 and 2008. However, of the 75 IDA countries evaluated, only 13 improved their scores over the period. 13. PSD. Regulatory obstacles to PSD were significantly reduced between 2006 and 2009. It is becoming cheaper and faster to register a new business in IDA countries, as the cost required to register a start-up has dropped by about one-quarter, and the time required has dropped by about one-fifth. Although on average it is still relatively more expensive to register a new business in IDA countries than in non-IDA developing countries, it takes fewer days to complete the procedures in IDA countries (see Figure 2). Progress in SSA was particularly notable: between 2006 and 2009, about two-thirds of SSA IDA countries made one or more business environment-related reforms per year, compared to less than one-third in 2005. Figure 2: Starting a Business in IDA Countries has Become Easier a) Cost of registering a new business is declining, b) Average time required to register business is but is still relatively high in SSA falling in IDA countries % of GNI per capita 250 Time required to start a business (days) IDA countries in Sub Saharan Africa 60 IDA only countries 2006 2009 200 All IDA countries Blend countries 40 Non-IDA developing countries 150 100 20 50 0 All IDA Blend IDA only IDA countries Non- IDA countries countries countries in Sub- developing 0 Saharan Africa countries 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: Doing Business Report 2010 and staff estimates 3. Infrastructure for Development 14. In 2006, 78 percent of the population in IDA countries had access to an improved water source, nearly 14 percentage points higher than in 1990. This rate of improvement, if maintained, is adequate to meet the MDG target. Mobile and fixed lines telephone subscribers in IDA countries more than tripled between 2004 and 2007, from 8 to 25 per 100 people. The telecommunications sector is growing rapidly in all regions, but the rate of expansion in IDA countries surpassed that in non-IDA countries over the same period. All except two IDA countries have seen progress in the number of phone subscribers per 100 people, and four-fifths have experienced an annual rate of change greater than 25 percent. Along with better access, phone calls also became more affordable in IDA countries. The average price of a standard 6 basket of mobile services5 was about 7.5 percent of GDP per capita in 2007, down significantly from 12.8 percent in 2004. In the roads sector, only a few data observations have been added to the "access to an all-season road" indicator since the last major data compilation effort in 2006. Because most countries still have only one data point available, it is not yet possible to assess progress. Management will undertake a review of this indicator and the expected short and medium-term data availability, and will make a recommendation as to whether it should continue to be used, or whether it should be replaced. Access to electricity by households also improved in this period, from 55 percent in 2004 to almost 62 percent in 2007 (see Figure 3). Figure 3: Access to Clean Water and Electricity Has Increased a) Access to electricity has increased in all IDA b) Access to clean water is a challenge in IDA only groupings countries, particularly in SSA % of households with electricity connection % of population having access to improved water sources 100 100 1990 2000 2006 1990 2004 2007 80 80 60 60 40 40 20 20 0 0 IDA countries IDA countries Blend IDA only All in South Asia in Sub- Blend IDA only All in Sub- Non-IDA countries countries Saharan countries countries Sahara Africa developing Africa countries Source: IDA14 RMS Source: WDI 2009 and staff estimates 4. Human Development 15. As the recent Global Monitoring Report, WDI, and other World Bank publications have discussed, progress toward many of the MDG human development goals lags behind intermediate targets in both IDA and non-IDA countries. 16. Health. Between 2005 and 2007, the under-five child mortality rate declined modestly in IDA countries from 107 to 103 per 1,000 live births. The pace of reduction for the IDA countries as a whole and for every IDA country individually is slower than needed to meet the MDG target of reducing under-five child mortality by two-thirds between 1990 and 2015. The adult HIV/AIDS prevalence rate appears to have stabilized in IDA countries at about 1.38 of the adult population of 15-49 years. However, the number of people living with HIV has increased as new infections outnumber AIDS deaths, leaving a growing population in need of chronic care. The annual rate of increase in the proportion of births attended by skilled health personnel in 5 Price basket for mobile services is calculated as the prepaid price for 25 calls per month spread over the same mobile network, other mobile networks, and mobile-to-fixed-line calls, and during peak, off-peak, and weekend times. It also includes 30 text messages per month. 7 IDA countries was also lower between 2003 and 2006 than during the period 1990 to 2003 (with almost no improvement in SSA, and far short of what is required to have a substantial impact on improving maternal mortality rates; see Figure 4a). Figure 4: Progress on Health and Education Outcomes Has Been Slow a) The gap in maternal care between IDA countries b) SSA IDA countries as a whole have the lowest and other developing countries remains large primary completion rates Births attended by skilled health staff (% of total) Children completing primay school (% of relevant age group) 120 100 1990 2003 2006 1990 2004 2007 100 80 80 60 60 40 40 20 0 20 IDA countries IDA countries Non-IDA Blend IDA only All in South Asia in Sub- All in East Asia in Eastern in South Asia in Sub- developing & Pacific Europe & Saharan countries countries countries Saharan Central Asia Africa Africa Source: WDI 2009 and staff estimates 17. Education. The primary school completion rate increased by 4 percentage points between 2004 and 2007 (see Figure 4b), and 75 percent of children of graduation age were able to complete their primary education. Although the ratio of girls' primary and secondary enrollment to boys' increased from 88.6 percent to 89.8 percent in three years, this rate of improvement is unlikely to achieve the MDG target of eliminating gender disparities in education by 2015. In SSA, the primary completion rate for girls is still 10 percentage points lower than that for boys, and fewer than 30 percent of girls go to secondary school. B. The Impact of Global Crises 18. The unprecedented surge in fuel and food prices from 2007 through mid-2008 had mixed impacts on IDA countries. For many commodity-importing countries, high prices led to deteriorating external balances, rising inflation, and falling private consumption. The impact was most severe for the poor, whose expenditure on food and fuel often represent more than half of total expenditure. World cereal prices have fallen rapidly since mid-2008, although food prices in August 2009 are still more than 50 percent higher than in 2005. According to the Food and Agriculture Organization's most recent estimate, the chronically malnourished population exceeds one billion due to income compression. 19. Effects of Financial Crisis. Although IDA countries were not in the epicenter of the financial crisis when it hit in late 2008, they were nevertheless adversely affected as the crisis spread from the financial sector to the real economy. The negative impacts could be both short- term and long-term. GDP growth in 2009 for all IDA countries is currently estimated to be around 2.2 percent down by 3.2 percentage points from 2008. Merchandise exports in dollar 8 terms are expected to fall more than 10 percent in 2009. Net private capital flows to IDA countries in 2009 are very likely to have a sharp fall from the 2007 peak. Taking into account net factor income, remittances, and other current account items, the current account balance is likely to deteriorate significantly. Although the global recession may be coming to an end, it will likely take longer for IDA countries to return to the previous trajectory of economic growth than for developed countries and emerging market economies. 20. Longer-Term Effects. As often seen in previous crises, this crisis is likely to have a larger and longer impact on the poor and the vulnerable groups in IDA countries, and progress to date toward achieving poverty and human development MDGs is likely to stall or even be reversed. The projected decrease in remittances from migrant workers in developed countries could have a serious impact on poverty, especially in countries and among households highly dependent on remittance flows. Layoffs from export-oriented industries due to falling global demand have already pushed large numbers of families back into poverty. The impacts on other social indicators will evolve over time, but there could be 30-50,000 additional infant deaths in SSA due to the current downturn. 21. IDA's Response. IDA has responded quickly to put in place initiatives that try to help IDA countries address these evolving crises. These initiatives include the IDA Financial Crisis Response Fast Track Facility approved in December 2008 to fast-track an initial US$2 billion of support from IDA15 through potentially front-loaded new credits and grants, as well as restructuring of existing operations; the Vulnerability Financing Facility which is a dedicated facility to streamline crisis support to the poor and vulnerable by organizing under one umbrella the existing Global Food Crisis Response Program (GFRP) and the new Rapid Social Response Program focused on social interventions; and the Infrastructure Recovery and Assets Platform which supports counter-cyclical spending on infrastructure and protects existing assets and priority projects. Box 1 highlights the GFRP's activities and results6. Impacts of the crises on IDA countries will be monitored, in order to respond effectively and rapidly. 6 Box 2 (in Chapter 3) also highlights selected results from IDA support in key sectors 9 Box 1: Global Food Crisis Response Program The Global Food Crisis Response Program (GFRP) is a US$1.2 billion facility approved in May 2008 to: (a) reduce the impact of high and volatile food prices on the lives of the poor; (b) support governments in designing sustainable policies that mitigate the adverse impacts of volatile food prices; and (c) support broad-based growth in productivity and market participation in agriculture. The recent decline in world grain prices has not translated into a matching decline in many countries where food markets are not fully integrated, as in landlocked Africa. World grain prices are still 50 percent (maize) to 100 percent (rice) higher than their 2003-06 averages, and the volatility of global grain prices has doubled over the last decade. The GFRP, notably through IDA, supports both short-term coping measures and efforts to increase sustainable food production. Often, requests for advice and technical assistance precede financial support, as in Cambodia and Haiti. More than 40 countries asked for and received support in the form of analytic and advisory activities (AAA). The GFRP has supported efforts to improve agricultural productivity in Bangladesh, Mozambique, and Rwanda, a "cash for work" program in Sierra Leone, and nutritional support for vulnerable mothers in Moldova. Agricultural investment programs in countries include fertilizer procurement and distribution in Ethiopia, the Kyrgyz Republic, and Niger, seed distribution in Tajikistan, and rehabilitation of small-scale irrigation schemes in Afghanistan. Through IDA and the Food Price Crisis Response Core Multidonor Trust Fund, commitments to IDA countries by June 30, 2009, amounted to US$956 million for 46 projects in 32 countries; about 61percent of that amount has been disbursed. By streamlining processing and approval, the GFRP provided timely support to meet critical needs. Average time from concept to effectiveness of GFRP operations was less than three months. Early results are encouraging; for example over 265,000 children have benefitted from school feeding programs, and over 239,000 farmers have benefitted from over 63,000 tons of fertilizer and over 2,400 tons of seeds. III. INCREASING FOCUS ON RESULTS ­ TIER 2 22. IDA has increased the results-orientation of its operations. The use of results frameworks for country strategy documents has become mainstreamed. At the project level, the development and use of results frameworks has become embedded throughout the project cycle. The quality of results frameworks at entry is improving, increasing the usefulness of these frameworks for in- country monitoring during implementation and in ex-post evaluation. This improvement is paralleled by improvements in the quality of IDA's operational work - at entry, during supervision, and most importantly on completion as reflected in satisfactory outcomes. 23. The IDA RMS, launched in 2002 during the IDA13 replenishment, was enhanced in IDA14 and IDA15 to strengthen IDA's focus on results. Tier 2 of the system monitors IDA's contribution to development outcomes using indicators of the quality and effectiveness of IDA programs and examples of the outputs that IDA produces in four sectors. It also monitors improvements in country statistical capacity. This chapter reports on efforts by IDA and by IDA country governments to strengthen the focus on results and the ability to manage for results. A. Improving Agency Effectiveness 24. Strengthened Results Reporting and Monitoring. Since the last report to the Deputies, IDA has undertaken a number of actions to strengthen results reporting and monitoring. On July 1, 2009, IDA introduced standardized core sector indicators--well defined and easily measurable--for the education, health, roads, and water supply sectors. IDA also introduced changes in the information capture and reporting systems to enable it, for the first time, to aggregate and report on project-level results from ongoing operations on a regular basis. These quantitative reports have been supplemented by updated IDA at Work stories and briefs, which 10 are being collected at the country, sector, thematic and project levels. Selected results from IDA support to key sectors are highlighted in Box 2. Box 2: Selected Results from IDA Support in Key Sectors Water Supply IDA has directly contributed to shrinking the gap in access to water supply by at least 22 million people during 2000-2006. For example in Burkina Faso, an IDA project supported the tripling of the number of city residents with household connections, from 300,000 in 2001 to 1,040,000 in 2007. Now 94 percent of the capital city's population has full-time access to safe water, and 85 percent report that they are highly satisfied with the water utility. In Burundi, an IDA operation contributed to the construction of 335 kms of a water supply network, providing water to 120,000 beneficiaries. Roads IDA projects completed over the last eight years have built, rehabilitated, or maintained some 240,000 kms of roads, with rural connectivity benefiting over 75 million people. For example in Vietnam, an IDA project supported the rehabilitation of some 7,600 kms of roads and 26 kms of bridges. Usage increased by 70 percent between 2002 and 2004 with a 12 percent drop in travel time. The roads contributed to more frequent health visits, higher school attendance, and greater access to local government. Restoration of basic infrastructure is often IDA's first involvement in a fragile state, and is an important pre- requisite for the social infrastructure and community development work that IDA supports and provides a tangible demonstration of the benefits of peace. With the support of IDA, millions of people have been connected or reconnected to main transport networks in fragile and post-conflict states. The development of the institutional framework and technical and managerial capacity is an important objective, but progress is slower and again highlights the contextual complexity of fragile environments. For example, 1,190 kms of road have been built or rehabilitated through IDA's assistance in Burundi, Cambodia, Eritrea, the Republic of Congo, and Yemen. These roads reduce transport costs and time, and thereby increase the access to markets, schools, hospitals, dumpsites, and places of employment. Isolated villages and neighborhoods are connected, in some cases for the first time, in some cases for the first time since the start of conflict. Electricity In energy, IDA has leveraged more than two times the amount of funds it committed. In Mali, for example, an IDA project has brought electricity to 40,000 homes, 1,080 enterprises, 1,025 rural schools and 107 health clinics. IDA and IFC have been helping the Government of Senegal to increase access to energy, notably for the poor, by developing an innovative approach combining privately operated concessions with output-based subsidies to leverage private financial resources and overcome barriers of high up-front connection and in-house installation costs. Back in September 2004, the Board approved an IDA credit of almost US$30 million and a US$5 million Global Environment Facility grant to support three of those concessions for rural electrification. Comasel is the first of these to come on stream, a rural electrification concession in Senegal won by Office National de l' Electricite (ONE), Morocco´s electricity utility, which has now asked IFC to invest equity in the project company. During the first three years, the project will supply electricity to almost 20,000 customers in 298 rural villages, both by extending the grid network and by installing solar kits. Health The IDA Poverty Reduction Support Credit series in Senegal supported the Government in improving access to and use of basic health services by strengthening outreach activities, implementing improved infant and child health care management, and extending services for maternal care (increased attended birth and development of obstetrical emergency care), and by making additional progress in the prevention of infectious diseases. The 11 authorities continued their efforts to improve the monitoring of health sector performance, both collecting data and measuring the quality of health services throughout the country (producing statistical reports for 2003 and 2004). These actions contributed to positive results: an increase in diphtheria, tetanus, and pertussis immunization coverage, improvements in the attended birth rate, and a reduction in the share of underweight children. Education In India, IDA has been part of a multi-donor effort to support Government efforts to bring 20 million out-of- school children into elementary education. Since 2001, universal access is almost achieved, the gender gap has continued to shrink, with 93 girls for every 100 boys in primary school in 2008, progress has been made on inclusion, transition rates have improved, and drop-out rates for first-generation learners have fallen. The IDA development policy credit series in Bangladesh contributed to reforms in the means-tested stipend program, which is believed to have contributed to increasing the share of resources to poor children at secondary schools from 24 percent (baseline) to 33 percent at the closing of the series. The operation also contributed to improving gender parity in school enrollment, and it was synchronized with the nationwide female stipend programs to increase females' enrollment and improve their learning levels. Access to education, and improved quality of education, are especially important in stabilizing the society in fragile states. Access in post-conflict situations for girls is often especially restricted as a result of security concerns, and IDA has been instrumental in improving gender parity at the primary level. In Afghanistan, for example, 2.4 million additional students were able to attend primary school through the support of an IDA operation; 901,000 of these students were girls. 25. Internal Controls Review. To assure stakeholders that IDA funds are used as intended and show measurable results, the Bank has completed an extensive in-depth review of IDA's internal controls.7 The review found that (with some important qualifications) IDA's internal controls framework operates to a high standard overall, giving reasonable assurance that the controls operate effectively; this is important for the achievement of development results. Management is implementing a robust plan to address the issues raised in the review, including ensuring adequate resources for higher-risk projects and strengthening risk management capacity, strengthening oversight over project implementation, and tightening fiduciary rules and integrating the prevention of fraud and corruption into operations.8 7 Independent Evaluation Group, Review of IDA Internal Controls: An Evaluation of Management's Assessment and the IAD Review, (Washington, D.C., December 22, 2008). 8 OPCS and Corporate Strategy and Resources Group, Management Response and Updated Summary of Management's Overall Assessment, (Washington, D.C., February 24, 2009). 12 1. Improving Quality and Results Orientation 26. At the time of the IDA15 replenishment, Deputies asked Management to monitor and report on 13 monitorable actions related to Tier 29, six of which are new (see Box 3). This section provides a comprehensive overview of the progress achieved on these indicators. Box 3: Tier 2 Monitorable Actions on Agency Effectiveness The 13 monitorable actions agreed at the time of the IDA15 replenishment are: (i) Enhance quality of Country Assistance Strategy (CAS) Results Frameworks by strengthening the emphasis on results in the corporate review process (NEW); (ii) Monitor self ratings in Country Assistance Strategy Completion Reports (CASCRs) and their independent validation by the Independent Evaluation Group (IEG) (NEW); (iii) Monitor and report percent of IDA projects with satisfactory quality at entry; (iv) Monitor and report percent of operations whose outcome indicators in the Project Appraisal Document (PAD) (for investment lending) and in the Program Document (PD) (for development policy operations (DPOs)) cover key aspects of the Project Development Objective (PDO) (NEW); (v) Monitor and report percent of first Implementation Status and Results Reports (ISRs) submitted during the fiscal year with adequate baselines for key outcome indicators; (vi) Monitor and report percent of ISRs with satisfactory outcome baseline data (i.e. baseline availability for either one outcome or one intermediate outcome indicator) (NEW); (vii) Develop a list of uniform/standard output indicators for 4-5 sectors (NEW); (viii) Monitor percent of IDA projects with satisfactory Implementation Completion and Results Report (ICR) quality; (ix) Monitor percent of IDA ICRs that report on key outputs and outcomes from the results framework (NEW); (x) Monitor percent of IDA projects with satisfactory outcome ratings; and (xi) Monitor progress on selected aggregate project outputs in four sectors: health, education, transport, and water. (xii) Continue to improve staff and Management incentives for implementing of the results agenda by aligning performance to results. (xiii) A summary assessment of previous analyses of the AAA, current issues, future directions, and implications for Tier 2 metrics. a. Results Frameworks in Country Strategy Documents 27. Two monitorable actions for IDA15 are related to IDA's focus on results in CASs: (i) Enhance quality of CAS Results Frameworks by strengthening the emphasis on results in the corporate review process; and (ii) Monitor self ratings in CASCRs and their independent validation by the IEG. 28. Results-based CASs. IDA's support to low-income countries is largely programmed at the country level, and the CAS is the central tool for guiding the country program. Results-based CASs (RBCASs) were piloted after a discussion at the Development Committee meeting in September 2002 on how to "ensure that the CAS explains the proposed choice of World Bank interventions in light of the ultimate outcomes they are intended to affect, the intervening `results 9 IDA, IDA, "Additions to IDA Resources: Fifteenth Replenishment," February 28, 2008,Washington, D.C., p. 23-24 and Table 3. 13 chain' for getting there, and the benchmarks for measuring and monitoring progress."10 After a successful pilot, RBCASs were mainstreamed on January 1, 2005. From then until end-FY09, a total of 56 IDA RBCASs have been prepared, for 51 IDA-eligible countries. Five IDA countries have already prepared "second-generation" RBCASs: Armenia (FY04, FY09), Mozambique (FY04, FY07), Sri Lanka (FY03, FY08), Yemen (FY06, FY09), and Zambia (FY04, FY08). Over time, CASs have become more flexible and indicative to allow teams to customize IDA services to the country's own needs and priorities and to changing circumstances. 29. Corporate Review Process. The focus on results during the corporate review process has also been strengthened. At least two corporate reviews are held for each CAS before its discussion at the Board: an upstream review of the draft document, and a final downstream review. The minutes of the review meetings of the 27 IDA CASs that were prepared in FY08-09 indicate that in almost all cases there was a discussion of the results framework. For upstream reviews, the results framework was discussed in 22 of the 26 cases (85 percent) for which minutes are available; and for final reviews, the results framework was discussed in 16 of the 18 cases (89 percent) for which minutes are available. 30. CASCRs. A tool for learning and accountability, the CASCR was introduced in 2002, at the same time as the RBCAS, to capture lessons from the previous program for the formulation of the new CAS. In a CASCR, the country team uses the results matrix of the previous CAS to evaluate the program's performance and contribution to the country development goals. The country team's self-evaluation and ratings in the CASCR are independently validated and rated by IEG. In FY08-09 IEG reviewed 20 IDA CASCRs, of which 10 had self-evaluation ratings: nine were rated Satisfactory or Marginally Satisfactory, and one was rated Marginally Unsatisfactory. All 20 have IEG evaluations and ratings: 12 are rated Satisfactory or Marginally Satisfactory; and 8 are rated Marginally Unsatisfactory. 31. Evolution of CASCRS. Most of the CASCRs are based on CASs that were not results- based and therefore lack such a framework for evaluation. As second-generation CASs are being prepared, CASCRs are starting to evaluate CASs on the basis of explicit results frameworks, and country teams are formally introducing self-evaluation ratings into the CASCR. In FY08-09, four of the CASCRs were based on RBCASs (see Table 2). However, since CASCRs are currently based on both non-RBCASs and RBCASs, it is not possible to draw any conclusions or to compare performance over time. Table 2: CASCRs Based on RBCASs (FY08-09) Country Date of Evaluation Self Evaluation Rating IEG Evaluation Rating Armenia FY09 Satisfactory Satisfactory Sri Lanka FY08 Marginally Satisfactory Marginally Unsatisfactory Yemen FY09 Marginally Satisfactory Marginally Satisfactory Zambia FY08 Marginally Satisfactory Marginally Unsatisfactory Source: IEG 10 Development Committee, Better Measuring, Monitoring, and Managing for Development Results, (Washington, D.C., September 18, 2002). 14 b. Quality at Entry 32. Two monitorable actions for IDA15 are related to the quality of IDA operations at the time they are approved (at entry): (iii) Monitor and report percent of IDA projects with satisfactory quality at entry; and (iv) Monitor and report percent of operations whose outcome indicators in the PAD (for investment lending) and in the PD (for DPOs) cover key aspects of the PDO. 33. Quality at Entry. Management has monitored quality at entry for operations approved by the Board between FY96 and FY07 through eight Quality at Entry Assessments (QEA). In FY09 this series was discontinued. Instead, the Quality Assessment of Lending Portfolio (QALP)11 was used to review a sample of operations that were at the midpoint of implementation, and it included the review of quality of design as one of the dimensions of quality. Since the QALP sample was drawn from operations approved between FY03 and FY06, the most recent comparative information on quality at entry is from the last QEA for operations approved in FY07 (see Table 3). The data are encouraging, with quality at entry for IDA operations improving over the 10-year period from the start of IDA11 in FY97 to the last year for which data are available, FY07. Going forward, Management will consider different approaches to the assessment of the quality at entry of IDA operations so as to be able to monitor and report on this Tier 2 indicator on a timely basis. Table 3: Quality at Entry for IDA Operations Number of IDA Number of Projects Percentage Rated IDA Period Projects Reviewed Rated HS/S/MS* HS/S/MS* IDA11 (FY97-99) 111 87 78.4 IDA12 (FY00-02) 91 80 87.9 IDA13 (FY03-05) 124 109 87.9 IDA14 (FY06-07) Partial 63 56 88.9 * HS means highly satisfactory, S means Satisfactory, and MS means Marginally Satisfactory. Source: Data from Quality at Entry Assessments, August 21, 2009 34. Quality of Results Frameworks. To better understand the quality of the results frameworks at entry, Management carried out a comprehensive review of all the 343 IDA investment lending operations that were approved in FY08 and FY09. The review first examined the quality of the PDO and whether it conformed to the key aspects of IDA's definition of PDOs.12 It found that: in 88 percent of operations, the group targeted for project 11 The QALP assesses four inter-related dimensions of quality: quality of design; quality of implementation; quality of World Bank supervision; and likelihood of achieving development objectives. 12 "The Project Development Objective (PDO) describes the intended benefits to a specific community/group of people or organizations/institutional changes that are to be realized through one or more development interventions. The intended benefits should be measurable and specific. By reading a PDO, one should be able to determine which group is being targeted directly by the project (e.g., students, farmers) and what they will be doing better or differently as a result of the project interventions (e.g., farmers adopting new techniques). The nature of the outcome described in the PDO should be based on a realistic (and evidence-based) assessment of what effect can be achieved with the available resources over the relevant time horizon using the approach being pursued." From "Results Terminology" on the OPCS Website, August 23, 2009. 15 activities was specified; in 71 percent the expected change in behavior of the target group as a result of the project was specified; in 83 percent the benefits were measurable and specific; and in 98 percent there was good alignment between the PDO and the activities supported by the operation. On the quality of the key outcome indicators13 used in results frameworks, the review found that in 89 percent of the operations the outcome indicators captured all aspects of the PDO, and that in virtually all operations, the outcome indicators captured at least one aspect of the PDO. In 86 percent of the operations the outcome indicators measured the uptake, adoption, or use of project outputs by the target group, and in 92 percent were clearly defined and measurable. Overall, however, only 42 percent of operations scored perfectly on every dimension that was reviewed, so there are still some gaps that need to be addressed. However, the high scores on each individual dimension indicate that task teams are aware of what constitutes a high-quality results framework, and are able to apply that knowledge. 35. The 343 operations reviewed included 86 additional financing operations (AFOs) and 52 emergency recovery operations (EROs). Since these operations have special requirements for the results frameworks, additional analysis was undertaken. AFOs are prepared for operations that are being successfully implemented; these operations already have results frameworks. Of the AFOs examined, 86 percent followed good practice by updating the project results frameworks, and in almost all cases the new results frameworks improve on the results frameworks of the parent projects. In virtually all AFOs, the PDO specified a target group, and the benefits were measureable and specific. For EROs, because of the rapid timeframe within which these operations are prepared and the fluid environment in which they are implemented, a strong results framework is essential for the continual reassessment of whether the operation is still on track to meet its development objectives, or whether changes are needed.14 The guidance allows flexibility to use access to services and goods as an objective, and to use output indicators instead of outcome indicators. The review indicates that 94 percent of EROs specified a target group, higher than the 88 percent for all projects. EROs also took advantage of the increased flexibility: 90 percent defined the PDO in terms of improved access to services and goods, and 45 percent defined key output rather than outcome indicators. 36. For the 75 DPOs approved for IDA countries between FY06-Q4 and FY08, Management reviewed the quality of the results frameworks, and the conformance of the results indicators to the program of policy and institutional actions supported by the operation.15 All DPOs reviewed 13 "A Project Outcome is the uptake, adoption or use of project outputs by the project beneficiaries." From "Results Terminology" on the OPCS Website, August 23, 2009. 14 "Ensuring a Results Orientation During Project Preparation Under OP/BP 8.00 -Guidance Note for Staff" April, 2009, OPCS Website. 15 Management carried out this review as part of the 2009 Development Policy Lending Retrospective. See 2009 Development Policy Lending Retrospective: Flexibility, Customization, and Results (forthcoming). In DPOs, the Program Development Objectives (PDOs) are represented by the results that the DPO expects to achieve, as specified in the operation's results frameworks. In essence, therefore, in the context of DPOs, reviewing the quality of the results frameworks and the conformance of the results indicators to the program of policy and 16 included a results framework. The results frameworks for IDA operations contained an average of 31 results indicators, but there was wide variation--from 6 in a specialized sectoral DPO to 121 in a development policy grant with a multidonor results matrix (the size of this matrix may be attributed to efforts to remain aligned with the broad donor harmonization framework in the country). The review found that in 56 percent of the IDA DPOs the results frameworks presented clear linkages between the program supported by the Bank and the expected results of the operation. In 44 percent of the results frameworks reviewed, the program supported by the Bank and the expected results could have been more clearly linked, and the results frameworks would have benefited from the inclusion of monitoring indicators with baselines and targets (although sometimes the nature of the reforms may make it harder to use quantifiable indicators). Finally, the review noted that in about 20 percent of all the DPOs reviewed, the results frameworks represented examples of good practice, demonstrating the proper causation between the actions supported by the Bank and the expected results of a DPO with quantifiable indicators accompanied by baselines and targets. c. Quality of Results Reporting During Supervision. 37. Two monitorable actions for IDA15 are related to the quality of results reporting during the supervision of IDA operations: (v) Monitor and report percent of first ISRs submitted during the fiscal year with adequate baselines for key outcome indicators; and (vi) Monitor and report percent of ISRs with satisfactory outcome baseline data (i.e. baseline availability for either one outcome or one intermediate outcome indicator). 38. These two indicators are very closely related: the first monitors baseline data in first ISRs, and the second monitors the baseline data for all operations under implementation. 39. First ISRs. Management reviewed the first ISRs filed in each fiscal year from FY07 to FY09. Since ISRs can be filed at any time within the first 12 months after Board approval of an operation, some of these ISRs are for projects approved in earlier fiscal years. About 57 percent of the ISRs reviewed have adequate baseline data for every PDO indicator, and there is a slight year-to-year improvement (see Table 4). An additional 27 percent have adequate baseline data for at least one PDO indicator. Thus, for first ISRs filed in FY07-09, in 84 percent of operations some baseline data is available in the ISR for use in monitoring project performance results. Reviewing the data by approval fiscal year instead of by fiscal year of first ISR, 83 percent of IDA14 operations (FY06-08) that are still in the active portfolio had adequate baseline data for at least one PDO indicator. The data for FY09 approvals are incomplete: for the 53 operations that have already filed first ISRs, 85 percent have at least one outcome or one intermediate outcome indicator, which is equivalent to the IDA14 results. institutional actions supported by the operation is equivalent to reviewing the extent to which indicators in the Program Document for DPOs cover key aspects of the PDOs. 17 Table 4: First ISRs with Adequate Baselines for PDO Indicators FY07 FY08 FY09 FY07 FY08 FY09 Category Number Percentage Number of First ISRs Filed in the Fiscal Year 149 160 150 First ISRs in Which All PDO Indicators Have 80 93 88 53.7% 58.1% 58.7% Baselines First ISRs in Which At Least One But Not 42 41 39 28.2% 25.6% 26.0% All PDO Indicators Have a Baseline First ISRs in Which None of the PDO 27 26 23 18.1% 16.3% 15.3% Indicators Have Baselines Source: OPCS Review, July 2009 40. Latest ISRs. For the second indicator, since the review of first ISRs (discussed above) indicated that 83 percent of IDA14 operations in the active portfolio had adequate baseline data for at least one PDO indicator, Management reviewed a representative sample (70 percent, i.e., 45 out of 64) of the remaining operations. The review of the latest ISR for these operations found that, as expected, task teams are adding baseline information to the supervision reports as it becomes available: 76 percent of the operations in this sample have added PDO baseline data for at least one PDO indicator. Within the sample, the percentage is higher for those that have been under implementation for a longer period. d. Development of Core Indicators to Monitor Results During Implementation 41. One monitorable action for IDA15 is related to the systematic capturing of aggregate data from ongoing operations: (vii) Develop a list of uniform/standard output indicators for 4-5 sectors. 42. To improve the ability to capture, aggregate, and report on the results achieved through IDA's support, the Deputies encouraged IDA to adopt uniform output indicators for key sectors--information that would complement more detailed project-specific results data, as well as country and sector results data. On July 1, 2009, IDA adopted standardized Core Sector Indicators to be used in IDA-supported investment operations for the education, health, road transport, and water supply sectors (Annex 3 provides the list of Core Sector Indicators). Systems changes were made to facilitate the collection and aggregation of data through the ISR template. 43. Implementation. Since July 1, 2009, staff have been working to review the IDA- supported operations in these four sectors that were approved under IDA14 (FY06-08) and IDA15 (FY09), to apply the core indicators as relevant, and to provide the available results data (baseline, progress, and target) in the ISRs. Of the 261 operations in the IDA14-15 active portfolio that are in these four sectors, task teams have found that the core indicators apply to 184 operations. The data collected so far are preliminary, based on readily available information; the indicators have not yet been incorporated into country monitoring and evaluations systems, and many projects have only recently been approved or started implementation. Nevertheless, these early results are encouraging, and show that IDA has already contributed to the achievement of significant results (see Box 4 and Annex 3). 18 Box 4: Core Sector Indicator Data The Core Sector Indicator data from August 21, 2009 show that active projects in the IDA14-15 portfolio have already contributed to the achievement of results: Education 1 million additional teachers are qualified to teach at the primary level. 600,000 additional classrooms are now available for use through new construction or rehabilitation. Health 7 million people have access to a basic package of health, nutrition, or population services. 275,000 health personnel have received training. 2,000 health facilities have been constructed, renovated or equipped. 7.2 million children have received a dose of Vitamin A, and over 800,000 children have been immunized. 7.8 million pregnant women have received ante-natal care. 18.5 million insecticide-treated malaria nets have been purchased and/or distributed. 21,400 adults and children with HIV have received antiretroviral therapy. Road Transport 2,480 km. of rural roads and 1,790 kms of non-rural roads have been constructed or rehabilitated. Water Supply About 8,500 community water points have been constructed or rehabilitated. About 60,500 new piped household water connections have been established, and another 146,500 have been rehabilitated. 1,360 water utilities and water service providers are being supported. e. Quality of Results Reporting in ICRs 44. Two indicators for IDA15 are related to the quality of IDA's self-evaluation of projects at exit through the ICR: (viii) Monitor percent of IDA projects with satisfactory ICR quality; and (ix) Monitor percent of IDA ICRs that report on key outputs and outcomes from the results framework. 45. IEG Findings. IEG evaluates the quality of ICRs. Over the last 10 years (FY99 to FY09), IEG has found that 90 percent of ICRs for IDA operations have had satisfactory quality. For evaluated FY07 exits, 92 percent of the ICRs had satisfactory quality, and for evaluated FY08 exits, 94 percent of the ICRs had satisfactory quality.16 46. Outcome and Output Data. Management reviewed the use of key outcome and output data in the 110 FY08 and 88 FY09 IDA ICRs. The review found that ICRs reported on actual data related to the achievement of the PDO in 78 percent of ICRs, with partial data available in another 15 percent of the ICRs. Data were fully used in the assessment of the PDO in 53 percent of the ICRs, and partially used in another 35 percent. Output data were available in 60 percent of the ICRs. 16 IEG data in Business Warehouse, August 17, 2009. 19 f. IDA Outcomes 47. Two indicators for IDA15 are related to the outcomes of IDA-supported operations: (x) Monitor percent of IDA projects with satisfactory outcome ratings; and (xi) Monitor progress on selected aggregate project outputs in four sectors: health, education, transport, and water. 48. Satisfactory Outcomes. IEG evaluations indicate that over the 12 years of IDA11 to IDA14, satisfactory outcomes at exit have been steadily improving. Based on the fiscal year in which the projects exited (which is the traditional way of reporting satisfactory outcomes at exit), outcomes improved from 64 percent during IDA11 to 76 percent during IDA14. Satisfactory outcomes at exit for operations that exited during FY08 were 82 percent. Based on fiscal year of approval, the same trend is evident, with satisfactory outcomes at exit for operations approved as part of IDA11 that have already exited the portfolio at 72 percent, and for operations approved under IDA14 that have already exited the portfolio at 87 percent (see Table 5). Table 5: Satisfactory Outcomes at Exit Based on IEG Evaluations Exit Fiscal Years Approval Fiscal Years IDA Period Satisfactory Number of Satisfactory Number of Projects Outcomes at Exit Projects Outcomes at Exit Evaluated Evaluated IDA11 (FY97-FY99) 64.4% 379 72.3% 264 IDA12 (FY00-FY02) 71.2% 392 73.8% 206 IDA13 (FY03-FY05) 74.2% 430 79.3% 88 IDA14 (FY06-FY08) 75.6% 332 86.8% 33 Source: IEG Data in Business Warehouse, August 17, 2009 49. Sectoral Outputs. The introduction of the core indicators, described above, will over time allow IDA to report not only on outputs for completed operations--as it now does--but also on outputs for operations under implementation. In the interim, for the purposes of this report, the data on selected aggregate outputs in the health, education, transport, and water sectors for projects that exited during the IDA14 and IDA15 periods have been manually tabulated from project ICRs (see Box 5 and Annex 4). These data show that through the support of IDA operations that closed in FY09, important results were achieved. Box 5: Outputs from IDA Operations that Closed in FY09 Through the contribution of IDA operations that closed in FY09, the following results have been achieved: Health 227 hospitals were built or rehabilitated; 2,735 health professionals were trained; 85,000 insecticide treated bed nets were distributed. Water Supply and Sanitation Over 2 million people benefited from water supply and sanitation projects; 3,887 new water connections were established; 92,541 new sanitation facilities were established. Transport 897 km of road were rehabilitated or maintained. 20 g. Staff Incentives 50. At the time of the IDA15 Replenishment, Deputies asked that Management: (xii) Continue to improve staff and Management incentives for implementation of the results agenda by aligning performance to results. 51. Management continues to refine corporate performance review and reporting. One of the key actions for improving the planning, budget and performance management process is the improvement of tools such as the Strategic Performance Contract. Management will consider incentives for implementing the results agenda as part of any such reform. h. Analytic and Advisory Assistance 52. At the time of the IDA15 Replenishment, Deputies asked for: (xiii) A summary assessment of previous analyses of the AAA, current issues, future directions, and implications for Tier 2 metrics. i. Assessment and Issues 53. Recent assessments of IBRD/IDA AAA by IEG, the Quality Assurance Group (QAG), and the Internal Audit Department (IAD)17 have found that AAA is of high quality and that clients appreciate its contribution. IEG also found that economic and sector work (ESW) and technical assistance (TA) tasks met their stated objectives, and were effective in shaping lending. Stakeholders in most countries indicated that ESW and TA tasks had made a difference to the reforms in their countries. These reviews also indicated that the role of AAA in enhancing development outcomes could be strengthened by improving: strategic focus, management practices, systems and processes, and results monitoring. Management is taking steps to address each of these issues. 54. Strategic Focus and Impact. While IEG evaluations and QAG assessments have noted that the overall quality of ESW and TA is high, they have also found that better integrated AAA programs that respond effectively to the country's development strategy are of higher quality and have higher impact. AAA products with limited strategic relevance to a given country program can also lead to fragmented and underfunded tasks, especially in IDA countries. Client 17 IEG, The World Bank's Economic and Sector Work and Technical Assistance FY00-06, July 2008; Annual Review of Development Effectiveness, (Washington, D.C., 2008); QAG, Quality of Country AAA (Phases I and II"), 2008; "Managing AAA Droppage: A QAG Assessment, (Washington, D.C., 2008); QAG, Annual Review of Portfolio Performance FY09, (Washington, D.C., 2009); IEG, Review of IDA Internal Controls: An Evaluation of Management's Assessment, 2009, (Washington, D.C., 2009); IAD, Audit of the Process for Managing ESW and Non-Lending Technical Assistance, (Washington, D.C., 2008). IEG, QAG and the IAD audit have all reviewed only ESW and TA, which account for an average of 80% of spending on AAA at the country level. The IDA Internal Controls exercise focused solely on a sample of large ESW reports (i.e. over $100,000). Therefore the use of the term AAA in this document is used interchangeably with ESW and non- lending TA. 21 ownership is a critical predictor of AAA products that are likely to have significant impact; obtaining client feedback, both during the design of AAA and after their completion, is important but it is not done frequently enough. Differences in ratings on "likely impact" can also be attributed to the quality of the follow-up dialogue. 55. Management Practices. The FY08 QAG assessment of country AAA programs (53 countries, of which 28 were IDA-eligible) found that there was a strong correlation between AAA program ratings and the level of management oversight from program inception to completion. Although oversight of country AAA is strong in some areas, greater emphasis is needed on ensuring that AAA programs are aligned with strategic priorities and have coherence. Greater attention and management oversight during dissemination and implementation would improve impact. The assessment recommended clearer accountability with respect to strategic choices and task implementation. 56. Systems and Processes. An IDA Internal Controls review assessed the adequacy of controls for effective management tracking of different stages of the AAA cycle--from initiation through delivery to the client and completion. This assessment noted that while controls are in place, the systems and processes are cumbersome, affecting the quality of information needed for managing and oversight of AAA products. The system through which AAA work is currently stored and archived needs improvements to make it easier to search and retrieve valuable knowledge outputs. The IDA Internal Controls review noted that institutional processes and systems for planning, monitoring, and tracking AAA tasks should be "simplified and rationalized for the wide scope of AAA activities."18 57. Results Monitoring. Finally, assessments of AAA have noted that results monitoring frameworks should be improved, both at the level of the individual activity and at the country program level. In FY05, the World Bank introduced a results tracking framework for individual ESW and TA activities that is based on self-assessment, to link a task's objectives (selected from a predetermined list shown in Table 6) to its interim results. The results for ESW and TA activities delivered to IDA-eligible clients in FY08 and FY09 for which an assessment has been completed are shown in Table 6. The most frequently selected development objective for ESW was to inform government policy (87 percent and 79 percent for FY08 and FY09, respectively); the self-assessments of two-thirds of these tasks indicated that these objectives were "fully" or "largely" achieved. For TA, the most commonly selected development objective was developing/strengthening institutions (at 88 percent and 74 percent for FY08 and FY09, respectively); 76 percent and 70 percent, respectively, of the activities were self-assessed to indicate that these objectives were "fully" or "largely" achieved. Overall, the majority of both ESW and TA achieved their selected development objectives in FY08-09. (Box 7 shows selected results of AAA products.) 18 IAD, Review of IDA Internal Controls: An Evaluation of Management's Assessment, (Washington, D.C., 2009); Audit of the Process for Managing ESW and Non-Lending Technical Assistance, (Washington, D.C., 2008). 22 Table 6: IDA AAA Results in FY08 and FY09 FY08 FY09 Number of Activities Percentage Number of Percentage of Reporting of Activities Activities Activities Percentage DO `Fully' Reporting Reporting Reporting DO of Activities or DO `Fully' Percentage DO `Fully' `Fully' or Development With This `Largely' or `Largely' of Activities or `Largely' `Largely' Objective (DO) DO1 Achieved Achieved2 With DO1 Achieved Achieved2 Economic and Sector Work Inform Lending 58% 70 80% 67% 12 75% Inform Government 87% 86 66% 79% 12 63% Policy Build Client Analytic 44% 26 61% 42% 7 70% Capacity Inform/Stimulate 65% 78 80% 67% 14 88% Public Debate Influence the 50% 39 57% 50% 8 67% Development Community Technical Assistance Assist in Client's 41% 43 86% 64% 23 72% Policy/Program Implementation Develop/Strengthen 88% 81 76% 74% 26 70% Institutions Facilitate Knowledge 39% 45 94% 46% 20 87% Exchange 1 Note: Because a given activity can contain more than one DO, the totals add to more than 100 percent. 2 Includes activities delivered to client in FY09 for which a results rating has been completed as of August 26, 2009. 23 Box 6: Selected Results from IDA AAA Products Albania. In 2005, approximately 19 percent of Albanians were living in poverty, including 3.5 percent on less than US$1 a day. For more effective efforts to reduce poverty, the Government required detailed and accurate information on poverty. Therefore, there was an increasing need to build government capacity to measure, monitor, and evaluate poverty with a view to identifying appropriate policy responses. IDA contributed US$400,000 to the Albania Programmatic Poverty Assessment Program, and an IDA team worked with the Government to conduct an effective analysis of poverty indicators and build local capacity for poverty measurement and monitoring. As a result, (a) the social assistance program is now better targeted--it reached one-third of the poor in 2005 compared to one-quarter in 2002, and a new social assistance reform strategy will target more poor people by updating the registry and revising the targeting method; (b) policymakers are using key findings and analyses to formulate poverty reduction action plans; and (c) the program contributed to the social inclusion strategy, which identifies key vulnerable groups (children, women, disabled, Roma) in need of special services. Nigeria. Nigeria's economy has relied disproportionately on oil revenue, making the country vulnerable to global price fluctuations. As a result, Nigeria has long aspired to grow additional sectors of its economy to diversify, spread the wealth, and insulate itself from booms and busts. During the period 2001-2007, Nigeria's non-oil economy recorded sustained, broad-based growth. In fact, the average growth was more than double the average growth between 1995 and 2000. On the surface, it seemed like good news. But several questions remained for public officials and policymakers alike: What was the impact on employment and income distribution? What did this mean for future growth strategies and industrial policy? Had the country indeed made progress in its competitiveness and trade? The Employment and Growth Study was a primary component of IDA's collaboration with the Nigerian Government. As a result, (a) the Growth Strategy component got the attention of Nigeria's Federal Minister of National Planning, who is now designing Nigeria's Vision 2020, which aims at developing a growth strategy for the country to become one of the world's 20 largest economies by 2020; he endorsed the study's key recommendations and cited it as a key input to Vision 2020; (b) the study's critical review of Nigeria's technical and vocational education system resonated strongly with several key policymakers, who subsequently advocated developing a more integrated framework for skills formation, building on those sectors with the highest growth and employment potential; and (c) following the study's presentation, the Minister of National Planning stated that import bans had not been effective, and should either be replaced by high tariffs or abolished (the study had found that import bans had adversely affected Nigeria's competitiveness and contributed to port congestion and customs inefficiencies, while not meeting their stated objective of protecting key domestic industries). ii. Future Directions 58. Management is engaged in a comprehensive effort to address the issues identified and summarized above. Since the assessments have found the technical quality of AAA to be good, the focus of the effort is on enhancing the effectiveness and impact of AAA through three mutually reinforcing measures: (a) upstream strategic discussion on the expected contribution of IDA-provided AAA to agreed country objectives, where possible, clustered around a given objective; (b) effective management of the AAA process; and (c) systems that provide greater transparency regarding the content of the AAA that is produced. In addition, although it is inherently difficult to evaluate the effectiveness and results of AAA,19 further efforts are needed to refine activity-level tracking and ensure more complete reporting by task teams. At the country program level, work is needed to improve the alignment of AAA with RBCASs, drawing on the principle of country ownership. To better track AAA activities while helping to reduce 19 IEG has acknowledged the difficulty to measure outcomes associated with analytic work (rather than inputs or even outputs). This difficulty is complicated by the fact that the impact of AAA may become apparent only in the long run. Other factors compounding the difficulty of measuring impact include attribution of a result to specific AAA and aggregating results of different kinds of AAA. 24 fragmentation and ensure adequate financing of AAA, the adoption of more programmatic approaches is under consideration. iii. Implications for Tier 2 AAA Metrics 59. It is not yet clear which indicators of AAA effectiveness and results could be incorporated into Tier 2 reporting. An approach will be presented to the Deputies at the time of the IDA16 Replenishment discussions. B. Improving Government Statistical Capacity 60. Weaknesses in statistical capacity constrain effective national policy analysis and decision making, and prevent the accurate monitoring of development progress. At the time of the IDA15 Replenishment discussions, Deputies asked for a report on four interrelated issues (see Box 8). Box 7: Improving Government Statistical Capacity Deputies asked IDA to report on four issues: Status (i) Develop web-based standards system to help IDA countries track progress towards Ongoing internationally accepted norms of data coverage, frequency, and timeliness; (ii) Explore the potential for expanding and re-designing the existing Trust Fund for Statistical Completed Capacity Building (TFSCP) to primarily provide a mechanism to finance and co-finance as well as to supervise investments in statistical capacity and implementation of National Strategies for the Development of Statistics (NSDSs); (iii)Jointly with donors and partners, intensify efforts to identify "lead donors" for work in Ongoing selected priority countries and accelerate efforts to develop a sector-wide approach to building statistical capacity in these priority countries; and (iv) In each IDA CAS, require a more comprehensive discussion of weaknesses in IDA Ongoing country statistical systems and their use in decision making. Where relevant, follow up in DPOs, Investment Lending, AAA, CASPRs, CASCRs, and ICRs. 61. Improved Statistical Capacity. Using a proxy indicator of statistical capacity (the "Statistical Capacity Score")20 maintained by the Development Data Group for measuring trends in country statistical capacity, some progress by IDA countries is evident. Of the 61 IDA countries with scores available in 1999 and 2008, 47 showed improvement in capacity; of those, one-quarter showed substantial improvement. 62. Variations among Country Groups. Change in SSA is less clear. Although on average SSA IDA countries improved over the same period, data collection activities and practices remained practically unchanged. In addition, while about two-thirds of the SSA countries showed improvement, progress was substantial for only two countries. Of the 14 IDA countries which declined or did not make progress, 13 were in SSA. By contrast, non-SSA IDA countries made significant progress in all aspects measured. 20 The Statistical Capacity Score is a composite indicator based on three dimensions: statistical practice, data collection, and indicator availability. Information about national statistical systems related to these three areas is collected from existing international data sources. 25 1. New Web-Based System 63. IDA's system to monitor statistical capacity is being significantly improved and expanded. A "Bulletin Board" system will allow countries to provide updated information using a web-based application. Indicators will provide an assessment of the extent to which specific methodological or timeliness standards are being followed. Areas covered include the institutional framework, data collection practices, methodology, and data dissemination. This new system will help track global improvements, and more importantly it will help countries themselves identify weaknesses and plan remedial action. Development of the database application is complete, and national statistical institutes in IDA countries have been advised of this new system, notified about the information it contains, and invited to provide updates or new data. 2. Strategies to Support NSDSs 64. Partnership Strategies. The focal partnership strategy for addressing weaknesses in statistical capacity, agreed internationally in 2004, is the Marrakech Action Plan for Statistics (MAPS). MAPS aims to help countries (a) implement country-owned national strategies for improving statistical capacity (now commonly known as National Strategies for the Development of Statistics or NSDSs); (b) conduct a population census during the decennial 2010 census "round"; (c) improve their household surveys and make data more accessible; and (d) better coordinate international efforts. One of the key successes of MAPS has been that almost all IDA countries now have a NSDS, which is an important prerequisite for making substantial and sustainable improvement in statistical capacity. Only six IDA countries are now without an NSDS, or are not planning one; and almost all countries in Africa now either have an NSDS or are working on one. 65. Helping countries develop an NSDS has been a major focus of the Partnership in Statistics for Development in the 21st Century (PARIS21) partnership--which has provided advocacy support and technical advice--and the World Bank's Trust Fund for Statistical Capacity Building (TFSCB), which has provided small grants directly to countries for NSDS development. In addition, the World Bank supports the international Capacity Scan (CAP-Scan) effort, which helps countries undertake a self-assessment of gaps in their capacity for results management, including statistical capacity; so far, CAP-Scan has been implemented in three IDA countries: Mauritania, Niger, and Senegal. A further tangible success of the MAPS strategy has been the Accelerated Data Program (ADP), a satellite program of PARIS21 supported by the World Bank's Development Grant Facility, which helps countries make survey data more accessible, and improves the quality of national survey programs. The ADP was originally started as a pilot program; however there has been strong demand from countries to participate, and 27 IDA countries are now involved in it. 66. Use of IDA Resources. Consistent with the IDA15 recommendation to support investment in this area, some countries used IDA resources to support statistical capacity building projects and the implementation of NSDSs, either through the World Bank Statistical 26 Capacity Financing Program (STATCAP)21 program or as part of larger operations. During IDA15, the focus has been on helping countries to move from the preparation to the implementation of NSDSs. As participants in the 2007 Third International Roundtable on Managing for Development Results noted, a key constraint is the availability of finance for investing in statistical capacity, from both national budgets and development partners. There are now active projects in Burkina Faso, Kenya, Nigeria, Sri Lanka, and Tajikistan. In Mongolia, a new five-year project was approved in June 2009 to use the NSDS to improve the national statistical system to provide better data to support Government's MDG-based National Development Strategy. Projects are also being prepared for Bolivia and Tanzania. 67. New Trust Fund. Some countries, however, are still reluctant to use IDA resources for this work. To encourage further investment at the country level, a new trust fund, the Statistics for Results Facility Catalytic Fund (SRF-CF), has been established, with funding from the Netherlands and the United Kingdom; it is administered by the World Bank. The SRF-CF, which became effective on July 30, 2009, aims to increase the capacity of IDA countries in formulating policies and decision-making through sustained improvement in the production, availability, and use of quality statistics for managing and measuring country development results. The SRF-CF will support improvements in NSDSs that will be linked to the needs of national and sectoral monitoring frameworks and that will promote improved dialogue and partnership between users and producers of statistics. In the first stage, the SRF-CF is expected to support five pilot IDA- eligible countries that have weak statistical capacity.22 A key focus of the SRF-CF is greater attention to aid effectiveness principles, including the use of a systemwide approach recommended in IDA15. The TFSCB and new SRF-CF will be complementary: TFSCB will provide resources on a limited scale to help countries implement priority actions and develop good-quality implementation plans, and the SRF-CF will help mobilize large-scale investment resources needed to put those plans into action. 3. Identification of Lead Donors 68. A criterion for receiving funds under the SRF-CF is the identification of an in-country partnership group consisting of lead and active donors to work together with the government on statistical capacity issues; the five pilot countries are now going through this process. A similar approach is also being encouraged in other countries with IDA-funded statistical capacity programs. 4. IDA CASs: Discussion of Country Statistical Systems 69. To highlight statistical capacity issues as part of the CAS development process, input is now routinely provided to CAS teams during the upstream review process. Moreover, to help the CAS team to address statistical issues during the initial stages of the process, OPCS has prepared a guidance note that includes a simple checklist to help staff identify: (a) key issues related to the supply and demand for official statistics in the country; and (b) sources of 21 STATCAP is a horizontal adaptable program loan, approved in 2004, designed to simplify procedures for developing and implementing statistical capacity building projects. 22 The following pilot countries have been selected by the SRF-CF Governing Council to apply for funds: Afghanistan, Democratic Republic of Congo (DRC), Ghana, Nigeria, and Rwanda. 27 information and methods for assessing national statistical capacity. Use of the checklist helps determine any potential areas of focus for IDA and other development partners, and possible operational interventions. To complement this note, a brief statistical module is being designed for the "CAS Academy" training course, and a new "e-learning" course has been developed for potential task team leaders of statistical capacity-building projects. C. Improving MDB Collaboration in Results Measurement 70. In 2003 the multilateral development banks (MDBs) established a Working Group on Managing for Development Results. The World Bank has been a member of the group since the beginning, and is currently the chair. It is collaborating with the Working Group members and other donor agencies on a comparable approach to results measurement and reporting, including the adoption of standardized core sector indicators. One of the products of the Working Group is the Common Performance Assessment System Report (COMPAS),23 which presents information on the results orientation of the MDBs and highlights new initiatives and progress made by each participating MDB in improving policies, operational processes, and systems toward achieving results on the ground. This year the World Bank led the effort to publish COMPAS. 23 See www.mfdr.org/COMPAS. 28 IV. NEXT STEPS 71. Overall, IDA countries (both IDA-only and IDA/IBRD blend) made progress on every indicator tracked in the Tier 1 IDA RMS during the most recent period for which data are available. However, progress was uneven across IDA countries and the rate of progress fell short of that needed to achieve most of the MDG targets. The Tier 2 indicators show that IDA has significantly increased its focus on managing for development results, and has collaborated with governments and partners to support their focus on development results. Moreover, IDA has responded quickly and effectively to short-term and long-term development needs. With the support and contribution of IDA operations, significant results have been achieved. 72. Monitorable Actions. All the monitorable actions have been completed, with the exception of the following, which are in progress: To review the Tier 1 PSD indicators in view of the increased emphasis on private sector in IDA15 and closer collaboration between IFC and IDA. To report on the process for capturing AAA results, and possible implications for Tier 2, on the basis of the work that is under way. To consider incentives for implementing the results agenda as part of any reform of the Strategic Performance Contract. 73. Management Reviews. Management will review the following: The short and medium-term data availability for the `Access to an all-season road (% of rural population)' Tier 1 indicator. Different approaches to the assessment of the quality at entry of IDA operations so as to be able to monitor and report on this Tier 2 indicator on a timely basis. 74. Proposed Change. Management has reviewed the PFM indicator and proposes to incorporate the following change into the RMS: To change Tier 1, Indicator 3 from "Public Financial Management (Number of HIPC Benchmarks Met)" to "Quality of Budgetary and Financial Management (CPIA13)" so that more recent and relevant data can be presented as part of the IDA RMS. 29 Annex 1: Changes in IDA Results Measurement System Tier 1 Outcome Indicators Year Outcomes c Benchmark rate of change(ROC) (%) Average Non-IDA Countries annual rate IDA past IDA required developing included a Coverage b Most Most of change ROC (1990 - ROC to countries Indicator Units (no.) (%) Baseline Recent Baseline Recent (ROC) (%)d baseline) achieve MDG recent ROC Recent progress exceeds the average ROC to reach MDG target 1. Proportion of population below Percent of population 64 91 2002 2005 46.4 42.4 -3.0 -1.7 -2.8 -15 US$1/day poverty line 3. HIV prevalence rate of adult Percent of population 53 96 2004 2007 1.5 1.38 -2.8 .. <=0 1.0 population aged 15-49 age 15-49 7. Proportion of population with Percent of population 55 98 2000 2006 73.1 78.4 1.2 1.2 1.1 1.0 sustainable access to an improved water source Recent progress exceeds IDA past ROC (1990--baseline) 2. Under-5 child mortality Per 1,000 live births 77 100 2005 2007 107 103 -2.1 -1.9 -4.4 -5.8 6. Primary school completion rate Percent of population 61 87 2004 2007 73.7 77.8 1.8 1.3 2.5 .. of official graduation age 8. Fixed lines and mobile telephone Per 100 people 75 100 2004 2007 8.0 25.4 38.7 17.0 n/a 16.4 per 100 inhabitants 12. GDP per capita Constant 2000 $US 7 97 2005 2008 511 607 5.8 2.7 n/a 6.2 14. Household electrification rate Percent of households 46 95 2004 2007 55.1 61.9 3.8 2.5 n/a .. IDA progress exceeds Non-IDA developing countries recent ROC 4. Proportion of births attended by Percent of births 71 91 2003 2006 42.7 44.4 1.3 1.6 3.8 0.9 skilled health personnel 5. Ratio of girls to boys in primary and Percent 50 79 2004 2007 88.6 89.8 0.4 1.4 1.3 0.3 secondary education 10. Time required for business start up Days 76 100 2006 2009 43 35 -7.0 .. n/a -5.7 IDA progress lags or no comparison possible 9. Formal cost required for business Percent of GNI per capita 75 100 2006 2009 90 68 -9.1 .. n/a -20.5 start up 11. Public financial management Number of benchmarks 22 27 2001/02 2004 6 6.5 2.7 .. n/a .. met 13. Access to rural population to an all- Percent of rural population 32 83 1995/2003 n/a 61 .. .. .. n/a .. season road Source: WDI 2009 and staff estimates. (..) means insufficient data, (n/a) means not applicable a 78 IDA eligible countries are considered in the aggregation for all indicators which were reviewed in May 2009. A 79th country has become IDA eligible since that time. b Percent of relevant population from total relevant population in the 78 IDA countries. c 2 indicators do not have sufficient information yet to measure progress between IDA periods. d The average annual growth is calculated between the baseline estimated value and the most recent estimated value. It is the most recent rate of growth experienced by the indicator. 30 Annex 2: Data Availability for IDA RMS Tier 1 Indicators Beginning of IDA14 a Mid-Term Review a May 2009 b # of IDA # of IDA # of IDA # of IDA countries Indicator countries # of IDA # of IDA countries # of IDA # of IDA countries included in included in countries countries included in countries countries with data calculation of calculation of with data in with data in calculation of with data in with data in in aggregate aggregate 2000-02 2002-04 aggregate 2000-02 2005-07 2000-02 performance performance performance GDP per capita (constant 2000 US$) 77 77 77 77 78 74 74 74 Population below US$1 a day (%) 15 48 25 21 48 31 23 62 Public financial management 24 24 23 29 22 23 29 22 d d Cost required for business start-up (% of GNI per capita) 55 38 0 66 65 0 76 65 d d Time required for business start-up (days) 55 39 0 66 65 0 76 65 e Access to an improved water source (% of population) 75 34 80 79 61 77 73 55 Fixed line and mobile phone subscribers (per 1,000 people) 80 79 80 79 80 77 75 76 Access to an all-season road (% of rural population) 11 .. 10 8 .. 10 8 .. Household electrification rate (% of households) 18 .. 21 2 .. ? ? ? e Under 5 mortality rate (per 1,000) 80 79 80 80 80 78 78 78 e c c d Prevalence of HIV/AIDS, (% adult population aged 15-49) 53 44 0 63 63 54 60 53 Births attended by skilled health staff (% of total) 53 52 61 40 64 61 55 72 Primary completion rate, total (% of relevant age group) 62 55 62 63 62 59 63 61 Ratio of girls to boys in primary and secondary education (%) 52 52 63 64 54 63 62 51 Source: WDI 2005, 2007, 2009 and staff estimates. a 80 IDA eligible countries in total, including Serbia and Montenegro but excluding Timor-Leste. b 78 IDA eligible countries in total in May 2009; a 79th country has become IDA eligible since that time. c The availability was based on prevalence of HIV/AIDS among women aged 15-24. d Previous estimates were removed as either the methodology changed or the methodology only provides estimates for benchmark years. e Availability is based on the official estimates reported by WHO, UNICEF, and United Nations Joint Programme on HIV/AIDS, and may under represent the availability of underlying primary data. 31 Annex 3: Data on Core Sector Indicators for IDA 14-15 Operations: August 21, 2009 Sector Indicator Incremental Number of Progress Projects Reporting Progress Education Decline in Shortfall of Qualified Teachers at the Primary Level (number) 1,012,354 5 Education Decline in Shortfall of Classrooms at the Primary Level (number) 617,251 11 Education System for learning assessment at the primary level (rating scale) [1] 18 Health People with access to a basic package of health, nutrition, or population services (number) 7,095,576 7 Health Health personnel receiving training (number) 275,371 30 Health Health facilities constructed, renovated, and/or equipped (number) 2,058 23 Health Children immunized (number) 870,422 6 Health Pregnant women receiving antenatal care during a visit to a health provider (number) 7,814,231 7 Health Children receiving a dose of vitamin A (number) 7,248,547 5 Health Long-lasting insecticide-treated malaria nets purchased and/or distributed (number) 18,572,851 9 Health Adults and children with HIV receiving antiretroviral combination therapy (number) 21,421 3 Health Pregnant women living with HIV who received antiretrovirals to reduce the risk of MTCT (number) [2] - Roads Roads constructed, non-rural (kms) 24 1 Roads Roads constructed, Rural (kms) 34 2 Roads Roads rehabilitated, Rural (kms) 2,454 16 Roads Roads rehabilitated, Non-rural (kms) 1,766 18 Roads Roads in good and fair condition as a share of total classified roads (% ) [3] 11 Roads Share of rural population with access to an all-season road (% ) [4] 8 Roads Average time from ship readiness to unload to final destination for an imported container, on the corridor [5] 1 targeted by the project (days) Water Supply People in project areas with access to Improved Water Sources (number) 2,379,452 28 Water Supply Improved community water points constructed or rehabilitated under the project (number) 8,498 15 Water Supply New piped household water connections that are resulting from the project intervention (number) 60,494 10 Water Supply Piped household water connections that are benefiting from rehabilitation works undertaken by the project 146,554 12 (number) Water Supply Number of water utilities that the project is supporting (number) 123 1 Water Supply Number of other water service providers that the project is supporting (number) 1,245 6 [1] Data needs further analysis by Sector Board. [2] Insufficient data with no operation reporting incremental progress. [3] The share of roads in good and fair condition as a share of total classified roads has increased by 1% to 43% in 8 projects for a total of 21% to 94%. [4] Insufficient data with only two operations reporting incremental progress. [5] Insufficient data with only one operation reporting incremental progress. 32 Annex 4: Outputs Table Unit of Measure- Sector Indicator ment FY01-03 FY04-05 FY06-08 a FY09 Output Hospitals built or rehabilitated examples in Health Number 79 97 12,804 b 227 Health professional trained Number 41,300 75,000 146,424 2,735 Children de-wormed Number (...) 67,000 69,044 (...) Insecticide treated bed nets distributed Number (...) 10,000,000 642,115 c 85,000 Childhood vaccines purchased or distributed d Number (...) (...) 509,000,000 e (...) Vitamin A doses purchased or distributed d Number (...) (...) 186,697 (...) Output Classrooms built or examples in rehabilitated Education Number 60,900 50,000 695,491 f (...) Teachers trained Number 47,500 263,000 602,856 (...) Teachers recruited Number (...) 37,500 926,833 f (...) Textbooks purchased and Number distributed (mill.) 25.9 4.2 233.6 f (...) Pupils benefiting from school feeding programs Number (...) 210,000 114,448 (...) Pupils receiving scholarships Number (...) 24,600 55,814 (...) Output Beneficiaries from water examples in supply or sanitation Water Supply interventions & Sanitation Number 7,545,000 4,760,000 11,131,819 2,088,500 New water connections Number (...) 15,000 96,851 3,887 New sanitation facilities Number 61,900 111,600 82,515 92,541 Sewers constructed Km 79 106 136 (...) Sewer household/communal connections Number 7800 14,000 200,002 (...) Output Total roads built examples in Transport Km 21,600 5,200 11,886 (...) Total roads rehabilitated and/or maintained Km 138,200 39,700 110,882 897 Total railroad constructed or rehabilitated Km 1,205 900 70 (...) Total culverts built or rehabilitated Number (...) 3,079 902 (...) People that gained increase access to services (e.g., health facilities, schools, markets) Number (...) 2,600,000 2,039,000 (...) Source: Staff calculations from IDA financed operations (does not include analytical activities). 33 (...) stands for "not available". Notes: a At the time of the review (July 9, 2009) only 57 ICRs out of a total of 84 FY08 exiting projects and 8 ICRs out of 63 FY09 exiting projects relevant to health, education, water and transport sectors were available including DPOs. b This figure includes hospitals, clinics, health posts and other health care facilities constructed, rehabilitated and/or newly equipped; however, it does not include blood banks, Voluntary Counseling and Testing (VCT) centers, nutritional sites, or laboratories. c Three operations reported purchase of 642,115 ITNs. Four other operations also reported to have supported Malaria prevention, but no specific numbers of Insecticide-Treated Nets (ITNs) procured were indicated. d This is one of the new IDA RMS core indicators for health sector. For this report, figure for this indicator was tracked only for FY07-09. Aggregate figures for this indicator for FY01-06 were not reported in previous IDA reports. e This figure represents the number of Oral Polio Vaccine (OPV) doses financed. f These figures include 650,442 classrooms constructed and 795,000 new teachers appointed and 220 million text books distributed through one Sector Wide Approaches (SWAp) education project with domestic financing and external donor financing including IDA. Simply based on the financing amount, the pro-rata share of IDA could be 41,765 classrooms constructed and 51,047 new teachers appointed and 15 million textbooks distributed.