IDA RESULTS MEASUREMENT SYSTEM: &&COMMENDATIONS FORIDA14 June 24,2004 ACRONYMS AND ABBREVIATIONS :AS Country AssistanceStrategy ZASCR Country AssistanceStrategy CompletionReport :FAA Country Financial Accountability Assessment :PAR CountryProcurementAssessment Report :PIA Country Policy and Institutional Assessment :WIQ Core Welfare IndicatorsQuestionnaire IECDG Development EconomicsData Group IHS Demographic HouseholdSurvey 3PI ExpandedProgrammeon Immunization 3sw Economic and sector work 'Y Fiscal year 3DP Gross Domestic Product 3NI GrossNational Income 4DN HumanDevelopmentNetwork 4IPC Heavily IndebtedPoor Countries 4IVIAIDS HumanImmunodeficiencyVirudAuto ImmuneDeficiency Syndrome 4NP Health,Nutrition, andPopulationTeam [BRD Intemational Bank for Reconstruction andDevelopment [CA Investment Climate Assessment ICP Intemational ComparisonProgram ICR ImplementationCompletionReport [CT Informationand CommunicationTechnology IDA Intemational DevelopmentAssociation [EAs Intemational EnergyAgency [MF Intemational MonetaryFund [SCED Intemational Standard Classificationof Education [SIC Intemational Standard Industrial Classification [TU Intemational TelecommunicationsUnion JSA Joint StaffAssessment KM Kilometer LPG Liquidpetroleumgas LSMS Living Standards MeasurementStudy M&E Monitoringand evaluation MCAs Millennium ChallengeAccount MDB Multilateral development bank MDGs Millennium Development Goals MEIP Monitoringand EvaluationImprovementPlan MICS Multiple Indicator Cluster Survey MRY Most-recent-year approach NEPAD New Partnershipfor African Development OECD-DAC Organisationfor Economic Co-operationand Development-DevelopmentAssistance Committee OED Operations EvaluationDepartment OPCS OperationsPolicyand Country Services PA PovertyAssessment PAD ProjectAppraisal Document PARIS21 Partnership in Statistics for the 21st Century PCR Primary completionrate PER Public ExpenditureReview PFM Public financial management PPP PurchasingPower Parity PRS Povertyreduction strategy PRSP PovertyReductionStrategy Paper PSA Public Service Agreements PSR Project Status Report QAG Quality AssuranceGroup QEA Quality at Entry R M S ResultsMeasurementSystem STATCAP StatisticalCapacity Program TBA Traditional birth attendant UIS UNESCO Institute for Statistics UN United Nations UNAIDS Joint United NationsProgrammeon HIVIAIDS UNESCO United Nations Educational, Scientific, and Cultural Organization UNICEF UnitedNations Children's Fund USAID United States Agency for Intemational Development WBI World Bank Institute WDI World Development Indicators IDA RESULTSMEASUREMENT RECOMMENDATIONSFOR IDA14 SYSTEM: CONTENTS ExecutiveSummary ........................................................................................................ ... 111 I Introduction................................................................................................................ 1 I1 Developmentof the IDA ResultsMeasurementSystem......................................... .. 4 4 B. IDA14 System: Background.................................................................................. A. IDA13 Results Measurement System.................................................................... 5 I11 IDA14 ResultsMeasurementSystem: A StrengthenedProposalfor Measuring . CountryOutcomes..................................................................................................... 9 I V IDA14 Results MeasurementSystem: Strengtheningthe MonitoringofIDA's . Contribution............................................................................................................. 11 V V I Conclusion................................................................................................................ ..Recommendations................................................................................................... 12 13 Box andTables Box 1. Tanzania: ImprovingDataQuality. Managkg for Results. andLinkingto theMDGs.....9 Table 1. Proposed Country Outcomes Indicators ............................................................ 6 Table 2. Indicators o f IDA Performance. FY97-04 ......................................................... 8 Annex A. Recent Analysis and IDA14 Baselines ............................................................ Annexes 15 Annex B. Possible Country Outcomes Indicators for IDA14.......................................... 19 Annex D. Data Issues for Proposed Indicators for IDA14............................................... Annex C. Additional Cost o fCollecting Indicators for IDAMonitoring System ...........37 43 55 Annex F. IDA Commitments by Major Sector and Themes, FY97-03 ........................... Annex E. Monitoring IDA's Contribution to Country Outcomes ................................... 57 Annex G. Technical Note for IDA13 Results Measurement System: Update on Primary 59 Annex H. Implementationo fthe Agenda on Managing for Results................................ Completion Rate andMeasles Immunization Coverage.................................. 71 IDA RESULTSMEASUREMENT RECOMMENDATIONSFORIDA14 SYSTEM: EXECUTIVE SUMMARY 1. The Monterrey Consensus underscored the shared responsibility for achieving development results, notably the Millennium Development Goals (MDGs), and the dual importance o f more and more effective development assistance. The World Bank has played a major role in the international partnership to manage toward development results, and has reflected that commitment in the IDA13 replenishment arrangement. IDA13 introduced a framework for measuring results, embodied in a system to monitor analytic inputs and country- level results during the IDA13 period, and undertook to develop an enhanced system to measure results for IDA14 and beyond. Buildingon the lessons learned from IDA13, the IDA14proposal consists o f a two-tiered approach to monitor (a) progress on aggregate country outcomes, and (b) IDA's contribution to country outcomes. 2. Strengthening the IDA Results Orientation. Deputies first discussed a preliminary Management proposal for an IDA14 Results framework in April 2003. Building on that discussion and on subsequent analysis, Management is now proposing to strengthen the first tier o f the IDA14 approach (progress on country outcomes) in two ways: (a) by focusing the list o f globally monitored aggregate outcome indicators on those that are most relevant to IDA'S business and most measurable in IDA countries, and (b) by introducing a mechanism to ensure reliable tracking o f selected outcome indicators in four key sectors where IDA has an active portfolio (health, education, water, and transport). Management i s also proposing to strengthen the second tier (monitoring IDA's contribution to country outcomes) to (a) ensure that a robust results-tracking mechanism is built into new IDA projects at the supervision and evaluation stages, and (b) measure and monitor improvements in selected project outputs and other related indicators as a direct result o f IDA'S interventions inhealth, education, water, and transport. A. IDA14 ResultsMeasurementSystem: MonitoringCountryOutcomes 3. The two-tiered proposal discussed inApril 2003 presented 15 aggregate country outcome indicators that reflected prevailing IDA country Poverty Reduction Strategy (PRS) priorities, were consistent with MDG monitoring, and captured the economic growth and human development priorities o f ongoing IDA programs. Criteria for selecting indicators included their relevance to desired outcomes, sensitivity to policy action, measurability, and cost o f data collection (to avoid excessive data collection burdens for low-income countries). Following consultations with Deputies, Executive Directors, and borrowers, Management was asked to explore additional indicators in the areas of governance, infrastructure and social equity. Analysis conducted for the IDA Mid-term Review inNovember 2003 led to the recommendation that the following three additional indicators be considered for possible monitoring over the course o f IDA14: access to all-season rural roads, access to electricity (which has strong near- term data availability), and use o f solid fuels (a more relevant indicator for poor households than accessto electricity, but one with less data availability inthe short term). 4. Focusing Country Outcome Indicators. Since November 2003, staff have undertaken a thorough analysis o f all 18 potential outcome indicators and have evaluated them on the basis o f their relevance to poverty reduction, their measurability and availability in IDA countries, their iv sensitivity to policy change, and the marginal cost o f generating a satisfactory database. To focus the set o f IDA14 outcome indicators more narrowly on those that are most relevant and measurable, and with a view to having a manageable set o f indicators for tracking aggregate country outcomes, Management recommends limiting the number of the indicators to 14. These outcome indicators would be reported every threeyears, in line with IDA replenishments. 5 . Monitoring through IDA Support. Inaddition, to support the collection and monitoring o f key outcome indicators in the four sectors where IDA is most active, Management recommends that all new IDA operations in these sectors be designed to monitor an outcome indicator identified by the relevant sector Board in the World Bank and included in the IDA14 results measurement system.I This process will contribute to building countries' capacity to generate regular data for these fundamental development outcomes, will use IDA operations to strengthen this capacity as necessary, and will help Bank teams remain focused on key outcomes influenced by IDA operations. B. IDA14 ResultsMeasurement System: MonitoringIDA's Contribution 6. InApril 2003 four indicatorswere identifiedfor assessingIDA's contributionto country outcomes: (a) adoption o f results-based CASs; (b) reporting on results-based CAS outcomes (beyond the IDA14 period); (c) the quality at entry o f IDA-supported operations; and (d) evaluation o f project outcomes for the IDA portfolio. Over the past year, progress in implementing the Bank's investment lending modernization program and the agenda on managing for results has strengthened results-oriented monitoring and evaluation at the project level, and has highlighted measurable indicators and empirical data in project reporting. As a result, the second tier o f the IDA results measurement system can be strengthened intwo ways: (a) by monitoring and targeting the extent to which solid results frameworks are incorporated in projects, and (b) by monitoring and reporting on improvements in selected project outputs and other related indicators as a direct result o f IDA's interventions in the same four sectors cited above under monitoring country outcomes-health, education, water, and transport. Management recommends that in addition to thefour indicators considered last year to gauge IDA's peformance, the following indicators be considered: the share of new Project Status Reports (PSRs) that have satisfactory baseline data for outcome monitoring; the share of Implementation Completion Reports (ICRs) with provision of ex post data on project outcomes; and a series ofproject output and other related indicators that will reflect the contribution of IDA operations in health, education, water, and rural road transport for completed operations. Management will provide information on these indicators for the July 2004 meeting o f IDA Deputies. C. Strengthening Statistical Capacity 7. Country capacity to measure progress toward core development outcomes is critical to country-led implementation o f PSRs, and is the foundation for global monitoring o f progress toward MDGs and complementary monitoring efforts such as the IDA results measurement system. It is recommended that, in addition to continuingsupportfor the PARIS21 consortium, IDA intensib support within its Country Assistance Strategies and projects for statistical ' The four indicators are the under-five child mortality rate, primary education completionrate, proportionof the population with access to an improved water source, and proportion o f the rural population with access to an all-season rural road. V capacity building, and work in partnership to implement the global action plan to strengthen statistical systems. D. Conclusion 8. This note suggests strengthening the proposed IDA14 results measurement system at both the country outcome level and the IDA contribution level. Outcomes at the country level are the ultimate measure o f development effectiveness, and regular monitoring o f a consistent set o f outcomes is essential for low-income countries and their external partners. While attribution o f outcomes to a particular actor or agency i s weak at this level, the donor community-including IDA-has a collective responsibility to ensure that external support helps developing countries move in the right direction on core development outcomes over the long run. Therefore, it is important for IDA Management to monitor the movement o f key development outcomes across the group o f IDA borrowers and to continually test and improve the links between IDA interventions and country-specific outcomes, even if progress in aggregate outcomes cannot be directly attributed to IDA support alone. To better assess IDA's contribution to the development process, the proposed IDA14 results framework now includes additional components to measure the institution's success at incorporatingrobust results frameworks insupervision and evaluation o f individual operations. Aggregate performance targets can be developed for these indicators as well as for those that were previously proposed to capture IDA's contribution to countries' development outcomes. Inaddition, IDA would monitor the outputs and other related indicators o fprojects inkey sectors. 9. Leveraging the IDA Results Measurement System. The IDA results measurement system can help low-income countries onto the path o f better data contributing to improved decisionmaking that results in faster progress toward their outcome goals. Strengthening the results focus o f CASs and IDA-supported operations will increase assistance to countries in buildingcapacity to manage for results andwill help ensure that core development outcomes are regularly monitored. The IDA system also has the potential to influence development partners in the emerging global partnership on managing for results. This underscores the importance o f making careful decisions about the IDA results measurement system. IDA RESULTSMEASUREMENT RECOMMENDATIONSFORIDA14 SYSTEM: I.INTRODUCTION 1. The IDA13 replenishment arrangement introduced a framework for measuring results, embodied in a system to monitor analytic inputs and country-level results during the IDA13 period, and in the undertaking to develop an enhanced system to measure results in IDA14 and beyond.' The development o f this system-which monitors progress on aggregated development outcomes and the contribution o f IDA programs to country outcomes-benefited from a number o f technical discussions and consultations with IDA Deputies and borrowers from November 2002 to April 2003, along with a technical briefing o f World Bank Executive Directors in February 2003. An update note assessing additional indicators was also discussed during the IDA13 Mid-term Review in November 2003.2 This paper builds on these earlier proposals for the IDA results measurement system, and offers specific recommendations for the upcoming IDA14period. 2. Objectives of the I D A Results Measurement System. The results measurement system serves to inform discussions about IDA's effectiveness and strengthen the focus o f IDA's programming on key development outcomes. The system does not influence the allocation o f IDA resources among eligible countries; this is guidedby IDA's performance-based allocation system, which uses the Country Policy and Institutional Assessment (CPIA) to evaluate the appropriateness o f countries' policies and institutions in supporting growth and poverty reduction. The performance-based allocation system provides incentives to countries to improve policies and strengthen institutions in pursuit o f longer-term results. Because the results measurement system tracks longer-term outcomes that are determined to a significant degree by sustained government policies and institutions, findings from these two systems-the IDA results measurement and the CPIA-could be expected to show significant correlation over time for individual countries. The IDA results measurement system does not, however, focus on individual country outcomes, but on IDA countries' aggregate progress toward the Millennium Development Goals (MDGs) and other goals, as well as on IDA's contributions to country results. In this respect, the IDA results measurement system contributes to the proposals prepared for the Development Committee by the Bank and the IMF with respect to global monitoring o f the policies and actions o f developing and developed countries for achieving the MDGs andother outcome^.^ 3. World Bank Agenda on Managingfor Results. The IDA results measurement system i s an integral part o f the World Bank's broad effort to enhance its results orientation. This effort was launched in 2002 in the follow-up to Monterrey, and it has drawn on the two international roundtables on managing for development results cosponsored by the multilateral development banks (MDBs)incollaboration with the Development Assistance Committee o f the Organisation ' SeeIDA Results Measurement System: Progress and Proposals (IDAiSecM2003-0159), April 7,2003. See IDA Results Measurement System: UpdateNote (IDA/R2003-0189), October 31,2003, See Achieving the MDGs and Related Outcomes: A Framework for Monitoring Policies and Actions (DC2003-0003), March 26, 2003, and Coherence, Coordination and Cooperation among Multilateral Organizations Progress and Challenges (SecM2004-0152), April 9,2004. 2 for Economic Co-operation and Development (OECD-DAC).4 The .World Bank's agenda on managing for results, endorsed by the Development Committee in September 2002, begins with the premise that sustained improvement in development outcomes at the country level i s the ultimate measure o f development effectiveness, and that these outcomes emerge gradually through a process that involves multiple internal and external influences. Improving country outcomes requires attention to results throughout the development process: during planning, implementation, and completion. Thus, the Bank's Implementation Action Plan for the results agenda5calls for action inthree areas: (a) incountries, to help countries strengthen both capacity and demand to manage for results; (b) in the Bank, to enhance the effectiveness o f our contribution to results; and (c) across development agencies, to harmonize results-based approaches andbetter align support to strengthen country capacity to manage for results. 4. Action Plan Implementation. Implementationhas started strongly on all three fronts.6 Supporting Countries in Strengthening Capacity to Managefor Results. Incountries, the Bank is helping countries strengthen the focus on results in national poverty reduction strategies and other development strategies while also helping countries builddemand within civil society for public sector accountability. The latest review o f completed PRSPs7found that countries have made progress in defining indicators and targets that are increasingly aligned with the MDGs-this has also strengthened the basis for harmonized reporting on results. The Bank has intensified its own efforts to help countries adopt results-based approaches to public sector management and strengthen core capacities for results-based strategic planning, policy formulation, public resource management, monitoring and evaluation, and statistics. For IDA countries, this support i s being provided through the work of IDA country teams as well as through World Bank Institute (WBI) learning programs, the Evaluation Capacity Development initiative o f the Operations Evaluation Department (OED), and the Bankwide Monitoring and Evaluation Improvement Program. Statistical capacity building i s being supported through the multidonor Global Trust Fund for Statistical Capacity Building and through the Bank's Statistical Capacity Program, which adopts a sectonvide approach. Enhancing the Bank's Contribution to Development Results. To be a more relevant and effective partner, the Bank is strengthening the focus on results in its strategies, instruments, incentives, andreporting systems. Central elements o fthis effort include piloting a results-based Country Assistance Strategy (CAS) to better link Bank programming to country development outcomes; piloting the CAS Completion Report, which evaluates CAS outcomes; and strengthening outcome monitoring in selected sectors and thematic areas. Project documents and procedures have also been revised to enhance the focus on outcomes and outcome-oriented management Information on the First and Second Roundtables on Managing for Development Results is available at ~~ El;ww.inanagingforcle\'elo~ni-resul. The Action Planwas discussed bv the Bank's Committee on Development Effectiveness (CODE) in December 2002. After that discussion, the paper Better-Measuring, Monitoring, and Managingfor Development Results: Implementation Action Plan (SecM2003-0038), January 29,2003, was revisedand submittedto the Board for information. In March 2004, CODE reviewed progress in implementing the results agenda; see Implementation of the Agenda on Managingfor Results (CODE2004-0017), March 23,2004. Poverty Reduction Strategy Papers-Detailed Analysis ofprogress in Implementation, (IDA/SecM2003-0468), World Bank and IMF (August 14,2003). during design and supervision. Staff learning and incentives are being adapted to more directly encourage staff to pursue sustainable results at the country level. Buildingon the IDAmeasurement system, more comprehensive results reporting will gradually be developed to better inform strategic choices and budgetary allocations. 0 Fostering a Global Partnership on Managing for Development Results. The Bank has cosponsored two international Roundtables on Managing for Development Results (June 2002 and February 2004), which have helped to raise awareness and develop a global framework and common principles for action on managing for results; and it has helped establish interagency mechanisms, including the MDB Working Group and the OECD-DAC Joint Venture on Managing for Development Results. The Joint Venture agreed to (a) hold regional workshops to buildawareness and discuss the challenges in managing for results, (b) develop a source book on managing for results, (c) pilot harmonized results reporting in selected African countries, and (d) explore the various ways development agencies are working to strengthen their internal effectiveness to improve their contributions to managing for development results, so as to reach a common understanding o f concepts o f effectiveness and performance. 5. IDA and the WorldBank Results Agenda. The IDA results measurement system draws on two key elements o f the Bank's agenda on managing for results. The first i s the increasing focus on country outcomes, and on the national capacity and statistical systems needed to measure and monitor progress toward desired outcomes. The second is the introduction o f a results-based CAS as the primary means to monitor the contribution o f Bank programming to selected country outcomes. For IDA-eligible countries, the CAS i s derived from a country-led poverty reduction strategy process, applying the principles o f the Comprehensive Development Framework.* 6. Organization ofthis Paper. Section I1summarizes the lessons learned during IDA13 and reviews progress indeveloping the system for IDA14. Section I11presents the findings from recent monitoring o fproposedcountry outcome indicators for IDA14, and proposes to strengthen IDA support for outcome monitoring within the IDA portfolio. Section IV reviews the proposal for monitoring IDA's contribution to country outcomes, and suggests two sets o f additional indicators: one to assess progress in measuring results within IDA-supported projects, and the other to monitor project outputs and other related indicators in selected sectors. Section V summarizes recommendations for strengthening the results measurement system during IDA14. SectionVIprovides a brief conclu~ion.~ * For more details on the CDF, see h t ~ . r . ' i m t ~ ~ ~ t t \ ~ ~ r ! ~ . b ~ ~ , ~ k . : ~ ~ I : ~ ~ . ~ ~ ~ . .I. There are also eight annexes that discuss recent analysis and IDA14 baselines (Annex A); possible country outcome indicators (Annex B); the additional cost of collecting indicators for the IDA monitoring system (Annex C); data issues related to the proposed IDA14 indicators (Annex D); monitoring IDA's contribution to country outcomes (Annex E); IDA commitments by major sectors and themes from FY97 to FY03 (Annex F); an update on the primary completion rate and measles immunization coverage in the IDA13 measurement system (Annex G); and progress on the implementation o f the Agenda on Managing for Results (Annex H). 4 11. DEVELOPMENT THE IDA RESULTSMEASUREMENT OF SYSTEM 7. This section begins by drawing the lessons o f experience from the IDA13 results measurement system and thenreviews initial development o fthe system for IDA14 and beyond. A. IDA13 ResultsMeasurementSystem 8. The results measurement system introduced for the IDA13 period tracked the preparation o f analytic inputs (economic and sector work such as Public Expenditure Reviews and Investment Climate Assessments) inIDA-eligible countries. It also monitored selected outcome- related indicators (measles immunizationrates, primary education completion rates, andtime and cost o f business start-up). It set targets for analytic inputs as well as for aggregate progress on the outcome-related indicators. Experience with this system and performance against these targets, yields a number o f useful lessons for the IDA14 system.'O 9. Lessonsfrom System Design. The IDA13 results measurement system has strengthened the foundations for results-oriented programming inIDA countries. At the same time, increased attention to outcome-related indicators has brought basic data and measurement issues to the forefront o f the global development agenda. Even as the IDA13 system was introduced, it was agreed that further development for IDA14 was desirable to better reflect country priorities articulated in poverty reduction strategies (PRSs), provide stronger linkage to the Millennium Development Goals (MDGs), and better assess the contribution o f IDA programming to sustainable country results. 10. Lessons from System Implementation. The challenge o f monitoring outcome-related indicators during implementation highlighted the need to refine statistical methodologies to provide more reliable information on development results in IDA countries. The evaluation o f short-term changes inindicator values proved particularly problematic for several reasons: 0 Irregular data collection and reporting at the country level ledto inconsistent aggregates over time. The inclusion or omission o f one influential country may be enough to affect measuredprogress over a short time period. To address this problem, gap-filling methods, such as interpolation or extrapolation, mustbe introduced. 0 Many indicators have unquantified measurement errors, which may compound errors inherent inthe interpolation and extrapolation o f low-frequency data to a common year for aggregate assessment. This introduces considerable uncertainty in comparing values or measuring trends over short periods o f time. Under these circumstances, target values are likelyto fall within plausible, butunknown, margins o f error. 0 Comparisons over time are further confounded by changes indefinitions andmethods that, although intended to improve the quality o f the resulting statistics, have caused breaks incontinuity o f time series data. 0 Indicator values are subject to revision as part o f the normal process o f compiling and validating statistical data. Preliminary estimates are typically finalized within a year, lo See IDA13 ResultsMeasurement System: Spring 2004 Update (SecM2004-0244; IDA/SecM2004-0327), May 18,2004. 5 but some estimation methods may entail more extensive backward revisions. When such revisions occur after baseline values have been agreed and targets set, the interpretation o fmeasured outcomes is further complicated. 11. Need for Monitoring. Outcomes at the country level are the ultimate measure o f development effectiveness, and therefore regular monitoring o f a consistent set o f outcomes i s essential for low-income countries and their external partners. While attribution o f outcomes to a particular actor or agency is weak at this level, the donor community-including IDA-has a collective responsibility to ensure that external support helps developing countries move in the right direction on core development outcomes over the long run. Therefore, it is important for IDA Management to monitor the movement o f key development outcomes across the group o f IDA borrowers and to continually test and improve the links between IDA interventions and country-specific outcomes, even if progress in aggregate outcomes cannot be directly attributed to IDA support alone. However, given the slow changes in development outcomes, the minimumpractical periodfor monitoringprogress is three years. B. IDA14 System: Background 12. InApril 2003, following consultations with IDA borrowers and Deputies and a briefing of the World Bank Executive Directors, agreement in principle was reached on a proposed system for IDA14 and beyond. The system i s a two-tiered approach that monitors (a) progress on aggregate country outcomes, and (b) IDA'Scontribution to country outcomes." 13. Country Outcome Indicators. The initial proposal presented 15 country outcome indicators that reflected prevailing Poverty Reduction Strategy Paper (PRSP) priorities, were consistent with MDG monitoring, and captured both the economic growth and human development priorities o f ongoing IDA programs.IZ Criteria for selecting indicators included their relevance to desired outcomes, sensitivity to policy action, measurability, and cost o f data collection (to avoid excessive data collection burdens for low-income countries). Following consultations with Deputies, Executive Directors, and borrowers, Management was asked to explore additional indicators in the areas o f governance, infrastructure, and social equity. Analysis conducted for the IDA Mid-term Review inNovember 2003 led to the recommendation that the following three additional indicators be considered for possible monitoring over the course o f IDA14: access to all-season rural roads, access to electricity (which has strong near- term data availability), and use o f solid fuels (a more relevant indicator for poor households than access to electricity, but one with less data availability inthe short term). This recommendation brought the list o f potential outcome indicators to 18. Table 1 summarizes these indicators and shows the availability o f data across IDA countries for each indicator. '' See IDA Results Measurement System: Progress and Proposals (IDNSecM2003-0159), April 7, 2003, and IDA Results Measurement System: UpdateNote (IDA/R2003-0189), October 31,2003. " Ibid. 6 Table 1. ProposedCountry OutcomesIndicators Number of Historical Annual Range of Percent of IDA average rate of most IDA countries annual change Percent of Percent of recentyear countries included to rate of required to PRSPs thatPRSPs that data with data calculate growth'for meet MDG include the cover the for IDA in 2000- growthfrom IDA targetfrom Indicator indicatorb subject countries 2002 1990 countries 199od 1. Proportion o f population below 14 100 1984-2002 19 48 -1.3 -2.8 ..................................................................................................................................................................................................................... $l/day poverty line " " .......................................................................................................................................... " 2. Prevalence of underweight children 44 69 1994-2002 53 46 -2.8 -2.8 ................ under..fi.e..yertrs.,of,a.~e.. ............................. .... ..".................................................................................................................................................................................... "............ ...3. Under-5 child.................................. mortality 75 97 100 77 -1.6 -4.4 "............. 4. Proportion o f I-year-old children 11 72 2002 "...".,,,.".......................................................................................................................................................................................1.8 2002 100 76 0.4 ................................................................................................................................................................................................................................................................V immunized against measles ................................. 5. HIV prevalence rate of women aged 6 67 1999-2001 66 (-1 (-1 (-1 ................................................................................................................................................................................................................................................................{ 15-24' ................... 6. Proportion o f births attendedby 55 100 1994-2002 66 39 2.7 4.3 .. skilled healthpersonnel ............................................................. ................. " 7. Ratio o f girls to boys in primary and 64 78 1990-2001 64 49 1.5 1.3 . .................................................................................................................................... secondary education " .................................. ............................................................................................................................................................................................ ....................................... ....................................................................................................................... 8. Primary school completion rate ............ .......................... 30 ......I ......... 100 1990-2002 .......................................................................................................................................................... 78 55 0.9 1.7 9. Proportion o f population with 86 94 2000 94 34 1.7 0.9 sustainable access to an improved ....................................................... water source .................................................................................................................................................................................................................................................. -..... ........................................................................................................................ 10. Fixed lines and mobile telephone per 33 48 2001-2002 100 78 13 (W ...................................................................................................................................................... 1,000 inhabitants .................................................................................................................................................................................................... 11. Formal cost required for business 3 17 2001-2003 69 (-1 (-1 (ni4 2i??s!P .......................... -....... ......... ........... .............................. " .................................................................................................................................................................................................... .!..?: ....Ti.me..e~uired..for..b~~ine~~..~ta~~~P 11 17.... ...................................................................... 2001-2003 ............................................................................................................................................ 69 (-1 (-1 1nia) ............................. .!.L Pul?!ic...ex.~e~diture...m~a~e.ment Ll.................................. ~ " .......... 0 97 2001-2002 .............. 30 (1.1) ....................................... .(dal.......... .!.41. ..A.~lc.!tura!..va!ue..added ........................................ 64......... ............. 90 68 2.4 .E..... 19 2000-2002......................................................................................................................................................................... 1nia) ..................... " .... GDP..Per..caP!!a .................................................................... 39 100 .................................... 2002 ......................................................................................................................................................................... 96 72 2.0 ............................ (-1 (-1 cn/a> 16. Access o f rural populationto an all- 14 42 1991-2003 14 (n/a) ................................................................................................................................................................................................................................................................R season road ^.... ............... ............................ K 1 ....Householde!ec,trificat~on.rae ............ 28 56 1995-2003 ..... 23 L.J........................................ (I:). ................................ in/a> .............................. 18. Proportion o f households using solid 11 36 1995-2003 20 (-1 (-1 t-> fuels Notes: (..)means insufficient data, (nia) meansnot applicable. a All data are taken from the latest 2004 World Development Indicatorsdatabase, except indicators 13, 16, 17, and 18. As o fMarch 2004,36 countries had full PRSPs. All growth rates have been calculated between 1990 and 2000 using the exponential growth method. Some indicators are excluded inthe analysis since the historical performance cannot be established because o f insufficient data. The annual growth rates required to meet the MDGtargets are calculated according to each specific target. e Due to differences in methodology, comparable data are not available over time. 7 14. IDA's Contributionto Country Outcomes. The initial proposal also identifiedindicators for assessing IDA'Sperformance and contribution to country outcomes through CASs and the associated IDA portfolio. Adoption o f results-based CASs in IDA-eligible countries was proposed as an indicator to show progress in linking IDA programming more clearly to country priorities. Inthe longer run, a second indicator would be based on eventual reporting on results- based CAS outcomes (to begin after the IDA14period). 15. Results-based CASs. A methodology has been developed to adopt a more results-based CAS that incorporates four elements: (a) a CAS Completion Report that provides a self- evaluation o f progress toward results under the previous CAS, to more systematically leam lessons and apply them going forward; (b) a results-oriented strategy; (c) a strengthened framework for monitoring and evaluating the CAS; and (d) analysis o f and support to countries to strengthen their capacity to manage for development results. The results-based methodology was piloted in FY03 and FY04. To date, the World Bank's Executive Directors have discussed-and favorably received-six results-based CASs. Another half-dozen are in preparation, and many other teams are beginning to adopt the methodology. Initial feedback from teams and clients on the design process has been positive, citing greater multisectoral problem solving, improved strategic relevance and coherence, clearer trade-offs, and greater selectivity. An evaluation o f the pilot phase i s beingprepared-but must at this stage be limited to a review o f the design process. The results-based methodology will be modified as needed in response to the evaluation, and a strategy for mainstreaming it inthe coming year will be agreed. The cumulative introduction o f results-based CASs in IDA-eligible countries would continue to be monitoredduringIDA14, to ensure progress toward results reporting at this level. 16. Quality at Entry and Project Outcomes. With respect to the IDA portfolio, the two indicators selected were the quality at entry o f IDA operations, which i s a leading indicator for positive outcomes, and the evaluation o f project outcomes upon completion. The Quality Assurance Group (QAG) examines quality at entry through peer assessments o f a random sample o f projects. This assessment provides information that i s useful in managing for results, with immediate impact on project implementation, including redesign and midcourse correction. The quality o f operational design has remained high in recent years, with the most recent data showing 85 percent satisfactory quality at entry for the IDA portfolio (see Table 2). The Operations Evaluation Department validates project outcome ratings by reviewing Implementation Completion Reports (ICRs) for all exiting IDA projects. These data, based on independent ex post evaluation, are currently the most reliable measure o f results across the IDA portfolio, and are available annually. The data are lagged because the average age o f projects exiting the portfolio is 5-6 years (see Table 2). 8 Table 2. Indicators of IDA Performance, FY97-04 LDA Samplesize and FY resultsindicator frequency 97 98 99 00 01 02 03 04 Introduction o fresults- Population d a d a d a d a d a d a 1 6a basedCASs (cumulative (15-20 CASs per year) no. ofcountries) Evaluationof................................................................................................................................................................................................................................................................< d a ........................ CAS Population d a d a d a d a d a d a d a outcomes (5-10 CASs per year (%satisfactory) toward the end o f ................................................................................................................................................................................................................................................................· IDA14) Quality at entry` Sample 70 82 88 rda 90 86 85 (..) ("hsatisfactory) (50-100 projects every ............................................................................................................................................................................................................................... two years) ~...................".................................................................................................................................. a...............-..................... Project outcome Population 74 65 61 74 76 74 75 (..) ("hsatisfactory) (120 projects per year) Notes: (..)meansdatanotyet available, (n/a) meansnot applicable. a As of May 7,2004, but will be 7 by end ofFY04. DerivedfromCAS CompletionReports, which assess the relevanceofthe CAS, achievementofexpectedoutcomes, and Bank's performanceduring the CASperiod. CASCRswill typically become available4-5 years after CAS approval. The evaluation ofCASCRswill bevalidatedby OED. An extendedquality-at-entryexercisewas conductedin FY00-01. As of May 3,2004. Source: OEDand QAGDatabases. 17. Strengthening Country M&E Systems. Development o f the IDA results measurement system has helped highlight the need to improve the availability and quality o f basic data, and strengthen monitoring and evaluation systems in IDA-eligible countries. Through the poverty reduction strategy (PRS) process, many countries have stepped up efforts to address the technical and capacity constraints that affect the measurement and monitoring o f development outcomes needed to assess progress. Tanzania provides one example o f a country-led effort to strengthen the focus on results through the PRSP (see Box 1). Donors, including IDA and the IMF, are actively supporting these efforts. 18. Partnership for Statistical Capacity Building. A significant multilateral effort is under way to improve the availability and quality o f data in IDA countries through PARIS21, an international consortium o f users and producers o f statistics that promotes evidence-based decisionmaking and emphasizes the need for investment in statistical systems, including household surveys. Through PARIS21, detailed case studies were undertaken to examine the constraints countries face in monitoring their PRSs and inproviding data to monitor the MDGs. The results have been used to develop plans for improving methods at all levels and to inform discussion among United Nations agencies on improvements in the international statistics system. A six-point global action plan to strengthen statistical systems was agreed during a meeting o f multilateral development banks and major international institutions at the Second Roundtable on Managing Development Results in Marrakech in February 2004. It calls for an unprecedented level o f international collaboration and support to rationalize existing instruments and resources, strengthen national statistical systems, and improve coordination and accountability within the international system. An estimated US$90 million per annum in incremental financing will be needed in the 2004-2006 period to strengthen national systems in 9 low-income countries, along with an estimated US$25 million per annum to improve the international system.I3 Box 1. Tanzania: Improving Data Quality, Managing for Results, and Linkingto the MDGs Starting in 2000, the Government o f Tanzania developed a comprehensive poverty monitoring system to (a) generate usable data over the short term to inform policymaking and budgeting in real time; and (b) assess the effectiveness o f policies and programs o n poverty reduction. T o a large extent the poverty monitoring system was developed inresponse to the analysis done for the PRSP. The Government developed a comprehensive monitoring planusing a joint funding mechanism between donors and the government. The result was a comprehensive database o f indicators for monitoring poverty and PRSP implementation, and the design o f improved statistical instruments, including both household surveys and the extraction of data fromroutine administrative systems. Tanzania was one of the first countries to produce a United Nations MDGreport. The PRSP includes many o f the MDG indicators, and it sets explicit targets for many o f the indicators proposed for the enhanced IDA monitoring system. Using the PRSP as a basis, Tanzania has prepared a comprehensive poverty monitoring master plan that defines the institutional arrangement for poverty monitoring as well as specific activities such as surveys and censuses, participatory poverty assessments, research, and analysis. The institutional structure in support o f the poverty reduction strategy and poverty monitoring process comprises four technical working groups-Surveys and Censuses, Administrative Data, Research and Analysis, and Advocacy and Dissemination. The Government has improved the statistical basis for monitoring many indicators, particularly as a result o f the Household Budget Survey conducted in 2000-2001, and the use o f administrative data, and has established a clear timetable, consistent with intemational recommendations, for updating PRSP and MDGindicators. Inthe future there is a need to strengthen: (a) the capacity o f the poverty monitoring institution; (b) the use o f poverty monitoring data in policy debates and resource allocation; and (c) the links between subnational and central poverty monitoring systems. 19. IDA's Support for Statistical Capacity Building. An important aspect in preparing a results-based CAS i s incorporating more support to strengthen country capacity to manage for results. On March 25, 2004, the World Bank's Executive Directors approved a new umbrella lendingprogram, the Statistical Capacity Program (STATCAP), which will provide an additional avenue for such support. The program aims to address countries' differing statistical capacity needs flexibly while offering a simplified preparation and approval process. STATCAP is being piloted in three to five countries this year, with Burkina Faso being the first IDA country to receive support from the program. 111. IDA14 RESULTSMEASUREMENT A STRENGTHENEDPROPOSALFOR SYSTEM: MEASURING COUNTRY OUTCOMES 20. This paper recommends further strengthening the proposed IDA14 Results-Measurement System on both the level o f outcome monitoring (tier 1) and monitoring IDA's contribution to results (tier 2). This section discusses tier 1. 21. Monitoring Country Outcomes. Recent analysis of aggregate country data has led to the establishment o f IDA14 baselines for the 18 proposed country outcome indicators. Results are l3 The details o f the action plan can be found in htm:'/\\;ww.niaiia~in~f~~rdevclonnientrest~lts.or~~(~~~~uni~nt~~~~ ActionPlanforStatistics.df. The action plan is in two parts. The first addresses national needs (e.g., mainstream strategic planning o f national statistical systems and prepare national statistical development strategies for all low-income countries by 2006, begin preparations for the 2010 census round, and increase national statistical capacity building). The second establishes intemational statistical responsibilities (e.g., set up an intemational household survey network, undertake urgent improvements needed for MDGmonitoring by 2005, and increases accountability ofthe intemational statistical system). 10 shown inAnnex A. Although these indicators have some o f the best available data and reporting mechanisms, serious constraints to measurement remain. As previous IDA reports on the results measurement system showed (and as Table A1 summarizes), for many o f the indicators, only about half the IDA-eligible countries have two observed data points with which to calculate a trend line for the decade o f the 1990s-much less assess change within a three-year PRSperiod or IDA cycle. For some countries, the most recent observed data are six or seven years old. Data availability and quality vary significantly across the indicators but remain a problem even for outcome indicators with relatively well-established monitoring andreporting systems. Adopting statistical norms and generating data in less-established areas-including public expenditure management, private sector development, and infrastructure-are proving particularly challenging, although IDA and developing countries have intensified their efforts inthe past few years, as part o f the global partnership for statistics. 22. Selection of County Outcome Indicators. As the IDA results measurement system has developed, the number o f country outcome indicators under consideration has risen to 18. Both IDA borrowers and Deputies have underscored the value o f tracking a limited number o f outcome indicators. Given capacity constraints in IDA countries, and the value o f focusing on a manageable set o f highly relevant indicators, it i s suggested that the number o f indicators monitored during IDA14 be reduced. To review the strengths and weaknesses o f each indicator, sector professionals and statisticians inthe Bank were asked to reevaluate each one according to the four criteria o f relevance, sensitivity, measurability, and cost (see Table D4 in Annex D). Most weight was given to relevance and measurability, and consideration was also given to the comparative strengths o f multiple indicators inthe same sector, such as health and infrastructure, and the availability o f data. 23. Relevance to Desired Outcomes. It was concluded that three o f the proposed indicators were somewhat less relevant than the others to the desired outcomes they are meant to represent. Measles immunization rate was originally selected for monitoring during IDA13 because o f its strong measurability over a short time period and its use as a proxy indicator for delivery o fbasic health services. However, it i s desirable to move from a relatively narrow intermediate indicator to an indicator that captures a broader result interms o f children's' health status. The under-five mortality rate i s perhaps the most relevant outcome indicator o f overall health status for children, and could serve as the indicator o f choice in the health area during IDA14. The proportion o f attended births is also a health service indicator that does not directly or fully capture the high- level outcome (e.g., maternal survival) it represents. However, given the lack o f reliable maternal mortality data in many low-income countries, no alternative to the births attended indicator i s readily available. Inanother domain, agricultural value-added was also judged to be less relevant to desired outcomes-such as agricultural productivity and rural incomes. On this basis, it may be possible to drop measles immunization rate and agricultural value-added as country outcome indicators for the IDA14period. 24. Measurability in IDA Countries. Two indicators, although considered highlyrelevant to desired outcomes such as poverty reduction, were deemed to have enough remaining measurement difficulties (frequency, accuracy, and high costs) to warrant reconsideration. Measurement o fthe prevalence o funderweight children i s dependent on an infrequent subsample o f household surveys in low-income countries. Likewise, data on household use o f solid fuels is currently available in only 28 IDA-eligible countries. Work will continue to incorporate this 11 area into more household surveys, as it is recognized as an important indicator o f household health and welfare. Nonetheless, for the IDA14 period, both the malnutrition and solid fuels indicators could be dropped due to their limited measurability. A third highly relevant indicator-the prevalence o f HIV infection among women aged 15-24-has so far used different methodologies over time to measure prevalence. Consequently, changes over time cannot be accurately determined. On the other hand, in July this year UNAIDS will publish estimates o f HIV prevalence among adults (15-49 years) for 2001 and 2003 that will be comparable over time, and Management will be working with UNAIDS to identify the most suitable HIV indicator for future monitoring inthe IDAresults measurement system. 25. A More Manageable Set of Indicators. On the basis o f the above staff analysis, it was concluded that the number o f country outcome indicators during the IDA14 period should be reduced from 18 to a more manageable 14. To summarize, the four following indicators would be dropped: prevalence o f underweight children, measles immunization rates, agricultural value- added, and household use o f solid fuels. 26. Supporting Outcome Monitoring through Projects. The experience with outcome monitoring under IDA13 underscored the importance o f using IDA-supported projects to help countries strengthen their capacity to monitor these outcomes. As part o f the broader agenda on managing for results, this goal has been pursued through various initiatives, including a phased plan to strengthen the results frameworks underpinning the Bank's sector strategies and identify core outcome indicators to be tracked through project monitoring systems. In an initial phase, eight Sector Boards launched an effort to better define the results framework and identify outcome indicators that are relevant across a wide range o f Bank borrowers. This work will be expanded to cover all sectors and themes in coming years. On the basis o f this work, all new IDA operations for FY05 and beyond infour key sectors-primary health care, basic education, household water supply, and rural road transport-would be designed to monitor an outcome indicator already identified by the appropriate Sector Board in the World Bank and included in the IDA14 results measurement system: under-five child mortality rate, primary education completion rate, proportion o f the population with sustainable access to an improved water source, and proportion o f the rural population with access to an all-season road. The purpose is to ensure that countries have the capacity to generate regular data for these fundamental development outcomes, to use IDA operations to strengthen this capacity as necessary, and to helpBank teams to remain focused onkey outcomes influencedby IDAoperations. IV. IDA14 RESULTS MEASUREMENT SYSTEM: STRENGTHENINGTHE MONITORING OFIDA'S CONTRIBUTION 27. The April 2003 proposal for the IDA14 results measurement system identified processes and indicators by which to assess IDA's contribution to country outcomes at the level o f the IDA portfolio and-as a central element o f the agenda on managing for results-at the broader level o f IDA's Country Assistance Strategies (CASs). Progress since then in strengthening results- oriented monitoring and evaluation at the project level will allow IDA to better monitor its contribution and performance during the IDA14 period. Management proposes two sets o f additional indicators: one to assess progress inmeasuring results within IDA-supported projects, and the other to monitor project outputs and other related indicators inselected sectors. 12 28. Assessing Progress in Measuring Results within IDA-supported Projects. In the past year, as part o f the investment lending modernization program and the broader agenda on managing for results, the Bank has revised project documents and procedures to emphasize outcome-oriented objectives and monitoring. Project Appraisal Documents (PADS)and Project Status Reports (PSRs) have been revised to focus more explicitly on expected outcomes, measurable indicators, and provision o f data to assess progress. This will permit a clearer assessment o f the causal links between IDA programming and outcomes, with a stronger attribution o f outcomes to the influence o f IDA-supported interventions. QAG's quality-at-entry assessments will explicitly assess the results framework underpinning project design and the provision o fbaseline data inPSRs. OED will continue to assess the evidentiary base provided in ICRs to evaluate achievement of outcomes. To better assess IDA's performance in introducing results-based monitoring and evaluation at the project level, it i s proposed that IDA would establish targets for new PSRs with satisfactory baseline data on project outcome indicators and ICRs with provision o f ex post data on project outcomes. These targeted indicators would be a useful gauge o f the extent to which measurable results are being specified and monitored throughout the project cycle. 29. Monitoring in Selected Sectors. Inaddition to monitoring for an improved results focus in project supervision and evaluation, we propose to monitor aggregate progress on selected project outputs and other related indicators infour sectors where IDA i s active across a range o f countries: health, education, transport, and water (the same four sectors inwhich IDA projects would help countries strengthen outcome indicators). These measures would rely in some cases on estimations of the impact o f project outputs on the target population, but they have the advantage o f being able to better capture changes that can be attributed to IDA's interventions. For example, in the water sector, we can measure the increase in the "number o f people with access to clean water" as a result o f IDA interventions. The baseline value would be based on the value o f the indicator associated with water supply projects exiting the portfolio inFY02-03, and progress would be monitored based on projects exiting the portfolio in FY05-06. A more detailed proposal for this component o f the IDA14 results-measurement system will be elaborated in a note to be prepared for the Deputies' meeting inHanoi (July 9-11). V. RECOMMENDATIONS 30. The lessons o f experience in implementing the interim results measurement system during IDA13, as well as the findings from additional monitoring and analysis o f the proposed indicators for the IDA14 system, have led to a set o f recommendations. These recommendations are summarized below: Country OutcomeIndicators 0 Monitor 14 country aggregate outcome indicators (the indicators cited in Section I11 above) during IDA14 and beyond, and report on the basis o f the three-year IDA replenishment cycle. 0 Ensurethat all new IDA operations infour sectors where IDA is most active will be designed to monitor a key sector outcome (under-five mortality rate in health projects, primary completion rate ineducation projects, proportion o f the population 13 with access to improved water supplies in domestic water supply projects, and proportion o f population with access to an all-season rural road in transportation projects). MonitoringIDA's ContributionandPerformance 0 Monitor and target the three indicators initially put forward to measure IDA performance (the introduction o f results-based CASs, project quality at entry, and project outcomes as rated by OED). Inthe longer run, a fourth indicator would be based on eventual reporting on results-based CAS outcomes (to begin after the IDA14 period). 0 Monitor and target the provision o f baseline data on project outcomes in PSRs and ex post data on project outcomes inICRs. 0 Monitor aggregate progress on selected project outputs and other related indicators in four sectors where IDA is active across a range of countries: primary health, basic education, domestic water supply, and rural roads. 31. Statistical Capacity Building. Getting better development results depends on countries' ability to use information to guide decisionmaking and steer development processes toward desired outcomes. Country capacity to measure progress toward core development outcomes is critical to country-led implementation o f PRSs, and i s the foundation for global monitoring o f progress toward MDGs and complementary monitoring efforts, such as the IDA results measurement system. IDA should continue to play a leadership role inthe global partnership for statistical capacity building. Therefore, it is recommendedthat, in addition to continuing support for the PARIS21 consortium, IDA intensib support within its CASs and projects for national statistical capacity building, and work in partnership to implement the global action plan to strengthen statistical systems, including householdsurveys. VI. CONCLUSION 32. The IDA program and its results measurement system can help low-income countries improve data collection contributing to better decisionmaking that fosters progress toward established goals. Strengthening the results focus o f CASs and IDA operations will increase support to countries for building capacity to manage for results and will help ensure that core development outcomes are regularly monitored. Consultations with IDA borrowers have indicated that countries welcome international efforts to develop comparable indicators for core development outcomes and improve the availability and quality o f basic data. Borrowers also encourage development agencies to harmonize results reporting requirements around national monitoring andevaluation systems for PRS implementation. 33. Importance of a Sound IDA Results Measurement System. The IDA results measurement system i s designed to inform discussions o f IDA's effectiveness and improve IDA's performance. At the same time, decisions made about the IDA results measurement system have the potential to influence other multilateral development banks and bilateral development agencies within an emerging global partnership on managing for results. Thus it is 14 all the more important to make careful decisions about the IDA system for the upcoming period, to ensure that goals are achieved in ways that are pragmatic and cost-effective. Designingthis system involves selecting a manageable set o f indicators, adopting an appropriate frequency for country outcome monitoring, having IDA performance measures that feed into the IDA management system to improve IDA performance, and increasing IDA'Ssupport to countries to strengthen their capacity-including statistical capacity-to manage for development results. ANNEX A RECENT CHANGESINPOSSIBLEOUTCOMESFOR IDA14 BASELINES 1. Despite the paucity o f data, an effort has been made to monitor aggregate progress among IDA-eligible countries for the possible country outcome indicators. This assessment was made for a longer-tenn trend (generally since 1990), and for the most recent period (generally by comparing 2000-2002 data with 1997-1999 data). 2. Longer-term Trends. Eleven o f the 18 possible indicators are also MDG indicated. As Table A1 and Box A1 indicate, progress toward achieving most MDGs was inadequate in IDA- eligible countries in the 1990s. While the direction o f change was generally positive, the pace was insufficient to achieve the 2015 targets. 3. Recent Changes. Recent progress can be assessed for 14 o f the 18 possible indicators, as indicated in Table Al. The others are relatively new indicators for which only baseline figures have been calculated. Analysis o f recent changes in the 14 indicators leads to the following observations: Thirteen o f the 14 indicators show progress in the most recent period-a favorable finding. Rates o f progress vary considerably, and for some indicatorssuch as childhood malnutrition and primary completion school completiobprogress has accelerated in recent years. This is an encouraging finding in the short term, which will needto be sustained inthe longer term. Although real per capita income across IDA countries rose by nearly 2 percent annually over the period 1997/99-2000/02, the proportion o f the populationbelow the US$l/day poverty line in IDA countries-estimated over a shorter period (1999- 2001) for far fewer IDA countries with limited new data-did not change significantly. Many factors could potentially account for this finding. The Bank's Development Economics Research Group is undertaking analysis over a longer timeframe and with a comparable set o f countries to more accurately assess the effect o f economic growth on US$l/day poverty rates worldwide. Growth in aggregate GDP from agriculture also appears low, which probably means that per capita growth in rural areas was negative in recent years-although a lack o f reliable data on the evolution o f rural populations prevents a firm conclusion. ' The factors that could account for this include, the periods o f computation are different; GDP may have increased significantly faster than household consumption expenditure; there may have been significant inequities in the way that GDP, employment, and income were distributed among consumers; and there may have been biases resulting from estimations o fmissing data. 16 ANNEXA Box Al. Progress Toward the MDGs in IDA Countries: Right Direction, Wrong Pace Population Below $US 1 Poverty Estimated values for 11 o f the 18 indicators in the IDA results Line measurement system provide an overview o f changes over time and in relation to the MDGs. With the exception o f HIV/AIDS and householduse o f solid fuels, all other MDG indicators show progress between 1990 and 2000 (see table below). O f the 11 MDG indicators for which time series data are available, 8 are directly related to quantifiable targets expressed in 'Ei z _I-, the Millennium Declaration. It is therefore possible to measure 1990 2000 2015 growth between 1990 and 2000 in the IDA countries and compare this historic rate with the overall rate required to reach the MDG Under-Five Mortality Rate targets. This rate can be calculated for the period 1990-2015 (overall required rate) or for the period 2000-2015 (catch-up rate). The table below shows the historic (1990-2000) rate, the overall rate required to achieve MDG targets (1990-2015), and the catch-up growth rates for eight MDG indicators. For three indicators (prevalence o f underweight children, ratio o f girls to boys, and access to water), growth rates between 1990 and 2 5 1 2000 would result in MDG targets being met. For the other five 1990 2000 2015 indicators (see graphs), progress has been insufficient to reach the MDG targets. The graphs show past performance and two future One-Year-Old Children Immunized scenarios: the predicted end point if historic rates o f change are Against Measles maintained (dotted line), and the rate (catch-up rate) from 2000 to 2015 that will achieve the MDGs (dashed line inbold). Average Annual Rates of Change: Historicversus Required (in percent) Overalt Catch-up 5 z _I-# Historic required required Indicator I 1990-2000 I I I 1990 2000 2015 1090-2015 2000-2015 Population below $US l/day -1.3 BirthsAttended by Skilled Health poverty line (PPP) -2.8 -3.7 Personnel Prevalenceof underweight -2.8 -2.8 (Wa) children under five years 1 I I 1 loo 75 1 Under-5 mortality -1.6 -4.4 -6.3 (number per 1,000 live births) - One-year-olds immunized 0.4 1.8 2.7 25 1Iqn 40 against measles (YO) I I I I - I I Birthsattendedby skilled I I I # I 1990 2000 2015 personnel (%) 2.7 4.3 5.4 Primary School Completion Ratio o f girls to boys in 1.5 1.3 (n'a) primary and secondary Ieducation (%) Primary school completion I 0.9 I 1.7 I 2.1 I Population with sustainable 1.7 0.9 (&a) accessto an improved water source (%) a u r cn I 1 I I I I I 1990 2000 2015 Source: DECDG,The WorldBank-prepared by ChangqingSun(DECDG) andPedroAriai (OPCRX). 17 ANNEXA Table Al. Changes in ProposedCountry Outcome Indicatorsin Recent Years Year Outcomes Average Countries annual includeda Coverageb Most growth Indicator Unit (no.) (%) Baseline recent Baseline Most recent rate(%)d 1. Proportion ofpopulation below I Percentof 48 91 ............E,f?P:............;........................................................... Percentof I 1 44 83 1;I.......................... ..................... i.............................................................. I........................... 1999 2001 33 33 i -0.3 1997199 120001021 41 37 : -3.3 ........................... childre ars.ofage...... .................... age 0-4 ......................................... ........................................... ........................... 3. Under : 1 ty ,000 live; 79 120 i -1.4 85 : 0.9 8. Primary school ............................................. ............................................ water source .......................................................... 12. Time required for b 13. Public expendi Notes: (..) meansinsufficient data, (da) meansnot applicable. a 80 IDA-eligible countries are considered in the aggregation for all proposed indicators. Timor-Leste is excluded because of its very limiteddata availability. Percent of relevantpopulation from total relevant population inthe 80 IDA countries. Four indicators do not have sufficient information yet to measureprogressbetweenIDA periods. The average annual growth is calculated between the baseline estimated value and the most recent estimated value. It is the most recent rate of growth experiencedby the indicator. e Note that, because o f the difference in estimation methods between years, changes in prevalence over time are not likely to be ANNEX B POSSIBLECOUNTRY OUTCOMEWDICATORS FOR IDA14 Indicator: Proportionofthe Populationbelow US$l/dayPovertyLine Definition Proportion of Population Population below US$ 1 a day is the percentage o f the population living on less below US%l/daypovertyline than $1.08 a day at 1993 intemationalprices. 8 Aggregation 2 100% d From the total of 80 IDA countries 48, representing 91 percent o ftotal population 2 ::g inthe IDA countries, were used to aggregate the figures inthe graph. Population 40% ineachcountry was usedto weight individual country data. $ ;20% 0% 1999 2001 The IDA Countries Included Albania, Armenia, Azerbaijan, Bangladesh, Bolivia, Burkina Faso, Burundi, Cambodia, Cameroon, Central African Republic, CBte d'Ivoire, Ethiopia, Gambia, Georgia, Ghana, Guyana, Honduras, India, Indonesia, Kenya, Kyrgyz Republic, Lao PDR, Lesotho, Madagascar, Malawi, Mali, Mauritania, Moldova, Mongolia, Mozambique, Nepal, Nicaragua, Niger, Nigeria, Pakistan, Rwanda, Senegal, Sierra Leone, Sri Lanka, St. Lucia, Tajikistan, Tanzania, Uganda, Uzbekistan, Vietnam, Yemen Rep., Zambia, Zimbabwe Relevanceto PovertyReductionand MDGs The ultimate goal of most development agencies and institutions is to eradicate extreme poverty and hunger. The first goal in the MillenniumDevelopment Declaration is to eradicate extremepoverty andhunger, andthe first target is to halve, between 1990 and 2015, the proportion of people whose income is less than U S 1 a day in purchasing power parity (PPP) terms. Ineffect, this indicator measuresprogress toward the reductiono f extreme poverty, which is also a goal o f most PRSPs. Sensitivityto PolicyChange As a result of revisions in PPP exchange rates, poverty rates cannot be compared with poverty rates reported previously for individual countries. Data showing as 2.0 signifies a poverty rate of less than 2.0 percent. However, the indicator is good proxy to capture changes in the number o fpoor people brought about by a govemment's actions and programs focused on reducingpoverty. Measurabilityand Reporting Estimates of this indicator are based on per capita incomes or expenditures derived from household surveys by the Bank's Economic ResearchGroup. Whenever possible, expendituresare used. The distribution o f per capita expenditure or income is estimated using empirical Lorenz (distribution) curves weighted by household size. Inall cases measures o f poverty to obtain Lorenz curves are calculated from primary data sources. Poverty in a country is estimated by converting the US$l a day poverty line to local currency using the latest PPP exchange rates for consumption taken from World Bank estimates. Localconsumer price indices are then usedto adjust the intemational poverty line in local currency to prices prevailing around the time of the household surveys. This intemational poverty line is used to identify how many people are below US$1 a day in PPP terms. The PPP-based intemational poverty line is required only to allow comparisons across countries and to produce estimates o f poverty at the aggregate level. Most countries also set their own povertj lines. The availability o f this indicator depends on the availability of household survey data with income or expenditure data, and the availability of PPP exchange rates. Household income and expenditure surveys are typically conducted every three to five years ir many developing countries, but in some countries less frequently. The most recent PPP exchange rates were calculated for 1993. Regional and global estimates o f poverty based on the US$l day poverty line are updated annually by the World Bank and published inWorld Development Indicators. Estimates are available every 3 years. The Bank i s working to compile an improved repository of national household survey data, which will provide data that are better harmonized across countries. Also, a nen round of the Intemational Comparison Programproject is planned, which is designed to update the PPP exchangerates. Costs (implicationfor IDA borrowers and the Bank) Estimating the proportion o f the population below a standardizedpoverty line requires a good quality incomeieqenditure survey, and estimates o f PPP. Over 75 percent o f IDA countries are participating in the new 2003-06 round of the Intemational ComparisonProgram, which will substantially improve the quality and coverage o f data for the calculation o fpoverty rates inPPP terms. Countries currently conduct incomeiexpenditure surveys roughly every five years or so. Surveys o f this type are driven b) domestic policy needs, for analysis to underpin PRSPs, or to form the basis for poverty assessments; more frequent surveys, for example to achieve the ideal three-yearly frequency of the IDA cycle, would require additional resourcesto carry our surveys of this type, which typically cost between $300,000 and $1 million per country. IThis note wasprepared by Martin Ravallion, Shaohua Chen, andJohan A. Mistiaen (DECRG) and the DECDG Team. 20 ANNEXB Indicator: Prevalenceof UnderweightChildrenUnder-fiveYearsofAge Definition Prevalence o f underweight children under-five years o f age, also known as Prevalence Of underweight prevalence o f child malnutrition (weight for age), is the percentageo f children under-fiveyears of age under-five whose weight for age is less than minus two standard deviations loo* from the median for the intemational reference population ages 0 to 59 80% months. The reference population, adopted by the World Health Organization 60% (WHO) in 1983, is based on children from the United States, who are assumed 40% to be well nourished. 20% Aggregation 0% From the total o f 80 IDA countries 44, represent 83 percent o f total children 1997-99 2ow-02 under-five years o f age in the IDA countries, were used to aggregate the figures shown inthe graph. The data are aggregated from country data, using the number of children under age five as weights. The IDA Countries Included Albania, Angola, Armenia, Azerbaijan, Bangladesh, Benin, Burkina Faso, Cambodia, Chad, Comoros, Congo Dem. Rep., Cote d'Ivoire, Entrea, Ethiopia, Gambia, Ghana, Guinea, Haiti, Honduras, India, Indonesia, Kyrgyz Republic, Lao PDR, Lesotho, Madagascar, Malawi, Mali, Mauritania, Mongolia, Nepal, Nicaragua, Niger, Nigeria, Rwanda, Senegal, Serbia and Montenegro, Sierra Leone, Sri Lanka, Sudan, Tanzania, Uganda, Vietnam, Zambia, Zimbabwe Relevanceto Poverty Reductionand MDGs Child malnutrition is linked to poverty, low levels of education, and poor accessto healthservices. Even moderate malnourishment even moderate, increases children's risk of death, inhibits their cognitive development, and affects health status later in life. The first MDG calls for reducing poverty, but also for halving the proportion o f people who suffer from malnutrition. Sufficient and ,good-quality nutrition is critical for development, health, and survival o f current and succeeding generations. Moreover, govemments will only be successful in their efforts to accelerate economic development in a sustained way when they ensure optimal child growth. Sensitivity to Policy Change Child malnutrition, as reflected in body weight, is selected as an indicator to measure malnutrition for several reasons. It is monitored more closely than adult malnutrition, can affect health in later life, and may be taken as an indicator of malnutrition in general. This indicator is sensitive to changes and policies leading toward better child nutrition and, thus, malnutrition reduction, in the IDA countries. Measurabilityand Reporting Data on malnutrition are compiled intemationally by UNICEF and WHO. The calculation of this indicator requires data on child weights, which are normally obtained from household surveys that include weight measurements. Surveys o f sufficient quality are typically undertaken every 5-10 years, although a greater frequency would be preferable. Some countries have few or no data. There can be difficulties associated with age reporting in some countries, and ages are estimated, in some cases. The weights of national child population are compared with the table o f weights from the National Center for Health StatisticsiWHO table for each age group. The percentageso f children whose weights are below the threshold o f minus two standard deviationsfrom the median are then aggregatedto form the total percentageo f the children under-five who are underweight. ICosts (implicationfor IDA borrowers and the Bank) IfthecurrentavailabilityofdataareconsideredsufficientforIDAmonitoring,thentherewouldbeverylittleadditionalcostfor including malnutrition in IDA RMS. More data would require more household surveys, which may cost from a $100,000 to $1 million, depending on their comprehensiveness. Surveys that collect data to calculate information on child malnutrition are normally more costly than standard socioeconomic surveys, because o f the need to accurately weigh children during the enumeration process. However, all o f the major intemationally sponsored surveys collect child malnutrition data (MICS, DHS, LSMS and CWIQ). Given past trends and future plans for these surveys it seemsreasonableto assume that most IDAcountries will conduct at least one o f these surveys ina three-year period, and at least one survey within the IDA14 period, and so the additiona cost equates to that o f ensuring that data are collected from different household surveys in a comparable manner-and that these data are then fully utilized by the intemational agencies responsible for compiling and reporting these data, inthis case WHO and UNICEF. This note wasprepared by Ed Bos and Emi Suzuki (HDNHE). 21 ANNEXB 1 Indicator: Under-Five Mortality Rate Definition Under-five mortality rate Under-five mortality rate is the probability that a newbom baby will die before reaching age five, if subject to current age-specific mortality rates. The probability i s expressed as a rate per 1,000. For example, as the graph shows, for every 1,000 <-8130 live births in IDA countnes in 1997-99, 125 children died before reaching age five, 120 GIZO whereas 120 did so in2000-02. E Aggregation ;xu0 S From the total o f 80 IDA countries 79, representing 100 percent of total estimated 2 1w births in the I D A countries, were used to aggregate the figures shown in the graph. 1997-99 2000-02 Estimatedbirths ineach country were usedto weight individual country data. The IDA CountriesIncluded Afghanistan, Albania, Angola, Armenia, Azerbaijan, Bangladesh, Benin, Bhutan, Bolivia, Bosnia and Herzegovina, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Congo Dem. Rep., Congo Rep., Cote d'Ivoire, Djibouti, Dominica, Eritrea, Ethiopia, Gambia, Georgia, Ghana, Grenada, Guinea, Guinea-Bissau, Guyana, Haiti, Honduras, India, Indonesia, Kenya, Kribati, Kyrgyz Republic, Lao PDR, Lesotho, Liberia, Madagascar, Malawi, Maldives, Mali, Mauritania, Moldova, Mongolia, Mozambique, Myanmar, Nepal, Nicaragua, Niger, Nigeria, Pakistan, Papua New Guinea, Rwanda, Samoa, Senegal, Serbia and Montenegro, Sierra Leone, Solomon Islands, Somalia, Sri Lanka, St. Lucia, St. Vincent and the Grenadines, Sudan, Tajikistan, Tanzania, Togo, Tonga, Uganda, Uzbekistan, Vanuatu, Vietnam, Yemen Rep., Zambia, Zimbabwe Relevanceto Poverty Reductionand MDGs The indicator relates directly to the fourth MDG ("Reduce child mortality") and measures child survival. Survival of a child is closely linked to the provision of primary health-care services; but malnutrition, a decline in breast-feeding, matemal education, use of improved water, and inadequacy sanitation and health are all associated with high child mortality. Sensitivityto PolicyChange The indicator is very powerful, since it reflects not only health care conditions, but also the social, economic, and environmental conditions in which children (and others in society) live. It is sensitive (in the medium term) to changes in policies affecting child mortality. Measurabilityand Reporting Since data on incidences and prevalence of diseases (morbidity data) are frequently unavailable, mortality rates are often used to identify vulnerable populations. Mortality rates are among the indicators most frequently used to compare levels of socioeconomic development across countries, and data on child mortality are more complete and more timely than data on adult mortality. To calculate this indicator, age-specific mortality rates are calculated from births and deaths data derived from vital registration, census, or household surveys, and an estimate o f the number o f children at risk. These are then summed for children under five years o f age. When using household surveys under-five mortality estimates can be obtained in a direct (using birth histories) andor indirect way ("Brass" method). The best source o f data are a complete vital registration system (one covering at least 90 percent o f vital events ir the population), but such systems are fairly uncommon in developing countries. Thus, estimates must be obtained from sample surveys or derived by applying indirect estimation techniques to registration, census, or survey data. Vital registration statistics are available generally once a year, but they are usually deficient in developing countries, and household surveys that include questions on birthsand deaths are performed every few years. Data from household surveys are also subject to a number o f sampling and nonsampling errors, including inadequate sample size (surveys estimating child deaths require large samples, because such incidences are uncommon), recall errors, and interviewer mistakes. Also, indirect estimates rely on estimated actuarial ("life") tables that may be inappropriate for the population concerned Finally, it should also be noted that when the under-fiveiinfant mortality rates are derived from survey data in a direct way (using birth histories), the latest rate generally represents an average for 0-4 years before the survey. UNICEF, WHO, and the World Bank produce harmonized estimates o f this indicator every year that make use o f all information available, including vital registration as well as direct and indirect estimates from household surveys and censuses. The estimates from household surveys and censuses are fitted into a weighted regressionfor the best estimates. Costs (implicationfor IDA borrowersand the Bank) If the modeled data from the interagency group are acceptable, there is little additional cost involved. Collecting information tc calculate child mortality rates tends to be the concem o f surveys specifically designed to collect demographic data, such as the MICS and DHS surveys, and population censuses. Some LSMS surveys include a mortality module, but LSMS samples used for these modules are typically too small to allow reliable estimates. Where available, data from vital registration systems may also be used, but these systems are not typically well developed in low-income countries. Incountries where they are conducted, the frequency o DHS surveys tends to be every five years; but their coverage o f the group o f IDA countries is not complete. MICS tend to cover morr IDA countries than DHS, but again these surveys have been conducted only every five years4ICSl in 1995 and MICS2 in2000 with MICS3 plannedfor 2005. This note wasprepared by Ed Bos and Emi Suzuki (HDNHE). 22 ANNEX B Indicator: ProportionofOne-year-old ChildrenImmunizedagainst Measles Definition Proportionof one-year-old children The proportion of one-year-old children immunized against measles is the immunizedagainst measles proportion of children aged one who received one dose of measles vaccine. A child is consideredadequately immunized against measles after receiving one doseo f vaccine. -EL 80% Aggregation 60% From the total o f 80 IDA countries 73, representing 100 percent of children ," 40% underage one inthe IDA countries, were usedto aggregatethe figures shown ap 20% inthegraph. Childrenunder age ofone ineachcountry were usedto weight 0% individual country data. 1997-99 2ow-02 The IDA Countries Included Afghanistan, Albania, Angola, Armenia, Azerbaijan, Bangladesh, Benin, Bhutan, Bolivia, Bosnia and Herzegovina, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Congo Dem. Rep., Congo Rep., Cote d'Ivoire, Djibouti, Eritrea, Ethiopia, Gambia, Georgia, Ghana, Guinea, Guinea-Bissau, Guyana, Haiti, Honduras, India, Indonesia, Kenya, Kyrgyz Republic, Lao PDR, Lesotho, Madagascar, Malawi, Maldives, Mali, Mauritania, Moldova, Mongolia, Mozambique, Myanmar, Nepal, Nicaragua, Niger, Nigeria, Pakistan, Papua New Guinea, Rwanda, Samoa, Senegal, Serbia and Montenegro, Sierra Leone, Solomon Islands, Somalia, Sri Lanka, St. Lucia, Sudan, Tajikistan, Tanzania, Togo, Uganda, - - Uzbekistan, Vanuatu, Vietnam, Yemen Rep., Zambia, Zimbabwe Relevanceto Poverty Reductionand MDGs More than 10 million children die each year in the developing world, the vast majority from causes preventable through a combination of good care, nutrition, and medical treatment. Mortality rates for children under-five dropped drastically in the past two decades, but the rates remain high in developing countries. This indicator is related to the fourth MDG ("Reduce child mortality") since immunization is an essential component for reducingunder-five mortality, and it serves as a proxy to measurethe coverage and the quality of the child health care system in the IDA countries. Sensitivity to PolicyChange Govemments in developing countries usually finance immunization against measles, diphtheria, pertussis (whooping cough), tetanus, andpolio as part o f the basic public health package. The indicator is sensitive to government's immunizations campaigns, and values fluctuate according to when immunization campaignsare carried out. MeasurabilityandReporting Estimates of immunization coverage are generally based on two sources o f empirical data: administrative data and coverage surveys. For estimates based on administrative data, the immunization coverage i s derived by dividing the total number o f vaccinations given by the number o f children in the target population. For most vaccines the target population is the national annual number of births or number of surviving infants (this may vary depending on countries' policies and the specific vaccine). Immunization coverage surveys are frequently used inconnectionwith administrative data. The indicator is estimated for children aged 12-23 months who receive a dose o fmeasleseither any time by the survey or before age 12 months. The two data sourcesare reports of vaccinations performed by service providers (administrative data), and household surveys containing items on children's vaccination history (coverage surveys). Routine data are compiled by the National Expanded Programme on Immunization programmanagers. The principal types of surveys usedas sourceso f information on immunizationcoverage are the EPI 30 cluster survey, the UNICEF Multiple Indicator Cluster Survey (MICS) and the Demographic Health Surveys (DHS). Administrative data are collected annually and surveys are generally less frequent. Data are collected intemationally through the Annual WHO/"ICEF Joint ReportingForm on Vaccine Preventable Diseases, and are then screened and standardized. The first dose o f measlesvaccine is supposed to be administered to all children at the age o f nine months or shortly after. By2000, most countries were providing a "second opportunity" for measles vaccination, either through a two-dose routine schedule or through a combined routine schedule and supplementary campaigns. Inmany developing countries lack o f precise information on the size of the cohort of children under one year o f age makes immunization coverage difficult to estimate, and survey data are also subject to erroneous maternal reports. Costs (implicationfor IDA borrowers and the Bank) Ifthe data collected through the WHOKJNICEF form are acceptable, there is little additional cost involved. Data are relatively easy to collect from any national household survey or census provided that definitions are clearly and consistently applied during enumeration. An analysis of past trends, and existing plans for the Bank's Living Standards Measurement Surveys and surveys based on the Core Welfare Indicator Questionnaire, the Demographic and Health Surveys sponsored USAID, and the UNICEF Multiple Indicator Cluster Survey reveal that almost all IDA countries conduct at least one major survey ina three-year period, and will do so during the IDA 14 period. Assuming that these surveys are already funded, data collection for this indicator will not require additional surveys. This note wasprepared by Ed Bos and Emi Suzuki (HDNHE). 23 ANNEX B Indicator: HIV Prevalence Among Women Aged 15-24 Years Definition HIV prevalence rate among HIVprevalenceat any given age is the differencebetween the cumulativenumbersofpeople woman aged 15-24 years who have become affected with HIV up to this age andthe numberwho died, expressedas a percentage of the total number of alive at this age. The basis of measuring infection is the incidence ofHIV among women 15-24at hospitals and antenatalclinics. zz 2:g Aggregation Fromthe total o f 80 I D A countries 44, representing 90 percent o f the population of females * 0 0 aged 15-24 in the IDA countries, were used to aggregate the figures shown in the graph. 0% The population o f females aged 15-24 in each country was used to weight individual 1999 2001 country data. The I D A Countries Included Angola, Bangladesh, Benin, Bolivia, Burkina Faso, Burundi,Cambodia, Cameroon, Central African Republic, Chad, Congo Dem. Rep., Congo Rep., Cote d'Ivoire, Ethiopia, Gambia, Ghana, Guinea-Bissau, Haiti, Honduras, India, Indonesia, Kenya, Lao PDR, Lesotho, Madagascar, Malawi, Mali, Moldova, Mozambique, Nepal, Nicaragua, Nigeria, Pakistan, PapuaNew Guinea, Rwanda, Senegal, Sierra Leone, Sri Lanka, Tanzania, Togo, Uganda, Vietnam, Zambia, Zimbabwe Relevance to Poverty Reduction and M D G s MDG number six is to combat HIVIAIDS, malaria, and other diseases. These diseases, which are among the world's most important killers, all have their greatest impact on poor countries and poor people. They interact in ways that make their combined impact worse. In addition, the economic burden of epidemics such as HIVIAIDS on families and communities is enormous. Estimates suggest that when the prevalence o f HIVIAIDS reaches 8 percent-about where it is for several African countries today-the cost in economic growth is estimated at about one percent a year. Prevention and treatment programs will save lives, help economic development and reduce poverty. Infection leads to AIDS, and without treatment, average survival from the time of infection is about nine years. Access to treatment is uneven, and no vaccine is currently available. About half of all new HIV cases are among people 24 years old or younger. In generalized epidemics (with prevalence consistently at more than 1 percent among pregnant women), the infection rate for pregnant women is broadly indicative o f the overall rate for the adultpopulation, provided adequate age, gender, and residence. Therefore, this indicator is ameasure of spread ofthe epidemic. It is worth mentioning that many ofthe IDA countries listedabove do not have a generalizedepidemic. Sensitivity to Policy Change Changes in HIV prevalence in women 15-24 is determined largely by the relationship between the number of women infected and the number o f deaths from AIDS. This leads to a number of conclusions. First, the larger the spread inage o f a group, the less is the impact of new infections, which typically occur at a young age. Second, regardlesso f the age o f the spread, an increase inthe death rate relative to the infection rate will reduce the prevalence rate and make it appear that the problems o f HIV infection is declining, even while infection maybe rising, though not at the same rate as deaths. Thus, reductions in HIV incidence associated with genuine behavioral change should first become detectable in HIV prevalence figures for people aged 15 to 19, particularly when incidence o f HIV is rising. Under these circumstances, the 15-24 year-old group is a less satisfactory early-warning indicator o f a growing epidemic. Parallel behavioral data should be used to aid interpretation o f trends in HIV prevalence. In countries, with concentrated or stable epidemics a relevant indicator could be "HIV prevalence among female sex workers," and in countries with significant injecting drug use transmission, "HIV prevalence among injecting drug users" could also be useful. In nascent or concentrated epidemics, antenatal surveillance may not capture HIV infections if the epidemic i s largely concentrated inHIV-vulnerable groups that are not representedas part o f the measurementof HIV incidence at hospitals and antenatal clinics. Inthis case, there may be an underestimate. Therefore, the indicator i s not a good tool for tracking within country impacts o fpolicies such as use o f condoms and behavioral change. Nevertheless, it does reflect inter country differencesunderlining the HIVIAIDSchallenge. Measurability and Reporting There are no reliable estimates of HIV infections among women aged 15-24. UNAIDS estimates this number from records of HIV infection among pregnant women aged 15-24. Data on HIV in pregnant women come from tests on blood samples taken as part of regular anonymous examinations of pregnant women aged 15-24 at antenatal clinics, which have been chosen as sentinel surveillance sites for HIV infection. The sentinel surveillance sites are typically chosen to reflect urban, rural, ethnic, and other socio-geographic divisions in a country, but in practice antenatal examinations are more frequent in urban and peri-urban clinics, which biases lesults toward HIV prevalence in urban areas. Also, by choosing pregnant women as the sample population, data are biased by a potential higher sexual activity of the subgroup, leading to a potential higher prevalence rate. The data are collated annually m most developing countries. These data are gathered by UNAIDS, UNICEF, and WHO, which have formed coalitions to coordinate global efforts to treat victims and prevent the diseases from spreading. The data used in the WDI are drawn from the Joint UN Progam (UNAIDS) and WHO'S AIDS Epidemic Update (2002) report. Surveillance i s being strengthened with donor support. UNAIDS and US Centers for Disease Control and Prevention are also examining improved testing protocols. The most important problem with this indicator, however, is that different methodologies have been used to calculate prevalence rate and hence the senes are not comparable over time. In early July 2004, UNAIDS will publish estimates of HIV prevalence among adults (15-49 years) for 2001 and 2003 that will be comparable over time. Costs (implication for I D A borrowers and the Bank) Data for HIV prevalence rates are basedon modeled estimates calculated by UNAIDS. Estimates have been made for all I D A countries for the years 2000 and 2002. It is likely that data from these estimation methods will be available in future at a frequency to allow changes in aggregates from one three-year period to another. Quality is also likely to increase, particularly since the inclusion of blood testing insome 18 DHS surveys. -. . .. - - .- .--_.. . 1 - - - . _ 1 , Indicator: BirthsAttended by SkilledHealth Staff Definition Proportionof births attended by Births attended by skilled health staff are the percentage of delivenes skilled healthpersonnel attended by personnel trained to give the necessary supervision, care, and 100% advice to women during pregnancy, labor, and the postpartum period, to conduct deliverieson their own, and to care for the newboms. y1 80% Aggregation 5 60% From the total o f 80 I D A countries 52, accounting for about 82 percent of % total estimated births inthe IDA countries, were usedto aggregate the figures 40% shown inthe graph. Estimated number o f births ineach country were usedto weight individual country data. 20% 0% 1997-99 2000-02 The IDA Countries Included Angola, Armenia, Azerbaijan, Bangladesh, Benin, Bhutan, Bolivia, Bosnia and Herzegovina, Burkina Faso, Cambodia, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Cote d'Ivoire, , Eritrea, Gambia, , Grenada, Guinea, Guinea-Bissau, Guyana, Haiti, Honduras, India, Indonesia, Lesotho, Madagascar, Malawi, Maldives, Mali, Mauritania, Mongolia, Nepal, Niger, Nigeria, Pakistan, Rwanda, Senegal, Serbia and Montenegro, Solomon Islands, Sri Lanka, St. Vincent and the Grenadines, Tajikistan, Tanzania, Togo, Tonga, Uganda, Uzbekistan, Vietnam, Zimbabwe Relevanceto Poverty Reduction and MDGs Every year more than 500,000 women die from complications of pregnancy and childbirth. Most of the deaths occur in Asia, but the risk of dying is highest inAfrica. Women inhigh-fertility countries inSub-SaharanAfrica have a I-in-16 lifetime risk of dying from matemal causes, compared with women in low-fertility countries in Europe, who have a kin-2400 risk, and in North America, who have a 1-in-3,500 risk o f dying. High matemal mortality rates in many countries are the result of inadequate reproductive health care for women and inadequately spaced births. The fifth MDG aims at improving matemal health, with a target of reducingby three-quarters, between 1990and 2015, the matemal mortality ratio. Sensitivity to PolicyChange The indicator monitors the ability o f the health system to provide good antenatal and postnatal care for women. "Skilled health personnel" are intended to include only those who are properly trained and who have appropriate equipment and drugs, but standardization o f the definition o f "skilled" health personnel is sometimes difficult because of differences in training o f health personnel among countries. Traditional birth attendants (TBAs), who may have leamed from other TBAs, even if they have received a short training course, are not intended to be included. In spite of the limitations, the indicator is sensitive to government's actions or policies that lead towards an increase in births attended by professionals, and it is assumed that mortality rates will subsequently drop. Measurabilityand Reporting Measuring matemal mortality accurately is notoriously difficult, except where there is comprehensive registration o f deaths and causeso f death. Some indicators track progress by focusing on professional care during pregnancy and childbirth (particularly for the management o f complications) and the most widely available indicator is the proportion o f women who deliver with the assistance of a medically trained health care provider. Data on this indicator are compiled by UNICEF and WHO. To calculate this indicator, the number o f births attended by skilled personnel (doctors, nurses, or midwives) is expressedas a percentage of deliveries (or births if those are the only data available) in the same period. Data are collected through household surveys, in particular the Multiple Indicator Cluster Survey (MICS) sponsored by UNICEF, the Demographic and Health Survey (DHS), and as well as other national householdsurveys. Household survey data on this indicator are generally available every five years. Costs (implicationfor IDA borrowers and the Bank) Ifthe data compiled by UNICEFNHO are acceptable, there is little additional cost involved. Data are relatively easy to collect from any national household survey or census, provided that definitions are clearly and consistently applied during enumeration. An analysis ofpast trends and existing plans for the Bank's Living StandardsMeasurement Surveys (LSMS) and surveys based on the Core Welfare Indicator Questionnaire (CWIQ), the Demographic and Health Surveys (DHS) sponsored by USAID, and the UNICEF Multiple Indicator Cluster Survey (MICS) reveal that almost all IDA countries conduct at least one major survey in a three-year period, and will do so during the IDA14 period. Assuming that these surveys are already funded, data collection for this indicator will not require additional surveys. This note wasprepared by Ed Bos and Emi Suzuki (HDNHE), 25 ANNEXB Indicator: Ratio of Girlsto BoysinPrimary and Secondary Education Definition Ratioof girls to boys in The indicator is defined as the ratio of the gross enrollment rate o f girls to primary and secondary education boys inprimary and secondary education levels inbothpublic and private schools. 6100% -B2 80% Aggregation d From the total of 80 IDA countries 52 were usedto aggregate the figures L 60% shown in the graph, and they represent79percent of total IDA. The D population ineach country was used to weight individual country data. 40% G I 20% 1997-99 2wo-02 The IDA Countries Included Albania, Azerbaijan, Bangladesh, Benin, Bolivia, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Chad, Comoros, Congo Rep., Cote d'Ivoire, Djibouti, Dominica, Eritrea, Ethiopia, Gambia, Georgia, Ghana, Grenada, Guyana, India, Indonesia, Kenya, Kyrgyz Republic, Lao PDR, Lesotho, Maldives, Mauritania, Mongolia, Mozambique, Myanmar, Nepal, Nicaragua, Niger, Papa New Guinea, Rwanda, Samoa, Senegal, Serbia and Montenegro, St. Lucia, St. Vincent and the Grenadines, Sudan, Tajikistan, Tanzania, Togo, Tonga, Vietnam, Yemen Rep., Zimbabwe. Relevanceto Poverty Reductionand MDGs Women have an enormous impact on the well-being o f their families and societies, but their potential is sometimes not realized because o f discriminatory social norms, incentives, and legal institutions. Although their status has improved in recent decades, gender inequalities persist. Education is one o f the most important aspects o f human development, and eliminating gender disparity at all levels o feducation would help to increase the status and capabilities o fwomen. The third MDGseeks to promote gender equality and the empowerment o fwomen, and this indicatorprovides a measure o fequality o f educational opportunity. Sensitivity to Policy Change The indicator is an imperfectmeasure o fthe relative accessibility o fschooling for girls for the following reasons: it does not allow assessment o fwhether slight improvements inthe ratios reflect an increase ingirls' school attendance (desirable) or a decrease in boys' attendance (undesirable); and it does not show whether those enrolled at school complete the relevant education cycles. The indicator, however, is sensitive to actions that lead toward a sustained increase in the access o f schooling for girls, and thus, toward gender equality and the empowerment o fwomen. Measurability and Reporting The unit o f measurement is girls' enrollment ratio as a percentage o f boys' enrollment ratio. The usual method of computation is to take the number o f boys and girls, regardless o f their ages, enrolled. The number o f enrolled girls/boys is then expressed as a percentage o f school age female/male population. Private education tends to be underreported. International coverage has improved in recent years, but reported with a two-year time lag. The enrollment data usually come from school records as reported to ministries o f education and/or national statistical agencies. The official data are collected by UNESCO from approximately two-thirds o f countries inthe world, using an annual questionnaire-based survey. The data collection is based on the International Standard Classification of Education classification, which allows for international comparability between countries and over time. Costs (implication for IDA borrowers and the Bank) Data for indicators based on school enrollments are normally produced from education management information systems, based on school records. There are often quality problems associated with the recording o f enrollment, suck as those caused by age misreporting; there would be development costs associated with addressing these issues in many countries. Thisnotewasprepared by SukaiPromJackson (HDNED). 26 ANNEXB Indicator: Primarv SchoolComDletion Rate Definition Primary school completionrate Primary completion rate (PCR) is the number of students successfully completing the last year of (or graduating from) primary school in a given year, divided by the number of children of official graduation age in the population. Because of 809. 76X difficulties with developing data based on this definition, data analysis is & %5@ generally based on the PCR proxy indicator-i.e., the number of childrenreaching the last year of primary school (as defined by a country) net o frepeaters. *0 a x 409. Aggregation ffi From the total o f 80 IDA countries 55, representing about 72 percent of the total 1997-59 number of children o f official graduation age in the population in the IDA countries, were used to aggregate the figures shown in the graph. The number of children o f official graduation age in the population in each country was usedto weight individual country data. The IDA Countries Included Albania, Armenia, Azerbaijan, Bangladesh, Benin, Bhutan, Bolivia, Bosnia and Herzegovina, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Chad, Comoros, Congo Rep., Cote d'lvoire, Djibouti, Eritrea, Ethiopia, Gambia, Georgia, Ghana, Guyana, Honduras, India, Indonesia, Kenya, Kyrgyz Republic, Lao PDR, Lesotho, Madagascar, Malawi, Maldives, Mali, Mauritania, Moldova, Mongolia, Mozambique, Nepal, Nicaragua, Niger, PapuaNew Guinea, Rwanda, Samoa, Senegal, Sri Lanka, St. Vincenl and the Grenadines, Tajikistan, Tanzania, Togo, Uganda, Uzbekistan, Vanuatu, Zambia. Relevanceto PovertyReductionand MDGs Education creates choices and opportunities for people, helps to reducepoverty and gender bias, lowers birth rates, and stimulates i better understanding of diseases. For countries, it creates a more dynamic workforce and well-informed citizens able to compete and cooperate globally-opening doors to economic and social prosperity. Universal primary education (measured through primary completion rate) i s the secondMDG. Information provided on actual completion captures one o f the most critical issues ir the education sector. Sensitivity to PolicyChange Vanous factors may lead to poor performance on the PCR indicator, including low quality of schooling, discouragement resulting from poor performance, the direct and indirect costs o f schooling, and the demands o f farm work which keep children out of schoo for extended periods. Students' progress to higher grades may also be limited by the availability o f teachers, classrooms, and educational materials. However, this indicator is the most direct measure of national progress toward universal primary education and it captures the final outcome o f the primary education system. But these factors take some time to change. Therefore. improvements (or declines) inresponseto policy change will only register slowly. Measurability and Reporting Although the World Bank and UNESCO's Institute o f Statistics are committed to monitoring the proxy PCR indicator annually systems for collecting and standardizing the data from all IDA countries are not yet inplace; as a result, the current database has many gaps, particularly for small countries and earlier years, and there are anomalies and estimates. Efforts to compile more current data from countries by staff o f the World Bank's Human Development Network reveal several inconsistencies that rendei the data difficult to interpret. The collection o f information for the actual PCR based on students who successhlly complete primary education requires focused discussion on the indicator and agreement on the bases for defining completion with quality. This discussion must take into consideration the objective assessment o f country-defined competencies for primary school graduation. Not all countries have such competencies, with implications for policy reforms in country curriculum and exit examinations. Costs (implicationfor IDA borrowers and the Bank) Data for this indicator basedon school enrollments are normally producedfrom education management information systems, basec on school records. There are often quality problems associated with the recording of enrollment, such as those caused by age misreporting; there would be development costs associated with addressing these issues in many countries. The primary schoo completion rate is a more difficult statistic to produce than data on enrollments, interms o f data sources required, since informatioi i s neededon completion and this is not always recordedby schools. I t also requires accurate estimates o f the numbers o f childrei in the population of school completion age, which in tum requires a good quality census or vital registration system, and goo( national capacity to analyze demographic data and make forecasts. Thisnote wasprepared by Sukai Prom Jackson (HDNED). 27 ANNEXB Indicator: Ponulation with SustainableAccess to an ImDroved Water Source Definition Proportionof populationwith sustainable Access to an improvedwater source is currently defined within the UNsystem accessto an improvedwater source as the percentageo f the population that can obtain at least 20 liters per person per day from an "improved" source that is within one kilometer o f the user's loo% dwelling. Improved water sources include household connection, public 5 standpipe, borehole, protected well or spring, and rainwater collection. 2 Unimproved water sources refer to water provided through vendors, tanker 60% trucks, unprotected wells, unprotected springs, andbottled water. B 40% s 20% Aggregation From the total of 80 I D A countries 34, representing 86 percent o f total I D A 0% 1990 zoo0 population, were used to aggregate the figures shown in the graph. The population ineachcountry was used to weight individual country data. The IDA CountriesIncluded Bangladesh, Bolivia, Burundi, Cameroon, Central African Republic, Comoros, Cote d'Ivoire, Ethiopia, Ghana, Guinea, Haiti, Honduras, India, Indonesia, Kenya, Madagascar, Malawi, Mali, Mauritania, Nepal, Nicaragua, Niger, Nigeria, Pakistan, Papua New Guinea, Senegal, Sri Lanka, Sudan, Tanzania, Togo, Uganda, Vietnam, Zambia, Zimbabwe Relevance to PovertyReductionand MDGs MDG 7 ("Ensure environmental sustainability") contains target 10 that aims at halving, by 2015, the proportion of people without sustainableaccessto safe drinking water and basic sanitation. Unsafe water and lack o f basic sanitation is the direct cause of many water-related diseases in developing countries. This indicator monitors access to improved water sources basedon the assumption that improved sourcesare likely to provide safer water. Specifically, access to safe drinking water is o f fundamental significance to lowering the nsk o f faecal contamination and the frequency of associated diseases. In addition, its association with other socioeconomic characteristics, including education and income, makes it a good universal indicator o f human development. Sensitivity to PolicyChange Ideally, an indicator o f "access to safe water" would refer to the percentageof the populationwith reasonableaccessto an adequate supply of safe water intheir dwelling or within a convenient distance of their dwelling. However, "access," quality, and volume of drinking water are difficult to estimate, and therefore sources of drinking water that are thought to provide "safe" water within a reasonable distance o f the householdare used as a proxy. Inusing this proxy indicator, it is assumedthat improved water sources (defined above) are likely to provide safe water. The existence o f an "improved" water supply does not alwaysmeanthat it is safe, nor that local people use it. The proposed indicator is still sensitive to change in the context o f governmentand community actions. The indicator will capture progress in the IDA countries from actions or policies that lead to an increase in the access to safe drinking water. The definitions and numbers will also be consistent with those adopted by the UNsystem for tracking the MDGs. MeasurabilityandReporting Since the late 1990s, data are routinely collected at national and subnational levels, in more than 100 countries, using censuses and surveys by national governments, often with support from international development agencies. Before these population-based data were available, provider-based data were used. Evidence strongly suggests that data from surveys are more reliable than the administrativerecords. There are many doubts about dataquality; despite officialWHO definitions, the judgment about whether or not water is safe is often made very subjectively. Administrative data are often available annually, but household surveys are less frequent. The compilation of the data from household surveys into country, regional, and global coverage estimates is conducted every two to three years. WHO and UNICEF compile official international estimates under the Joint Monitoring Programand their latest report Global Water Supply and Sanitation Assessment 2000 Report w o r l d Health Organization and UNICEF Joint Monitoring Programme,for Water Supply and Sanitation: Geneva) hasbeenusedas the data source. Costs (implicationfor IDA borrowers and the Bank) The marginal cost is minimal. The Bank is involved inan advisory capacity to the Joint MonitoringProgramme o f WHOiUNICEF which collates this information, and it participates intwo or three meetings a year to discuss how best to improve the indicator's measurement. However, the Bank is not involved in collecting such national data, and does not anticipate any significant additional costs inadopting such an indicator. This note wasprepared by Peter J.Kolsky, William Kingdom, Jonathan D. Halpern, and Caroline VanDen Berg (EWDWS). 28 ANNEXB Indicator: FixedLines and Mobile TeleDhonesDer 1.000 Inhabitants Definition Fixed lines and mobile telephone This indicator is a composite of two measurements, namely, the number o f per 1,000 inhabitants telephone fixed lines connecting a customer's equipment to the public switched telephone network (per 1,000 inhabitants), and the number of subscribers to a c LW mobile telephone service using cellular technology that provides access to the M public switched telephone network (per 1,000 inhabitants). The composite is the simple addition ofthe two indicators. 8 A- 10 Aggregation k %a, From the total o f 80 IDA countries 79, representing 100percent o fpopulationin the IDA countries, were used to aggregate the figures shown in the graph. The 2000-02 population counts in each country were usedto weight individual country data. 1997-99 The IDA Countries Included Afghanistan, Albania, Angola, Armenia, Azerbaijan, Bangladesh, Benin, Bhutan, Bolivia, Bosnia and Herzegovina, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Congo Dem. Rep., Congo Rep., Cote d'Ivoire, Djibouti, Dominica, Eritrea, Ethiopia, Gambia, Georgia, Ghana, Grenada, Guinea, Guinea-Bissau, Guyana, Haiti, Honduras, India, Indonesia, Kenya, IOribati, Kyrgyz Republic, Lao PDR, Lesotho, Madagascar, Malawi, Maldives, Mali, Mauritania, Moldova, Mongolia, Mozambique, Myanmar, Nepal, Nicaragua, Niger, Nigeria, Pakistan, Papua New Guinea, Rwanda, Samoa, Sa0 Tome and Principe, Senegal, Serbia and Montenegro, Sierra Leone, Solomon Islands, Somalia, Sri Lanka, St. Lucia, St. Vincent and the Grenadines, Sudan, Tajikistan, Tanzania, Togo, Tonga, Uganda, Uzbekistan, Vanuatu, Vietnam, Yemen Rep., Zambia, Zimbabwe Relevanceto Poverty Reductionand MDGs The increasing significance of telecommunications for economic development and poverty reduction is well recognized. Telecommunications provide farmers, workers, and entrepreneurs opportunities to reduce transaction costs, increase market coverage, and improve competitiveness across borders. People living in rural and remote areas tend to be poor and socially isolated. They lack information relevant to their particular situation and have difficulty interacting with other community members or other communities. Telecommunicationscan help the disenfranchised voice their concerns, demand their rights, take control of their own lives, and improve information flows and communication services to make government and organizations serving the poor more efficient, transparent, and accountable. Information and Communication Technology (ICT), such as telephone and e mail, can be of great value in bringing people together, bridging geographic distances, and providing relevant information about and to the poor. The importance of telecommunications is reflected in the Millennium Development Goals. In particular, the eighth goal focuseson developinga global partnership for development, in which accessto new technologies is highlighted (Target 18). Three key indicators are used to track this goal: (a) telephone lines and cellular subscribers per 100 population; @) personal computers in use per 100 population; and (c) Internet users per 100 pcpulation. In addition to Goal 8 o f the M D G indicators, telecommunicationsiICT can contribute to improving the efficiency of delivering progress toward the MDGs in other sectors and accelerating the achievement ofdevelopment targets. Sensitivityto Change The two components of the indicator (telephone mainlines and mobile telephone service) are related to the country's macroeconomic situation and government decisions and actions, which take place at different speeds in the deregulation o f markets. Wars or economic decline cause destruction o f fixed lines. Teledensity stagnates where barriers to the provision o f demanded services exist, and increases where successful telecommunication sector reform has taken place, particularly in the mobile sector. These factors make the indicator sensitive to change in policy and valuable for measuring progress in the development o fmodem communications technologies. Measurability and Reporting This indicator measures telecommunications; it does not measure the quality o f service or the volume o f traffic. Information on mainline and mobile telephone connections is provided by the International Telecommunications Union (ITU), which carries out annual surveys of telephoneregulators and telephone companies in its member countries. The I T U cleans and reports the data from its surveys. This indicator will be measuredby countingthe number of tangible phone lines and subscribers. Costs (implication for IDA borrowers and the Bank) Baseline, updates and maintenanceof the data are covered by ITU's regular work program. I t is therefore possible to obtain annual data o f this indicator from the ITU, available for most IDA countries. Data on mainline and mobile telephone subscribers are contained in the World Telecommunications Development Report and associated databases, published annually by the ITU; and in different format they are in re-published in the Bank's World Development Indicators. Additional costs for the Bank would involve staff time to analyze the data, track the performance, andproduce reports as requested. This note wasprepared by Charles J. Kenny (CITSr), Christine Zhen-Wei Qiang(CITST), and Anat Lewin (CITPO). 29 ANNEXB Indicator: Time and Cost for BusinessStart-uD Definition Time and cost f o r business start-up Time and cost of business start-up measures the time and cost for a new limited liability 91% company to fulfill all necessary requirements to legally start operations inthe specified country. loo 8o It is a performance measure o f investment climate institutions. aL M 40 Aggregation From the total o f 80 IDA countries 39 (38 for cost), representing 89 (88 for cost) percent o f the 20 total IDA population, were used to calculate the population-weighted aggregate days and cost Jan 02 Janai data for IDA countries. I-Timc i d w i -cost i%.i CNI olr c.nttaii The IDA Countries Included Albania, Armenia, Azerbaijan, Bangladesh, Benin, Bolivia, Bosnia and Herzegovina, Burkina Faso, Cameroon, Cote d'Ivoire, Ethiopia, Georgia, Ghana, Honduras, India, Indonesia, Kenya, Kyrgyz Republic, Madagascar, Malawi, Mali, Moldova, Mongolia, Mozambique, Nepal, Nicaragua, Niger, Nigeria, Pakistan, Senegal, Serbia and Montenegro, Sri Lanka, Tanzania, Uganda, Uzbekistan, Vietnam, Yemen Reo.. Zambia. Zimbabwe ~~ Relevance to Poverty Reduction and MDGs. Long-term economic growth is a critical requirement for poverty reduction. Inthis processo f growth, privateenterprises have an essential role. Private businesses, small and large, are the engine o fjob creation and income growth and thus provide poor people with the chance to improve their living standards. But huge obstacles to growth exist in the form o f red tape, weakjudicial systems, over-zealous licensing, and inefficient banking systems. Why do such impediments exist? Wouldn't the government eam tax revenue ifthey were abolished? Are there any recent cases o f successfulreform in improving the environment for doing business? What catalyzes improvements in the environment for doing business? The time and cost o f business start-up are gatheredas a part of the Doing Business project inthe Private SectorVice Presidency,by tracking the processfor astandardizedcompany to complete all of the necessary regulatory requirements to register a business formally. Formal cost o f business start-up (registration and licensing procedures), measuresthe official payments requiredto fulfill all requirements for businessstart-up, scaled as a percentageo f GNI per capita. The cost o f registration is a major barrier to entry in poor countries, deterring participation in the formal economy. Time required for business start-up (registration and licensing procedures) measures calendar days for a firm to obtain all necessary permits, and to notify and file with all requisite authorities, in order to legally operate a business. Time delays in registratun are a major impediment to formal entry of private firms and increase the potential for corruption. Sensitivity to Policy Change The time and cost o f business start-up indicators are unique in that they can be linked directly to policy action. Behind the total time and cost measures is a full description o f the procedures, rules, and regulations for business start-up. Policymakers can review this information, identify specific bottlenecks to business entry, and know what to reform. For example, in January 2002, Ethiopia was one o f the most expensive countries in the world in which to start a business, at 437 percent o f GNI per capita. The breakdowno f the business entry process showed that the high cost was driven mainly by the requirement to publish an official notice in the newspaper. Following dialogue with the World Bank, the Ethiopian Government eliminatedthe public notice requirement, and the cost plummetedto only 85.2 percent o f GNI per capita. Between 2002 and 2004, over 20 o f the 39 I D A countries have improved the efficiency o f business start-up. Many o f these reforms were directly supportedby the policy dialogue o f the World Bank. The indicators are increasingly being adopted in country work, including in adjustment lending, PRSPs, and country assistance strategies. Measurability and Reporting As o fJanuary 2004, the data are available for 134 countries, including 55 IDA countries. Data are available for 39 countries as of January 2002. For consistency o f comparisons, only the data for the 39-country sample are reportedinthe above chart. The data are updated annually, with a time lag o f only a few months from the point o f measurement-Le., January 2004 data are available by April 2004. The indicators are built through a combination o f desk researchand expert assessment. The project team starts by studying the laws and regulations in force on business regulations, as well as reviewing publicly available summaries and descriptions o f the business start-up process. From this research, a detailed list o f the steps, time and cost for business start-up are compiled. This list is then sent to business start-up experts in the country, who are asked to verify the data, identify any missing stepddata, and make any necessary corrections. If there are differences in their answers the project staff go back to the respondents until the data can be reconciled. To be comparable across countries, the indicatorsmeasure the time and cost for business start-up under specific assumptions about the company size, industry,legal identity, and location, as well as the procedures followed. The data cover only the generic entry requirements and do not capture industry-specific licenses or utility hookups. They cover only mandatory official procedures and costs, and therefore exclude voluntary procedures and delays as well as informal payments. Actual start-up time and cost reportedby individual firms, for example in firm-survey based investmentclimate assessments, may therefore vary from the indicators to the extent that a particular firm does not match the assumptions in the standardized case. The Doing Business project reports the indicators on a website http://rm.worldbank.orgldoingbusiness/and in its annual publication. The DoingBusiness series represents a collaborative effort. The team works with leading scholars linking theory, academic rigor, and practice. Each assessment involves a partnership with an association o f practitioners or an international company (e.g., law firm o f Baker and McKenzie). The Doing Business project receives the invaluable cooperation of local partnerscg., municipal officials, incorporationlawyers, businessconsultants, etc. Once an assessment is completed, the results are subject to a peer-review processin leading academicjournals. This collaborative and continuous process o f refinement produces indicators that have been scrutinized by the academic community, government officials, and local professionals.I Costs (implication for IDA borrowers and the Bank) The data are collected as part o f the Doing Business project, and the marginal cost for IDA has been zero. The cost o f an additional country is around $3,000, and the size o fthe country does not affect the cost. This note was prepared by Axel Peuker(CIC1C) and Caralee McLiesh (CICMA). I Time and cost o f business start-up include procedures beyond registration at the company registrar-such as filing for different taxes, registering employees, obtaining health department clearances, and all other procedures requiredto legally start operations. 30 ANNEXB Indicator: Public Financial Management Definition Public Financial Management (PFM) Public financial management (PFM) is measured through 16 benchmark indicators that were average number o f benchmarks met chosen as critical elements o f effective PFM and deemed necessary for tracking poverty- per country reducing public spending The recently added 16th benchmark indicator is for procurement, but will not be included in the comparison due to lack o f 2002 data (See Table Bl.) =go z3 1990 2000 2002 Primarycompletion rate(%) 28. However, the number o f countries with a positive growth rate can only partially reflect the improvement made inthe IDA countries during this period. It does not inform us about how many countries are making progress inthis period and how far these countries are to the goal o f universal primary education. For example, the fact that two thirds o f the IDA countries have made positive growth since 2000 seems to be at least as informative and relevant as the second target. Figure 2 shows that the progress has been made across the board. Countries22are grouped into five classes depending on whether their completion rates were less than 30 percent, or between 30 percent and 50 percent, or between 50 percent and 70 percent, or between 70 percent and 90 percent, or 90 percent or higher. The distribution o f the number o f countries among the five classes is plotted for the three benchmark years - 1990, 2000 and 2002. The numberso fcountries belonging to the lowest two classeshas decreased steadily, from 12 in 1990 to 5 in2002 for the lowest class and fi-om 19 to 13 for the second lowest class. At the same time, the other three classes see an increasingnumber o f members duringthis period. 29. Table G4 presents the progress ina more dynamic way. Countries are grouped according to the classifications o f both their 1990 primary completion rates and their 2002 PCRs. Each row shows the 2002 distribution out o f a group o f countries with a specific classificationin 1990, while each column shows the 1990 distribution out o f a group o f countries with a specific classification in 2002. So the cells on the right o f the diagonal cells represent improvement in the completion rate classification, while the cells below the diagonal cells represent a deterioration. Altogether there are twenty-nine countries that have improved, and only five that have moved down. These five countries are Burundi, Central African Republic, Congo Democratic Republic, Nigeria, and Zambia, all inthe Sub-Saharan Africa. Amongthe seventycountries includedinthe "New Spring2004 Result", eight haveno 1990value. Another sevencountries are excludedincountingthe number of countrieswith positivegrowthrelative to 1990becausetheir 1990values exceed 100percent. 22 The seven countries whose 1990values exceed 100percentare includedin Figure 2 andTable F4. 69 ANNEXG Table G4: Change in Distribution 2002 value Average completion rate (not weighted) 1990 1995 1999 2000 2001 2002 Ifcountriesgroupedby: 2000value>1990value 42 51 57 60 61 63 2000value<= 1990value 65 55 53 53 55 56 Ifcountriesgroupedby: 1990value<70% 42 47 52 53 55 57 1990value>=70% 85 81 82 85 85 84 31. Table G5 strongly suggests two different growth paths for the IDA countries. The "performing" countries, as defined above, have made impressiveprogress inthe 90s as well as in the 2000-02 period, but from a low starting point. The "non-performing" countries, on the contrary, have experienced a decline in the 90s from a much higher starting point, but reversed the decline decisively inthe recent years. There are fifteen "non-performing" countries24,with a primary completion rate in 1990 ranging from 34 percent (Rwanda) to 92 percent (Zambia). Three "non-performing" countries (Cambodia, Cameron and Guyana) have surpassed their 1990 values in the 2001-02 period. Even if the two groups are defined based on whether the 1990 value i s greater than 70 percent or not, the difference in their growth paths remains. The countries with 1990 values less than 70 percent have made progress continuously in the 1990s, and similar improvement inthe 2000-02 period. Butmost countries whose 1990 values are equal to or greater than 70 percent25have seen their primary completion rates fluctuating around their 1990 values since 1990. 23 The seven countries whose 1990values exceed 100percent are excluded here. 24 Armenia, Burundi, Cambodia, Cameroon, Central African Republic, Congo Dem. Rep., Congo Rep., Ghana, Guyana, Kenya, Nigeria, Rwanda, Sudan, Vanuatu, Zambia. 25Armenia, Georgia, Guyana, Indonesia, Nigeria, Vanuatu, Serbia and Montenegro, Zambia. 70 ANNEXG 32. The difference in growth potential in primary completion rates among IDA countries should be taken into consideration in setting any new target in the future. Countries with different initial levels o f completion rates may face very different policy options. The impact o f similar policy inputs is likely to be different across countries. So initial condition should be considered insetting targets. D. Conclusion 33. IDA countries have made significant progress on both measles immunization coverage rate and primary completion rate by 2002. The targets for both indicators havebeenmet. 34. One lesson that can be drawn from this experience is that the target-setting process should be based on clear assumptions and principles inorder to be informative inthe later policy debate. A clear understanding o f the logic underlying the expected progress could facilitate an assessmento f actual progress against a given target when the baseline value i s revised or when a different set o f countries have to be included due to data availability constraint. Even more important, when clear links cannot be made between policy interventions and the targeted results, we cannot say what has worked in the case o f reaching the target and what has not worked inthe case o f failure. 35. Another lesson is that targets need to be comprehensive, allowing a straightforward interpretation, and intended to measure progress during a relevant period. The target o f the number o f countries with positive growth relative to 1990 should be reassessed in the future results-measurement system. The change in the number o f countries with positive growth relative to 1990 fails to capture the progress made by the majority o f countries which either have surpassed their 1990 value at the beginning o f the measurement period and stayed above their 1990 value duringthe period, or have not surpassed their 1990 value at the ending point but are on the way to breakthrough. Therefore, the benchmark o f 1990 value used should be replaced with one that is comparable across countries and more relevant for the MDG target-universal primary education. ANNEXH EXECUTIVE SUMMARYOFTHE IMPLEMENTATIONOF THE AGENDAONMANAGING FORRESULTS' 1. The need to better manage for results-to use information to improve decision-making and steer country-led development processes toward clearly-defined goals-has emerged at the forefront o f the global development agenda in the post-Monterrey period. The conceptual framework for the results agenda was definedinthe course o f 2002 as the next step inthe World Bank's long-standing efforts to improve delivery management, operational quality, and aid effectiveness. Implementation o f the Bank-wide agenda on better managing for results began in early 2003 with an Implementation Action Plan that called for actions across three pillars: (a) in countries, where development results are achieved, to strengthen both capacity and demand to manage for results; (b) in the Bank, to enhance the relevance and effectiveness o f our contribution to results; and (c) across development agencies, to harmonize results-based approaches andbetter coordinate support to strengthen country capacity to manage for results. 2. Questionsfor the Committee. Progress across the three pillars o f the results agenda i s summarized below. In reviewing this report, members o f the Committee on Development Effectiveness may wish to address the following questions: 0 How could the Bank further bolster its efforts to help countries strengthen their capacity for strategic planning, public sector management, statistics and monitoring and .evaluation? 0 To what extent has the design o f early results-based country assistance strategies met the need for a more monitorable and evaluable way to track the Bank's contribution to country results? 0 What more could the Bank do to encourage the global partnership to harmonize results reporting and increase coordination o f support to countries to strengthen capacity? A. Recent Progress 3. Over the past year, the Bank has moved beyond the conceptualization phase into the implementation phase o f the results agenda. Within this short period, progress has been made on all pillars o f the Implementation Action Plan. Nonetheless, much remains to be done to filly implement an agenda that requires medium-term institutional development at the country level, systems development and a further change in mindset within the Bank, and an easing o f constraints to harmonized action among donors. 4. Strengthening Country Demand and Capacity to Manage for Results. Developing countries need both sustained political will and national institutional capacity to manage for results. This is the fundamental issue o f the results agenda and its most difficult challenge. To address this challenge, the Bank has focused on helping countries strengthen national strategic planning (including for poverty reduction strategies) and provided support for results-based public sector management, statistical capacity and monitoring and evaluation systems. Initial progress includes strengthened guidance from the Poverty Reduction and Economic ' Implementation of the Agenda on Managingfor Results, (R2004-0071, IDAR2004-0097), May 6,2004. 72 ANNEXH Management Network (PREM) on design and monitoring and evaluation o f Poverty Reduction Strategy Papers (PRSPs), and demand-driven support from the Bank's Monitoring and Evaluation Improvement Program (MEIP) for countries to assess their institutional readiness and to strengthen results-based approaches to public sector management. An important step this past year was development by the Development Economic Data Group (DECDG) o f the Statistical Capacity Building Program (STATCAP)-approved by the Board on March 25- which provides a sectorwide approach to building capacity based on a strategic statistical plan for providing reliable and timely data on countries' core development outcomes as articulated in their PRSPs or national development strategies. 5. Enhancing the Bank's Contribution to Development Results. Significant first year progress has been made in increasing the focus on results in Bank strategies, instruments, incentives, and reporting systems. Inparticular, country teams have made progress inpiloting a central element o f the agenda: moving to a results-based country assistance strategy (CAS) that shifts the focus to country-level results and more explicitly links these results to the choice o f products and services within Bank programming. Sector Boards also made progress in strengthening the results frameworks and outcome monitoring for sector and thematic strategies. Revision o fbasic documents and procedures has begunto strengthen the articulation o f outcome- oriented objectives and monitoring for Bank operations. Further, broad agreement on an Intemational Development Association (IDA) results measurement system that focuses on progress toward country outcomes and on IDA'Scontribution through the CAS has been an important step toward improved reporting on results. However, development o f a more comprehensive results reporting system will be sequenced in coming years to take advantage o f enhanced monitoring and evaluation o f CASs, sector strategies, and global programs. Interms o f staff leaming and incentives, a Bank-wide event, Results: Everybody's Business, held in early January 2004 highlighted emerging good practices and sent strong signals throughout the institution onthe importance o fimplementing the results agenda. 6. Fostering a Global Partnership on Managing for Development Results. A formal partnership has been established through the Multilateral Development Bank (MDB) Working Group and, more recently, through the MDB/Organization for Economic Co-operation and Development-Development Assistance Committee (OECD-DAC) Joint Venture on Managing for Development Results. For the global statistical community, more than a year o f preparation resulted in agreement on a medium-term global action plan to strengthen intemational statistical systems. The Second Intemational Roundtable on Managing for Development Results, held in February 2004, helped foster an emerging consensus on priorities for the global partnership. Sponsoring agencies endorsed a Joint Memorandum, core principles and action plan that can serve as a foundation to broaden this consensus and take further action incoming years. B. LookingAhead 7. In FY05, the Bank will deepen implementation of the results agenda. A continued change in mindset within the institution will be critical for this deepening to occur across all three pillars o fthe actionplan. 8. Country Capacity. Supporting countries in strengthening capacity to manage for results remains the most difficult longer-term challenge o f the results agenda. In the near term, the 73 ANNEXH Bank will increase advocacy and outreach on the importance o f managing for results through regional workshops, and teams will intensify support within country assistance strategies to strengthen country capacity to manage for results. Regions and Networks will work more closely with the World Bank Institute, PREM, DEC, OPCS and other central units to provide clients with a better integrated menuo f services to support the strengthening o f core capacities in public sector management, statistical systems, and monitoring and evaluation. 9. Internal Focus. Within the Bank, implementationwill focus on expanding coverage and improving the quality o f results-based CASs and CAS Completion Reports, strengthening the monitoring and evaluation framework for sector strategies and global programs, beginning to develop a more comprehensive results reporting system, reinforcing technical support for task teams and rolling out a revised staff learning program. Getting staff incentives right- particularly informal ones-will remain the greatest internal challenge in supporting countries to achieve sustainable development results. The Bank will further analyze this internal issue inthe coming year, drawing on the work o fthe new Task Force on Organizational Effectiveness. 10. Global Partnership. The global partnership requires greater commitment and coordination o f resources for strengthening country capacity, as well as increased donor willingness to harmonize reporting requirements. In 2004, the MDB Working Group and OECD-DAC Joint Venture will identify and disseminate good practices inmanaging for results, raise awareness through regional activities and support country-led processes to harmonize results reporting inat least four countries. The Results Secretariat will coordinate preparation o f a global good practice guide on managing for development results, drawing on country and partner experiences to date. 11. Near-term Progress Indicators. Inlooking ahead to the next Progress Report which will report on implementation o f the results agenda through end-FYOS, the Bank has established monitorable indicators that reflect the concerns o f the three pillars. By end-FYOS, Committee members can expect the following: 0 Ten country studies on poverty monitoring systems completed, to draw lessons andgood practice on the design, establishment and functioning o f such systems in the PRSP context; 0 Strategic statistical plans completed for at least halfo f all PRSP countries; 0 STATCAP programs effective inat least five countries; 0 At least 20 CASs with adequate results frameworks and Completion Reports; 0 At least 75 percent o f projects under implementation are satisfactory or better for monitoring and evaluation; 0 Bank participation in country-led processes to harmonize results reporting around national M&E systems inat least four African countries (per Marrakech Action Plan). 12. Driving Progress across the Three Pillars. At the Results: Everybody's Business event inJanuary 2004, Bank staff noted that "the focus on results is here to stay." As one Manager 74 ANNEXH concluded, "it is no longer a question o f `if,but rather, o f `how' we manage for results.''2 This represents an important-albeit nascent-change in mindset that i s critical to a sustained and successful focus on results. Nurturing this change will be essential, and the Bank will take further action to ensure that managers at all levels of operations send consistent signals on the importance o f managing for country results, and provide clear rewards for doing so. With appropriate signals from managers, and continued support from Executive Directors, Bank staff will become increasingly results-focused in their day-to-day work, incorporating in Bank programming greater support to strengthen country demand and capacity to manage for development results. Ultimately, nothing can substitute for countries' capacity and will to steer their own development processes toward desired outcomes. Much can be done, however, to help countries achieve better development outcomes through the Bank's unwavering commitment to implement the agenda on managing for results. See ResultsEverybody's BusinessNewsletterfor January 12,2004, http OLXS \It rz iie\is:results jar112 w e e t i pdf