IDA13 Performance Management in IDA International Development Association April 2002 Performance Management in IDA 1. This note attempts to summarize the issues that emerged at the technical meeting on performance management on April 18, 2002 and outlines an approach for Deputies' consideration at the upcoming IDA13 meeting on May 1-2, 2002. 2. Participants at the technical meeting recognized the need to establish a system to measure and monitor the results of IDA assistance and to link the design and delivery of IDA lending and policy advice to development outcomes. There was also a shared sense that any such system should be multilateral in nature and thus should be agreed upon by all IDA Deputies. 3. Timeline. Many participants noted that the development of a performance measurement, tracking and management system has just begun, that there is much to be learned over the next few years, and that it would be useful to review progress during the IDA13 period. Accordingly, Management concludes that the balance of advice was that Deputies review the system, its inputs, and its outputs, at two points during the IDA13 period: at the IDA13 mid-term review (Spring 2003) and at the start of the IDA14 replenishment process (Spring 2004). At the first review, Deputies could assess progress on agreed targets for IDA inputs (see Table 1) and could review the Bank's overall performance management system along with its associated baseline data, identified outcome indicators, and expected progress targets. At the second review, Deputies could assess further progress on agreed targets for IDA inputs and could review the first-year results of the performance management system. In addition, Deputies could also assess progress on the set of output indicators identified in the discussion paper "Measuring Outputs and Outcomes in IDA Countries," February 2002. Donors could have the option of linking additional IDA contributions to progress captured by either the input or output indicators. This sequenced approach to tracking outputs and accounting for results could inform the design of a performance management system going into IDA14. 4. Borrower involvement. A concern was raised at the technical meeting regarding the need to ensure borrower involvement in the development of a performance management system. Because the performance management system will rely on the outcome and output targets identified in countries' PRSPs, borrower involvement will be a critical component. Nevertheless, efforts will still need to be made at the country level to promote consensus building for the PRSP in order to ensure broad ownership of the outcomes and of the strategies for achieving them. 5. Input indicators. Many participants felt that Management should focus on tracking a core set of IDA inputs, such as fiduciary ESW. The key requirement was to better be able to measure the performance of IDA as an institution in delivering on its commitments. It was felt that since core analytical work and policy advice constitute the basis on which IDA provides assistance to its clients, tracking the delivery of these products could help gauge the performance of IDA as an institution. Along these lines, - 2 - taking into account the estimated work program during the IDA13 period, the following schedule of inputs could be put in place. The inputs suggested here have been selected on the basis of their importance to the country dialogue and their role in informing the design of policy reforms in key areas. The completion targets are cumulative. Table 1. IDA Inputs to be Tracked during IDA13 Product Number Number Completed by Completed by Spring 2003 Spring 2004 Country Financial Accountability 30 40 Assessment (CFAA) Country Procurement Assessment 24 38 Review (CPAR) Public Expenditure Review (PER) 29 40 Investment Climate Survey (number 15 25 initiated) 6. A number of participants emphasized the need to improve the quality of ESW in IDA countries and to ensure that quality is not compromised in an effort to deliver on quantity. The quality of ESW is a result of many factors including country capacity which often presents a greater challenge for IDA countries. However, the Bank's Quality Assurance Group (QAG) will review the ESW proposed below along the following five dimensions: relevance, internal quality, Bank process, country dialogue, and likely impact. These QAG assessments will help IDA Management identify areas for increased attention going forward.1 7. Output Indicators. There was also broad recognition that country output indicators would form an important part of any performance measurement system but that these indicators should ideally mesh with ongoing work at the Bank, among the donor community, and within IDA countries, including the PRSP process and efforts to achieve the MDGs. There was a shared understanding that the mechanics of a sensible and functioning system will present a significant challenge and will require a long-term effort that will extend well beyond IDA13. Many participants stressed that the tracking and assessment of outputs and outcomes should be closely related to the efforts being taken by IDA countries themselves under their PRSPs, and hence objectives for such measures should be closely tied to expected PRSP (and by extension, IDA CAS) outcomes. 1. In addition, as Deputies are aware, the Financial Sector Assessment Program (FSAP), a joint Bank- Fund exercise, analyzes, in individual countries, the sensitivity of financial institutions to macroeconomic shocks; measures the depth, development and diversity of the parts of the financial system; evaluates the adequacy of legal and institutional infrastructure; assesses compliance with selected international standards; and proposes priorities for work to strengthen the financial sector. Several such assessments have been carried out for IDA countries, and while the program pertains to Industrial, IBRD and IDA countries, it is expected that roughly 5-8 FSAPS per year will be carried out in IDA countries over the next few years. - 3 - 8. Participants at the meeting recognized the difficulties associated with selecting a small number of output indicators, given the depth and breadth of issues that are important to poverty alleviation and economic development. It was recognized that the output indicators proposed in Management's discussion paper2 are intermediate indicators for three of the seven MDGs3 and were selected based on their ability to capture development effectiveness, to link to IDA and country inputs, and to be measured across countries and over time. As such, it could be expected that these indicators would in many cases be the same as those used in countries' PRSPs. Nevertheless, some questions were raised as to whether the set was too narrow and whether the targets specified were related to country objectives. Some participants proposed additional output indicators, but many felt that the selection of indicators should be handled in a measured way. 9. Participants acknowledged the difficulty of attributing progress in a country output indicator to IDA's influence, given the many external factors involved and the time lags associated with some of the data. A number of participants expressed concern about the possible distortionaryeffects of focusing attention on a few selected output indicators. The group of indicators in Management's discussion paper was selected with this concern in mind and recognizing that there is a trade-off between attribution on the one hand and potentialdistortionary effects on the other. It was felt that the distortionary effect of these indicators is minimal given the many factors (most of which are under governments' control) that must be in place to achieve progress (such as functioning public health systems, economic incentives for children to remain in school, passage and implementation of legislation governing private sector investment, etc.). 10. Nevertheless, as with any measure of progress, judgment plays a role in interpreting the indicators, and supplemental, qualitative discussion around the results will be needed. Even if the proposed outputs are adopted in IDA13 as early indicators of IDA's progress in reaching specific outcomes, it would be important to fold them into a more comprehensive set of measures in the near term. Deputies would need to consider enriching this set of outputs with measures that will have emerged from the overall performance management system in time for the start of the IDA14 replenishment discussions. 11. Set of Countries to Achieve Outputs. Another issue that was raised at the technical meeting concerned the set of IDA countries for which the output indicators would be tracked. IDA Management's proposal, as set out in the discussion paper, noted that progress on outcomes could more easily be attributed to IDA's influence in countries that received substantial amounts of IDA resources because they were among the better 2Discussion has focused on the following four indicators: primary school completion; measles immunization; cost of business start-up; and time required for business start-up. 3The primary school completion rate is an indicator of progress towards universal primary education; the measles immunization is an indicator of progress towards reducing child mortality; and the reduction in inefficiencies associated with starting new businesses is an indicator of reduced barriers to productivity- enhancing investment which leads to economic growth and poverty reduction. - 4 - performers. Management had therefore suggested that the set of countries be limited to the upper three performance quintiles.4 As noted at the technical meeting however, participants were concerned that this approach could run the risk of possibly marginalizing the remaining countries where a focus on achieving results is clearly important. It was generally the sense of the meeting that if the output measures were to be adopted, they should, where possible, be defined for the entire group of IDA countries with the understanding that somewhat less ambitious targets would be reached. Attachment 1 sets out the broader list of countries and the adjusted performance targets for the output indicators. 12. Guidance from Deputies. On the basis of the above, the following guidance is sought from Deputies.: · Do Deputies agree on the overall timeline to measure performance, with reviews at the mid-term review, and at the start of IDA14? · Do Deputies agree on the selection of inputs to be tracked and the targets for progress on their delivery (as set out in Table 1)? · Do Deputies agree to defer the specification of output indicators until a broader and richer set of indicators emerges from the ongoing work at the Bank to develop a performance management system (Spring 2003)? · If it is desirable to define some output indicators in the near term, do Deputies agree with the approach set out in Attachment 1, derived from Management's earlier paper but measured over all active IDA countries where data exists? 4At the technical meeting, it was clarified that IDA Management had proposed that input targets would pertain to all IDA countries. Only output measures were linked to a subset of better performing IDA countries. - 5 - Attachment 1 IDA Countries to Which Output Estimates Would Apply Afganistán 1/ Lesotho 1/ Albania Liberia 2/ Angola 1/ Madagascar Armenia 2/ Malawi Azerbaijan Maldives 2/ Bangladesh Mali Benin Mauritania 1/ Bhutan 1/ Moldova Bolivia Mongolia 2/ Bosnia & Herzegovina 2/ Mozambique Burkina Faso Myanmar 2/ Burundi 1/ Nepal Cambodia 1/ Nicaragua Cameroon Níger Cape Verde 1/ 3/ Nigeria Central African Republic 1/ Pakistan Chad 1/ Rwanda 1/ Comoros 1/ 3/ St. Lucia 1/ Congo, Democratic Republic of 1/ St. Vincent & Grenadines 2/ Congo, Republic of 1/ Samoa 2/ 3/ Cote d'Ivoire Sao Tome & Principe 2/ Djibouti 1/ Senegal Dominica 2/ Sierra Leone 2/ 3/ Eritrea 1/ Solomon Islands 1/ 3/ Ethiopia Somalia 2/ Gambia, The 1/ Sri Lanka Georgia 2/ Sudan 2/ Ghana Tajikistan 2/ Grenada 2/ Tanzania Guinea * Togo 1/ Guinea Bissau 1/ Tonga 2/ 3/ Guyana Uganda Haití 1/ Uzbekistan 2/ Honduras Vanuatu 1/ India Vietnam 2/ Indonesia Yemen 2/ Kenya Yugoslavia FR Kiribati 2/ 3/ Zambia Kyrgyz Republic 2/ Zimbabwe Laos 1/ Note: 1/These countries are not includedin the benchmarks for the PSD measure due to lack of available data. 2/These countries are not included in the benchmarks for the education measure due to lack of available data or existence of only one survey at this time (they could enter data set at a later date). 3/These counties are not included in the benchmarks for the health measure due to lack of available data or existence of only one survey at this time (they could enter data set at a later date). - 6 - Performance Targets for Selected Output Indicators (based on all IDA countries where data exists) Education: (a) Population-weighted average completion rate: Target improvement: increase from 68% in 2002 to 69% by 2004 (b) Number of countries with positive growth rates in primary completion rates: Target improvement: increase from 32 countries in 2002 to 38 countries by 2004 Health: (a) Overall coverage rate (population weighted) of measles immunization: Target improvement: increase from 58% in 2002 to 60% by 2004 (b) Number of countries with 80% coverage of measles vaccination (a threshold beyond which measles can be contained): Target improvement: increase from 27 countries in 2002 to 29 countries by 2004 Private Sector Development (a) Time required for business start-up (registration and licensing procedures) Target improvement: 7% reduction in the number of business days in the specified countries (weighted by population) from end-2001 to Spring 2004 (b) Formal cost of business start-up (registration and licensing procedures) Target improvement: 7% reduction in official cost (in percent of GDP per capita) in the specified countries (weighted by population) from end-2001 to Spring 2004