23016 r y-cis WORLD BANK OPERATIONS EVALUATION DEPARTMENT WINTER 2001 NUMBER 204 ARDE 2000 From Strategy to Results his year's Annual Review of Development Effectiveness (ARDE) confirms that significant improvements have taken place in the Bank's development performance over the past five years. Despite a challenging and volatile world economy, portfo- lio performance is likely to exceed the Strategic Compact target of 75 percent satisfactory in FY00. The ARDE finds, however, that out- comes of country programs frequently differ from the aggregate port- folio performance of the country. Development effectiveness at the country level depends on the adaptation of strategies and instru- ments-including partnerships-to country conditions, not just on project performance. Moreover, OED reviews of sector and thematic strategies suggest that while the Bank usually articulates a coherent strategic framework, these strategies frequently encounter difficulties in implementation. ARDE 2000 draws on the findings of the debt relief, and global public goods. The Operations Evaluation Department's Bank has also taken upon itself the goals of (OED's) most recent evaluative work to help generating and disseminating development guide management of tensions and knowledge, engaging its partners in global objectives in projects, country assistance policy debates, and dealing with global programs, and sector and thematic activities. development challenges that cannot be tackled effectively at the country level. An Increasingly Complex Mandate These institutional responses have given The Bank's mandate has expanded and rise to tensions and tradeoffs among goals. deepened over the past decade. In response to changes in the international economy Performance Trends and the development system, the Bank is Despite the challenging business environ- placing more emphasis on institutional ment, the Bank's efforts to enhance project development, good governance, social quality in the past five years have borne $ g development, post-conflict reconstruction, fruit. Preliminary results for FY00 exits 2 World Bank Operations Evaluation Department Figure 1. Trends in Satisfactory Project Outcomes Figure 2. Regional Trends in Satisfactory Operations, by Projects Percent satisfactory * Exit FY 90-94 90 Percent satisfactory Exit FY 95-98 100 N Exit FY 99-00 80 ~By disbursements *8 70 6 By projects [60 60 40 20 - 50 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Exit fiscal year ECA LCR EAP SAR MNA AFR Bankwide show an increase to 77 percent satisfactory outcomes adjustment programs should increasingly link to explicit from 72 percent in FY99. Weighted by disbursements, development outcomes, with the accountability of project performance deteriorated from FY99 to FY00, various actors clearly delineated. reflecting the exit of two large, unsatisfactory oil and gas projects in Russia (figure 1). Performance by Addressing Tensions and Tradeoffs disbursement is likely to recover once the rest of the As it seeks to implement an increasingly crowded FY00 cohort is evaluated. Project performance remains agenda, the Bank faces a number of tensions and below Bank average in Africa, but Quality Assurance challenges. It must learn to reconcile client ownership of Group (QAG) data from the current portfolio could activities with the Bank's corporate priorities; adapt signal a promising turnaround in response to portfolio global knowledge to local conditions; support countries improvement efforts (figure 2). with strong enabling environments and those with poor These gains have been achieved even though projects enabling environments, but a high incidence of poverty; are becoming increasingly complex and demanding, and be efficient and selective in implementing a holistic especially in low-income countries. More than four of vision of development, making judicious use of every five recently evaluated projects were substantially partnerships. demanding for the borrower. Since more than one in The Review concludes that reforms and three projects show unsatisfactory borrower development programs are more likely to be sustained implementation, further progress will require parallel if there is consensus within the country to support improvements in capacity building and adapting each them and a shared vision between the Bank and the project's design to borrower capacity. country. At the country level, the Bank needs to use its Institutional development outcomes have reached a lending and nonlending tools more deliberately to new high, at 43 percent substantial. Sustainability is up foster consensus through increased local involvement to 57 percent. But there remains substantial room for and more widespread dissemination of analytic work; improvement in both Bank performance and borrower through more use of pilot projects to test new capacity development. approaches; and through greater use of advisory Adjustment lending continues to perform strongly, services and flexible lending instruments to nurture reaching 86 percent satisfactory in FY99/00. But findings policy change processes. In sector and corporate from project audits and country evaluations suggest that strategies, the Bank should acknowledge areas of the achievement of the immediate policy objectives in disagreement and identify priorities for knowledge adjustment loans does not necessarily translate into a creation, cross-country learning, and strategic use of long-term impact on incomes and poverty. Such an its convening role. impact requires government commitment and enough Development effectiveness also depends critically on consensus to sustain reform. Outcomes can also be adapting institutional priorities and programs to undermined by exogenous factors. As the Bank moves individual countries' constraints and opportunities. Such toward a results-based framework, the objectives of adaptation requires an up-to-date knowledge base about Pr6cis 204 3 each country's operating (and enabling) environment. Bank advice, willingness to undertake difficult reforms, Despite an increased emphasis on knowledge management, and capacity to implement recommended measures. In current resource allocation patterns do not give sufficient some cases the Bank has lent to support reforms-either priority to economic and sector work. through adjustment or investment lending-before Countries with stronger policy frameworks receive more sufficient consensus had emerged. Stakeholder analysis is lending resources per capita. But for each individual coming into wider use, but remains the exception rather country, especially those with poor policies and than the rule. institutions, the Bank's effectiveness depends not solely on Lending operations remain the basic building blocks of lending volume, but also on the mix of instruments, the country programs, but Bank nonlending services- timing of policy-based loans, and the judicious use of including economic and sector work, aid coordination, partnerships. and partnerships-also influence the effectiveness of The Bank has initiated a wide range of productive lending and overall program outcomes. The Bank is partnerships, but will realize their full potential only by working to be more selective, relating lending volumes to being more businesslike and selective about what they are country performance ratings. But continued lending intended to achieve, what risks they entail, and how they despite poor performance contributes to unsatisfactory can yield operating economies for the Bank through outcomes. sharper delineation of its comparative advantage. In middle-income countries, particularly those with access to private capital, the Bank's financial contribution Adapting to Country Context tends to be countercyclical and affected by the cost of In recent years, the Bank has sought to shift its focus from doing business with the Bank. Of course access to private individual projects to the higher plane of country capital varies by country and sector, and is not binary. programs. The outcomes of country programs often differ But lending trends suggest that the Bank should revise its from the country's aggregate portfolio performance (see lending role in countries with strong capacity and ready table 1). And country program outcomes are not access to private capital, and refine its approach to necessarily determined by country income (figure 2). OED nonlending services in countries for which, for reasons of Country Assistance Evaluations (CAEs), project evaluation poor performance or external shocks, capital markets have data, and other evidence suggest that the two most closed. Bank lending and nonlending activities should important determinants of country program outcomes are focus on catalyzing support for selected reforms and the level of borrower commitment to the country program institutional development, facilitating cross-national and the extent to which Bank strategies, programs, learning, and promoting global public goods. processes, and partnerships are adapted to the country In many low-income countries, borrower capacity and setting. local financial resources are binding constraints. In these Country evaluations confirm that the Bank has cases, the Bank should first strengthen capacity for aid strengthened its poverty focus, but the linkage between coordination, help harmonize donor standards, and assist Bank country programs and poverty outcomes was not borrowers in enhancing the coordination and coherence always clear. More needs to be done to integrate broad- of donor activities. Second, even in "good performing" based poverty reduction strategies into macroeconomic low-income countries, simplifying project and program and sectoral programs and to strengthen poverty design and sequencing reforms are priorities. Third, as monitoring and evaluation. CAEs also find a pattern of confirmed in CAEs for Burkina Faso and Uganda, Bank over-optimism about borrower governments' receptivity to efforts to strengthen country offices in low-income Table 1. How Country Program Outcomes Stack Up Against Portfolio Performance Country program outcome Portfolio performance Partially (% projects satisfactory) Fully satisfactory satisfactory Marginal Unsatisfactory >85 Argentina (1995-00) Kazakhstan (1991-99) Costa Rica (1990-00) Uruguay (1990-00) Burkina Faso (1989-99) 70-85 Mexico (1989-99) Indonesia (1s993-98) Paraguay (1990-00) Tanzania (1996-00) 50-l9 Ghana (1995-00) Egypt (1990-00) oanuatoT1986-h Bn i Ukganda (1986-99) India (1990-00) <50 Camerpon (1996-00) Caeroon (1982-95) ______________________________________ __________________Papua New Guinea (1989--00) Note Income hold=upper-midle; standardmeower-midute, talicplow. 4 Precis cED OED PARTNERSHIPS AND KNOWLEDGE GROUP countries should continue, since this helps to improve Editor-in-Chief: Elizabeth Campbell-Pag6 project supervision and enhance country dialogue. A special challenge is to find ways to reduce poverty in low-income countries with weak governance and Assistant: Juicy Qureishi-Huq policy frameworks. The Bank needs to strengthen and clarify its approach, diagnosing the factors affecting weak performance and using its convening role to build consensus for reform. Adjustment lending should Pricis are available at no charge by contacting the OED be avoided pending concrete actions that signal Help Desk: tel: 1-2021458-4497, or fax the form below to: commitment to change. Depending on country 1-202/522-3125. You can also e-mail your request to: circumstances, limited, targeted grants or lending elinc@worldbank.org operations to strengthen public sector accountability systems or to provide support to targeted and closely Mail this form to: monitored poverty programs may be appropriate. Pr&is-OEDPK, World Bank 1818 H Street, N.W. Sector and Thematic Performance Washington, D.C. 20433, U.S.A. The Bank's shift toward the higher plane of country Quantity Title Pr6is# strategy has been paralleled by a shift toward improving sector and thematic performance at the country and global levels. The challenge has been (1) to strengthen the strategic focus and policy content of Bank lending and nonlending services within particular Please add my name to the Pricis mailing list. E sectors and (2) to integrate cross-cutting thematic priorities-including poverty, participation, and Name gender-into a range of Bank activities. Address The Bank's ability to articulate a broad strategic vision is one of its strengths, and it has an obligation to City State provide consistent advice, taking account of the special Postal Code - Country circumstances of individual countries and Regions. Sector and thematic strategies are important vehicles Telephone for articulating corporate objectives and achieving international development targets, but must be 0 For large orders, please contact the World Bank Info- reconciled with the increasingly demand-driven Shop at tel: 1-2021458-5454, fax: 1-202/522-1500, or approach of Bank country programs. Agreement on through e-mail at books@worldbank.org the priorities identified in the Bank's strategy and those of the borrower-or other stakeholders-fosters the This and other OED publications are on the Interent, at implementation of Bank sector or thematic strategies. http://www.worldbank.orghtmloed But few Bank strategies and policies provide explicit guidance on what instruments to use and what posture 0 To order our books, please contact: to take in the country dialogue if governments do not The World Bank Fulfillment Center want to address an issue of global interest-for P.O. Box 960 example, gender equity or forest conservation. Herndon, VA 20172-0960, U.S.A. Customizing strategies and policy recommendations Tel: 1-703/661-1580 is an important determinant of local program success. Fax: 1-703/661-1501 Country-specific analytic work is crucial to support Internet: http://www.worldbank.org such adaptation, and the decline in funding for From the World Bank homepage, select publications. economic and sector work is of great concern. So is the e-mail: books@worldbank.org chronic weakness observed in monitoring and DISCLAIMER: OED Pricis are produced by the World Bank Opera- evaluation arrangements-essential for adapting and tions Evaluation Deparment, Partnerships and Knowledge Group updating strategies. While Bank-sponsored economic (OEDPK), Outreach and Dissemination Unit. The views in this and sector work is usually of good technical quality, paper are those of the Operations Evaluation staff and editors and greater emphasis needs to be given to local involvement and dissemination of findings. should ot its affiliate Diracioa- To organize and build on the findings of the sector and thematic evaluations, OED evaluators assessed ISSN 1564-6297 eight recently evaluated sector and thematic strategies Pr6cis 204 ARDE 2000: From Strategy to Results OED Publications Recent OED Precis 203 Striking the Right Balance: World Bank Forest Strategy along several dimensions. Most of the strategies and 202 Poverty Reduction in the 1990s: The World Bank Strategy policies were developed in the early- to mid-1990s. 201 The Drive to Partnership: Aid Coordination and the World Bank The quality and relevance of the strategic framework 200 Evaluating Gender and Development at the World Bank receive the highest average rating, and the quality of 199 Ghana: Building a Stronger Transportation System plans for implementation the lowest. 198 Agricultural Extension: The Kenya Experience With the introduction of Sector Strategy Papers in 197 Toward a Comprehensive Development Strategy 1996, the Bank began to standardize the process for 196 Evaluation and the Development Challenge developing sector strategies and to address some of the 195 Poverty Assessments: Maximizing Impacts weaknesses observed in the earlier policy development 194 Involuntary Resettlement: The Large Dam Experience process But most budgetary resources and 193 Partnership for Education in Jordan decisionmaking reside in country departments, and 192 Reforming Bolivia's Powver Sector 192 eforingBoliia'sPowr Setornone of the sector strategies evaluated included an 191 Bangladesh: Progress Through Partnership explicit, verifiable plan for implementation. Recently, 190 Transport in Indonesia 190 ranportin ndonsiasome Strategy Papers have given greater attention to 189 Health Care in Brazil: Addressing Complexity the business implications of sectoral and thematic 188 Health Care in Mali: Building on Community Involvement goals, but the continued imbalance in the matrix 187 Health Care in India: Learning from Experience structure renders results-oriented management of 186 Global Health: Meeting the Challenge sector and thematic strategies difficult to achieve. 185 Aid Coordination and Post-Conflict Reconstruction: The West OED's analysis suggests that much remains to be done Bank And Gaza Experience to deliver on the four "Ps" of successful sector 184 Land Administration and Rural Development: Two Cases from strategies: a clear Policy framework; an action plan to Thailand Thailandimprove on past Performance; a specific Program of Precis aussi disponible en franfais development assistance at the country and global @ http://www.worldbak.orghitml/ocd levels; and a definition of the Bank's role, taking Precis en espahol tanbien disponible account of Partners' activities. @4 http://www.worklbanik.org/htm1/ioed ("on fronting Dilemmas OED Study Series ARDE 2000 concludes that the Bank can further 2000 Annual Review of Development Effectiveness improve its development effectiveness by directly Agricultural Extension: The Kenya Experience confronting the dilemmas it faces: Bangladesh: Progress Through Partnership Bank strategies should explicitly acknowledge Developing Towns and Cities: Lessons from Brazil and the Philippines differences between client and Bank priorities, and Financing the Global Benefits of Forests: The Bank's GEF Portfolio and Bank instruments should be mobilized more the 1991 Forest Strategy strategically to build consensus at the country and Investing in Health: Development Effectiveness in the Health, Nutrition, global levels. and Population Sector The Bank should concentrate on the effective Nongovernmental Organizations in World Bank-Supported Projects implementation of its policies and strategies through Poverty Reduction in the 1990s: An Evaluation of Strategy and Performance The World Bank's Forest Strategy: Striking the Right Balance and political environments. The World Bank's Experience with Post-Conflict Reconstruction The Bank should clarify and strengthen its approach to poor-performing countries and to middle-income OED Multilingual Series countries with access to capital markets. Assessing Development Effectiveness: Evaluation in the World Bank and the International Finance Corporation Becoming More Selective Appr&iation de l'efficacite du developpement: The Bank must also adapt its organizational structure, L'dvaluation a la Banque mondiale et 3 la Socidte financiere business practices, and operational instruments to internationale reduce the enormous stress from which the organization Determinar la eficacia de las actividades de desarrollo: is suffering-stress that, unless remedied, could La evaluaci6n en el Banco Mundial y la Corporacidn Financiera undermine its development effectiveness: Internacional Internciona Bank strategies, programs, budgeting processes, and C6te d'Ivoire Revue de Paide de la Banque mondiale an pays staff incentives should be more explicitly linked to Philippines: From Crisis to Opportunity the achievement of results, consistent with the Filipinas: Crisis y oportunidades Bank's mission and conparative Rebuilding the Mozambique Economy: Assessment of a Development Partnership processes should be reformed to align corporate Reconstruir a Economia de MoCaTbique resources with corporate strategies. Approved Country Assistance Strategies, sector strategies, and 6 World Bank Operations Evaluation Department Figure 3. Sector Project Outcomes Vary by Country Income Percent 100 90 80 70 60 50 40 30 20 1 10 Low income Lower-middle income Upper-middle income global partnerships should be fully costed, and the costs monitor and track the quality and coherence of all linked to the annual budget process. Staff incentives Bank knowledge activities. should be linked to results and performance, not inputs To be responsibly selective through businesslike and processes. partnerships, the Bank must seriously and Maintaining the Bank's knowledge base is critical, but continuously assess its core competencies and the Bank should strike a more appropriate balance comparative advantage relative to international and between knowledge management and knowledge client country partners.This may entail letting go of creation, at both the country and global levels. It should activities other partners can carry out. o T'his pT(cis is based on 2000,Aaiiiiid Rcn2I'cir I 1 ,[;caiiy: 1yromR o Ib rp i , by Timothy bsnslk Johnston, with William Battaile. Available to Bank Executive Directors and staff from the Internal Documents Unit and from regional information service centers, and to the public from the World Bank InfoShop. www.worldbank.org/htnl/oed