WORLD BANK OPERATION9 EVALUATION DEPARTMENT 1999 ANNUAL 20180 REVIEW OF November 1999 DEVELOPMENT EFFECTIVENESS 4, OPERATIONS EVALUATION DEPARTMENT ENHANCING DEVELOPMWENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDEYCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. WORLD BANK OPERATIONS EVALUATION DEPARTMENT QM 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECT IVENESS Nagy Hanna with Ramgopal Agarwala William Battaile Deepa Chakrapani Ruchira Corcoran Federico Mini 1999 The World Bank http://www.worldbank.org/htm/oed Washington, D.C. Copyright © 2000 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First edition November 1999 The opinions expressed in this report do not necessarily represent the views of the World Bank or its member governments. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. The World Bank encourages dissemination of its work and wilt normally grant permission promptly. Permission to photocopy items for internal or personal use, for the internal or personal use of specific clients, or for educational classroom use is granted by the World Bank, provided that the appropriate fee is paid directly to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, U.S.A., telephone 978-750-8400, fax 978-750-4470. Please contact the Copyright Clearance Center before photocopying items. For permission to reprint individual articles or chapters, please fax your request with complete information to the Republication Department, Copyright Clearance Center, fax 978-750-4470. All other queries on rights and licenses should be addressed to the Office of the Publisher, World Bank, at the address above, or faxed to 202-522-2422. Photographer: Still Pictures (Father carrying daughter - Nepal) ISSN 1019-4363 ISBN 0-8213-4706-3 Printed on recycled paper. Contents vii Acknowledgments ix Foreword, Prefacio, Pr6face xi Executive Summary, Resumen, Rdsumd Analytique xxiii Abbreviations and Acronyms 1 1. Toward a Comprehensive Development Strategy 2 Long-Term, Holistic, Balanced 3 Ownership 3 Partnership and Development Cooperation 3 Results, Learning, Accountability 4 Challenges and Tensions 5 2. Challenges at the Project Level 5 Keeping up with Complexity 6 Varied Performance among Regions, Sectors, and Types of Lending 7 Institutional Development and Sustainability Flagging 8 Bank and Borrower Performance Must Improve 9 Project Performance and CDF Principles 11 3. Challenges in Sectoral and Thematic Programs 11 Short versus Long Term 12 Comprehensiveness versus Selectivity 13 Speed versus Broad-Based Ownership 13 Partnership versus Country Capacity and Transaction Costs 14 Accountability for Results versus Local Capacity 17 4. Challenges at the Country Level 17 Addressing Social Issues 18 Addressing Institutional Capacity and Structural Issues 19 A Long Way to Go 20 Managing the Tensions in the CDF 21 Country Cases 23 5. Promising Approaches 23 Learning Process, not Blueprints 24 Managing Complexity by Sequencing 24 Comprehensive Analysis and Selective Actions 25 Sectorwide Approaches 25 Adaptable Conditionality 26 Time to Build Consensus 26 Broadening Participation 27 Information for Accountability and Learning 27 Capacity Building to Manage for Results 28 From Aid Coordination to Development Partnership 28 Linking Global and Country Strategies 31 6. Implications for Development Effectiveness 32 Implications for the Bank iii 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Annexes 35 Annex 1: Project Performance Trends and Outstanding Projects 47 Annex 2: Challenges in Implementing the CDF Principles at the Sector Level 49 Annex 3: Development Performance in Selected Countries 55 Annex 4: Emphasizing People-Centered Development in a Holistic Framework 57 Annex 5: Ownership, Participation, and Conditionality 61 Annex 6: Managing for Results 65 Annex 7: Partnership and Strategic Selectivity 71 Annex 8: International Public Goods and Aid Effectiveness 75 Annex 9: Results-Based Management Glossary 77 Annex 10: Managing Development Effectiveness: An Overview from the CODE Chairperson 79 Endnotes 81 Bibliography 81 Background Papers 81 Selected Bibliography Figures 4 1.1 Interaction of CDF Principles 6 2.1 Demandingness, Complexity, Riskiness, and Outcome, by Exit Fiscal Year 18 4.1 The Bank's Contribution to Implementing CDF Principles 36 A1.1 Aggregate Project Performance Index 36 A1.2 Satisfactory Project Outcomes 37 A1.3 Outcome, Sustainability, and ID Impact, by Region and Exit Fiscal Year Group 37 AI.4 Satisfactory Operations by Sector and Exit Fiscal Year Group 38 A1.5 ID Impact 38 A1.6 Sustainability 39 A1.7 Borrower Performance 39 A1.8 Bank Performance 47 A2.1 Sector Scores on CDF 48 A2.2 Sector Scores on Individual CDF Principles 49 A3.1 Disappointing Performance in Reducing Poverty... 49 A3.2 ... And in Promoting Economic Growth 55 A4.1 Sustainable Livelihood Framework Boxes 8 2.1 Updated Determinants Analysis Puts Spotlight on Bank Performance 24 5.1 Challenges and Promising Approaches 58 A5.1 Conditionality Scenarios 61 A6.1 Barriers to Performance Orientation 62 A6.2 Pooling and Joint Reviews in Ghana's Health Sector Reinforce Accountability 63 A6.3 Malaysia: Sustainable Penang Initiative 67 A7.1 Learning from Business Alliances 68 A7.2 Capacity Building Assistance to Kazakhstan and Vietnam for Aid Coordination 69 A7.3 Advantages and Disadvantages of In-Country Aid Coordination Meetings 71 A8.1 Public Goods Primer 72 A8.2 Progress in Controlling River Blindness iv Contents Tables 2 1.1 Three Development Paradigms 41 A1.1 Outstanding and Poor Performers among Recently Evaluated Projects 42 A1.2 Outcome, Sustainability, ID Impact, and Aggregate by Sector, Network, Lending Type/Source, Region, and WDI Income Group for Exit Fiscal Years 1990-93, 1994-97, and 1998-99 (by projects) 44 A1.3 Outcome, Sustainability, ID Impact, and Aggregate by Sector, Network, Lending Type/Source, Region, and WDI Income Group for Exit Fiscal Years 1990-93, 1994-97, and 1998-99 (by real disbursements, FY96US$) 50 A3.1 Relevance of Sample of Evaluated Countries 51 A3.2 Country Performance: Per Capita Growth, Poverty, and Life Expectancy 52 A3.3 Country Performance: Savings 53 A3.4 Country Performance: Investment Efficiency 65 A7.1 Country Characteristics and Aid Coordination 66 A7.2 Donor Characteristics and the Aid Coordination Environment V ACKNOWLEDGMENTS his report was prepared by a team led by Shafik, William E. Stevenson, and Michael Walton. Nagy Hanna, including Ramgopal Agarwala, Pablo Guerrero also provided valuable comments. William Battaile, Ruchira Corcoran, Deepa The report benefited from comments of the DGO Chakrapani, and Federico Mini. The report benefited and the OED management team: Robert Picciotto, from the contributions of OED staff, as well as Wendy Jarvie, Elizabeth McAllister, Gregory Ingram, background papers from the Institute of Development Ruben Lamdany, and Osvaldo Feinstein. Bruce Ross- Studies, the Overseas Development Institute, and Ox- Larson and Meta de Coquereaumont provided editorial ford Policy Management. These contributors and par- assistance. Parveen Moses provided excellent support. ticipants are listed in the Bibliography. Special thanks The report was published by the Partnerships and to Keith Bezanson and Robert Chamber of IDS, to Knowledge group by the Dissemination and Outreach Simon Maxwell and Mick Foster of ODI, and to unit, directed by Elizabeth Campbell-Page (task team William Branson of Princeton University. leader), including Caroline McEuen (editor), Kathy Valuable guidance was provided by an Advisory Strauss and Lunn Lestina (graphics and layout), Bar- Committee including Robert M. Buckley, Carl Dahlman, bara Yale (editorial assistant), and Juicy Qureishi-Huq Christian Delvoie, Shantayanan Devarajan, Ishac (administrative assistance). Diwan, David R. Dollar, Marco Ferroni, Alan Harold Gelb, Ian Johnson, Geoffrey B. Lamb, Pierre Landell- Director-General, Operations Evaluation: Robert Picciotto Mills, Ashoka Mody, John Page, Anthony J. Pellegrini, Director, Operations Evaluation Department: Elizabeth McAllister Guillermo Perry, Guy Pfeffermann, Ian C. Porter, Manager, Corporate Evaluations and Methods: Wendyjarvie Wolfgang H. Reinicke, Neil D. Roger, Nemat Talaat Task Manager: Nagy Hanna vii Foreword FOREWORD PREFACIO PREFACE Despite the potential benefits A pesar de los posibles beneficios Malgré les avantages potentiels de of globalization and technological de la globalización y del cambio la mondialisation et du progrès change, world poverty has in- tecnolôgico, la pobreza mundial ha technologique, la pauvreté dans le creased and growth prospects have aumentado y las perspectivas de monde avance et les perspectives de dimmed for developing countries. crecimiento se han oscurecido en los croissance s'assombrissent pour les pays Inequality and instability have in- paises en desarrollo. La desigualdad y la en développement. Les inégalités se creased. Aid flows have stagnated. inestabilidad han aumentado. Los flujos creusent, l'instabilité s'accroît et le vo- Public dissatisfaction with the effi- de ayuda se han estancado. La opinión lume d'aide se tasse. Le manque cacy of development assistance has pública cada vez desconfia más de la d'efficacité de l'aide au développement grown. eficacia de la asistencia para el suscite un mécontentement grandissant. The Comprehensive Develop- desarrollo. Le cadre de développement intégré ment Framework (CDF), launched El Marco Integral de Desarrollo (CDI), lancé par le président de la by the President of the World Bank (MID), puesto en marcha por el Banque mondiale en janvier 1999, vise in January 1999, is a response to Presidente del Banco Mundial en enero à corriger cette évolution inquiétante. these alarming trends. It is far too de 1999, es la respuesta a esas S'il est encore bien trop tôt pour early to evaluate the initiative. In- tendencias alarmantes. Es todavia évaluer les résultats de cette initiative, stead, this year's Annual Review of demasiado pronto para evaluar esa la présente édition de l'Examen annuel Development Effectiveness exam- iniciativa. Pot ello, en el Examen anual de l'efficacité du développement dresse ines the lessons of development ex- de la eficacia en términos de desarrollo le bilan de l'année écoulée en perience through the lens of the CDF de este aio se analizan las ensefianzas appliquant les principes du CDI. principles. obtenidas de las actividades de Comme par le passé, nous As in past years, the Review desarrollo desde la perspectiva de los mettons en lumière le résultat des tracks the Bank's operational perfor- principios del MID. opérations de la Banque à partir des mance based on the findings of Como en afios anteriores, en el conclusions d'évaluations récemment recent OED evaluations. The over- Examen se consideran los resultados réalisées par l'OED. L'évolution de la all performance trends are positive, operacionales del Banco teniendo en performance globale est positive, alors despite a major increase in the cuenta las comprobaciones de las même que les opérations de la Banque relevance, demandingness, and evaluaciones recientes del sont d'une portée beaucoup plus large complexity of Bank operations-a Departamento de Evaluaciôn de et qu'elles sont devenues plus substantial achievement. But future Operaciones (DEO). Las tendencias exigeantes et plus complexes. C'est là gains will be more difficult to generales observadas son positivas, Io un résultat non négligeable. Les achieve, and quality at exit appears que representa un logro notable si se progrès seront toutefois désormais to have plateaued below the Strate- tiene en cuenta que al mismo tiempo plus difficiles à réaliser, comme le gic Compact target of 75 percent ha aumentado notablemente la montre l'évaluation de la qualité à satisfactory. A growing gap between pertinencia, dificultad y complejidad l'achèvement des projets, qui semble operational complexity and local de las operaciones del Banco. Pero los plafonner au-dessous du niveau fixé capacity confirms the need for the progresos futuros serán más dificiles par le Pacte stratégique (75 % enhanced capacity building efforts de conseguir y la calidad final parece d'opérations jugées satisfaisantes). Le highlighted in Mr. Wolfensohn's haberse estabilizado pot debajo del fossé qui se creuse entre la complexité Annual Meetings speech. objetivo del Pacto Estratégico, es decir, des opérations et les moyens locaux ix 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECT[VENESS Implementing the CDF un 75% de proyectos montre bien qu'il est nécessaire principles, both within and - calificados como satisfactorios. de redoubler d'efforts pour outside the 13 pilot countries, La diferencia creciente entre la renforcer les capacités, ainsi que raises tough challenges for the complejidad operacional y la l'a rappelé M. Wolfensohn dans Bank and its partners. The capacidad local confirma la son allocution devant Review draws on extensive necesidad de impulsar las l'Assemblée annuelle. evaluation evidence, litera- iniciativas de fortalecimiento de L'application des principes ture reviews and research findings, la capacidad destacada por el Sr. du CDI dans le cadre des opérations country assistance evaluations, Wolfensohn en su discurso con pilotes, et au-delà de celles-ci, est une background papers, and workshops ocasión de las Reuniones Anuales. tâche ardue pour la Banque et ses to identify the tensions and dilem- La aplicación de los principios del partenaires. Pour mettre en évidence mas likely to be involved in CDF MID, dentro y fuera de las iniciativas les tensions et les dilemmes que la mise implementation. It identifies prom- experimentales, plantea dificiles retos en œuvre du CDI risque de faire ising approaches to meet the para el Banco Mundial y sus surgir, le présent examen s'appuie sur challenges implicit in CDF imple- asociados. En el Examen se evalúan de nombreux faits observés dans les mentation. Finally, it draws broad atentamente los testimonios évaluations, sur des études implications for the Bank's future disponibles, se examinan las documentaires et des travaux de re- development effectiveness. publicaciones y los resultados de la cherche, sur des évaluations de l'aide investigación, las evaluaciones de la apportée aux pays, sur des documents asistencia a los paises, los documentos de référence et sur des ateliers. Il de antecedentes y los seminarios présente des formules prometteuses realizados con el fin de descubrir las pour surmonter les difficultés tensiones y dilemas que acompafiarán, inhérentes à cet exercice. Enfin, il tire probablemente, a la aplicaciôn del les grandes conséquences de la situa- MID. Se localizan planteamientos tion du point de vue de l'efficacité que prometedores para responder a los les opérations de la Banque pourront desafios implicitos en la aplicación del avoir sur le développement des pays. MID. Finalmente, se extraen algunas conclusiones generales sobre la eficacia futura del Banco en términos de desarrollo. Robert Picciotto Director-General, Operations Evaluation X Executive Summary EXECUTIVE RESUMEN RESUMÉ SUMMARY ANALYTIQUE Changes in the global envi- Los cambios registrados en el Les changements survenus sur la ronment over the past decade have entorno mundial durante el pasado scène internationale au cours des dix profoundly altered the context of decenio han modificado profundamente dernières années ont profondément Bank operations. Many developing el contexto de las actividades del Banco modifié le contexte dans lequel la countries have undertaken major Mundial. Muchos paises en desarrollo Banque opère. De nombreux pays ont political and economic reforms, han emprendido grandes reformas entrepris d'importantes réformes qui se opening their economies and politi- políticas y econômicas, que han sont traduites par une ouverture sur le cal processes. At the same time, representado una apertura de sus plan économique et politique. Au même moment, la mondialisation ouvrait des perspectives de gains mais faisait aussi naître des risques d'instabilité et creusait _ f les inégalités sur le plan tant national qu'international. Malgré la montée en - flèche de la demande d'aide publique, les NWI luiflux d'assistance ont fléchi. En dépit d'une amélioration des indicateurs -OI¯ sociaux partout dans le monde, la pro- gression du développement a été -.- décevante, les statistiques sur la pauvreté se sont aggravées et la plupart des pays à faible revenu sont restés très dépendants de l'aide. Pendant la période - comprise entre 1985 et 1995, le revenu par habitant a en fait baissé, de 1,4 % par an dans les pays à faible revenu (sans la Chine, ni l'Inde) et de 0,7 % dans les pays à revenu intermédiaire. globalization has brought new op- economias y de los procesos politicos. Dans ce contexte, le manque portunities for gain-and new risks AI mismo tiempo, la globalizaciôn ha d'efficacité du développement est devenu of instability and rising inequality traido consigo nuevas oportunidades, y un sujet de préoccupation croissante within and among states. Aid flows nuevos riesgos de inestabilidad y tant pour les autorités des pays have stagnated, even as new de- aumento de la desigualdad dentro de développés et des pays en mands for official resources have cada pais y entre unas naciones y otras. développement que pour le public en escalated. Despite worldwide im- Los flujos de la ayuda se han estancado, général. Face à cette situation, on a provements in social indicators, aun cuando se han multiplicado las recherché différents moyens d'améliorer development progress has been dis- nuevas demandas de recursos oficiales. la façon dont la coopération pour le appointing, poverty trends have A pesar de los progresos mundiales de développement est abordée. Le Cadre de worsened, and most low-income los indicadores sociales, el avance del développement intégré (CDI) reprend les countries remain heavily dependent desarrollo ha sido decepcionante, las grands thèmes autour desquels on aid. Per capita incomes actually tendencias de la pobreza se han s'organise le consensus de la fell during 1985-95-by 1.4 percent agravado y la mayoria de los paises de communauté du développement. Le a year in low-income countries (ex- ingreso más bajo continúan cadre stratégique de lutte contre la xi 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS cluding China and India) and dependiendo fuertemente de la pauvreté, qui sera conjointement by 0.7 percent a year in - ayuda. Los ingresos per càpita élaboré par la Banque mondiale, middle-income countries. retrocedieron de hecho en 1985- le Fonds monétaire international Inevitably, governments of 95, un 1,4% al aio en los paises et les pays emprunteurs, est une developed and developing de ingreso bajo (con exclusión de intervention étroitement liée à countries, as well as the public China y la India) y un 0,7% al cette question. at large, have become more año en los paises de ingreso Depuis son apparition, il y concerned about development effec- mediano. a un an, le CDI suscite un intérêt tiveness. This bas led to a wide- Inevitablemente, los gobiernos de considérable. Les éléments sur lesquels ranging search for improved los paises desarrollados y en il se fonde ne sont pas nouveaux. En approaches to development coopera- desarrollo, asi como la sociedad en revanche, la façon dont ils s'articulent tion. The Comprehensive Develop- general, se muestran ahora màs les uns avec les autres pour servir de fil ment Framework (CDF) encapsulates preocupados por la eficacia en conducteur à l'aide au développement major themes around which a consen- términos de desarrollo. Ello ha dado est une donnée nouvelle. sus of the development community is lugar a una bùsqueda generalizada de Premièrement, les problèmes auxquels crystallizing. The Poverty Reduction nuevos enfoques que permitan se heurte le développement étant de Strategy Papers-to be prepared orientar mejor la cooperación en mate- nature structurelle et sociale, les jointly by the World Bank, the Inter- ria de desarrollo. El MID compendia mesures de stabilisation économique national Monetary Fund, and bor- los grandes temas en torno a los et le recadrage des politiques ne rowing countries-are a closely cuales se està cristalizando el consenso suffiront pas pour les surmonter; related response. de todos los interesados en el l'approche doit être globale et Since its unveiling a year ago, desarrollo. En ese mismo contexto se multisectorielle. Deuxièmement, ni les the CDF has evoked considerable estàn preparando los documentos de réformes, ni le développement interest. The basic elements of the estrategia para la reducciôn de la institutionnel ne peuvent être importés CDF are not new. What is new is their pobreza, obra conjunta del Banco ou imposés : la pérennité des réformes joint articulation as a guide to devel- Mundial, el Fondo Monetario et des investissements passe par une opment assistance. First, develop- Internacional y los paises prestatarios. adhésion sans réserve au niveau local. ment constraints are structural and Desde que se propuso, hace un Troisièmement, le développement ne social and cannot be overcome afio, el MID ha suscitado considerable peut réussir que s'il fait appel à des through economic stabilization and interés. Sus elementos básicos no son partenariats entre les pouvoirs publics, policy adjustment alone-they re- nuevos. Lo novedoso es su les populations locales, le secteur quire a holistic, broad-based ap- articulación conjunta como guía de la privé, la société civile et les organismes proach. Second, policy reform and asistencia para el desarrollo. Primero, d'aide au développement. Enfin, les ac- institutional development cannot be los obstàculos al desarrollo son tions de développement doivent avoir imported or imposed: without domes- estructurales y sociales y no pueden le souci du résultat. Le CDI est lancé à tic ownership, reforms and invest- superarse ùnicamente con politicas de titre pilote dans 13 pays et il n'a pas ments are not sustainable. Third, ajuste y de estabilizaciôn econômica, encore revêtu sa forme définitive. Bien successful development requires part- sino que requieren un planteamiento qu'il soit encore trop tôt pour dresser nerships among government, local global, de base amplia. Segundo, la un premier bilan, la présente édition communities, the private sector, civil reforma de las politicas y el desarrollo de l'Examen annuel de l'efficacité du society, and development agencies. institucional no pueden importarse ni développement s'appuie sur les Fourth, development activities must imponerse: si en los propios paises no principes du CDI pour apprécier le be guided and judged by results. CDF hay una identificación con las nuevas chemin parcouru et tirer des pilot activity is beginning in 13 coun- propuestas, ni las reformas ni las enseignements qui faciliteront tries, and its dimensions are still inversiones seràn sostenibles. Tercero, l'application des dispositions prévues. evolving. Although it is too early to un desarrollo eficaz requiere el assess the CDF as such, this year's establecimiento de asociaciones entre Évolution générale des projets Annual Review of Development Ef- los gobiernos, las comunidades lo- Les faits observés dans les évaluations fectiveness examines development cales, el sector privado, la sociedad montrent que le résultat des projets est xii Executive Summary experience through the lens of civil y los organismos de étroitement lié aux politiques CDF principles, with a view to , desarrollo. Cuarto, las nationales et aux facteurs drawing lessons for CDF imple- actividades de desarrollo deben institutionnels, aspects au cœur mentation. orientarse y juzgarse teniendo du CDI. Les projets étant l'un des en cuenta los resultados. La grands moyens d'assistance de la Project Trends aplicación experimental del Banque, la performance à ce Evaluation evidence confirms MID ha comenzado en 13 niveau continuera à influer de the importance of country policy países, y sus dimensiones están adn en façon déterminante sur l'efficacité du and institutional factors addressed evolución. Aunque es todavia développement. Sur les dix années by the CDF to project performance. demasiado pronto para evaluar el écoulées, le résultat des évaluations de Projects are important vehicles of MID en cuanto tal, en el Examen projet est en amélioration ce qui, compte Bank assistance, and most perfor- anual de la eficacia en términos de tenu de l'exigence et de la complexité mance at the project level will desarrollo de este año se examina la toujours croissantes des opérations, continue to be a critical element of realidad del desarrollo desde la constitue un bilan remarquable. Les development effectiveness. Project perspectiva de los principios del MID, prêts à l'ajustement, en particulier, ont evaluations have shown improve- con el fin de extraer ensefñanzas para donné lieu, pour une assez large part, à ments over the past decade, along su aplicación. des résultats jugés satisfaisants. On a with a considerable increase in toutefois constaté récemment un certain project demandingness and com- Tendencias de los proyectos tassement: 72 % des projets évalués qui plexity--a considerable achieve- Los datos relativos a las evaluaciones sont sortis du portefeuille pendant les ment. Adjustment lending, in confirman la importancia de los factores exercices 98 et 99 ont enregistré des particular, has delivered a rela- normativos e institucionales de los résultats satisfaisants, mais moins de la tively high share of satisfactory paises contemplados en el MID para moitié sont susceptibles de confirmer outcomes. But there has been a determinar los resultados de los durablement cette performance. recent plateauing in performance: proyectos. Los proyectos son vehiculos Les projets étant devenus plus 72 percent of evaluated projects importantes de asistencia del Banco, por complexes, la performance des exiting in FY98-99 show satisfac- Io que la multiplicación de esos emprunteurs au niveau de l'exécution tory outcomes, while the share of resultados continuarà siendo un des opérations est restée stationnaire, exiting projects likely to sustain elemento critico de la eficacia del ce qui contribue à la stagnation des their benefits remains below half. desarrollo. Las evaluaciones de los résultats. Les analyses montrent que As the complexity of exiting proyectos han demostrado los cette performance est un élément qui projects has risen, borrower imple- progresos conseguidos en el pasado vient immédiatement après la supervi- mentation performance has been decenio, a pesar de que ha aumentado sion de la Banque dans les facteurs qui stagnant, which has contributed to también notablemente la dificultad y déterminent le succès d'un projet. Le the plateauing of performance. complejidad de los proyectos, bo que creusement de l'écart entre la Analysis shows borrower imple- representa un importante logro. En par- complexité des opérations et les mentation performance to be a key ticular, el financiamiento para fines de capacités locales souligne l'importance determinant of project success, sec- ajuste ha conseguido una proporción de ces relais et la nécessité d'une ond only to Bank supervision. The relativamente elevada de calificaciones meilleure prise en compte des widening gap between complexity satisfactorias. Pero se ha observado contraintes institutionnelles. Malgré and local capacity highlights the también una estabilización de los une nette amélioration au cours des importance of these inputs, as well resultados: el 72% de los proyectos dernières années, seuls 40 % des as the need for intensified attention evaluados finalizados en los ejercicios de projets sortant du portefeuille ont to institutional constraints. Despite 1998-99 presentan resultados actuellement un impact important sur substantial improvements in recent satisfactorios, mientras que la parte de le développement institutionnel. years, only 40 percent of exiting los proyectos concluidos con projects currently have substantial probabilidades de continuar Une tâche ardue institutional development impact. produciendo beneficios se mantiene por Le CDF repose sur des principes qui debajo de la mitad. tirent parti des synergies. Mais celles-ci xiii 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Tough Challenges Ahead A medida que ha génèrent aussi des antagonismes The CDF principles are syner- aumentado la complejidad de qu'il faudra gérer prudemment. gistic. But their combination los proyectos concluidos, el Le court terme et le long carries with it critical tensions desempeño de los prestatarios terme. La transformation that will require careful man- en la ejecución de los proyectos sociale et le développement agement. se ha estancado, Io que ha institutionnel supposent un ef- Short term versus long contribuido a la estabilización fort de longue haleine. term. Social transformation and in- de los resultados. Los análisis Pourtant, les mécanismes d'incitation stitutional development require efectuados revelan que los resultados et le mode de fonctionnement qui long-term and sustained efforts. Yet de la ejecución de los proyectos son un prévalent dans les pays en the incentives and processes in de- determinante fundamental del éxito de développement et dans les organismes veloping countries and aid agencies éstos, cuya importancia sólo es infe- d'aide ont tendance à privilégier le have tended to focus on the short rior a la supervisión del Banco. La court terme. La planification à l'aide term. Blueprint planning and the diferencia creciente entre la de modèles et l'approche-projet project approach embody this short- complejidad y la capacidad local pone s'inscrivent dans cette perspective. Les term focus. The absence of a long- de manifiesto la importancia de esos dimensions structurelles des réformes term perspective has been especially aportes, asi como la necesidad de (privatisation, réforme de la fonction detrimental in dealing with the intensa atención a los obstáculos publique et déréglementation du structural dimensions of reform, institucionales. A pesar de las secteur financier, par exemple) ont such as privatization, civil service considerables mejoras de los últimos particulièrement souffert de cette ab- reform, and deregulation of the afños, sólo el 40% de los proyectos sence d'optique à long terme. financial sector. concluidos tienen actualmente Exhaustivité et sélectivité. Du fait Comprehensiveness versus selec- importantes repercusiones en el de l'expansion du programme de tra- tivity. The Bank's expanding agenda desarrollo institucional. vail de la Banque, les projets et les has increased the complexity, de- programmes se sont fait plus com- mandingness, and risk of projects and Arduos desatios para el futuro plexes, risqués et exigeants. Face à programs. In the face of these pres- Los principios en que se fundamenta el cette situation difficile, on ne procède sures, selectivity has not always been MID son sinérgicos, pero la pas toujours de façon sélective. exercised. Yet evaluation findings combinación de éstos conlleva tensiones Pourtant, les évaluations montrent highlight the risk of excessive com- criticas que se deberàn manejar con qu'une complexité excessive comporte plexity, especially when it is built into sumo cuidado. des risques, surtout lorsqu'il n'a pas the design without regard to domestic Corto plazo versus largo plazo. été tenu compte des capacités ou de la capacity or ownership. Projects that La transformación social y el volonté d'appropriation au niveau lo- involve several implementing agen- desarrollo institucional exigen cal. Les projets faisant intervenir cies or cross-sectoral assets tend to esfuerzos sostenidos y a largo plazo. plusieurs organismes d'exécution ou perform poorly. Sin embargo, los incentivos y procesos des actifs dans plusieurs secteurs Speed versus broad-based own- en los paises en desarrollo y en los obtiennent généralement des résultats ership. Sustainable reform requires organismos de ayuda se han centrado médiocres. broad-based ownership. Yet part- en el corto plazo, tendencia que se ha Rapidité et large adhésion. Pour ners may disagree about the distinc- caracterizado por una planificacién qu'une réforme soit durable, il faut tive roles of the state, the private detallada y un enfoque en los que l'adhésion à l'action menée soit sector, and civil society. Achieving proyectos. La falta de una perspectiva totale. Pourtant, les partenaires ne agreement among partners may de- a largo plazo ha sido especialmente sont pas toujours d'accord sur les lay reform. Governance reforms perjudicial para abordar las rôles respectifs de l'État, du secteur may be required to institutionalize dimensiones estructurales de la privé et de la société civile. La réforme participation-a long-term process. reforma, como la privatizacién, la pourra prendre du retard du fait du Ownership versus conditionality. reforma de la administracién péblica y temps nécessaire aux différents acteurs How should the apparent tension la desreglamentacién del sector pour parvenir à un accord. Il faudra between country ownership and do- financiero. parfois réformer le mode de gestion xiv Executive Summary nor interest in performance (of- Integridad versus des affaires publiques pour ten enforced through condition- - selectividad. El programa de institutionnaliser la participa- ality) be resolved? Conditional- actividades del Banco, que va en tion, une entreprise de longue ity is widely viewed as an aumento, presenta una mayor haleine. attempt to impose reforms and complejidad, dificultades y Adhésion au projet et blueprint solutions on clients. riesgos de los proyectos y conditionnalité. Comment Yet ownership is essential to programas. Ante tales l'apparente contradiction entre sustainable reform. New forms of presiones, no siempre se ha l'adhésion du pays à l'action menée et conditionality-reflecting the CDF aplicado el criterio de la selectividad. les résultats demandés par les bailleurs principles-should be instituted. Con todo, en las conclusiones de las de fonds (souvent au moyen de la Partnership versus country ca- evaluaciones practicadas se destaca el conditionnalité) peut-elle être résolue ? pacity and ownership. Countries in riesgo de una excesiva complejidad, La conditionnalité est très souvent need of partnership often lack the sobre todo cuando ésta se incluye en el perçue comme une tentative non capacity to coordinate aid. Govern- diseño sin tener en cuenta la capacidad déguisée d'imposer des réformes ou un ments, civil society, the private sector, local o la identificación del pais con las modèle aux pays clients. Et pourtant, and external donors may have differ- propuestas. Los proyectos que il ne peut y avoir de réformes durables ent agendas, requiring a gradual entraian la participación de varios si le pays ne s'identifie pas à l'action approach to consensus building. The organismos de ejecuciôn o activos menée. Il conviendrait de repenser la larger and more diverse the partners, intersectoriales tienden a arrojar conditionnalité, sous une forme plus the higher the transaction costs and resultados deficientes. respectueuse des principes du CDI. the greater the difficulty of combining Velocidad versus amplia Partenariat, capacité locale et ownership and partnership. Where identificación. Para que las reformas adhésion du pays. Les pays devant country commitment is lacking, co- sean sostenibles es preciso que exista faire appel à des partenaires multiples herent views among external partners una amplia identificación con ellas. No sont rarement à même de coordonner may be perceived as "ganging up." obstante, los asociados pueden l'aide reçue. Les objectifs des Country-led partnership versus discrepar con respecto a las diferentes gouvernements, de la société civile, du donor accountability. All develop- funciones del Estado, el sector privado secteur privé et des bailleurs de fonds ment assistance agencies are account- y la sociedad civil. El logro de un peuvent en effet être différents, ce qui able to their authorities, so placing an consenso entre los asociados puede obligera les partenaires à rechercher unresponsive government "in the demorar las reformas. Puede ser progressivement le consensus driver's seat" involves risks. The necesario introducir reformas de la nécessaire. Plus les partenaires sont challenge is to find common ground función de gobierno para nombreux et différents, plus les coûts and achieve results over the long term institucionalizar la participación, un de transaction sont importants et plus through patient nurturing of reform proceso a màs largo plazo. il est difficile de concilier adhésion et and capacity building. Identificación versus partenariat. En l'absence, de volonté Results-orientation versus local condicionalidad. ¿Cómo debe politique de la part du pays capacity. The performance of the resolverse la aparente tensión que bénéficiaire, une communauté de vues Bank and most developing countries existe entre la identificación del pais y des bailleurs de fonds peut être perçue in monitoring and evaluation bas el interés de los donantes en el comme une « coalition ». been weak. Yet the international desempefño (el que a menudo se exige a Partenariat piloté par le pays et development goals, the recent atten- través de la condicionalidad)? Existe la responsabilité des bailleurs de fonds tion to governance, and the move to impresiôn generalizada de que la envers leur autorité de tutelle. Tous les programmatic lending reinforce the condicionalidad es un burdo intento organismes d'aide au développement need for results-based management de imponer reformas y soluciones ayant à répondre de leur gestion and stronger evaluation capacities planificadas a los clientes. Sin em- devant leur autorité de tutelle, « passer and local accountability systems. bargo, la identificación con las les rênes » à un gouvernement peu Country-based programs versus reformas es fundamental para que motivé comporte des risques. Il s'agira global public goods. Development éstas sean duraderas. Deben instituirse de trouver un terrain d'entente et assistance efforts and approaches nuevas formas de condicionalidad, d'obtenir des résultats à long terme en xv 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS are focused at the project and que reflejen los principios del renforçant patiemment les country levels. With global- - MID. capacités et en laissant mûrir les ization, development prob- Asociaciones versus réformes. lems require multilateral capacidad e identificaciôn de los Obligation de résultat et solutions and stronger links paises. Los paises que necesitan capacités locales. Le travail de between national strategies establecer asociaciones a suivi et d'évaluation de la and international policies. menudo carecen de la capacidad Banque et de la plupart des pays para coordinar la ayuda. Los en développement laisse à désirer. Et Promising Approaches gobiernos, la sociedad civil, el sector pourtant l'existence d'objectifs In order to resolve the above dilem- privado y los donantes externos internationaux de développement, mas and implementation chal- pueden tener distintos programas, bo l'accent récemment mis sur le mode lenges, it will be necessary to em- que hace necesario adoptar un d'administration des affaires ploy creativity and innovation and planteamiento gradual para la publiques et le passage à des prêts to heed the lessons of experience. formaciôn de consenso. Mientras más programmatiques rendent d'autant Several promising practices emerge numerosos y diversos sean los plus nécessaires des mesures visant à from this review. All are predicated asociados, mayores serán los costos axer la gestion sur les résultats, à on country commitment to poverty de transacción y las dificultades para améliorer les capacités d'évaluation et reduction and sustainable growth, combinar la identificación y las à renforcer les systèmes de contrôle. and a few are quick to take to scale. asociaciones. Cuando en un pais falta Programmes au niveau des pays • An adaptive learning process el sentido de compromiso, la et biens publics de caractère mondial. starts small and combines top- coherencia de opiniones entre los Les actions et stratégies d'aide au down direction with bottom-up asociados externos puede percibirse développement sont conçues au niveau experimentation and learning. como una "alianza". des projets et des pays. Du fait de la This shifts the emphasis from Asociación de esfuerzos mondialisation, la solution aux up-front analysis and detailed encabezada por el pais versus problèmes de développement passe design toward developing flex- responsabilidad de los donantes. par l'adoption de solutions ible solutions, building local Todos los organismos de asistencia multilatérales et par un renforcement capacity, and relying on social para el desarrollo son responsables des liens existant entre les stratégies processes and monitoring sys- ante sus autoridades, de manera que nationales et les politiques tems for adaptation and learn- poner a un gobierno que no reacciona internationales. ing during implementation. en el " asiento del conductor" plantea • One way out of the excessive ciertos riesgos. El desafío consiste en Des possibilités intéressantes complexity of projects and encontrar un terreno común y lograr Pour sortir des dilemmes mentionnés programs is to sequence inter- resultados a largo plazo fomentando plus haut et résoudre les problèmes de ventions within a long-term con paciencia la reforma y el mise en œuvre qui en découlent, il strategy that builds on past desarrollo de la capacidad. faudra faire preuve de créativité et learning. Such sequencing can Orientación a los resultados ver- d'innovation et savoir tirer les leçons de start by piloting comprehensive sus capacidad a nivel local. La l'action menée. Plusieurs possibilités approaches at the local level, actuación del Banco y de la mayoria de intéressantes se dégagent du présent then scaling them up as part of los paises en desarrollo en Io que examen. Toutes reposent sur la volonté a long-term process of capacity respecta a las actividades de du pays de faire reculer la pauvreté et de building and decentralization. seguimiento y evaluaciôn ha sido promouvoir une croissante durable, • Tensions between comprehen- deficiente. Con todo, las metas mais rares sont celles qui peuvent être siveness and selectivity can be internacionales de desarrollo, la rapidement portées à une plus grande eased by combining compre- reciente atención a los asuntos échelle. hensive analysis with strategic relativos a la función de gobierno y el • Tout processus d'apprentissage actions. Participatory ap- avance hacia el financiamiento para évolutif doit démarrer à petite proaches can enable poor programas refuerzan la necesidad de échelle et allier une démarche people to analyze their realities una gestión basada en los resultados, venant d'en haut à une acquisi- xvi Executive Summary in a comprehensive way, and asi como de una mayor tion des connaissances et à une then express their priorities capacidad de evaluación y de expérimentation partant du and choose interventions se- sistemas para asegurar la bas. Cela permettra de moins lectively. An example of a responsabilidad a nivel local. mettre l'accent sur les analyses holistic yet selective approach Programas basados en los initiales et les plans détaillés et is the sustainable livelihoods paises versus bienes públicos de privilégier la recherche de so- approach of the U.K. Depart- mundiales. Las iniciativas y lutions souples, le renforcement ment for International Develop- enfoques relativos a la asistencia para des capacités locales et le recours ment, CARE, and the United el desarrollo se centran al nivel de los aux systèmes de suivi et aux Nations Development Program proyectos y de los paises. Con el mécanismes encadrant les (UNDP). proceso de globalizaci6n, los conduites au sein des collectivités • Sectorwide and programmatic problemas del desarrollo exigen pour favoriser l'adaptation et approaches can help reduce the soluciones multilaterales y el l'apprentissage pendant rigidities and burdens associ- establecimiento de vinculos más l'exécution des activités. ated with the proliferation of estrechos entre las estrategias • L'un des moyens de remédier à la donor-financed projects. Learn- nacionales y las politicas complexité des projets et ing and innovation pilots fol- internacionales. programmes est d'échelonner les lowed by adaptable program interventions en appliquant une loans can support priority Planteamientos prometedores stratégie qui intègre les acquis sur goals, particularly in sectors Para resolver los dilemas y retos en ma- le long terme. Pour échelonner les where flexibility is critical. But teria de aplicaciôn sefialados interventions, on peut commencer moving prematurely from anteriormente, serà necesario recurrir a par expérimenter des méthodes projects to full-scale sector ap- la creatividad y la innovación, y poner globales au niveau local, pour les proaches is risky. Capacity atención a las ensefñanzas de la appliquer ensuite à plus grande building must accompany the experiencia. Del presente examen surgen échelle dans le cadre d'un proces- process of scaling up. More- varias pràcticas prometedoras. Todas se sus de renforcement des capacités over, sectorwide approaches fundamentan en la determinación de los et de décentralisation qui s'inscrit should be tailored to countries paises para reducir la pobreza y lograr el dans la durée. and sectors. desarrollo sostenible, y unas pocas se e En complétant l'analyse globale • Conditionality should be recon- pueden aplicar de inmediato en mayor d'actions stratégiques, on peut ciled with country ownership. escala. atténuer l'antinomie existant entre Empirical analysis of past • Un proceso de aprendizaje exhaustivité et sélectivité. De country reforms over a long adaptativo comienza en forma même, en associant les pauvres à horizon and of the recent modesta y en él se combina una l'action menée, on peut leur higher-impact adjustment lend- dirección de arriba hacia abajo permettre d'analyser la réalité de ing in Africa confirms that con- con experimentación y façon globale, puis de fixer leurs ditionality is best managed as a aprendizaje de abajo hacia arriba. priorités et de sélectionner les in- flexible, noncoercive policy Con esto se cambia de énfasis al terventions en conséquence. La compact adapted to different pasar de un anàlisis inicial y un façon dont le Département pour stages of reform. disefio detallado a la formulación le développement international • Broadening ownership across de soluciones flexibles, la creación (Royaume-Uni), CARE et le many stakeholders with diverse de capacidad a nivel local y el PNUD abordent la question des interests and capabilities requires apoyo en los procesos sociales y moyens d'existence viables est un tirne and early mobilization, par- en sistemas de seguimiento que bon exemple d'une approche à la ticularly of communities and the permitan la adaptaciôn y el fois globale et sélective. poor. It implies the systematic use aprendizaje durante el proceso de • Les approches par secteur et par of participatory processes; in- aplicación. programme peuvent aider à forming and giving voice to the • Una manera de evitar proyectos y réduire les rigidités et les weak partrers, particularly programas excesivamente contraintes liées à la prolifération xvii 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS women and marginalized complejos es establecer un de projets financés par des populations; and creating an - orden de sucesión de las bailleurs de fonds. Des enabling environment for scal- intervenciones dentro de una opérations pilotes pour le ing up and institutionalizing - estrategia a largo plazo que se développement des participation. fundamente en las lecciones connaissances et l'innovation, • Partnership requires two extraidas del pasado. Tal complétées de prêts à des complementary roles for the secuenciamiento puede programmes évolutifs, peuvent Bank: supporting country lead- comenzar con la aplicación ex- aider à la réalisation d'objectifs ership and building the capac- perimental de planteamientos prioritaires, surtout là où la lati- ity to exercise that leadership, globales al nivel local, que luego se tude d'action joue un rôle and engaging its development pueden ir incrementando como déterminant. Il est toutefois risqué assistance partners to promote parte de un proceso a largo plazo de transposer prématurément au selectivity, coordinate interven- de desarrollo de capacidades y niveau de l'ensemble d'un secteur tions, and harmonize proce- descentralizacion. les activités réalisées à l'échelon dures. Partnerships may imply • Las tensiones entre la integridad y d'un projet. Le passage à l'échelle up-front investments and must la selectividad pueden suavizarse supérieure doit s'accompagner be effectively monitored and mediante la combinación de un d'un travail de renforcement des managed if transaction costs to análisis cabal y acciones capacités. Uapproche devra en donors and countries are to estratégicas. Los enfoques outre être adaptée au pays et au decline over time. participatorios pueden permitir secteur considérés. que la gente pobre analice su • Il faut concilier conditionnalité et Evaluating Development realidad en forma integral, y luego adhésion du pays à l'action Effectiveness expresar sus prioridades y menée. L'analyse sur une longue The changing global economy and escoger selectivamente las période des réformes entreprises the CDF suggest several principles intervenciones. Un ejemplo de un dans les pays et l'examen des for evaluating development effec- planteamiento integral pero récentes opérations de prêt à tiveness. selectivo es el enfoque sobre l'ajustement, qui ont eu des Evaluation should focus on re- medios de vida sostenibles del conséquences importantes en sults, and this depends on accurate Departamento de Desarrollo Afrique, montrent que la tracking of progress toward devel- Internacional del Reino Unido, conditionnalité gagne à prendre la opment goals, with a clear focus on CARE y el PNUD. forme d'un programme souple, poverty reduction and growth. • Los enfoques sectoriales y non imposé, qui évolue au fil de Tracking development outcomes programàticos pueden ayudar a son application. should comply with the comprehen- reducir las rigideces y la carga • Pour amener un grand nombre de sive development agenda agreed by asociadas a la proliferación de parties prenantes sans the government and its partners. proyectos financiados por communauté de moyens ni As the focus of the development donantes. Los préstamos d'intérêts à souscrire pleinement à effort moves from projects to the experimentales para el aprendizaje l'action menée, il faut compter avec higher plane of country programs, y la innovación seguidos de le temps et mobiliser rapidement les so must the evaluation process. Re- préstamos adaptables para intéressés, tout particulièrement la sources and skills should be invested programas pueden apoyar la population et les pauvres. À cette in developing appropriate indica- consecución de metas prioritarias, fin, il faudra systématiquement faire tors and information systems. The sobre todo en sectores en los que appel à la participation, s'adresser current preoccupation with project la flexibilidad es fundamental. et donner la parole aux partenaires performance and evaluation should Pero pasar prematuramente de les plus faibles, notamment aux be complemented by a sectoral and los proyectos a enfoques femmes et aux exclus, et créer les countrywide focus. Public sector re- sectoriales integrales plantea conditions qui permettront form needs to include building the riesgos. El proceso gradual debe ir d'élargir et d'institutionnaliser cette evaluation capacity of countries. acompaiado de la creación de participation. xviii Executive Summary Development effective- capacidad. Además, los • Pour promouvoir le ness should be evaluated in enfoques sectoriales deberian partenariat, l'action de la terms of shared objectives, adaptarse a las circunstancias Banque doit être double. Elle joint responsibility for out- de los paises y los sectores. doit aider les pays à jouer un cornes, reciprocal obligations e La condicionalidad debería rôle moteur en contribuant au to achieve results, and distinct conciliarse con la identificación renforcement de leurs capacités accountability for perfor- de los paises. El anàlisis et elle doit inciter ses partenaires mance. In particular, donors empírico de las reformas bailleurs de fonds à encourager la and governments should team up to introducidas en el pasado durante sélectivité, à coordonner les inter- involve civil society and the private un prolongado horizonte tempo- ventions et à harmoniser les sector in monitoring and evaluation, ral y el financiamiento para fines procédures. Les partenariats and to help all stakeholders acquire de ajuste otorgado recientemente impliquent parfois un the needed skills and attitudes. Par- a África confirman que la mejor investissement initial et doivent ticipatory monitoring and evaluation manera de gestionar la être efficacement suivis et régulés hold significant promise for social condicionalidad es mediante un pour que les coûts de transaction learning and managing for results. pacto de politicas flexible, no à la charge des bailleurs de fonds Evaluation should be informed coercitivo, adaptado a las et des pays diminuent au fil du by the global perspective of the diferentes etapas de la reforma. temps. International Development Goals • Para que la identificaciôn de endorsed by the development com- muchas partes interesadas con Évaluation de l'efficacité du munity. Far from implying rigid, diversos intereses y capacidades développement top-down global planning, this two- pueda extenderse se requiere La mondialisation de l'économie et le way link means adapting the inter- tiempo y un proceso temprano de contenu du CDI donnent à penser que national goals to country conditions movilización, especialmente de las plusieurs principes doivent s'appliquer à and priorities and enhancing part- comunidades y de los pobres. l'évaluation de l'efficacité du nerships at all levels. Para ello es preciso recurrir développement. sistemàticamente a procesos L'évaluation doit être axée sur les Implications for the Bank participatorios; informar y dar résultats. Pour cela, il faut pouvoir The CDF is not a blueprint-the oportunidades de expresión a los suivre avec précision les progrès hallmark of the planning era and asociados màs débiles, sobre todo réalisés par rapport aux objectifs de the adjustment era. Its aim is to a las mujeres y a las poblaciones développement, en s'intéressant tout launch a process that adapts con- marginadas, y crear condiciones particulièrement à la croissance et à la tinually to changing circumstances. propicias para incrementar e lutte contre la pauvreté. La mise en Custornization should thus replace institucionalizar la participación. lumière des fruits du développement the "one-size-fits-all" mindset. • Para las asociaciones se requieren doit correspondre au programme To this end, the Bank must do dos funciones complementarias d'ensemble arrêté d'un commun ac- more to learn and listen. The CDF de parte del Banco: el respaldo a cord par le pays et ses partenaires. points to the pivotal role of knowl- los dirigentes nacionales y el Le recentrage de l'effort de edge and learning in development. desarrollo de la capacidad para développement, qui est passé des projets Too often, it is assumed that the ejercer ese liderazgo, y la aux programmes à l'échelle du pays, Bank bas the answers, and that the participación de sus organismos doit s'accompagner d'un recadrage ana- only problem is to sell the preferred asociados de asistencia para el logue du processus d'évaluation. Il solution to clients. But the Bank is desarrollo a fin de promover la convient de mobiliser les ressources et les not a storehouse of universally ap- selectividad, coordinar las compétences nécessaires pour mettre au plicable knowledge just waiting to intervenciones y armonizar los point les indicateurs et les systèmes be transferred. That is why it should procedimientos. Las asociaciones d'information voulus. Outre l'attention empower its clients to tap global pueden entrañar inversiones actuellement portée à la performance et knowledge, connect clients to one desde un comienzo y deben à l'évaluation des projets, il faut mettre another and to other sources of vigilarse y gestionarse de manera l'accent sur les résultats au niveau de xix 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS experience, and learn with eficaz a fin de que los costos de l'ensemble du secteur et du pays. them about what works. - transacción para los donantes y À cette fin, les réformes du Practicing selectivity los paises puedan disminuir a lo secteur public doivent through partnership will largo del tiempo. comprendre le renforcement de la require continuous change. capacité d'évaluation des pays. Bank management has initi- Evaluación de la eficacia en Des objectifs communs, une ated many changes under the términos de desarrollo responsabilité conjointe des Strategic Compact. More change La cambiante economia mundial y el résultats à obtenir, une obligation de will be needed to sharpen the Bank's MID plantean varios principios para résultat réciproque et des impératifs focus and improve the development evaluar la eficacia en términos de distincts en ce qui concerne les effectiveness of its operations. That desarrollo. comptes à rendre sont les critères à will involve new skills, new ap- La evaluación debe centrarse en los utiliser pour évaluer l'efficacité du proaches, new incentives, and new resultados, y para ello es necesario un développement. Il faut, en particulier, attitudes-to make room for genu- seguimiento exacto de los progresos que les bailleurs de fonds et les me partnerships. Above all, the hacia la consecución de las metas en ma- pouvoirs publics s'emploient de con- Bank will have to be realistic and teria de desarrollo, con un claro enfoque cert à associer la société civile et le strategic about its evolving com- en la reducción de la pobreza y en el secteur privé au travail de suivi et parative advantage in a fast- crecimiento. El seguimiento de los d'évaluation, et à aider toutes les par- changing environment. resultados en términos de desarrollo ties prenantes à acquérir les debe ajustarse al programa integral de compétences et l'état d'esprit desarrollo convenido con el gobierno y nécessaires. Ici, l'approche participa- sus asociados. tive laisse bien augurer de l'avenir, car A medida que el énfasis de las elle est instructive pour la société et iniciativas en pro del desarrollo avanzan permet une gestion soumise à des del nivel de los proyectos al plano màs critères de performance. elevado de los programas nacionales, lo Le travail d'évaluation devra propio debe ocurrir con el proceso de s'inscrive dans la perspective mondiale evaluaciôn. Se deben invertir recursos y des Objectifs internationaux de conocimientos para formular développement approuvés par la indicadores y sistemas de información communauté du développement. Loin adecuados. La actual preocupaciôn por de déboucher sur une planification los resultados de los proyectos y la imposée d'en haut et rigide au niveau evaluación debe complementarse con un mondial, ce double lien doit permettre enfoque sectorial y nacional. La reforma d'adapter les objectifs internationaux del sector público debe incluir el à la situation et aux priorités du pays desarrollo de la capacidad de evaluación et de renforcer les partenariats à tous pot parte de los paises. les échelons. La eficacia del desarrollo debe evaluarse en términos de los objetivos Répercussions pour la Banque compartidos, la responsabilidad Le CDI n'est pas un modèle et ne vise conjunta por los resultados pas à marquer de son estampille l'âge de obtenidos, las obligaciones reciprocas la planification et de l'ajustement. Il a para alcanzar resultados y la pour objectif de lancer un processus qui distinciôn de responsabilidades con ne cessera de s'adapter à l'évolution de la respecto al desempefño. En particular, situation. Le « sur mesure » est donc los donantes y los gobiernos deben appelé à remplacer la « taille unique ». unirse para hacer participar a la Pour cela, il faut que la Banque sociedad civil y al sector privado en las cherche davantage à apprendre et à actividades de seguimiento y écouter. Le CDI insiste sur le rôle xx Executive Summary evaluación, y para ayudar a déterminant que la diffusion du - todas las partes interesadas a savoir et le développement des adquirir y desarrollar los connaissances jouent dans le conocimientos y las actitudes développement. Trop souvent, necesarios. El seguimiento y la on part du principe que la evaluación con participación Banque détient la réponse et ofrecen grandes esperanzas qu'elle n'a plus qu'à vendre la para el aprendizaje social y la gestión meilleure solution aux clients. Mais la de los resultados. Banque ne stocke pas des La evaluación debe fundamentarse connaissances d'application générale en la perspectiva global de las metas qui attendent d'être transférées. C'est internacionales en materia de desarrollo pourquoi elle doit donner aux pays la aprobadas pot la comunidad del possibilité d'exploiter les desarrollo. Muy lejos de implicar una connaissances existant à travers le planificación global rigida, de arriba monde, mettre ses clients en rapport hacia abajo, este vinculo bidireccional les uns avec les autres, leur fournir significa adaptar las metas d'autres sources d'information et internacionales a las condiciones y apprendre, avec eux, les formules qui prioridades de cada pais y mejorar las donnent des résultats concluants. asociaciones en todos los niveles. Pour agir de façon sélective dans le partenariat, il faut être constamment Repercusiones para el Banco prêt à s'adapter. La direction de la El MID no es un plan detallado, bo cual Banque a inscrit de nombreux caracterizó a la era de la planificación y changements dans le Pacte stratégique. la era del ajuste. Su objetivo es poner en Mais l'institution devra continuer à marcha un proceso que se adapte évoluer pour mieux circonscrire son continuamente al cambio de action et améliorer l'impact de ses circunstancias. En consecuencia, la opérations sur le développement. Cela adaptaciôn debe reemplazar a la passe par de nouvelles compétences, mentalidad de "modelo ùnico". de nouvelles approches, de nouvelles Para ello, el Banco debe esforzarse incitations et un nouvel état d'esprit, màs para aprender y escuchar. El MID autant d'éléments qui permettront la destaca el papel fundamental que naissance de véritables partenariats. Et desempefñan el conocimiento y el surtout, la Banque devra faire preuve aprendizaje en el desarrollo. Con de réalisme et faire des choix demasiada frecuencia se supone que el stratégiques pour tenir compte de Banco tiene todas las respuestas, y que l'évolution de son avantage el único problema es vender la comparatif dans un environnement en solución preferida a los clientes. Pero pleine mutation. el Banco no es un almacén de conocimientos de aplicación universal en espera de ser transferidos. Pot esa razôn, el Banco debe dar a sus clientes los medios para aprovechar los conocimientos, poner en contacto a sus clientes entre si y con otras fuentes de experiencia, y sacar conclusiones con ellos acerca de Io que da resultado. xxi 1999 ANNUAL PEVIEW OF DEVELOPMENT EFFECTIVENESS Para poner en práctica la selectividad mediante las asociaciones se requerirán C cambios continuos. La administración del Banco ha emprendido muchos cambios en el marco del Pacto Estratégico. Serán necesarios otros cambios para definir mejor el enfoque del Banco y para aumentar la eficacia de sus operaciones en términos de desarrollo. Para ello se requerirán nuevas aptitudes, nuevos planteamientos, nuevos incentivos y nuevas actitudes, para dar cabida a auténticas asociaciones. Por encima de todo, el Banco deberá ser realista y estratégico con respecto a su cambiante ventaja comparativa en un entorno en rápida evolución. xxii ABBREVIATIONS AND ACRONYMS AFR - Africa Region APL - Adaptable Program Loan APPI - Aggregate Project Performance Index ARDE - Annual Review of Development Effectiveness BP - Background Paper CAE - Country Assistance Evaluation CARE - Cooperative for Assistance and Relief Everywhere, Inc. CAS - Country Assistance Strategy CDF - Comprehensive Development Framework CG - Consultative Group CGIAR - Consultative Group on International Agricultural Research DAC - Development Assistance Committee DANIDA - Danish Cooperation Agency DFID - Department for International Development DGO - Director-General, Operations Evaluation EAP - East Asia and Pacific Region ECA - Europe and Central Asia Region ESSD - Environmentally and Socially Sustainable Development Network FERD - Foreign Economic Relations Department (Vietnam) GDI - Gross Domestic Investment GDP - Gross Domestic Product GDS - Gross Domestic Savings HIAL - Higher-Impact Adjustment Lending ICR - Implementation Completion Report ID - Institutional Development (Impact) IDA - International Development Association IDS - Institute of Development Studies at Sussex University LCR - Latin America and Caribbean Region LIL - Learning and Innovation Loan M&E - Monitoring and Evaluation MENA - Middle East and North Africa Region NAFI - National Agency for Foreign Investment (Kazakhstan) NEP - New Economic Program (Bolivia) NGO - Nongovernmental Organization ODI - Overseas Development Institute OECD - Organization for Economic Cooperation and Development OED - Operations Evaluation Department PREM - Poverty Reduction and Economic Management QAG - Quality Assistance Group RBM - Results-Based Management SA - South Asia Region SDC - Swiss Agency for Development SPA - Special Program of Assistance to Africa UN - United Nations UNDP - United Nations Development Program UNICEF - United Nations Children's Fund USAID - United States Agency for International Development WBI - World Bank Institute WDR - World Development Report xxiii TOWARD A COMPREHENSIVE DEVELOPMENT STRATEGY lobal development during the 1990s presents a paradoxical picture. Markets and democratic principles continue to extend their reach, and global markets for trade, finance, and information are increasingly integrated. These trends should have helped developing countries to accelerate their growth and reduce poverty. Yet the fight against poverty is being lost, and the efficacy of the development assistance system is being questioned. A necessary condition for poverty reduction- set of international goals for the first decades of the growth in per capita incomes-is not being achieved by next century that stress poverty reduction, education, a majority of developing countries: health, and the environment. Most bilateral agencies have realigned their policies to emphasize these priori- * During 1985-95, the aggregate of low-income ties. OED's evaluations (ARDE 1998) have stressed the countries (excluding China and India) experi- importance of looking beyond projects, getting clients enced a decline in per capita income of 1.4 to own and broaden their programs of reform, coordi- percent a year-and middle income countries, a nating the Bank's work with that of other donors, and decline of 0.7 percent.' injecting more accountability into the development * The number of poor people living on less than effort-both for countries and for donors. US$1 a day rose from 1,197 million in 1987 to The Comprehensive Development Framework 1,214 million in 1998. Excluding China, there (CDF), proposed by President Wolfensohn in January are 100 million more poor people in developing 1999, pulls together many strands of thought within countries than a decade ago. four overarching themes: These numbers have led to much soul-searching in * Holistic. A development strategy has to go the development community. The United Nations De- beyond macroeconomic management and incor- velopment Programme (UNDP) has adopted human porate governance, human, and social develop- development as its focus. The Organization for Eco- ment objectives. nomic Cooperation and Development (OECD) Devel- * Ownership. The country has to be in the driver's opment Assistance Committee (DAC) has formulated a seat in formulating and implementing develop- 1 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS TABLE 1.1 THREE DEVELOPMENT PARADIGMS Comprehensikc D'%elopmtnr Planning Adjustment Framework * Pervsive market tailure * Pervasive Po%erinmenE railLift, * Situ.in-dependen[ LIIlurec * Go%ernment-led dc%l.'pmnEl! * NI.irker-led devc!'pmntcr * COUinri-WJ deeclopiivin thi-.t0h partncrships * Centr iliv drien: detadld * Shiirt-rern idlusEments * Long-tcrm vi,icn. '. .la blucprints rai;torintion, adapic*el.1ifl!ii * In'. er1tnE-cd deve kpi.i* ln,ctue-I Cd dol c lpnen * Inlest eti. incCMitikc ¾ .ind * RCe0urLC all cu'ln hi * ln%estments and in,tilutihit in,titLi[IunnS C'fi deiYd ''i-tic idninisrtrartr at fliIlo it * Planner1 3d cnlClie%CS .-nr.in1iti * Economist, ind tinancil e'.pert, * NIuilJisopinar apprch.ih donliri.int * Indti,r,arianon -w%th imporr * Liheralti'a.n and priaatioTin * Liberalzation. regulain. and ~ u b ri ir' n idtismri:d p,'liC:% [o iii.irClh StItc caupabilit. * D-.ior, 1.11 re%i *rce 4ip ' deterrn rc.'rc* C ounrr driv,i id cordn.ut el elOpe based onl 'mlparrife di'an[r.vim * Dn*o'rs pl.rce tori,n Wxpcris *f. Dlnior, impose pcIic ic, * Donor, proide aJ i-r stakeholders %'ith c-pti tn, 1 MIarginal rol t.r e 11olrn.z 6 [rior-drilen nwonicrt.lih'' .t pcl * P3rricipl[..r n.-nitoric irid Oild e1.flu.inri inplvile ni oi i 1 u.111*1 cllto ci hinclle IL..irib lt" ard ment strategy, and it must involve the private The principles of the CDF differ significantly from sector and civil society. those of the planning and adjustment eras (table. 1.1). Partnership. Donors and multilateral institutions The planning era focused on investment, and neglected should harmonize their policies. The adjustment era focused on policies, and The principles of the programs and prac- neglected public investment. Both neglected institu- tices, concentrate on tions. By contrast, the CDF emphasizes institutions, COF differ significantly areas of comparative including governance, the judicial framework, finan- from those of the advantage, and work cial intermediation, and social capital creation. It planning and with country partners draws on both old and recent lessons of development. adjustment eras. in a framework of mu- tual accountability. Long-Term, Holistic, Balanced * Results-orientation. A recurrent theme of OED reports has been that Success has to be judged by progress on the institutions matter (Picciotto and Wiesner 1998). Evalu- ground toward equitable and sustainable ation evidence confirms the need for a broad-based growth. concept of development, including effective public 2 Toward a Comprehensive Development Strategy expenditure management, suitable institutional capa- changes in its rationale, strategy, and mode of opera- bilities, and safety nets. Recent research also highlights tion. The development assistance system is too frag- the value of a comprehensive approach to create mented and onerous, particularly for poor and weak virtuous, circles (Dollar 1999; Thomas and others countries. Recent studies have concluded that there is 1999). Governance and institutional quality are key to an urgent need for a country-led partnership approach growth, poverty reduction, and project success (Evans to development assistance (World Bank 1998i, 1999m). and Battaile 1998; Buckley 1999). An OED study recommends that the Bank "let go" in Poverty is multidimensional and location-specific. areas where it does not have a comparative advantage, It can be reduced only through programs tailored to the and concentrate instead on mainstreaming promising local context (Narayan 1999; World Bank 19991). An programs, relying on other agencies to pilot new OED case study of health determinants suggests the approaches (World Bank 1999c). Similarly, for need to go beyond the confines of the health sector- multicountry, grant-based collaborative programs, an transport, pollution, communications, the environ- OED study concludes ment, and education must all be considered to get that the Bank should Lack of ownership results (World Bank 1999b). Similarly, getting girls team up with private into schools and keeping them there transcends the foundations and develop- has compromised education sector-concurrent investments in transport, ment assistance agencies development water, sanitation, and domestic labor-saving technolo- with a long tradition of effectiveness. gies may also be necessary. awarding grants. Conflicting interests Ownership among donors and barriers to progress within countries Lack of ownership has compromised the development must be addressed in order to achieve reforms. The effectiveness of many Bank-supported operations (World Bank can help forge strategic alliances with other Bank 1998e, 1998f, 1998m, 1999c). Projects have tended lenders and donors to overcome these constraints and to perform poorly when they were prepared by outsiders, nurture consensus on policy changes and capacity did not engage stakeholders and beneficiaries, exceeded building requirements (World Bank 1995b). Active local implementation capacity, and did not engender involvement by senior management and well-staffed borrower commitment. Success in both investment and Resident Missions facilitate cooperation (Kreimer and adjustment operations has been associated with owner- others 1998). The OED aid coordination study also ship and beneficiary participation.2 It is important to highlights the need to build local capacity for effective nurture ownership among senior government officials aid management (World Bank 1999m). involved in negotiations, civil servants concerned with implementation, and those affected by the intervention, as Results, Learning, Accountability well as among Bank staff (World Bank 1998m). The aid business has been overly focused on inputs, Decentralization involves risks, but can help im- financial commitments and disbursements, and supply- prove ownership (World Bank 1999o). Over time, driven technical assistance. An OED study on resettle- decentralized systems create commitment to reform as ment suggests that disappointing outcomes were largely lower-level officials, accountable to elected officials, the product of reliance on "plans" rather than "results" as move to higher posts, bringing with them enthusiasm the touchstone of quality management (World Bank for pluralistic, consensual, and responsive modes of 1998k). Better performance indicators and learning pro- governance (Crook and Manor 1999 BP).1 Decentrali- cesses can help manage projects and strategies toward zation also broadens the scope for partnerships among successful outcomes (World Bank 1999d). local government institutions, civil society, and An extensive evaluation literature provides evi- grassroots communities. dence of the crucial role of monitoring for development effectiveness (World Bank 1999d). Results orientation Partnership and Development Cooperation connected to a learning process requires mechanisms to Spurred by global economic and political change, generate and share knowledge, both within a country development cooperation is undergoing fundamental and among partners. OED's assessment of public 3 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS expenditure reviews recommends a focus on the goals FIGURE 1.1 INTERACTION OF CDF PRINCIPLES of expenditure reform from the beginning, both for proper selection of expenditure instruments and for better implementation (Datta-Mitra 1997). n Challenges and Tensions The CDF principles, a regular staple of research and evaluation findings, are rarely practiced. We need to Resuks understand why and find ways to overcome the implementation challenges. There is tension as well as [.rrH I n.rJi) synergy among the CDF principles. The distinctive contribution of the CDF is to have brought these well- established principles together. Each is valid on its own, but it is their combination that promises high the development assistance system. Leadership, a new rewards, as well as significant risks (figure 1.1). authorizing environment, and broad-based ownership The challenges arise from current incentives and of the CDF principles must be combined with system- capacity constraints within developing countries and atic learning. 4 2 CHALLENGES AT THE PROJECT LEVEL J rend analysis shows that the quality at exit of Bank-supported development projects has improved over the 1990s, but the most recent evaluations suggest a stalling of gains - below the Strategic Compact goal of 75 percent satisfactory.' The quality at exit has taken a slight downturn from a peak of 74 percent satisfactory for projects exiting the portfolio in FY97 to 72 percent for FY98-99 exits. Moreover, institutional development impact and sustain- ability of project benefits continue to lag. The demandingness, complexity, and risk of ment, governance). The increased complexity and the projects have also climbed over the past decade, more adverse operating environment suggest that the making the modest performance gains more impres- modest improvement in results during the 1990s repre- sive.2 But stagnant performance in borrower imple- sent a substantial improvement. But the gains in project mentation has led to a growing gap between complex- performance appear stalled, highlighting the need for ity and capacity, contributing to the recent downturn in intensified attention to institutional constraints. project outcomes. Improving project performance will OED evaluators assess project results along three require greater emphasis on developing borrower ca- related dimensions-outcome at the time of evaluation, pacity. Lessons from project experience also point to sustainability of ben- the continuing importance of the Bank's appraisal and efits, and institutional Demandingness, supervisory roles. development impact. complexity, and risk of Last year's ARDE intro- Keeping up with Complexity duced a new perfor- projects have climbed The changing nature of Bank-financed projects must be mance indicator that over the past decade. kept in mind when considering results trends. The combines these three di- exiting portfolio has become more ambitious relative mensions into an aggregate project performance index to the implementation environment (figure 2.1). Devel- (APPI). The latest evaluation results show a broad- opment work in traditional sectors has become more based improvement in the aggregate performance of complex, and the Bank has expanded its activities in exiting projects. more challenging areas (environment, human develop- Across exit years, the APPI shows a leveling off in 5 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS performance since the improvement from the decade's downturn in project outcomes combined with the steady nadir in FY94, both by projects and by disbursements (see increase in complexity suggests a possible tradeoff be- figure A1.1). Since FY97 the APPI by projects has tween the two under current conditions. To ensure future remained at 6.7, and by disbursements at roughly 7.1. performance gains, greater project complexity must be The recent results reflect met with improvements in borrower capacity and in- To ensure future stability in overall per- creased vigilance in quality assurance. formance, but also hide Over the short term, outcome performance is performance gains, the dynamics among the expected to remain below Strategic Compact levels. greater project three dimensions of the The Quality Assurance Group (QAG) reports an in- complexity must be met index. A modest down- creased share of projects exiting the portfolio with with improvements in ward shift in the out- unsatisfactory ratings, including most of the projects to comes of recently evalu- be evaluated by OED over the next six months. But borrower capacity and ated projects has been among the remaining active projects, a declining share increased vigilance in offset by an increase in are at risk of failing to achieve their development quality assurance. institutional develop- objectives, which is a promising signal for improve- ment impact (although ments over the long term. from a low base). The percentage of projects with satisfactory outcomes Varied Performance among Regions, Sectors, and at exit-projects that have efficiently met or exceeded Types of Lending their major relevant objectives-has increased over the Regions fall into three groups according to the perfor- 1990s, rising above the 70 percent threshold of the early mance of projects exiting in FY98-99 (see figure A1.3). 1990s (see figure A1.2). The share of satisfactory projects The top performers are Latin America and the Caribbean, exiting in FY98-99 is 72 percent, confirming a move Europe and Central Asia, and East Asia and the Pacific- beyond the trend for FY90-96, when outcome perfor- all with more than 80 percent of projects rated satisfactory mance averaged 66 percent satisfactory. But while FY98 in outcome, and more than half as likely to be sustain- is the second consecutive year in which more than 70 able. The Middle East and North Africa and South Asia percent of exiting projects had satisfactory outcomes, the rank in the middle, with roughly two-thirds of projects share is down from 74 percent in FY97, and preliminary having satisfactory outcomes. Sub-Saharan Africa ranks results for FY99 (which may not be statistically signifi- lowest on all three dimensions, despite solid improvement cant) show a further decline to 70 percent. The recent in project outcomes and sustainability. The Region had FIGURE 2.1 DEMANDINGNESS, COMPLEXITY, RISKINESS, AND OUTCOME, BY EXIT FISCAL YEAR Percent 90 80 - 70 - 60 - 50 - 40 - 30 20 - 10 FY90 FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 E Demandingness (% substantial) 0 Complexity (% substantial) N Riskiness (% substantial) - Outcome (% satisfactory) Note: Results for FY99 are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. 6 Challenges at the Project Level the second-largest gain in the share of projects with provement in outcomes since the 1980s and early 1990s- satisfactory outcomes between FY94-97 and 1998-99, satisfactory outcomes rose from 69 percent in FY90-93 to from 55 to 61 percent. 85 percent in FY98-99. Performance has improved in Performance has improved dramatically in Latin Sub-Saharan Africa (79 percent satisfactory, up from 69 America and the Caribbean over the past decade, and percent in FY94-97) and in the Finance, Private Sector, more recently in Europe and Central Asia. The gains have and Infrastructure Net- put projects in these regions on par with those in East Asia work (94 percent, up and the Pacific, where the financial crisis has led to a from 74 percent). But Fewer than half the deterioration in outcomes and sustainability. The Latin sectoral adjustment evaluated projects America and the Caribbean Region has seen improve- lending has accounted exiting in FY98-99 are ment in all three dimensions. The gains in institutional for the bulk of the gains; development impact are particularly promising for the structural adjustment likely to have sustained Region's future performance; 52 percent of projects lending has shown stag- results, a performance exiting in FY98-99 were rated as having substantial nating performance. similar to that impact, the highest share among all the Regions for this Despite the overall throughout the 1990s. period. In Europe and Central Asia, recently exiting improvement in the projects show solid gains in outcomes and sustainability performance of adjust- (83 percent with satisfactory outcomes and 69 percent ment lending, concerns have been expressed about its with likely sustainability in FY98-99). But performance social and environmental content. A 1996 evaluation of in institutional development impact has improved only completed adjustment operations by OED has identi- slightly, and self-evaluations of the active portfolio in the fied a need for greater emphasis on cost-effective Region suggest a diminished development impact in the management of public social expenditures, along with future. Active commitments at risk rose from 25 percent in effective safety nets to protect the extremely poor and FY98 to 47 percent in FY99, making the Europe and those vulnerable to the effects of adjustment Central Asia portfolio the riskiest in the Bank. The (Jayarajah, Branson, and Sen 1996). Management increased riskiness reflects a rapid deterioration in the reviews of the extent of recent progress are under way, Russia portfolio. following QAG and Environmentally and Socially Outcome performance varied significantly among Sustainable Development Network (ESSD) assess- sectors between FY94-97 and 1998-99, deteriorating in ments. Better dissemination of relevant Bank policy six sectors while improving in five (see figure A1.4).4 But directives to increase the congruence between policy several of the improving sectors made tremendous gains, and practice should be considered. lifting Bankwide performance. Gains were particularly strong in three sectors: the share of projects with satisfac- Institutional Development and Sustainability Flagging tory outcomes rose by 41 percentage points in public Sustainability of project benefits and institutional de- sector management (to 93 percent), by 25 points in urban velopment impact remain major concerns (see figures (85 percent), and by 17 points in transportation (93 A1.5 and A1.6).6 Fewer than half the evaluated projects percent). The latest self-evaluation data show that public exiting in FY98-99 are likely to have sustained results, sector management is a key problem area for the Bank, a performance similar to that throughout the 1990s. A however, suggesting much lower performance in the volatile external environment has suppressed improve- future. The two best-performing sectors in the FY94-97 ments in sustainability, particularly in East Asia and exit cohort-social sector and multisector-both experi- the Pacific, because of the recent financial crisis, and in enced deterioration in project performance. Performance Europe and Central Asia. Operations with large dis- declined most for multisectoral projects, with the share bursements perform somewhat better on sustainabil- showing satisfactory outcomes falling from 83 percent for ity-two-thirds of evaluated disbursements for fiscal projects exiting in FY94-97 to 65 percent for those exiting 1998-99 are judged likely to have sustained results. in FY98-99. The decline reflected lower than average An emphasis on institutional development is criti- performance for structural adjustment and technical cal if the Bank is to move beyond simple project assistance loans. financing to long-lasting improvements in developing Overall, adjustment loans have shown strong im- countries and full ownership by borrowers. At the 7 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS project level, the Bank has been making slow progress produce an institutional development impact at the in institutional impact. Still, only 39 percent of the national level.' Among countries in which more than exiting projects in FY98-99 show substantial institu- half the evaluated projects achieve substantial institu- tional development impact. Better ways to achieve this tional development impact, three-quarters show corre- crucial objective must be found. sponding improvements in institutional quality at the Recent evaluations show marked improvement in the national level. This impact is strongest for projects that institutional impact of projects directed primarily to advance the regulatory framework, streamline public institutional development. Half deliver substantial im- sector operations, and improve the enabling environ- pact, a significantly higher share than the Bankwide ment for private sector activity. average. These focused institutional development inter- ventions, which make up only a quarter of the exiting Bank and Borrower Performance Must Improve portfolio, thus contribute disproportionately to the gains For this Review, OED conducted econometric analyses in the institutional impact of Bank-supported projects. of the key determinants of success for investment and Promising evidence suggests that projects can adjustment lending (box 2.1). The results for invest- BOX 2.1 UPDATED DETERMINANTS ANALYSIS PUTS SPOTLIGHT ON BANK PERFORMANCE T his Review tory-as the dependent rower preparation turned Satisfactory quality at revisited variable. out to be insignificant. entry increases the esti- the economet- Investment projects. Compared with the mated likelihood of ric analysis of the The results for the most results for projects exiting success by 12 percent- determinants of perfor- recently exited investment in FY95-97, the most age points, Bank super- mance for investment projects (FY98-99) striking difference is the vision by 13. Political projects using a meth- broadly confirm the greater importance of economy factors also odology employed in conclusion of the earlier Bank supervision; its help predict whether a the 1994 and 1997 analyses: borrower and contribution to the likeli- project will succeed: a ARDEs. And for the Bank performance are the hood of success increased one standard deviation first time, a similar most important determi- fourfold. This finding is increase in the treatment was given to nants of project intuitive given the incumbent's time in adjustment operations, success, with Bank super- increased complexity of power at the start of using a methodology vision and borrower Bank projects and the the operation reduces developed in the Devel- implementation perfor- stagnant performance in the likelihood of opment Research mance both crucial in borrower implementation. success by 13 percent- Group. The main goal determining the final Adjustment operations. age points, and a one was to assess the rela- outcome. Other things Bank and borrower standard deviation tive impact of Bank being equal, improved performance also turned increase in the number and borrower inputs on Bank supervision increases out to be key determi- of average government the success of projects, the likelihood that a nants of the success of crises reduce it by 5 while controlling for project will be successful adjustment projects, percentage points. The country factors such as by 51 percentage points- along with political other political the macroeconomic and satisfactory borrower economy factors. economy variable, a environment and qual- performance by 43. Qual- Borrower performance dummy capturing ity of governance. ity at entry increases a during implementation is whether the incumbent Single-equation probit project's likelihood of critical: when it is satis- was democratically analysis was applied, success by 28 percentage factory, a project's likeli- elected, turned out to with outcome-satis- points, and borrower hood of success increases be insignificant. factory or unsatisfac- compliance by 20. Bor- by 50 percentage points. 8 Challenges at the Project Level ment projects show that Bank supervision has become ensuring the flexibility to adapt to changes in the the most important factor in determining outcomes. implementation environment and in working with and This makes intuitive sense, given the increased com- benefiting from partners. This process role is critical in plexity of Bank projects and the volatility of the developing a learning environment while maintaining operational environment. Bank and borrower inputs a focus on development impact. are both important for adjustment operations, but borrower performance is of greatest significance. This Project Performance and CDF Principles finding confirms the prime importance of country A sample of recent project evaluations was reviewed to ownership to successful policy reform. assess the importance of CDF principles in project For projects exiting in FY98-99, overall borrower performance. For investment projects, the review cov- performance-based on the average of preparation, ered all four pillars of the framework. A smaller number implementation, and compliance with legal covenants- of adjustment project evaluations was reviewed to assess rose to 73 percent satisfactory, up from 68 percent the balance between attention to social issues and the satisfactory in FY94-97 (see figure A1.7). This improve- macroeconomic components of the reform program. ment in performance, consistent with last year's findings, India's Agricultural Development Project in Tamil keeps borrower inputs on par with Bank inputs. Three of Nadu provides a good example of the way synergies four evaluated projects now show overall satisfactory among policy reforms, institutional development, and Bank and borrower inputs. public investment can help achieve broad-based, sus- Yet while overall borrower performance is up, tainable rural development. The project shows how a implementation performance continues to hover around committed borrower, with a coherent policy framework 60 percent satisfactory, and remains the weakest aspect of and decentralized project execution, can implement a borrower project processing. This stagnant performance successful, long-term, holistic program. The Tamil reflects the difficulty of improving borrower capacity and Nadu Project now serves as a model for rural develop- underscores the need for sustained emphasis on capacity ment projects in other Indian states. building. The determinants analysis suggests that im- The Ghana Feeder Roads Project shows the benefits proved borrower implementation would lead to major of ownership and strong partnership among donors. gains in the development impact of projects. The project aimed to help the rural poor by improving Bank performance shows mixed results for evaluated access to markets and agricultural areas and by projects exiting in FY98-99 (see figure A1.8). Compared promoting labor-inten- with results for FY94-97 projects, identification perfor- sive construction meth- mance has declined from 82 percent satisfactory to 78 ods to provide employ- Determinants analysis percent, appraisal performance has stagnated at 62 ment. It also sought to suggests that improved percent satisfactory, and Bank supervision has improved integrate the transport borrower to 76 percent satisfactory. Disaggregating the Bank and agricultural sectors . supervision measure turns even this positive result into a more closely. The De- mixed signal, however: while the share of projects partment of Feeder lead to major gains in reporting satisfactory supervision was 80 percent for Roads, the implement- the development FY98, it dropped to 69 percent for the partial FY99 ing agency, exhibited impact of projects. sample. As the econometric determinants analysis con- strong ownership and firms, this drop is closely associated with the downturn in commitment during all outcomes for evaluated projects exiting in FY99. stages of the project. The project's institutional devel- CDF would have significant implications for Bank opment activities improved the department's work appraisal and supervision. Upstream activities are the programming and resulted in a thriving construction entry point for several key principles of the framework. and consultancy industry in Ghana. Partnership with Project appraisal would include consideration of ben- the Danish Cooperation Agency (DANIDA) and the eficiary participation and local capacity to ensure U.K. Department for International Development was ownership. And it must build in a focus on results from exceptional, with periodic formal meetings enhancing the beginning. Equally important, if not more so, is coordination. Bank supervision, which would assist the borrower in Bank loans often supply the institutional frame- 9 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS work for expanded activities that attract other multilat- Chile's Primary Education Improvement Project eral and bilateral donors. This additional funding can and India's Industrial Technology Development Project help multiply project benefits and impacts, as in highlight the benefits of building flexibility into project Poland's Environmental Management Project. implementation. An adaptable design gave India's Bangladesh's Agricultural Support Services Project and project the flexibility to respond to a changing eco- Poland's Energy Resource Development Project, both nomic environment, contributing strongly to its suc- carried out in partnership with another donor, illustrate cess. And in Chile's education project, adapting instruc- the benefits of donor tional processes in response to the results of monitoring Chile's Primary cooperation. The suc- by the Bank and the borrower ensured highly satisfac- Education Improvement cess of these projects tory outcomes. But adequate resources for effective and can be attributed in permanent monitoring are seldom provided. Many Project and India's part to joint missions, projects, such as Mauritius's Industrial and Vocational Industrial Technology the smooth exchange Training Project, fail to design clear performance Development Project of information, and the targets for project activities-and even when targets highlight the benefits of coordination of techni- are identified, supervision missions often ignore them. cal assistance. Adjustment operations face the challenge of bal- Where coordina- ancing a focus on macroeconomic issues with an equal project implementation. tion is lacking, projects focus on their social implications. Panama's Economic suffer. This was the case Recovery Project shows how this can be done. The for Kenya's Forestry Development Project. Originally project included a poverty reduction program to miti- designed, in collaboration with five donors, to foster gate the employment effects of labor market reforms donor coordination and promote balanced development and to reduce the cost of basic consumption needs, of the forestry sector, the project was drastically pared including public transportation. The government also down after donors withdrew. But even the reduced project set up a social emergency fund to protect those most suffered from lack of cooperation, this time between the susceptible to malnutrition and disease. Bank and the borrower. 10 3 CHALLENGES IN SECTORAL AND THEMATIC PROGRAMS T he difficulties involved in practicing the CDF principles vary across sectors (see Annex 2). CDF principles have been easier to practice in technology-driven sectors, such as infrastructure, than in the softer sectors, such as human development, or the social and structural dimensions of hard sectors. The challenges involved within and across CDF principles reflect dilemmas the Bank has been grappling with for decades. Five key tensions are at play: short versus long and deregulation of the financial sector (Mathieu 1998; term, comprehensiveness versus selectivity, speed ver- Stiglitz 1999b; World Bank 1999c). Privatization in- sus broad-based ownership, partnership versus country creases inequality if the appropriate regulatory frame- capacity, and accountability for results versus local work and environment for private sector development are capacity. Resolving such tensions lies at the core of missing. In transition quality management in development assistance. economies, the rush to mass privatization, Many factors contribute Short versus Long Term without establishing the to the short-term Many factors contribute to the short-term orientation of underpinnings of capi- orientation of development efforts: the project approach, financial talism, led to corrupt crises, political instability, the election cycle, and the sales, lack of restructur- development efforts, incentive systems of the civil service in developing ing, insider-dominated yet the most countries, as well as the incentives and planning transactions, and un- fundamental problems processes of donors and the Bank. Yet the most regulated actions by in- of development require fundamental problems of development, such as institu- vestment funds. tional development and governance, require long-term The more ambi- long-term strategies strategies and sustained efforts. tious the reform, the and sustained efforts. A long-term perspective is especially important in more time and re- dealing with the structural dimensions of reform. Among sources are needed to prepare the way. A long-term recent examples of failures to take the long view are commitment is essential to success. OED's study of privatization in transition economies, civil service reform, financial sector reform found that countries with 11 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS unsatisfactory outcomes averaged only 1.5 reform programs' weak links to existing government structures. operations dealing with the financial sector over a 12- The Bolivia Social Fund's emphasis on speed and au- year period; countries with satisfactory outcomes aver- tonomy from line ministries worked against fitting aged 2.3 operations. Equally, resettlement operations projects into sectoral plans. The assessments for social call for involvement of affected communities many funds in Armenia, Ecuador, and Peru highlighted the need years before the infrastructure investments take place for complementary actions, such as funding educational (World Bank 1998j). material, equipment, and other inputs along with infra- structure in school projects or including training for water Comprehensiveness versus Selectivity and sanitation projects. According to the Ecuador benefi- More comprehensive approaches frequently imply ciary assessment, if the social fund does not permit the greater complexity and implementation difficulties for financing of complementary works, serious consideration sectorwide and multisectoral programs. Thematic, should be given to not financing the project. "Inattention cross-sectoral, and structural dimensions are particu- to complementary requirements can . . . put at risk the larly challenging for sector-bound aid agencies and impact of the project, not to mention the satisfaction with, government ministries. For example, it took decades of use, and maintenance of projects." external pressure and top management leadership to Multisectoral program services have been difficult to bring environmental considerations into the Bank's sustain without good coordination with ongoing pro- operations. Gender, public sector management, and grams of sectoral ministries. In Mali's integrated Health, private sector development issues raise similar chal- Population, and Rural Water Supply Program, imple- lenges (Murphy 1997). Cross-sector themes are often mented by different line agencies, the population and ignored or resisted by sector-bound organizations, both water supply components were not always well coordi- within the Bank and in developing countries. nated with the health component. With the involvement of Following the debt crisis, the Bank broadened its many agencies, programs proved difficult to monitor- view to address systemic problems. Although now more and implement. The challenges of managing multisec- relevant, the success rate of the more demanding toral, multiagency programs are compounded by weak financial sector operations dropped to 50 percent, even incentives and mechanisms for intersectoral coordination, prior to the 1997 financial crisis.' An OED review of both in countries and within the Bank. recently closed opera- Integrated programs may also generate tensions The chalanges O tions supporting finan- between line agencies and oversight bodies, such as cial sector reforms ministries of finance. Activities requiring recurrent manging shows that success is funding can create ownership conflicts between central mnulisecora], often attributable to and local governments, especially in non-revenue- muliagercy programs government ownership earning operations such as highways. Implementation are compounded by and commitment, con- of a Thailand highways project, an integrated interven- weak e afif sensus-building, a fa- tion designed to address cross-sectoral issues, was vorable political cli- marred by conflicts among agencies. Implementing mechan sm.s or mate, and good policy agencies were strongly committed to the physical interse cqDa dialogue (Mathieu works, but less committed to policy reforms, in part nordihnafin, both mn 1998). because policymaking rested elsewhere in government. A focus on discrete Similarly, urban development projects involving mul- investments rather than tiple sectors have provoked detrimental competition 8 8 integrated packages of among oversight agencies, making them unmanage- investments has been a able. While the Bank has approved no new Integrated frequent feature of multisectoral programs, as noted in a Urban Development Projects since 1986, the perfor- recent review of social protection activities based on mance of some recently completed operations, such as beneficiary assessments. Going around ministries has the Brazil Salvador Metropolitan Development, suf- advantages, but these are often enjoyed at the expense of fered from needlessly complex designs involving dis- sustainability. Services provided through social fund parate activities implemented by too many agencies. agencies are particularly vulnerable because of the Sectorwide approaches are necessarily ambitious, 12 Challenges in Sectoral and Thematic Programs complex, and demanding of Bank involvement, espe- Plans and programs to strengthen national and local cially supervision. In the energy sector, for example, environmental institutions. But a 1996 OED review both the sector development model and the Bank's revealed that the environmental plans had generally agenda have grown in complexity. The global move not elicited local ownership. Many of the plans were from public monopolies toward privatization and prepared in haste and driven by deadlines that left little deregulation has required tackling a much broader time for participation.2 Making the plans a requirement range of issues: sector unbundling, private participa- for lending further eroded country support. tion, regulation, competition, interregional trade, re- The interests of different ministries (and the priori- settlement, environment, access by the poor, and ties of the center and the districts) can vary, and even renewable energy sources, among others. Operational- conflict. Uneven stakeholder commitment and weak izing a sectorwide approach through programmatic capacity can pose risks. OED evaluators found that the lending has had mixed results. Energy sector loans to success of health sector projects was significantly Pakistan, the Philippines, and Turkey fell short of their correlated with how well program designers had objectives because they were too complex-the FY89 assessed ownership Pakistan energy sector adjustment loan had more than by key stakeholders, Speed often compromises 40 conditions. Phased or incremental approaches have including concrete been more successful. In China a succession of power evidence of commit- W Ship. sector operations of incrementally increasing policy ment. In education, intensity succeeded because of a realistic assessment of too, the number of stakeholders is very large, with institutional capacity, judicious use of technical assis- many agencies and institutions involved in executing tance loans to build greater capacity, and effective use policies. Responsibility for selecting policy reform of economic and sector work. But a gradualist ap- measures and deciding on mechanisms to encourage proach may be difficult for the Bank to replicate in support must come from within the country and be countries where the assistance strategy can accommo- grounded in broad-based support for reform. The clear date only one energy operation every few years, or implication is that borrowers should be encouraged to where strong government ownership may develop only take a leadership role in the preparation of projects and during brief periods of political opportunity, as in that all stakeholders, including women and the poor, Argentina and Bolivia. should be fully engaged. Thematic strategies have a better chance of being Speed versus Broad-Based Ownership successfully implemented when a range of public and The lack of government ownership of reforms or commu- private stakeholders participate. Lessons from The nity ownership of local projects has undermined develop- Gambia, Ghana, Madagascar, and Mauritius suggest ment efforts. OED evaluations show that ownership is that this holds true for the environment. In Madagascar difficult to achieve in sectors that have a broad array of and Mauritius, national environmental action plans stakeholders with different interests, such as health and were able to increase local environmental capacity education; in thematic and structural areas, such as more than in some other countries because of substan- environment, rural development, and civil service reform; tial local ownership. The more participatory plans and in sectors such as agriculture, where resources were also successful in information gathering and provided through state channels are under pressure. public education. Partners may have different views of the roles of the state, the private sector, and civil society. Coalition building Parharship versis coumry sapa:iY and and media campaigns to overcome vested interests or Trasation COSIs hold the bureaucracy to account may not be feasible. Partnerships of external partners may be essential to Governance reforms may be required to institutionalize coordinate sectorwide programs and reduce demands participation, and this may take decades to accomplish. on government capacity. OED evaluators have spot- Speed often compromises ownership. In a spirit of lighted examples of effective coordination with donors priority setting and capacity building, the Bank has in such sectors as transport, telecommunications, and helped a number of governments address environmen- energy. In other areas, however, partnerships have a tal issues through National Environmental Action long way to go, especially in rural development, 13 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS education, and private sector development. * Discrepancies in sector policies and procedures While aid coordination can reduce demands on for program implementation government capacity, a lack of implementation capacity * Little interest in contributing to the Implementa- can still undermine reform. An evaluation of the Ghana tion Completion Reports (ICRs), depriving the Private Sector Adjustment concluded that "required exper- reports of the views of cofinanciers tise should be on board before the process begins, and * Problems with cofinanced technical assistance, potential legal issues, like land transfers, which proved such as reluctance of cofinanciers to replace problematic in Ghana, should be carefully reviewed in inappropriate technical assistance personnel advance." An evaluation of Jamaica's Private Sector * Dropping out of the project at appraisal after Development Adjustment highlighted the need to assess expressing support during preparation. borrower capacity to implement reforms, as well as borrower commitment. The borrower's own evaluation Accountability for Results versus Local Capacity was highly critical of unrealistic demands by the Bank on In tracking the development impact of projects, the Bank Jamaica's weak implementation capacity. has been weak, almost across the board. Monitoring and Partnership and coordination imply high transac- evaluation (M&E) for results has been easier to achieve in tion costs. In a Ghana health operation, the Bank infrastructure lending than in structural lending (institu- adopted a sectorwide approach but provided only tions, public and private sector development), social marginally higher supervision resources than for a lending (health, education), or thematic lending (environ- conventional investment project. This limited the ment, rural development). OED evaluators have consis- Bank's ability to establish local presence, include tently identified weak M&E capacity (including tracking appropriate technical expertise in supervision missions, inputs and outputs) and the need for greater attention to or adequately partici- sectorwide and thematic efforts. Decentralization, pate in coordination privatization, and weak regulation have all compromised In tracking the meetings. monitoring and evaluation. Chile's power sector is development impact of A better alterna- typical: the combination of unbundling, privatization, projects, the Bank has tive may be to pro- and weak regulation led to a deterioration in monitoring been weak, almost mote strategic selectiv- and evaluation. A clear lesson is that the Bank should ity-that is, to share focus on sectorwide M&E as an integral part of its responsibility among assistance to setting up regulatory frameworks. A broader donors, to rely more M&E issue is that of cross-sectoral links and the dearth of on pooled technical assistance support under govern- data on the social impact of energy policies and sector ment control, and to resist the inclination to take on all reform. tasks and cover all bases.' More intense supervision of Identification of relevant indicators has been a increasingly complex Bank-assisted projects will un- challenge, both at the technical level (balancing process doubtedly raise their effectiveness, but complex with outcome indicators, and ensuring coverage of key projects also involve opportunity and transaction costs issues) and in negotiating a limited list among key for governments: the attention of senior officials is stakeholders. A concern expressed in Ghana's health directed to Bank missions at the cost of neglecting other sector program was that the indicators placed greater issues that may be more pressing. Donor pressure also emphasis on donor priorities than on the priorities of diverts a disproportionate share of scarce local budgets national stakeholders or consumers. In addition, institu- and staff to service a bewildering number of donor tional mechanisms have rarely been crafted in order to projects. create incentives for increased performance at the district An OED review of completed infrastructure and facility levels. Excessive emphasis on process indica- projects raises a number of issues surrounding the role tors (decentralization, budget allocations) may compro- of external financiers: mise achievement of outcomes (as in Zambia) (World * Too little involvement of partners and cofinan- Bank 1999d). ciers in project design and supervision M&E in thematic programs has also been weak, * Divergence, sometimes serious, between external particularly in newer dimensions such as gender, financiers and the Bank on stated project objectives informatics, and the cross-cutting areas of environment 14 Challenges in Sectoral and Thematic Programs and poverty. The Bank has rarely used its lending enhanced monitoring of resource allocations to the poor. portfolio to systematically collect evidence on what works Indicators used were generally input measures, such as the in reducing poverty, what does not, and why. Few rural number of personnel trained or wage expenditures, rather projects have supported those who work with the poor or than outcomes. Seldom were these gender-disaggregated. 15 4 CHALLENGES AT THE COUNTRY LEVEL 'I his chapter assesses Bank Country Assistance Strategies (CASs) from the perspective of the CDF It focuses on results-growth and poverty reduction over the long term. It also highlights the tensions in implementing principles of the CDF at the country level and identifies examples of good practice. The analysis covers 28 countries that have received OED Country Assistance Evaluations (CAEs). These evaluations were prepared to fit the Bank's CAS schedule, but relevance, regional coverage, and importance to the Bank's portfolio were additional selection criteria. The sample thus includes These findings confirm the view underlying the developing countries that range widely in size and CDF: that the battle against poverty is being lost and income, and achieves regional balance.' that business as usual will not accomplish the objec- The facts of growth and poverty in the 28 countries tives of the development community. between 1981 and 1997 are sobering (see Annex 3): Addressing Social Issues * In 40 percent of the countries, per capita income The CDF conceives of development as a social transfor- either failed to grow or shrank. mation process. Increased emphasis on social services * In 25 percent, the share of the population in forms part of the new development consensus. The absolute poverty increased. United Nations Children's Fund (UNICEF) and the * In 23 percent, life expectancy declined. International Labor Organization, along with other * In 54 percent, the people experienced stagnating observers, have argued that the inadequate progress in per capita income, rising poverty, declining life reducing poverty can be traced to neglect of social expectancy, or a combination of these events. priorities under adjustment programs (World Bank * In 85 percent, per capita income grew 1 percent a 19991). Yet the country evaluations show no significant year or less in the 1990s. decline in the share of social spending in government * In 59 percent, gross savings as a percentage of GDP budgets during the period under study; countries such were low (less than 10 percent) or declining. as C6te d'Ivoire, Ghana, Morocco, Poland, and Togo * In 67 percent, investment efficiency was less than fully sustained their social spending. 10 percent or declining.2 OED's forthcoming study on poverty shows that 17 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS many countries launched social protection and devel- expense of girls. Health sector spending reflects similar opment programs during the 1990s, some with finan- biases: more is spent on hospitals than on basic health cial support from the Bank. An earlier OED report on centers, on curative than on preventive care, in capital the social dimensions of adjustment concluded that in cities than in rural areas, and on wages than on basic most countries that supplies (particularly generic drugs). Social problems stem implemented adjust- ment policies, "real Addressing Institutional Capacity and Structural Issues more from the spending per head on Disappointing growth is associated with neglect of inefficiency and anti- health, education, and structural issues. Many OED reports emphasize a gap poor bias of social social security and between institutional capacity and policy reform. A programs than from welfare programs ei- 1996 review highlighted the importance of institutional ther rose during the ad- and capacity development issues over and above lack of budgetary justment period or re- macroeconomic stability, price liberalization, trade allocations. bounded soon after ... reform, and wage and interest rate liberalization.4 The few of the Bank's early report emphasized the need for prudential regulation adjustment programs provided for safety net programs, and banking supervision, laws for entry and exit of but such programs have become more common" firms (bankruptcy), technology development, technol- (Jayarajah, Branson, and Sen 1996: xi). ogy transfers and licensing, arbitration mechanisms to Social problems stem more from the inefficiency settle labor market disputes, labor force training in and anti-poor bias of social programs than from lack of cooperation with the private sector, and improved budgetary allocations. The CAS for C6te d'Ivoire notes information on market opportunities, particularly for that the unfavorable cost-benefit ratio in education was exports.5 produced by four factors-abnormally high wages for Sequencing is an important part of the CDE teachers, low internal efficiency caused by high drop- Problems arise when financial sector liberalization out rates, low employment potential, and inequitable precedes the development of a regulatory and supervi- access.' Public spending on education typically benefits sory system, or when privatization comes before a the wealthy at the expense of the poor, and boys at the sound framework for regulation and competition. FIGURE 4.1 THE BANK'S CONTRIBUTION TO IMPLEMENTING CDF PRINCIPLES N~lancr~ in pi lls- dialgrggue and~cto'r work- - H anlancn oi ,dr d pr.loe s |atior5 D l nt ldn l -. paci for Mtriakle Outnr:r-;tt orb indar ton I i .n~ru:t ro -.rili Owjnership t - r.d -c-[ i i _ t I, i ir. . j I.ril,rc _____________________________ ..0 10 20n 30 401 50l 60 70 80 90 10 rI,,r, - 11 -1--, __________________________________ Partnership Harmoization of domir procedtire TracLking Q.19[Ainabk- growth TfaCk ung poverrn, Results Focus Dekelopment of Lo&:31 capacii" for MI&E ________________________________ Outcomes,. [mpact, and rnontior-2hic imhicaLor, 0 10 20 30 40 50 60 70 80 90 100 Percent of countries rated modest-negligible 18 Challenges at the Country Level These problems were highlighted in several CAEs. In fell significantly in the 1990s, to less than 2.5 percent of addition, OED's forthcoming study on poverty high- the portfolio in FY99. Still, few CASs explicitly address lights the role of the composition of growth in poverty issues of income distribution, or propose interventions alleviation-rural growth, in particular, was found to that directly address inequality (as distinct from inequi- be critical in reducing poverty. ties in access to services). Despite the stated goal of pro- poor growth, policy prescriptions have deviated little A Long Way to Go from those of the 1980s. The CDF principles imply a different approach to OED's 1999 review of poverty assessments finds managing country assistance programs. For this report, that nearly half do not adequately assess individual a new calibration was tested to determine what the elements of the poverty reduction strategy.' They fail performance rating of the Bank would have been if the to address the links between poverty and such macro- CDF principles had been the agreed benchmark. To this economic policies as trade and exchange rate policy, or end, the principal authors of CAEs were asked to rate such sectoral issues as food and agricultural policy and the Bank's contribution in helping client countries rural development. They give only limited attention to implement CDF principles on a scale of 1 (poor) to 4 local dimensions of poverty reduction, including in- (excellent) (figure 4.1). These experienced evaluators vestment decisions. And while all assessments recog- rated the Bank's contribution to implementing the nize the importance of labor-intensive growth to pov- principles of the CDF as modest or poor in nearly two- erty reduction, few have analyzed this issue. thirds of cases. This suggests that country assistance Slightly more than a third of the sampled poverty practices will have to change substantially when the assessments achieved a high country-level impact-as CDF is mainstreamed. While the principles are not measured by the influence on the poverty debate, new, the Bank's commitment to giving them priority is. policy design and An upcoming OED report on poverty has reviewed implementation, and country assistance strategies, and preliminary findings institutional develop- Despite the stated goal indicate that (World Bank 19991): ment in the country- of pro-poor growth, while slightly under a policy prescriptions * More than 70 percent focus on three main policy third had a low impact. have deviated little areas-trade and tariff reform, macroeconomic The impact of poverty from those of the balance, and liberalization and deregulation. assessments is strongly * Fewer than 50 percent address other aspects of correlated with their 1 the structural agenda, such as regulatory poli- quality, with the ad- cies, support for the informal sector, or wage and equacy of time for their preparation, and with the employment policies. degree of partnership, consultation, information shar- * Roughly one in five contains a good discussion of ing, and knowledge transfer at the country level. A distribution or equity issues; one in three covers survey of stakeholders shows that while six of seven issues relating to the sectoral composition of were satisfied overall with the poverty assessment, they growth. viewed knowledge transfer and local partnership and * Fewer than 40 percent emphasize policy support consultation as the least satisfactory aspects. for the informal sector or wage and employment An OED study of public expenditure reviews found polices directed at removing labor market con- that, except in the cases of Ghana and Zimbabwe, the straints for the poor. reviews remain primarily internal Bank documents that evince little sense of ownership from the government. There has been a significant shift in Bank lending, Client governments do not believe that they can away from traditional investments in agriculture, and influence the process or the output. Some perceive the toward investments in human capital development. reviews as an attempt by the Bank to micromanage Agricultural lending declined from 27 percent in the expenditure policies. Even where the government is 1980s to 16 percent in the 1990s, while lending for heavily involved in the process, the impact on policy is human capital development increased from under 7 often weak because of inappropriate sequencing or percent to more than 21 percent. Lending to industry governance constraints. In brief, implementation of the 19 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS CDF principles will call for continuing improvements dependent countries, these burdens can add up to in country assistance practices. thousands of reports and missions each year. Managing the Tensions in the CDF Poor accountability record versus scaling up. The principles of the CDF pose important challenges. Despite continued emphasis by OED on the Four are highlighted here. (Chapter 5 proposes some importance of M&E for learning and account- promising approaches to address them.) ability, the record remains far from satisfactory. The international dvelopment goals and recent Ownership versus conditionality. How should attention to governance reinforce the need for the apparent conflict between country ownership enhanced accountability for results. and donor interest in performance (often en- forced through conditionality) be resolved? How should the need for ownership be reconciled with Past activities to build M&E capacity have focused the need for policy reform and sound develop- at the project level to satisfy donor requirements. The ment priorities when country commitment is resulting lack of domestic ownership of M&E has lacking? undermined the use of its findings and the acceptance of performance measurement for sound governance. The limited capacity created through donor-driven, project- Conditionality is widely viewed as a crude attempt based M&E has been dissipated at project completion. to generate policy reform in exchange for grants or The CDF raises the bar by emphasizing learning and loans. OED studies have shown that when applied as a accountability for results at all levels-well beyond one-sided, coercive instrument, conditionality can be projects. counterproductive and incompatible with ownership. Some observers have rushed to declare all forms of * Country focus versus global public goods. With conditionality a failure. This conclusion is not sup- globalization, develcpment problems increasingly ported by the evidence. But the principles of ownership require multi-country efforts and strengthened links and partnership clearly call for reconciling the ac- between national strategies and international poli- countability of donors and countries through reciprocal cies and programs. How can international efforts conditionality. aimed at global public goods complement national efforts through the C DF? Country-led partnership versus country capacity and commitment. How should donor demands for financial accountability be reconciled with client- As global integration deepens, the number of devel- led partnership, particularly when countries lack opment problems calling for supranational policy re- capacity? How can donors play to their compara- sponses grows. These cross-border challenges arise from a tive advantage, and thus maximize the effective- combination of market failure, government failure, and ness of the development assistance system? systemic failure. The challenge of overcoming such failures creates a new role for development assistance. The country focus remains critical, but official develop- Two enabling conditions for country-led partner- ment finance also must help meet the growing deficit in ship are often missing: country commitment to sound the supply of international public goods. policies and development priorities, and institutional Global forces (including technological change) are capacity to manage and coordinate aid (World Bank creating not only far-reaching growth opportunities, 1999m). Most donors and international agencies, in- but also a host of potential problems-including capi- cluding the Bank, continue to impose barriers to tal flight, financial contagion, illicit drug trade, cross- country-led coordination. These aid-delivery transac- border environmental problems, the spread of disease, tions costs include: divergent and complex administra- waves of migrants and refugees, and loss of bio- tive procedural requirements and the numerous diversity and cultural heritage. Building purposeful missions countries must accommodate. For highly aid- and inclusive partnerships is as important in addressing 20 Challenges at the Country Level these transboundary development challenges as it is in state-society relations (particularly service delivery). tackling national challenges under the CDF. Albania's experience highlights the need to go beyond traditional monitoring of macro and fiscal trends. Country Cases Development assistance strategies in several countries Bangladesh have gone some way toward implementing the prin- Bangladesh was thought to have little chance of survival ciples of the CDF and resolving some of the inherent after independence in 1971. The Bank, in partnership with tensions. Six such cases offer lessons of good (and other stakeholders, did much to help the new country sometimes bad) practice. grapple with its myriad challenges by following a holistic, long-term approach in providing support. Since Albania then the country has made remarkable gains in both The Albania review highlights some important CDF macroeconomic and so- lessons. While it stresses the need to be holistic, client- cial development. The Despite-or perhaps focused, and results-oriented, it reveals that, with a Bank ensured that the Despite- es CDF approach (like most things), the devil is in the country was not just the because of-its details. 'Three lessons emerge. owner of the develop- breadth, the strategy First, a holistic approach requires selectivity. The ment program, but also lacked depth, as well 22 projects supported by the Bank between 1992 and its author. It played a as a focus on the 1996 covered the social sectors, infrastructure, power, supportive role in key financial sector and agriculture, the enterprise and financial sectors, and sector programs-food the environment. Ostensibly the Bank was being production, family plan- governance. holistic. Despite-or perhaps because of-its breadth, ning, and education- the strategy lacked depth, as well as a focus on the within the framework of a comprehensive development financial sector and governance, which would have program prepared by the country. The Bank encouraged produced the greatest impact on long-term develop- broad participation and active NGO involvement in ment prospects. formulating and implementing programs. Finally, it Second, it may not be easy to identify who is in the allocated adequate resources to the country assistance drivers seat. During 1992-96, there was strong owner- program and contributed to improved aid coordination. ship by the government of many key components of the Bank's program, including the initial stabilization Bolivia program, enterprise reform, and agricultural adjust- The case of Bolivia demonstrates the need to go beyond ment. Yet in the year preceding the crisis, the govern- improved macroeconomic management to achieve ment exhibited arbitrary behavior that contributed to poverty reduction. Although the New Economic Pro- polarization. By the time the pyramid schemes col- gram (NEP) launched in 1985 with strong Bank and lapsed in 1997, the country was poised for a conflict donor support was remarkable in achieving stabiliza- that pitted the president and his predominantly north- tion of the economy, it did little to achieve poverty ern supporters against groups from the south. Owner- reduction. Social and structural reforms lagged until ship by government alone is not enough; it must be 1993-97, when a "made in Bolivia" program to broad-based. change the role of the State was launched; it included Third, keeping track of results without assessing health, education, and income-generating programs. In sustainability can be misleading. In the early years of August 1997 the new government initiated a participa- Bank involvement, Albania registered high growth tory process known as the national dialogue, and rates (nearly 9 percent annually). Agricultural produc- crafted a comprehensive framework for development, tion boomed. The government successfully completed a presaging many features of the CDE mass privatization program. By all accounts, Albania The Bolivian experience offers three lessons with was the hope of southeastern Europe. By 1997, it was broad application. First, ownership of the reform apparent that the good news on growth and agricul- program evolved slowly, finally extending to all major tural production obscured serious problems with the segments of civil society after 14 years-after the institutions governing financial intermediation and benefits of the early reforms had become apparent. 21 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Second, establishing partnerships among aid agencies into agriculture, education, and social and transport was a huge challenge. Many donor-financed consult- infrastructure. When oil revenues dropped, demand ants took line management rather than advisory posi- restraint combined with a flexible exchange rate policy tions, which reduced, rather than reinforced, ownership strengthened incentives for non-oil exports. Over time, of reforms. Finally, vested interests often stalled the the dependence on oil declined, poverty was reduced, reforms. The struggle against corruption and toward and social services improved. There was also good reform of the civil service, Customs, and the judiciary partnership and strong government ownership; loan faced many difficulties that could be resolved only conditionality was no longer essential. through the continued involvement of all development In the years leading to the crisis, complacency set participants. in, and both the Bank and government ignored gover- nance weaknesses, financial sector dysfunctions, and a Ethiopia fragile social situation. The eventual crisis, rooted in With a GNP per capita of US$110 and the vast the regional recession, declining export markets, and majority of its 60 million people living on less than excessive foreign borrowing, exposed cracks in the US$1 a day, Ethiopia is one of the world's poorest Bank's assistance strategy. countries. Yet since 1991, when the country opted for a reformist approach to development, it has made im- Uganda pressive economic gains. The Ethiopian experience The CDF emphasis on client ownership, a holistic and highlights the importance of the commitment of key long-term approach, and stakeholder partnership is not stakeholders, and their ownership of development new to Uganda. The country team focused on many key programs, as well as concerns of the CDF as early as 1987. Having The Ethiopian the need for relation- identified a group of reformers in government, the experience highlights ships built on trust. Be- International Development Association (IDA) sup- cause of the strong re- ported the government's rehabilitation program. In the importance of the lationship between the addition to financial support, IDA assisted with policy commitment of key country's leadership dialogue, training, and technical assistance, not only to stakeholders, and the Bank team, the identify reforms, but also to set out the analytical as well as the need for Bank sometimes ac- rationale in favor of market reforms and establish cepted government po- benchmarks for the measurement of progress. relationships built sitions (on the size of The Bank adopted a holistic approach. Well aware on trust. sector programs, for that the success of the macroeconomic reform program example), although hinged on the adoption of far-reaching reforms in both not entirely convinced they were the correct choice. An the financial sector and the key agricultural sector, the important lesson: policy advice is more likely to be Bank reinforced the government's reform program with effective if the client feels that its views are considered, a battery of macroeconomic and sector adjustment and the tone is neither hectoring nor patronizing. operations and technical assistance credits. The Bank also made extraordinary efforts to keep donors in- Indonesia volved in reform through co-financing and technical In Indonesia, the Bank's long-standing strategy success- assistance. fully supported many of the development outcomes Since FY92, the country's reform program has been summarized under the CDF Growth was substantial, transiting from macroeconomic adjustment programs comprehensive, and balanced. Economic management into a plethora of microeconomic, sector, and thematic kept the economy on track through boom and bust. strategies. While holistic, these strategies are difficult Dutch disease was avoided. The use of oil windfall to pursue simultaneously and pose a challenge to revenues was spread over time, and they were plowed consensus-building. 22 PROMISING APPROACHES -1 his chapter outlines some promising approaches to address the challenges and tensions at the project, sector, and country levels (box 5.1). These approaches are synthesized , from the lessons of experience and evaluative evidence presented in the background papers prepared for this report. More systematic learning is needed to enrich the tool kit. Learning Process, not Blueprints a significant shift of resources from program design to The blueprint approach has been a common pitfall. It supervision, implementation support, and participatory seems to simplify decisionmaking and reduce uncer- M&E. A fundamental lesson from both development tainties, while it imposes standard solutions in poorly planning and corporate understood sociopolitical contexts and varied local experience is that strate- realities and ignores social capital and local institu- gies emerge from con- tions, sustainability and learning, and the capacity to tinuous interactions of learn and act adapt during implementation. top-down and bottom- strategically is as OED lessons indicate that adaptive experimenta- up learning processes. important as preparing tion and sustainable learning through multiple initia- Top-down approaches tives yield better results than a one-size-fits-all best are best combined with plans. practice blueprint.' A learning process means starting bottom-up approaches small, building in a bias for action, avoiding new to enable local initiatives, identify and scale up successes, bureaucracies, and supporting multidisciplinary teams. promote stakeholder commitment, and ensure learning It emphasizes flexible, evolutionary, participatory, and adaptation over time. goals-oriented, and client-driven processes. It calls for Adaptable Program Loans (APLs) embody some thinking thematically and managing across sectors, aspects of the learning process approach-agreement without undercutting professional rigor and account- on long-term objectives and broad directions, with ability. Building capacity to learn and act strategically design limited to start-up processes and institution is as important as preparing plans. building. A learning process was embedded in Brazil's Inevitably a tradeoff between detailed analysis and Water and Sanitation for low-income settlements (the up-front design and adaptation during implementation is PROSANEAR Program). Design was demand-based involved. The new approach implied by the CDF requires and iterative, shaped during implementation by benefi- 23 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX 5.1 CHALLENGES AND PROMISING APPROACHES Challenges and tensioni Approaches * Short versus lonz term * Learning proce;, not blueprints * Comprehensiveness versus select]%tI * Nnagin complexity by sequencing * C unprehensive anilyis and 4elecoive actions * Sector%%!de approaches tailored to countries and wec or,, * ) nership versus conditionaliY * Adaptable condiIuonalir for a mutual commitment proce-s; * Speed versus broad-based ownership * Building consensus * Broadening participation * Accountabiliey for results tersuS local capacity * Information for accoLintabilrn and learning * Poor accountabilItm recorJ %er.u; scaling up * Capacity building to manage tor results * Partnership versus country capacity and * From aid .oordination to de%elopment transaction costs partncrship aind capacrit building * CountrN, tocus versu global public good * Links bmeen global and country strategies ciary participation, feedback, and learning. The pro- approval process. Career and budgetary incentives gram developed partnerships among residents for the encourage the design of large projects and discourage selection and management of water and sanitation priority setting (World Bank 1998a). At the country systems. Community mobilization and group decision- level there is often a desire for large resource transfers, making were carried full-scale national coverage, and "empire building" by One way out of crippling out differently in each the implementing agency. complexity is to take a community, depending One way out of crippling complexity is to take a on factors such as lev- long-term view by sequencing a series of projects long-term view. els of social cohesion within a long-term strategy that builds on past learn- and organization. In ing. The real issue is often premature complexity. an adaptive learning approach, the program encour- Projects that build on past learning and are integrated aged ongoing evaluation of each community's experi- into existing practices can be complex and successful, ence for rapid feedback to the next subproject. as the Bangladesh Population Project and the Brazil Health Program show.2 "Repeater" projects have Managing Complexity by Sequencing higher rates of success, especially when carried out Project complexity has been rising since the early within a strategic and long-term framework. 1990s (Chapter 2). Managing the tradeoffs between Another promising approach is to phase in cover- comprehensiveness and selectivity requires an under- age, and to expand geographical scope in line with standing of the sources of complexity. Bank thinking government capacity to manage policy reform and about development has become more complex. Insuffi- implement the program. Accordingly, growth and cient "upstream" sector work before project prepara- poverty alleviation interventions should be piloted tion and appraisal leads to inadequate selectivity. regionally, and progressively tailored and broadened. Fiduciary requirements are becoming more demand- This regional-focus approach is enhanced by explicit ing. There is pressure to add components to deal with attention to capacity building and decentralization, as the Bank's expanding agenda and survive the internal implied in the Bank's Rural Vision to Action Strategy. 24 Promising Approaches Learning and Innovation Loans (LILs) offer a suitable Sectorwide Approaches tool for managing complexity by starting small. Ad- A sectorwide approach provides an opportunity to shift equate supervision resources should be available to attention from inputs to monitoring against agreed ensure learning during implementation. intermediate and outcome indicators. The Ghana and Bangladesh sectorwide approaches are good examples. Comprehensive Analysis and Selective Actions They have been supported by two programmatic Comprehensive analysis can be combined with strate- investment instruments: sector investment and mainte- gic investments. It is best conducted with key partners nance loans and APLs. The sector investment and with a view to exercising selectivity in line with a maintenance loan is intended to bring sectoral invest- partner's comparative advantage and country capacity. ments, policies, and performance in line with economic Participatory approaches can be used to enable priorities and to ensure efficient operation and mainte- poor people to analyze their realities, express their nance of investments. The focus is on institutional priorities, and make explicit their choices and tradeoffs capacity to plan, implement, and monitor investments to improve selectivity and results. The Participatory (Jones and Lawson 1999 BP). Poverty Assessment process holds promise, as evi- APLs are especially well suited to the support of denced by its recent application to the Ghana and sectorwide approaches, particularly in sectors where Uganda programs, where it brought the realities and financial flexibility is needed. A recent review con- priorities of the poor to the attention of national cluded that several operations would probably not have policymakers. For participatory approaches to lead to been brought to the approval stage without this new superior selectivity and results, participants must have instrument, because of difficulties in accurately predict- access to information, options, and learning experi- ing activities, costs, implementation arrangements, ence, as social fund programs have shown.3 and results beyond three or four years. An example of this approach is the sustainable Moving from projects to a full-scale sectorwide livelihood approach, which emphasizes people-cen- approach (with pooling of donor finances) is risky if done tered development in a holistic framework. Sustainable prematurely (Jones and livelihoods provides an analytical structure for under- Lawson 1999 BP). Such The fiduciary risks are standing the factors that influence poverty and to a change takes time identify where interventions might best be made. This and systematic capac- higher f seCt0rwide or approach has been adopted by the U.K. Department for ity building. Its pace programmatic lending International Development, the United Nations Devel- should vary according than for project lending, opment Program (UNDP), and CARE, among other to the quality of macro- since more fungible groups. It proposes an integrated and dynamic way of economic and public forms of financing are understanding poverty and thinking about poor expenditure manage- people's livelihoods-the capabilities, assets (human, ment, sector-level poli- included. But these risks natural, financial, social, and physical), and activities cies and resources, should be balanced required to earn a living (see Annex 4). The approach quality of sector-level against the costs of builds on what people have and how they live their management, degree of business as usual. lives to add to their accumulation of assets and remove aid dependence, and barriers to the realization of their own livelihood other country- and sec- choices. tor-specific factors. Because sectorwide approaches add Central to this approach is a recognition of people's to program complexity for donors, more resources are diverse livelihood goals-such as health, income, or required for supervision and implementation assistance. reduced vulnerability-and the complex household Risks should be managed by supporting capacity build- strategies adopted to meet them. Strategies are driven ing, setting clear performance targets and safeguards, by preferences and priorities shaped by vulnerability to strengthening financial accountability, and emphasizing shocks and seasonal variations. Options are also M&E. influenced by structures (the form and organization of The fiduciary risks are higher for sectorwide or government and the private sector) and processes programmatic lending than for project lending, since (policies, laws, institutions). more fungible forms of financing are included (SIDA 25 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS 1999). But these risks should be balanced against the tomers: as long as the customer projects a credible path costs of business as usual. The proliferation of projects of earnings, lending continues. puts an enormous burden on weak administrations, Assessing ownership should lead to selectivity. often undermining local capacity building and Energy lending to reforming states in India in the 1990s sustainability. is a promising example of selectivity used to reinforce reform. Taking advantage of India's decentralized Adaptable Conditionality decisionmaking to demonstrate the benefits of reform, Conditionality should be understood as a credible the Bank shifted its policy dialogue from the federal to indicator of commitment by the Bank and its partners, the state level and engaged only reforming states. not as an attempt to force externally designed policy Ownership and partnership were strengthened by wait- changes on unwilling governments (see Annex 5). The ing until the climate was right. The Bank halted Bank and external partners act as enabling agencies to lending to the power sector in India for three years, support the country's motivation for a reform process until it found evidence of real ownership of policy that is guided by genuine learning from successes and reform in selected states. With other partners, it then failures (Branson and Hanna 1999 BP). This kind of engaged in capacity building to solidify local owner- conditionality is agreed and consensual. It represents a ship. Subsequent lending produced more sustainable policy compact based on mutual commitment. results than the earlier approach. Support for conditionality as a commitment pro- cess comes from a reevaluation of the Dollar-Svensson Time to Build Consensus data (in World Bank 1998o), using the country as the Mobilizing the support of beneficiaries cannot start unit of observation, early enough. Pilot projects do not always proceed Conditionality is part of and a recent study on smoothly, even when rooted in strong community a policy reform higher-impact adjust- support. Flexibility is essential, along with a willing- ment lending (HIAL) ness to listen and develop a program incrementally, compact: the Bank and (Dollar and Svensson informed by lessons learned. the borrower first 1998; World Bank Recent irrigation operations in India, the Philip- develop and then 1999i). When the data pines, and Turkey show the importance of allowing nurture mutual are re-analyzed using time for interventions to take effect on a socially the country as the unit appropriate scale: of analysis, past suc- as reform proceeds. cess becomes a highly * In Andhra Pradesh, India, in the early 1990s, significant predictor of irrigator groups were formed around pipe com- future success. This result supports the view of condi- mittees of 20-100 farmers. This group size tionality as a process of mutual commitment, since the allowed the local elite to continue to dominate Bank can refer to lending history in formulating future and led to water allocation disputes among pipe conditionality. Such conditionality is adapted as a committees. The democratic election of much country increases its ownership of reform, assisted by larger groups in the late 1990s overcame this capacity building to achieve parity in the relationship. problem. Ongoing reformers can be offered the option of ex post * In the Philippines, large national irrigation conditionality, while credible new reformers might schemes were effectively no more than fee- choose to adopt floating tranche loans, as in the HIAL collection groups for the government agency and approach in Africa. had limited responsibility for operations and In this context, conditionality is part of a policy maintenance. Water-user groups were more suc- reform compact: the Bank and the borrower first cessful in small communal irrigation projects develop and then nurture mutual trust and commitment that had more autonomy. as reform proceeds. Conditionality is the Bank's side of * In Turkey these lessons were taken into account. a continuing relationship, while ownership is the Efforts were made to build a consensus among country's side. A model for this approach is the stakeholders-a process greatly facilitated by the relationship between a commercial bank and its cus- World Bank Institute-before irrigation systems 26 Promising Approaches were turned over to water-user groups. Larger approaches to work, attitudes and behavior must groups were also more likely to be financially change among all actors and at all levels viable and could be built around existing institu- (Blackburn and Holland 1998). The bottom line in tions, such as municipalities. participatory change is individual and personal. Broadening Participation The CDF expands the ability of the Bank to better How should participation be broadened across interest deal with the difficult governance issues that often groups and scaled-up to the national level? Extensive involve stakeholders other than government. For ex- evaluation and research findings point to several ample, in the area of clean government and improved lessons (see Annex 5): public financial accountability, the Bank's concerns extend well beyond * Integrate a learning process. A well-known government, toward . p success in broadening participation in a govern- other elements of the ment bureaucracy is the Philippines National state, such as civil so- approaches to work, Irrigation Administration, which adopted a step- ciety, the media, and attitudes and behavior by-step approach to building the capacity of user the watchdog agencies must change among all associations to manage local irrigation systems. charged with monitor- actors and at all levels. * Beware of procedural inflexibility. Changing ing and protecting the organizational systems and procedures to facili- public interest under tate participatory development is a complex the laws of the country. The judiciary and legislature undertaking. But the costs of not doing so can be also take on new emphasis in capacity building. heavy, as in the government of Uttar Pradesh's A recent OED study finds mixed results for efforts to Doon Valley Project in the Himalayas (Shepherd broaden NGO interventions (Gibbs, Fumo, and Kuby 1999). 1998). Many NGOs are reluctant to increase the scope of * Avoid rushing. Participation cannot be rushed- their operations or to enter large-scale partnerships. and the larger the scale, the greater the risk. Scaling-up can pose a challenge to an NGO's manage- When the Indonesian government tried to insti- ment capacity and create obligations to members that are tute nationwide village development planning in difficult to sustain. Any scaling-up initiative must be less than a year in 1995-96, there was little or no preceded by an assessment of capacities and commitment. ownership of the process at the village level. Too A critical step is to involve all stakeholders in developing little time was spent building partnerships with performance indicators, a process that clarifies expecta- NGOs, whose skills might have enabled the tions and priorities. government to do a better job. * Persevere. The district-level Rural Integrated Information for Accountability and Learning Project Support Program in the Lindi and Targets for monitoring progress are effective tools for Mtwara regions of Tanzania has evolved into a guiding decisionmakers during implementation. Recent holistic program involving local government, education projects have invested heavily in setting and agriculture, natural resource management, monitoring targets. The Mozambique Second Educa- transport and marketing, education, savings and tion Program used indicators to help sustain the credit, health, and water. This broad participa- operation after the credit closed. It used targets set at tion emerged from the accumulated experience appraisal and added new ones to establish a five-year of repeated mistakes and learning over 25 years. plan agreed with the borrower. It also identified * Identify a champion. A pronounced shift toward outstanding policy and implementation issues, leading participation by government or NGOs can often to agreement on remedial actions. be traced to a single leader or strong alliance of Experience in health, nutrition, and population supporters. Taking the lead, these champions projects also shows the importance of effective M&E have often battled institutional inertia and politi- design. Selecting a limited number of well-chosen cal pressures from wealthy elites. indicators and attending to capacity for data collection * Change attitudes and behavior. For participatory and analysis increase the focus on results and the 27 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS likelihood of achieving development impact. Strength- From Aid Coordination to Development Partnership ening borrower systems for the collection, analysis, Effective aid coordination guides countries and donors and use of health information in policymaking takes toward agreement that all partners will accept mutual time. It requires attention and resources during pro- responsibility and distinct accountability for develop- gram design and implementation and strengthened ment outcomes. For countries, this requires commit- incentives to achieve results and use the information. ment to developing sound policies and effective institu- tions. For donors, it calls for adoption of a development Capacity Building to Manage ior Results orientation, the exercise of selectivity that reflects Developing a results-oriented public sector is a key comparative advantage, accommodation of country- challenge in many developing countries (Annex 6). An led efforts to achieve coherence and selectivity, and the initial focus on performance monitoring in selected provision of effective capacity-building assistance to sectors or ministries can create a demonstration effect, create a level playing field among partners. easing the way to broader application to program Although many countries have expressed a strong evaluation as opportunities arise (Marcel 1999). desire to take the lead in aid coordination, only a few, The emphasis on managing performance for such as the Republic of Korea, Malaysia, and Thai- projects and programs is being complemented by a land, can be said to have fully assumed the role. Many wider focus on governance and intersectoral synergies. countries lack the capacity to take the lead, and some Dissemination of M&E information on government still lack the commitment and resolve to do so. The performance can support the active involvement of time has come for a real change-rapid, but deliberate civil society in the assessment of that performance. and finely executed. A logical and critical step is for the Civil society also has a role to play in influencing the Bank-in consultation with other donors and coun- evaluation agenda. tries-to make country responsibility a reality wher- Key stakeholders such as NGOs, the media, and ever this is feasible. The Bank's chief role would be to parliaments-particularly those representing and em- support country leadership and to help build the powering the poor-gradually learn how to use perfor- capacity needed to exercise that leadership effectively. mance concepts and tools and to understand their By giving the country a chance to exert leadership, the limitations and weaknesses. Donors and governments Bank would be better positioned to assist in building can help build such ca- long-term capacity, in concert with the principles of the pacities among these CDF and the OECD/DAC (see Annex 7). Participatory M&E is stakeholders. The costs of poor partnership and inefficient aid one step toward Participator y coordination-in decisionmakers' time, gaps in assis- building capacity to M&E is one step to- tance, and distortions in country priorities-are borne learn and manage for ward building capacity primarily by developing countries. For donors and the to learn and manage Bank, the tensions between practicing partnership and r for results and apply- reducing the transaction costs involved may be high in the ing the CDF principles short term. There are up-front costs, but they should be (Estrella and Gaventa 1998). As multiple stakeholders viewed as long-term investments in building the infra- work together to develop indicators of success, their structure of partnership and creating skills, trust, and differing expectations and priorities are brought into learning. And the costs are likely to decline sharply if the open. Stakeholders must then negotiate to develop a partnerships are pursued selectively and strategically. The more generally applicable framework, thereby build- M&E of partnership and coordination can be strength- ing ownership in the outcomes and reflecting partner- ened to assess the cost-effectiveness of different ap- ship in the assessment itself. Participatory M&E also proaches. Much can be learned from the experience of allows the tracking of holistic goals, both tangible and private firms in selecting and building strategic alliances intangible. Developing large-scale participatory M&E (Doz and Hamel 1998; Picciotto 1998). requires skills unlike those needed for traditional evaluation. Stakeholders need help to acquire skills in Linking Global and Country Strategies indicator development, monitoring, facilitation, and Mission-oriented transnational networks addressing conflict resolution. highly visible and urgent human priorities-such as the 28 Promising Approaches Onchocerciasis Control Program and the Consultative regional and global partnerships should follow its Group on International Agricultural Research-can key tenets of inclusiveness and comprehensiveness. serve as a rallying device for contributing partners. Setting priorities at the national level under the CDF Selectivity is ensured up-front by the choice of public can help identify ar- goods to be created, shared learning occurs as a matter eas where interna- Setting priorities at the of course, and motivation and coordination among tional programs are donors and partners may be achieved more easily than needed to supple- in multisectoral national programs (see Annex 8). ment national ef- the ODF can help Attention needs to be given to the interface between forts. And building identify areas where international and national public goods-a new dimen- capacity in national international programs sion of aid coordination. The implication is not that and local institutions are needed ta investments in international public goods should wait (state and nonstate) until conditions are right in most countries, but that will be critical in supplement national conditions need to be nurtured through transnational achieving the devel- efforts. collaboration programs. opment goals of glo- The CDF provides guidance. Efforts to build bal programs. 29 IMPLICATIONS FOR DEVELOPMENT EFFECTIVENESS ncreased relevance implies greater complexity. Managing complexity effectively calls for new forms of partnership that facilitate strategic selectivity in line with comparative advantage. Sus- tainability implies ownership of reforms, empowerment of domestic actors, and adequate domes- tic capacity. Thus institutional development-which provides the foundations for balanced develop- ment, domestic ownership, participation, and partnership-emerges as the linchpin of the CDE Development effectiveness also depends on a tiveness. First, a results-based approach should be results orientation, a critical aspect of the CDF. The designed up front. That is, programs should be framework is a compass, not a blueprint. Managing it "evaluable" and the tracking of development outcomes effectively will therefore require accurate monitoring and leading indicators should be carried out jointly of progress toward development goals. with the government The results orientation of the CDF reflects hard- and its partners, with a Programs should be won lessons of Bank lending. OED's independent clear focus on poverty "evaluable" and the assessment of operational performance trends shows reduction and growth. that quality management must be strengthened further: Process indicators for tracking of development key principles, such as outcomes and leading * A plateau seems to have been reached in the ownership and part- indicators should be share of projects with satisfactory outcomes. nership, should also be carried out jointly with * Institutional development continues to improve, monitored. but from a very low base. Second, as the de- the government and its * Sustainability remains low, and the operating velopment effort shifts partners, with a clear environment is becoming riskier. its focus to the higher focus on poverty * The Bank is being called on to deal with more plane of country pro- reduction and growth. complex and demanding development problems. grams, so must the evaluation process. The changing global economy and the CDF sug- Resources and skills need to be invested in developing gest four principles for evaluating development effec- appropriate indicators and information systems. The 31 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS current preoccupation with project performance and repeating the pattern of the planning and adjustment eras. evaluation should be complemented by a sectoral and And the framework will have to be continually adapted. countrywide focus. Rigidity is a real implementation risk.' Third, evaluation must become participatory. Devel- Learning will complement dissemination. In the opment effectiveness must be considered in the context of planning and adjustment eras, donors and central shared objectives, joint responsibility for outcomes, recip- governments too often adopted a hegemonic planning rocal obligations to achieve results, and distinct account- mentality that excluded vital local knowledge and ability for performance. practical know-how.2 Similarly, in the adjustment era Every effort will be Donors and govern- it was assumed that the Bank had all the answers-the ments should team up to only problem was to sell them to its clients through link M&E systems with conditionality. Under the CDF, Bank staff can no longer CDF does not become civil society. Building pretend that the Bank is a storehouse of universally another blueprint, country capacity for applicable knowledge. Instead, they will be induced to repeating the pattern of M&E is imperative. empower their country counterparts and learn with the planning and Fourth, the growing them how to achieve development results effectively. interdependence of the The CDF is certain to increase the demand for adjustment eras. world's economies calls nonlending tools and advisory services, to engender for evaluating country- ownership, partnership, and long-term holistic think- based development programs from a global perspective, ing. The use of these tools will either empower clients taking account of international development goals. Coun- or lead to cognitive dependency-the end result will try assistance strategies should link country goals, depend on the skills and attitudes of users. Tendencies progress indicators, and national strategies with these toward bureaucratization and excessive documenta- goals. But these links should not imply top-down, target- tion-the pitfalls of the planning era-will need to be based planning. Instead, the international goals must be overcome by participatory and creative approaches to adapted to country circumstances and priorities (Max- strategy development. Donor-led economic work and well, Foster, Naschold, and Conway 1999 BP). A global policy prescriptions-the hallmarks of the adjustment perspective also calls for greater harmonization of evalu- era-should give way to country-led approaches that ation across development agencies. would build on local processes and experiences and Finally, given the emphasis on the learning and develop a strong commitment to policy reforms. process aspects of the development programs, Bank Bank processes have been attuned to a different performance indicators should emphasize professional- paradigm, and will have to change for implementation ism and partnership, creativity and innovation, pru- of the CDE Bank management has taken steps to adapt dence and probity, and flexibility and responsiveness. the organization under the Strategic Compact, but a Sharing the knowledge emerging from country-based survey of Bank staff (and local donor representatives) programs and pilots should leverage their learning found that more than half did not consider an active costs and accelerate the diffusion of lessons and government role a prerequisite for in-country aid promising approaches within and among countries. coordination (World Bank 1999m). Further internal changes, both subtle and demanding, will be necessary Implications for the Bank to fulfill the potential of the CDE While the Bank has experience in implementing indi- Finally, a new development architecture is needed to vidual CDF principles, it has yet to mainstream their joint address the crisis of global poverty and mainstream application across the board. In effect, the Bank is moving practice of the CDF principles throughout the develop- into uncharted territory. To equip itself to implement the ment system. The Bank should promote the development framework effectively, the Bank will have to continuously of this architecture, in partnership with other multilateral examine the results of ongoing experiments. institutions and developing countries. Among the first The CDF thus implies continued change in the Bank. steps is to harmonize donor procedures, build an informa- A "one-size-fits-all" mentality will be replaced by a tion-sharing network, and innovate ways to strengthen "customization" mindset. Every effort will be made to client voice and build local capacity for partnership, ensure that the CDF does not become another blueprint, broad-based participation, and learning from results. 32 ANNEXES ANNEX 1. PROJECT PERFORMANCE TRENDS AND OUTSTANDING PROJECTS OED evaluates all closed projects. These evalua- tory outcomes: an index measure of 6 represents the tions are the basis for the estimated trends in project divide. A project with an unsatisfactory outcome will performance and lessons from outstanding projects never score higher than 6, no matter what ratings it presented here. The number of projects exiting the receives along the other two dimensions. portfolio has risen significantly over the past two fiscal years, and the number of independent project evalua- Outcome tions has increased nearly 20 percent since the last Outcome assessments are based on a determination of ARDE, greatly adding to the evaluation knowledge whether the project achieved most of its major relevant base. goals efficiently and with few shortcomings. An The newly evaluated cohort consists mainly of evaluator's judgement about outcome essentially boils investment projects approved in the early 1990s and down to answering the question: Did the project fast-disbursing adjustment operations approved in achieve satisfactory development results, considering FY96-98. This yields a complete picture of the FY98 the importance and relevance of its major stated cohort of exiting projects and a partial look at the FY99 objectives and the associated costs and benefits? The cohort.' The composition of the evaluated portfolio outcome rating takes into account relevance (to check has evolved over time, with the share of exiting whether the project's objectives were consistent with projects in Europe and Central Asia and in the human the country's development strategy), efficacy (to exam- development sectors more than doubling during the ine whether the operation achieved its stated goals), 1990s. The strong performance of adjustment opera- and efficiency (to assess results relative to inputs by tions detailed in tables A1.2 and A1.3 is especially costs, implementation times, and economic and finan- notable and will require further analysis. cial returns). Outcome is rated on a six-point ordinal scale: highly satisfactory, satisfactory, marginally sat- Performance Trends isfactory, marginally unsatisfactory, unsatisfactory, OED evaluators assess project results along three and highly unsatisfactory. related dimensions-outcome (which combines rel- evance, efficacy, and efficiency), sustainability of Significant Variation among Regions and Sectors project benefits, and institutional development (ID) Figures A1.3 and A1.4 show the wide variation in impact. Last year's ARDE introduced another perfor- project results by region and sector. The three main mance indicator for quality at exit that combined these OED dimensions of outcome, sustainability, and ID key dimensions: the Aggregate Project Performance impact are presented in three separate graphs, with the Index (APPI). In parallel, OED assesses Bank and regional information segregated into the three exit borrower inputs during key stages of the project cycle. fiscal year periods of 1990-93, 1994-97, and 1998-99. Figures A1.1 and A1.2 present the aggregate and Of the 16 sectors in the Bank, the sectoral figure outcome trends by year of exit from the portfolio. highlights 13-these are the sectors with at least 10 projects exiting in FY98-99. Like the regional repre- Aggregate Project Performance Index sentation, each sectoral performance is segregated into The APPI is defined by assigning cardinal weights to the three exit fiscal year periods of 1990-93, 1994-97, the ratings of each of the three results-oriented dimen- and 1998-99. sions (outcome, sustainability, and ID impact), then summing them in a simple way. Thus, the APPI ranges Institutional Development from 2 (for projects with a highly unsatisfactory Emphasis on institutional development is critical to outcome, benefits that are unlikely to be sustainable, enabling the Bank to move beyond project financing and negligible institutional development impact) to 10 and to achieve long-lasting improvements in develop- (for projects that performed strongly on all three ing countries, with ownership by borrowers. OED measures). It is noteworthy that of the three dimen- measures ID impact as the extent of the improvement in sions, outcome is the key force behind the index. The the ability of an agency or a country to use its human index draws a line between satisfactory and unsatisfac- and financial resources effectively and to efficiently 35 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS FIGURE A1.1 AGGREGATE PROJECT PERFORMANCE Thus, sustainability is defined as the likelihood, at the INDEX time of evaluation, that the project will maintain its APPI average results in the future. Assessments of sustainability take 9 into account a wide variety of factors, including country conditions (for example, the government's commitment to the future of the project, as documented 8 in the operational plan); government economic and financial policies, including policies on cost recovery; 7du eavailability of funds for operation and maintenance; 7e Bthe political situation; sector conditions; and conditions specific to the operation (particularly the quality of 6 By projects project management and the capacity of project institu- tions). Sustainability is rated on a three-point scale: likely, uncertain, and unlikely. Aggregate results are 5 1portrayed in figure A1.6. FY90 91 92 93 94 95 96 97 98 99 Note: By exit fiscal year. Results for FY99 are preliminary (43 Borrower Performance percent coverage). Borrower performance is defined as the assumption of Source: World Bank, Operations Evaluation Department data. prformance assumpio ownership rights and responsibilities and delivery of the inputs needed to prepare and implement the project. organize economic and social activities prompted by OED assesses borrower performance along three di- the project. ID impact is rated on a three-point scale: mensions-preparation, implementation, and compli- substantial, modest, and negligible. The most recent ance with legal covenants. Of particular importance trend in ID impact is presented in figure A1.5. are the assignment of qualified personnel, the provision of the required financial resources, and compliance Sustainability with the conditions of the loan agreement, including Conceptually, sustainability differs from outcome, be- reporting and auditing requirements and specific pro- cause it focuses on the features that determine whether curement rules. The performance of implementing the operation will last over its intended useful life. agencies is rated on such elements as the quality of FIGURE A1.2 SATISFACTORY PROJECT OUTCOMES By Exit FY (percent satisfactory) By Approval FY (percent satisfactory) 90 - 90 - 50 - Blyl 5 Iibreet 80 I By dibreet 00 By projects 60 60 By projects 50 1 1 1 1 1 1 1 1 1 5 FY90 91 92 93 94 95 96 97 98 99 FY82 84 86 88 90 92 Note: By exit and approval fiscal year. Results are preliminary for FY99 exits (43 percent coverage) and FY92 approvals (57 percent coverage). Source: World Bank, Operations Evaluation Department data. 36 Annex I. Project Performance Trends and Outstanding Projects FIGURE A1.3 OUTCOME, SUSTAINABILITY, AND ID FIGURE A1.4 SATISFACTORY OPERATIONS BY SECTOR IMPACT, BY REGION AND EXIT FISCAL YEAR GROUP AND EXIT FISCAL YEAR GROUP 0 Exit FY90-93 U Exit FY94-97 M Exit FY98-99 E Exit FY90-93 0 Exit FY94-97 E Exit FY98-99 Percent Outcome (percent satisfactory) By Projects (percent satisfactory) 90 - Public sector 80 - management Transportation 70 -Urban 60 - development 50 - Social sector 40 - Education 30- Finance 20 - Oil & gas ________ _ E_ 10 -_____________ 0 Multisector AFR EAP ECA LCR MNA SAR Population, Region health & nutrition Percent Sustainability (percent likely) Agriculture Electric power 90 & other energy 80 - Water supply & sanitation 70 -Industry 60- 50 - Percent 0 20 40 60 80 100 40 -By Disbursements (percent satisfactory) 30 - Public sectot 20 -management 10 - Transportation Ni 0 Urban AFR EAP ECA LCR MNA SAR development Region Social sector Percent Institutional Development Impact Education 90 - (percent substantial) Finance 80 - Oil & gas 70 - 60 Multisector so0 Population,__________________ 50 - health & nutrition 40 - Agriculture 30 - Electric power & other energy Water supply 10 & sanitation 0 Industry AFR EAP ECA LCR MNA SAR I I I I I Region Percent 0 20 40 60 80 100 Note: The figures for FY99 exits are preliminary Note: Includes only sectors with at least 10 projects exiting in (43 percent coverage). FY98-99. Source: World Bank, Operations Evaluation Department data. Source: World Bank, Operations Evaluation Department data. 37 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS management and staff associated with the operation; which it has primary responsibility, such as appraisal the use and effectiveness of technical assistance, includ- and supervision. OED assesses Bank performance ing training, advisers, and contractual services; the along three dimensions-identification, appraisal, and adequacy of M&E systems; and the extent and quality supervision. These three important considerations in of participation by the intended beneficiaries, including judging Bank performance encompass the provision of their contribution to the project's outcome. Perfor- appropriate advice as an integral part of the appraisal mance by fiscal year groups is shown in figure A1.7. and supervision process, the ability to adapt to chang- ing circumstances (if necessary, by restructuring a Bank Performance project), and the oversight of loan/credit conditionality. Bank performance is defined as the quality of service Performance by fiscal year group is shown in figure delivered by the Bank, especially in those tasks for A1.8. FIGURE A1.5 ID IMPACT By Projects By Disbursements Percent Percent 70 - 70 - 60 - 60 - 50 -Modest 50 Moet000+ 40 m 4 ~Substttal 30 30 Sbrna 20 20 Negligible Negligible 10 -10 - 0 l l il I 0 I l l i l l l l l FY90 91 92 93 94 95 96 97 98 99 FY90 91 92 93 94 95 96 97 98 99 Note: By exit fiscal year. Results for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. FIGURE A1.6 SUSTAINABILITY By Projects By Disbursements Percent Percent 70 -70 60 - 60 50 ..... . soL 50Lkely 40 40- SUncertain. a a Uncertain 30 30 - 20 -20 - Unlikely 10 10 Unlikely 0 Il llI 011 11 FY90 91 92 93 94 95 996 97 98 99 FY90 91 92 93 94 95 96 97 98 99 Note: By exit fiscal year. Results for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. 38 Annex 1. Project Performance Trends and Outstanding Projects FIGURE A1.7 BORROWER PERFORMANCE By Projects (percent satisfactory) By Disbursements (percent satisfactory) Percent Percent 100 - 100 - 80 - 80 - 60 - - 60 - 40 - 40 - 20 - 20 - 0 - - 0 - Preparation Implementation Compliance Preparation Implementation Compliance El Exit FY90-93 * Exit FY94-97 E Exit FY98-99 Note: By exit fiscal year. The figures for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. FIGURE A1.8 BANK PERFORMANCE By Projects (percent satisfactory) By Disbursements (percent satisfactory) Percent Percent 100 - 100 - 80 - 80 -- 60 - 60 - 40 - 40 - 20 - 20 - 0 0 - Identification Appraisal Supervision Identification Appraisal Supervision Ml Exit FY90-93 * Exit FY94-97 0 Exit FY98-99 Note: By exit fiscal year. The figures for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. Lessons from Outstanding Projects judged to be potentially replicable in other countries or Of the 289 operations evaluated since last year's sectors. Their success can be traced to borrower ARDE, OED assessed 21 (7 percent) as outstanding and ownership, consistent monitoring, good supervision, an 8 (3 percent) as having particularly poor performance innovative design allowing flexibility in responding to (table A1.1). The selection criteria for outstanding changing conditions, and continuity in the Bank staff projects include a highly satisfactory outcome, likely assigned to them. sustainability, and substantial ID impact. These Several of these characteristics are amply demon- projects are judged to have met or exceeded all their strated by Argentina's Provincial Reform Loan, the first major goals, and many had innovative designs or are Bank adjustment operation targeting subnational enti- 39 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS ties. The project's main objective was to support the This project offers lessons directly applicable to the federal government's effort to promote the reform of increasingly important area of subnational lending and provincial finances. The loan's contributions went serves as a model for operations in Latin America and beyond the conditions for each tranche by providing a elsewhere. A telecommunications project in Sri Lanka forum for the provinces to agree on a reform strategy. demonstrates the importance of staff continuity and The operation also set in place support mechanisms strong borrower commitment in the difficult area of that not only advanced reform among the provincial institutional development. The project sought public governments, but also helped to curb the contagion reforms and restructuring in telecommunications, and effects of the Mexican crisis at the provincial level. The regulation and promotion of private investments in the operation's success is directly related to the consistency sector. Initially classified as a problem project because and commitment of both the national government and of slow disbursement, the project eventually exceeded the Bank. The government's strategy was transparent all its stated objectives, thanks to a strong commitment and uniform across provinces. And the Bank's lending to dialogue on both sides. The Bank team responsible portfolio was fully consistent with the government's for the project remained unchanged from identification strategy, with common rules for all provincial lending. to completion. 40 Annex I. Project Performance Trends and Outstanding Projects TABLE A1.1 OUTSTANDING AND POOR PERFORMERS AMONG RECENTLY EVALUATED PROJECTS C oun Ir IProject Loan or credir Outstainding peil'oYmers Argentina Pr- ,ncial Reform L3836 Argentina Pro%incial Penin Retorm AdJiurent L4116 Bolivia L aparilizacion Proir.m C2761 Bolivia H%drocarbon scctor Retorm C2762 China shkouk( H%droclectric 2 L3515 Georgia Trarsport Rvtabdiaon C2809 Ghana Feeder R,od, C2319 Hungary Erierpri,v .ind Financial Seccor :Juruient L4141 India indu,ral Techrioloi[% Dc%clopment C2064 India Technician Educmon C2130 India \ilhrashrra Eniergen,r F.rthquik C2594 Indonesia Villae Intra-rricturr L3888 Korea, Rep. En%iranrnental Rc.earch and Ediicmin L3612 Latvia [rucrura Adjusrinr L4126 Macedonia, FYR Tran,i Facihtmor L3868 Morocco Erners4enc Dr. us:hr RFe,mvei L3935 Mozambique Fconomic Rcco%cr% - CN010 Panama Ecnonic Rccovcri L3438 Poland Environmuncal Mananemwnt L3190 Romania Trar!poural..n L3593 Sri Lanka Telcomrnmunicatiw, 2 C2249 Poor performers Bulgaria Agr!cLtral Du cinpmrer L3771 Bulgaria Pri%are IntoeimenE ind Export Finance L3631 Indones! F1manal NSc tor Dcl.A-opmen. L3526 Morocc, NanonIl Ar-riltilUr.11 C redir L3088 Morocc.. Niuionil Ruril Finarice L3662 Nigeria NarionAl Pupulation C2238 Philippi-ic Rural Elccrrificaniiin L3439 Vanuatu HOuLIsing C2262 Note: Covers only the 289 projects evaluated since last year's ARDE. Source: World Bank, Operations Evaluation Department data. 41 TABLE A1.2 OUTCOME, SUSTAINABILITY, ID IMPACT, AND AGGREGATE BY SECTOR, NETWORK, LENDING TYPE/SOURCE, REGION, AND WDI INCOME GROUP FOR EXIT FISCAL YEAR GROUPS 1990-93, 1994-97, AND 1998-99 (BY PROJECTS) z Exit FY90-93 Exit FY94-97 Exit FY98-99 Outcome Sust. ID impact Outcome Sust. ID impact Outcome Sust. ID impact Projccts Share % % % Aggregate Projects Share % % % Aggregate Projects Share % % % Aggregate # % sat. likely sub. index # % sat. likely sub. index # % sat. likely sub. index 0 Agriculture 252 30 59 34 27 6.1 235 24 69 42 35 6.3 71 19 62 42 39 6.4 Education 67 8 82 64 38 7.0 79 8 72 44 27 6.4 3,1 10 79 49 33 6.8 Electric Power & Other Energy 79 9 69 59 34 6.5 84 9 60 54 28 6.1 2-1 6 61 39 30 6.0 Environment 0 - - - - - 9 1 56 44 22 6.5 1 80 40 20 6.5 Finance 53 6 57 43 26 6.2 51 5 59 49 38 6.3 2- 74 58 48 7.0 Industry 50 6 58 46 24 6.2 41 4 51 39 32 5.9 I 1i 40 33 30 5.4 Mining 12 1 55 50 50 6.2 9 1 78 78 67 7.1 3 I67 67 100 7.6 Multisector 62 7 71 52 35 6.7 100 10 83 54 27 6.8 11 N 65 48 13 6.2 Oil & Gas 27 3 78 52 67 7.3 24 2 79 58 42 7.0 13 69 69 31 6.8 Population, Health & Nutrition 21 2 67 43 14 6.2 42 4 67 55 26 6.4 2 - 64 50 25 6.3 Public Sector Management 35 4 56 40 31 6.0 57 6 56 44 23 5.8 29 ) 97 72 52 7.6 Z Social Sector 1 0 100 100 100 8.3 16 2 88 25 56 6.9 1i 5 80 20 45 6.7 Telecommunications 11 1 64 73 36 6.7 22 2 82 73 36 7.0 4 1 100 100 100 9.6 Transportation 95 11 74 55 34 6.6 108 11 76 50 33 6.7 29 8 93 72 69 7.9 Urban Development 44 5 77 41 27 6.5 50 5 60 36 16 5.9 21 6 85 45 45 6.9 Water Supply & Sanitation 37 4 69 40 43 6.7 38 4 55 32 18 5.7 23 6 48 26 30 5.7 Environmentally & Socially Sustainable Development 252 30 59 34 27 6.1 244 25 68 42 35 6.3 76 20 63 42 37 6.4 Finance, Private Sector & Infrastructure 408 48 68 50 35 6.5 427 44 65 48 30 6.3 154 41 71 52 46 6.8 Human Development 89 11 78 59 33 6.8 137 14 72 45 30 6.5 87 23 75 43 33 6.6 Poverty Reduction & Economic Management 97 11 66 48 34 6.4 157 16 73 50 25 6.5 60 16 80 60 32 6.9 TABLE A1.2 OUTCOME, SUSTAINABILITY, ID IMPACT, AND AGGREGATE BY SECTOR, NETWORK, LENDING TYPE/SOURCE, REGION, AND WDI INCOME GROUP FOR EXIT FISCAL YEAR GROUPS 1990-93, 1994-97, AND 1998-99 (CONTINUED) (BY PROJECTS) Exit PY90-9 E\i[ FY94-9- Exir FY9S-99 )urc'le %uq ID ipaci (unme SiuI. II) impact OuC,l, Nu,[ Iim ii Pr-,i1~ . Nhar , f- .- ... "hare ', . , :wcg.uc Proiici, shar , .. ''. '. .. , . hik,1h ,uh mid,.\ i- a. kh ,b. uridk\ ' ¯ .1. lu. , % uh. mJ& ' Adjustment 112 j iA' il s "." 132 jr -i 50 2s ." 33 14 l- ni 4 - Investmenr ~l o'e 45 32 -. 11; S3 Nrl 44 i< n.s 3'4 4n -h 4e 3s '., Li *.\ u 4.S 52. t S¯ Th - 1S 31 41 55 4 l 5"45 - ID.\nj d r\ 4 -3 2 '.i 4~ 4 -. IS 2" n 2- S n1> 41 4 -. Alid ¯S | I 4 25 2I S. 2- lu S 5 3l Ss -2 3,2 bl 34 '> -. nJ N-t- Ahii v I q 44- ih lIm 1" ¯ 33 2. n 4 1"1.1 13 ' 4 3¯ i i |1 A- l2 s2 f.1 L. ., cr 41I '4" hI Ir,,44 2 i 2' " - L.ocr 2.'i, 4< 5" 2.ii 2a n i 4¯ 4'4 604i 2 n 2 il 1s3 40 66 3', '5 <. L ieimidde u 3 4 3i . 3 14 4 S° Si -.i Uppeml.iddb 142 I~- 6' n2 b . i in 1 1 nY 45 ¯i i S - 4' .2 Toial/atrage s4, 1ii ni, 4., 12 , 4 <6S 1il 65 4 .1 ".4 V¯ 10 ¯ 44 su - Note: Percentages exclude projects not rated. The data for FY99 exits represents a partial sample (118 out of 277) and reflccts the processing of all ICRS received through July 1999. The processing of the remainder of the FY99 sample is ongoing; completion is expected by spring 2000. Access the table on-line at TABLE A1.3 OUTCOME, SUSTAINABILITY, ID IMPACT, AND AGGREGATE BY SECTOR, NETWORK, LENDING TYPE/SOURCE, REGION, AND WDI INCOME GROUP FOR EXIT FISCAL YEAR GROUPS 1990-93, 1994-97, AND 1998-99 (BY REAL DISBURSEMENTS, FY96US$) z z r \i[ fN90-93 NExi Fi91-9- Eur FY98-J99c Dibur-e 0i.¯ume Su,. ID mP.ici D hur u, uitU[ ID im,pat I>shui,e (uuOnu NI. I 'Ip hi % haIII .. . . .. . r ',Iar '. "6 - .. zzretare 5 Share '., '. 'c: h reu millin, " ,, ,i h ,ub. ImdL milhøn, ,a I1L h 'Uh mi n uIhn .. ýai. ikeh ,ub. ir-Id A l 4.ii.i 21 " 42 . n.4 13,13' 1¯ 9 il 4' b.¯ ,4. 12 64 54 II e. EJic d 2,~1 4 sN b ~ .4 t~ ; 4 ; " aS 5S i r Lln I-e x ørIe Lu : 9,'i2 14 øn - 4> '.5 9,hI 1 3 ø" "I I i S I,5i S i 4 2s t. Fm rnneti - - 2x5 % ' is 3> ¯.lt1 i0 s2 44 l~r. IIodustr ¯·11 1- - 4 2k 1.~ W,14 lIi ¯ 4 hi 4" . ." S4 2 5 -l~ 4 -. 5 4- S,5 - Sit., 2 1 A- 4 l~1 .: a >1, i995 ~4 I 4 l- ¯¯,6sSi 2 1n ¯ 41 *. 11 ø O s4 , H p iiiinn,. Hc.1ith N1un[I..n1 -1 b I -I iS . -." 1.-21 2 Ni 65 ' . I,sm n S1 ~ ¯ . ie' I. 'eU' \I. i un. i j,v,S 2 ¯ 4 ii 4 ,k> -I ti i- ni' 2.12 ¯ 91- ¯; 11,1 1.2 ' 5,.. i! LUtr 2> u 1 10 l .432 1,'22 ' '9 55 4 < 6. N l -4 l3 ¯ ¯ Tvïmp,'i(:..1 ¯,431 11 sl 'a? Y~ ¯. 4,453 I? 56 54 .fs '." 2,~'¯> 4 ht¯ > e¯ ¯. lhu i e.cIopn t . ¯'C l - ~l S "j -." 1,~ 11 ¯14 I l4 .3 1,i -4 5 C¯ ' ¯.- \\akr lupph l Qiii i.,n I,>1 ' 3 1' 4'' 1 n 1,6' 4 S 23 I S. I,SnS i 44 is 23 i. Environmer ills 1 ca1 Sustainabk lDk,1..piliili 14,8'1,' 21 b' 42 3 ø 4 13,421 1 S -" '2 41 r.S %.49 I? "4 4" 6.4 Finance, Pri% i, Sector&]ur%iru,urr 3,22 i4 s "3 si '. 42.3S 56 -2 i 35 b. IS d6 SS 52 ¯ i; ¯ Human De; c1 hpioc nt ,o 5 sa ¯ 54 -.4 n,id3 Ni 5 ; . 5,30s I - oi 43 ¯. Poverty Reclustu.1 & Economic :In.ïexniu 13,14 I s2 b3 SI ¯.z 13,24', il 3. 14 C? ¯ N 4,272 13 87 63 45 7.4 TABLE A1.3 OUTCOME, SUSTAINABILITY, ID IMPACT, AND AGGREGATE BY SECTOR, NETWORK, LENDING TYPE/SOURCE, REGION, AND WDI INCOME GROUP FOR EXIT FISCAL YEAR GROUPS 1990-93, 1994-97, AND 1998-99 (CONTINUED) (BY REAL DISBURSEMENTS, FY96US$) Emur FN 90-91 -it b-) 94-9-' Exit fl 95-99 D,.r,e O inm Stu,r. I) Lmp.m Di,-bur,c Ourcomc Susi. 1L) Impa,1 DPuhu, O rc S,, ID unpaci % Shar -. '., ., s Sharc ~·.. , .gerCuau S Share "o .. .. \W re.. m3hn1" a. hhh,1 gub mid, Ilhium, ". at. l- h 4, 4u b. mldex milhon, " L , . l,.h sub. 11d& \ Alit'rm m 21.¯45 H fl4 n2 4 -. 22,S 2 NI 4 41 ¯.l iI "4 IlC-.n Ime,anemoli 4-,5I <''- -I 5s 3- n.- S'3,50 -- -5 53 7.3 n. 2d.S¯4 s1 fl Sh .4! . IlRi .nh 51,14 -S 2 nA 42 n. , -- ni lC nN 3,3¯i1 34 ~I i I -. ID )\Iblcd -, -- I ~I 42 3 I .4 2..,11, 4 -', - h' n ''., l 3 l - 54 s~ n. 1J' z.' ~ S 5 12S I. in~ 33 ' -.ý 4')0 f. - \ A4 i mid P'ikl 14,-2 21 S, i 3 fl' 15,5-I4 fl S n - -%. 10,V- S i i s' --. Iurmps nol <_uil \n, 5.512. '~ n 54 41 ". .3-s 14 SJ -I in -. 3.~l9 5 -3 -5 3 -. [Li AmenI a ind i:i'Ixin 21,12 31 on ni 4 '.s Is.lI' 2a '2 is 4i ¯.1 ¯,ii, Ic il s - ~.i Al\ dlI1 Fir ,d1N.rri Ntru. 4.411·1 - - f ~ 45 ni ,,4yii h4 34 [4 Ss i. sI 2 s ¯ Sn - .ud .\-i 11,546 fl -, 52 3' '. I3,2-1 1", 2 S 24 32 5,_ S h' Su 1 ". 11N53 1~ 11 1 1'-':. ti 44 2n1 w,~ 1-4 n9 4S 2 S0 l Imrixmddkc2M f 1,s n' §' 1.; ' i m 5'' 11 L452 11 N I 5 1 niaa erage 6'>,396 1 . ~ 59 3' 6. ~',,6s 100 -~ 6 6. 32,48 100 1 66 44 '.0 0 Note: Percentages exclude projects not rated. The data for FY99 exits represents a partial sample (118 out of 277) and refleets the processing of all ICRS received through July 1999. The processing of the remainder of the FY99 sample is ongoing; completion is expected by spring 2000. Access the table on-line at httpn://whblnG(0 8.wordbaik.or/o/ed or:hh.nj1/hiimdini', tS 2 .' 24.2 I 2' '0 37.3414¯ . .. 1334 .l¯ N lJ c,fil ý tiC, 1 Nell.:m .u 46 04 2 P. i.nJ nlk. 4.4 ?3.12 I"¯ 24 ..32 TulmJ'inCS[ i .46 22.I~ 24.S4 35 'l fll._ T..-a . 11n 4 4 '.N1 . "44 Ilililsi lki 2ì23 Zn mb.241.4 I 3 4.1 4 21' 4..4 1.03 m ou i c;ih·' i i NIe*. lii!ïquL 2.46 I Cu411stm 1[2 .- ,hIl/4 lo tanaofitetmn 2e lii'-. 2225''' 21.35 D1chInr,lov ettiieen,c -t ni L tme nt 6· 16i241 a. No data available. Note: The Investment Efficiency Ratio is defined as the GDP growth rate divided by the investment GDP ratio. This is the inverse of the conventional Incremental Capital Output Ratio (ICOR), but is adjusted for changes in terms of trade between investment goods and overall GDP. Simple averages are used. 53 ANNEX 4. EMPHASIZING PEOPLE-CENTERED DEVELOPMENT IN A HOLISTIC FRAMEWORK The sustainable livelihood approach to develop- The approach identifies five kinds of capital assets ment adopted by the U.K. Department for International on which people can build or draw: human, natural, Development, CARE, and the UNDP proposes a holis- financial, social, and physical. It aims to build on what tic understanding of poverty and an integrated and people have and how they live their lives, to add to the dynamic way of thinking about poor people's liveli- accumulation of assets, and to remove the barriers to hoods. This approach defines a livelihood as "the people's ability to advance their livelihood choices. capabilities, assets (including both material and social Early experience with the sustainable livelihood resources), and activities required for a means of approach has shown that its common framework living. A livelihood is sustainable when it can cope facilitates cross-sectoral and multidisciplinary think- with and recover from stresses and shocks, and main- ing. It makes explicit the choices and tradeoffs in tain or enhance its capabilities and assets both now and planning and execution. It emphasizes a small number in the future, while not undermining the natural of entry points, with multisectoral links evolving over resource base" (Chambers and Conway 1992). time. The iterative approach demands learning and The sustainable livelihood approach provides an sequencing, and thus requires that funding partners be analytical structure to guide the understanding of flexible and responsive. factors that influence poverty and to identify where But the experience has also raised new challenges. interventions might best be made (figure A4.1). It The approach can result in a huge agenda, and thus recognizes people's diverse livelihood goals-better risks becoming too multifaceted and time-intensive. health, more income, reduced vulnerability-and the Institutional issues-such as developing closer opera- complex strategies that households adopt to meet these tional links with the social sectors-need more atten- goals. These strategies are shaped by preferences and tion. And while the approach may improve the under- priorities that arise in a context of vulnerability standing of the problems faced by the poor, effecting resulting from shocks, changing trends, and seasonal change in a world of entrenched power structures variations. They are also affected by structures (such as remains a huge challenge. Another challenge is to the form and organization of government and the develop effective tools for managing tradeoffs and private sector) and processes (policies, laws, and sequencing-and for measuring change in people's institutions). livelihoods and security. FIGURE A4.1 SUSTAINABLE LIVELIHOOD FRAMEWORK Key H = Human capital S = Social capital N = Natural capital P = Physical capital F = Financial capital LIVELIHOOD ASSETS LIVELIHOOD TRAkNSFORMNI[NC OUTCOMES STRLCTURES & - - OUTCOMES VULNERABILITY PROC ESFS; In * More income CONTaEXT H STRUCTURES order * Increased -,SHOCKS S N Infuence * Levels of LIELIHOODwell-being * TRENDS & access government STRATEGIES * Reduced k SEAONL F* Private * PLs STRTEIE vulnerability * SEASONALITY P F sector *PoliciesI I * Culture to* Improved food . security * Institutions achieve PRO CESSES '-------- * M ore sustainable use of natural resources 55 ANNEX 5. OWNERSHIP, PARTICIPATION, AND CONDITIONALITY Ownership at its broadest involves commitment by difficulties that the Bank and its clients are likely to the public sector, the private sector, and civil society to face in trying to scale up participation (and promote a national development effort. At its narrowest it consultation) in national development and country involves beneficiaries, local government, and commu- assistance strategies, or even in advisory and analyti- nity organizations in project design and implementa- cal activities. Not all NGOs want to increase the scale tion. Ownership can apply to a national development of their operations to work on Bank-supported projects. strategy, country assistance strategy, sectoral reform Scaling up can also strain an NGOs management programs, or a project. capacity and create financial obligations to staff and clients that are difficult to sustain. Ownership and Participation Any scaling up initiative must be preceded by a Participation is important because it increases the careful assessment of the capacities and commitment of likelihood of a project's relevance and effectiveness. all parties. A critical step is to involve all stakeholders OED findings suggest that participation promotes in developing success indicators, a process that can ownership and broad, long-term attention to human expose differences in expectations and priorities. Nego- issues. It improves sustainability of development pro- tiating to develop a common framework helps to build grams and promotes learning and results-based orienta- ownership of outcomes. tion-key goals of the CDE OED findings suggest several lessons for success in scaling up participation: Benefits of Participation At the project level there are clear benefits to ownership * Integrate a learning process. Perhaps the best and participation. Participation promotes a more holis- known success in scaling up participation is the tic approach. For instance, in Bangladesh and India, Philippines National Irrigation Administration, participation quickly uncovered the necessary adapta- which adopted a step-by-step approach to build- tions that turned projects in roads, lighting, reliable ing user associations' capacity to manage local power, and better services into good investments. This irrigation systems. The use of a learning process in turn facilitated a longer-term view. And the more approach and the introduction of a number of projects that are controlled by community clients managerial innovations allowed local priorities rather than by the Bank or government, the more likely and processes to guide the activities of the the projects are to have a long-term perspective. bureaucracy-rather than the other way around Experience also shows that participation increases (Korten 1980, 1988). A working group was transparency and accountability-and reduces corrup- formed to act as a catalyst for change. It brought tion. When civil works belong to the people who use together frontline workers and higher-level staff them, there is no tolerance for siphoning off project to define a common vision of change, and then funds in bribes. Contractors are grateful for community worked through a learning cycle to achieve that support and protection from extortion. While close change over a number of years. community supervision might lead to more complaints, * Beware of procedural inflexibility. Changing contractors were able to fix problems more quickly and organizational systems and procedures to facili- cheaply, reducing complaints on completion and im- tate participatory development is a complex proving relationships with communities. undertaking. But the costs of not doing so can be heavy, as in the Government of Uttar Pradesh's Scaling-up Participation Doon Valley Project in the Himalayas (Shepherd It takes time and effort to foster participation, particu- 1999). Examples across the world show that larly to scale up participation from a project to simply adding participatory rural assessments programs, sectorwide reform, multisectoral efforts, or onto existing programs will not lead to partici- a national strategy. There have been few successes in pation. scaling up NGO involvement from projects to wider * Avoid rushing to scale up. Participation cannot programs. Bank evaluations and research name the be rushed, and it is especially dangerous to try to 57 1999 ANNUAL REV[EW OF DEVELOPMENT EFFECTIVENESS do so on a large scale. The Indonesian or a single person. These champions have often government's attempt to implement nationwide led the battle against institutional inertia, mndi- vi'llage development planning in less than a year vidual hostility, and political pressures from the in 19 95-96 illustrates the pitfalls of forced wealthy elite. For example, the regional commis- participation. There was little or no ownership of sioners of the Mtwara and Lindi regions in the process at the village level, and too little time Tanzania allied themselves with the Rural Inte- was devoted to building partnerships with grated Project Support Program to introduce NGOs, whose skills might have enabled the participatory approaches. government to do a better job. *Change attitudes and behavior. For participatory "Persevere. The Rural Integrated Project Support approaches to work, actors at all levels will need Program in the Lindi and Mtwara regions of to change attitudes and behavior (Blackburn and Tanzania is one of the best documented cases of a Holland 1998). Otherwise a new authoritarian government scaling-up program. This district- manager can come into an organization and set level holistic program involved participation by the clock back, ruining a participatory culture stakeholders in local government, agriculture, and the practices patiently nurtured by a prede- natural resource management, transport and cessor. The bottom line in participatory change marketing, education, savings and credit, health, is individual and personal. and water. The successful collaboration and results visible today are an outgrowth of the Ensuring attitude and behavior change also poses a accumulated experience of repeated mistakes major challenge for the Bank. The Bank may be asked and learning over some 25 years. to help the government develop participation-for * Identify a champion. A strong shift toward example, through World Bank Institute seminars for a participation in government or NGO practice wide range of participants, including members of can often be traced to an alliance of individuals parliament and the press. Cooperation with local BOX A5.1 CONDITIONALITY SCENARIOS A At least four *A higher-impact struc- tranches that could be *Advice, continuing conditionality tural adjustment lend- converted to floating contact, and readi- scenarios can ing model of floating tranches as ownership ness to commit sup- be imagined, ranging tranches. This model develops or experi- port if circum- from the most flexible might apply to a coun- mental single-tranche stances change. This to the most restrictive, try seen to be a cred- operations. This model could apply E x post conditions ibte reformer with model could apply to to a country with a for lending to an strong ownership, but countries with government not yet ongoing reformer no real track record. nascent or developing ready for reform. with a track record The government ownership, but with All the scenarios are of success. Here the chooses the sequence recent changes in consistent with adjust- process of mutual and timing of the regime or other able conditionality and commitment is conditions and tranche sources of credibility tranches. As countries fully under way. release. Four coun- in reform. The single- progress, they would The country is tries with floating tranche operation move up the condition- making progress in tranches-Cameroon, could apply to a ality ladder. The possi- ref orms and the C6te d1Ivoire, potential reformer, bility of this kind of Bank is lending. Malawi, and Tanza- with the promise of sequencing should An example may ma-outperformed additional lending to strengthen the be Indonesia during others. follow if reform actu- movement toward 1985-95. *Regular multiple ally comes about. mutual commitment. 58 Annex 5. Ownership, Participation, and Conditionality research organizations can also help develop policy The government would choose the sequence and timing knowledge and contribute to ownership through the of sector reforms as external support is calibrated to the participation of local researchers and officials.' quality of the program. A model might be the relation- Broad participation in program development, with ship between a commercial bank and its customers. As cooperation between the Bank and the government, long as the customer can project a credible path of should strengthen ownership on both sides. Broad earnings, lending continues. Thus conditionality is the support can lend greater credibility to government Bank's side of a continuing relationship, and ownership ownership, which in turn can result in greater Bank is the country's side. flexibility regarding conditionality. It is difficult to establish criteria for ownership.2 The frameworks commonly used to assess ownership- Ownership and Conditionality leadership analysis and stakeholder analysis-are help- Conditionality, through its relationship to ownership, ful, but insufficient. Reform-readiness analysis is often can foster mutual commitment to ongoing reform. proposed as an alternative, but it does not eliminate Traditionally the Bank has attached conditionality to uncertainty. Broad participation in program develop- tranche releases on specific, single loans. The CDF ment, with cooperation between the Bank and the suggests a broader view of conditionality as the government, strengthens ownership on both sides. outcome of an ongoing process that allows the bor- Broad support can lend credibility to government rower and the Bank to develop and nurture mutual trust ownership, leading, in turn, to greater flexibility on and commitment, a process that adjusts as country conditionality. ownership grows. Empirical evidence on successful adjustment lending suggests different forms of condi- Conditionality as a Commitment Process tionality for countries at varying phases of the reform Two recent studies support the view of conditionality as a process. It proposes a new view of flexible conditional- commitment process: a study of HIAL and a reevaluation ity, combined with capacity building, to level the of the 1998 Dollar-Svensson data using the country as the playing field for all parties in the relationship. unit of observation (Dollar and Svensson 1998; World Bank 1999i). Both studies support the view of condition- Reconciling Conditionality and Ownership ality as a process of mutual commitment. Under the CDF, conditionality can play a key role in The HIAL study found better policy outcomes and the Bank's dialogue with reforming borrowers. Rather economic impacts for countries receiving HIAL than than an attempt to force externally designed policy for other groups of IDA countries. This performance changes on unwilling governments, conditionality gain was attributed to greater selectivity in lending and should be used as a credible indicator of commitment more flexibility in disbursement arrangements. HIAL by the Bank and its partners. Under the CDF, condition- introduced floating tranches, with the government ality should represent a transparent and explicit under- deciding when to meet conditions, which were gener- standing of sustained Bank commitment in support of a ally sectoral, and when to draw on the corresponding new or ongoing policy reform program formulated by tranche. Several loans had an initial tranche based on the country, with wide participation by the private selection criteria (generally macroeconomic stabiliza- sector and civil society in cooperation with external tion), followed by floating tranches. Analysis for this partners. The program would be owned by the country, report of the relationship between conditionality and and conditionality would define the parameters of tranche arrangements and outcomes found that coun- external support. tries with higher-impact adjustment loans performed Viewed in this way, conditionality could be applied better during the loan period than before, with the to actions already taken-with further support expected largest average increase in growth of 3.5 percent. on the assumption that the reform process will continue While the samples are small, this provides encouraging (box A5.1). As the reform process and ownership support for flexibility in conditionality and for condi- deepen, conditions would become more flexible. A new tionality as a process of mutual commitment. reformer might begin with a single up-front experimen- The 1998 study by Dollar and Svensson of the tal tranche, or with normal conditions that could be importance of country political economy variables converted to floating tranches as the process develops. compared with variables under Bank control in predict- 59 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS ing success or failure (as measured by OED ratings) process of mutual commitment, since the Bank can use concluded that "the results have clear implications for lending history to formulate future conditionality. the management of policy-based lending. They suggest This view of flexible conditionality differs from the that the role of adjustment lending is to identify view of conditionality as coercion-an attempt to reformers, not to create them." The study uses the loan induce the borrower to take action against its will. as the unit of analysis. But structural adjustment Participation, by involving other stakeholders, reduces lending tends to be repetitive: 27 percent of countries the risks of coercion implicit in a secret negotiation receiving adjustment loans since 1979 received more between a hard-pressed borrower and a powerful than five loans. Past successes could be predictors of lender. Equally, transparency and participation should future successes. When the data are re-analyzed using minimize the risks of shirking or policy reversals. the country as the unit of analysis, past success Under the CDF, this form of conditionality is comple- becomes a highly significant predictor of future suc- mentary to ownership. cess. This result supports the view of conditionality as a 60 ANNEX 6. MANAGING FOR RESULTS There are growing pressures on both governments and donors to make and track progress in achieving BOX A6.1 BARRIERS TO PERFORMANCE development objectives. These pressures include: ORIENTATION * Poor demand and ownership in countries * Citizen demands for more accountability and * Lack of a culture of accountability (often re- greater value for the money lated to ethics or corruption) * Scarce donor funds and growing competition * Absence of evaluation, accounting, or audit- among countries ing skills * Market perceptions of national economic com- igsil * Poor quality of performance information, and of petitiveness accounting and auditing standards and systems * Greater willingness of civil servants to take a * Little integration of M&E findings and systematic approach to learning from experience. mechanisms into decisionmaking. Developing countries are placing more emphasis on good governance and are adapting approaches and Countries and donors often disagree on ways to models from developed countries. One popular model is track implementation and impact, and few local results-based management (RBM), which focuses on institutions can bridge the gap between the technical goals, objectives, and the results chain-the linkage of task of research and evaluation and the political task of resources, processes, outputs, client reach, outcomes, and policymaking. The experience of the U.S. Agency for impact. It emphasizes the perspectives of clients and other International Development (USAID) suggests that coun- stakeholders, and can help build a client-oriented service tries are more likely to monitor policy implementation culture. At the core of RBM is a performance measure- and systematically use the findings in decisionmaking ment system focusing on results, not just inputs, processes, when: or narrowly defined outputs. A number of governments have implemented elements of RBM, and their experi- * Think tanks and other credible, nongovernment ence-particularly in measuring government perfor- parties help monitor. mance-provides key lessons for a more performance- * Explicit responsibility is assigned for translating oriented approach to government (Mackay 1998b; research and evaluation findings into policy OECD 1995, 1997a). implications. * Evaluation findings are discussed in public forums. RBM and the COF The CDF implies an RBM approach: keep the focus on It may take a decade or more to build a strong, achieving development effectiveness. RBM is a means to systematic approach to measuring government perfor- evaluate CDF processes, and thus to help governments mance. The challenge goes beyond building skills and and donors learn what works, what does not, and why, civil service infrastructure. It also encompasses build- from the CDF pilots and their approaches. CDF countries, ing sustained government and civil service commit- which will adopt different approaches built around the ment to performance orientation. four core CDF themes, can share lessons about the Incentives are crucial to ensuring a performance strengths and weaknesses of these approaches. orientation (Picciotto and Weisner 1998). To under- Experience shows that countries must demand a stand a country's incentive frameworks, one must focus on results (UNDP 1997); donor-driven efforts are diagnose its institutions and consult closely with its not enough. Donors can "sell" a greater performance government (as in Zimbabwe) (Mackay 1998a, 1998b) orientation to countries, but an infrastructure that and, where agreeable to government, with civil society measures performance-through systematic M&E- and the private sector. M&E findings can enhance will founder unless the government accepts the donor's accountability, improve manager performance, and arguments or reaches its own conclusions about the help guide budget allocations (including budget cuts). merits of such an approach (box A6.1). Where government performance is poor, M&E can 61 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX A6.2 POOLING AND JOINT REVIEWS IN GHANA'S HEALTH SECTOR REINFORCE ACCOUNTABILITY G hana's health evaluated and allows an * Incomplete evalua- to the sector. sector demon- emphasis on joint account- tions. In-depth evalu- * Lack of evaluations strates an ability for the donors and ations of selected on the outcomes or innovative approach to the government. problem areas-pub- impact of health donor funding and The government and lic health, institu- ministry programs. reviews of sector the donors conduct a tional care, and * Inability of the performance. Donors joint review of the health systems of manage- review to take full have agreed to pool a sector every year by ment and support- account of the significant proportion of assessing 20 performance had been planned as activities and their funds for this sec- indicators. They also part of the 1998 services provided tor, giving up the right identify priorities for the review, but were not by bodies external to select which projects coming year. Two such completed. to the health minis- to finance in exchange joint reviews have been * Deficiencies in the try-such as other for a voice in develop- conducted since 1997. health ministry's ministries, NGOs, ing sectoral strategy and Although close govern- capacity to collect, and the for-profit allocating resources. ment-donor collaboration analyze, use, and private sector. This pooling approach has been effective, there report performance * A need for greater means that the cost-ef- is still work to be done. monitoring data, dialogue with civil fectiveness of specific The joint review faced relating not only to society, including donor funds cannot be problems such as: the ministry but also the district level. help to improve accountability. Measuring govern- to start with performance monitoring in selected sectors ment performance increases the pressure for a more or ministries, and then extend the approach to others, responsive public sector. In Bangalore, India, for and add measurement tools such as project and example, an NGO surveys citizens on the quality of program evaluation as opportunities allow. municipal government services and the extent of Chile took such an incremental approach (Marcel corruption in delivery (Paul 1998). 1999: 265-325). Initial monitoring identified some RBM has synergies with many types of public problem programs, and this led to questions about sector reform. These include public expenditure man- program outcomes and impact that have since been agement (medium-term expenditure frameworks and answered by in-depth evaluations. performance budgeting), civil service reform (perfor- An emphasis on performance management for mance contracts for senior civil servants), intergovern- projects and programs should be complemented by a mental fiscal decentralization, commercialization and focus on governance and intersectoral synergies. Do- private sector delivery of public services, service nors have attempted to foster good governance and a delivery or beneficiary surveys, participation and the performance culture at the national and ministry "voice" of civil society, and anticorruption efforts that levels. But attempts to strengthen M&E have focused strengthen financial management and oversight bodies. on the project level, typically only to satisfy donor Efforts to foster RBM are thus likely to benefit from requirements. The lack of domestic ownership of M&E close links with related public sector reforms. has undermined acceptance of performance measure- A holistic approach to governance encourages ment for sound governance. Moreover, the limited greater performance orientation. A cross-sectoral per- M&E capacity created through donor-driven, project- spective puts pressure on lagging ministries to keep up based activities was often lost once the project was with leading ministries. But experience also suggests completed. Ghana is starting to work with donors, that a holistic approach to RBM will not be easy to including the Bank, to build more robust national and trigger across all ministries. It might be more realistic sectoral approaches to performance management and 62 Annex 6. Managing for Results measurement. Donor adoption of a sectorwide ap- ments now conducted in many countries provide such proach helps this process along (box A6.2). opportunities. Civil society can also influence the The CDF provides an opportunity to involve civil evaluation agenda by identifying government activities society in assessing government performance and in in need of assessment. Box A6.3 demonstrates another discussions about national and sector priorities and type of civil society involvement. progress in achieving development goals. Civil society Donors and governments both have a role to play in has a number of potentially important roles to play. It building the capacities of key stakeholders such as NGOs, can provide a deeper understanding of the constraints the media, and parliaments. These stakeholders need to and limitations on government activities. It can pres- have some familiarity with performance concepts and sure governments and civil servants to improve man- M&E tools to make full use of information on govern- agement of inefficient or ineffective government activi- ment performance; they also need to understand its ties.' It can use and learn from information on limitations and weaknesses. In South Africa, for example, performance indicators and the findings of evaluations an NGO works directly with parliament on a fee-for- and reviews. It can voice views of ordinary citizens, service basis to help it understand and digest information NGOs, and the private sector-the consumers of on the performance of government ministries. government goods and services. The beneficiary assess- BOX A6.3 MALAYSIA: SUSTAINABLE PENANG INITIATIVE in late 1997, the the extent to which citizens ments and made a list of People's Report on Malaysian State of feel they have a voice in key performance indica- Penang, and reviewed Penang initiated the the changes affecting their tors for its topic. The by the state assembly. Sustainable Penang neighborhoods. groups chose indicators The intention is that the Initiative, with support The initiative was based on their rigor, roundtable groups will from the Canadian designed as a pilot relevance, and availabil- reconvene each year to International Develop- project for community ity. In community health review improvements ment Agency, UNDP, indicators. State assembly care, for example, indi- and identify problem and the UN Economic members, state govern- cators included cancer areas. and Social Commission ment officials, business rates, infant mortality, The initiative for Asia and the Pacific. representatives, NGOs, quality of health care, appears to have been The initiative is a long- community-based organi- levels of patient satisfac- successful in generat- term pilot response to zations, academics, and tion, and the ratio of ing stakeholder community concerns concerned individuals health care facilities and ownership of evalua- about the costs of organized into five professionals to the popu- tion, showing that Penang's rapid develop- roundtable groups of 50 lation. The groups used RBM is possible at ment: the sustainability people each. The groups data to identify trends the community level. of growth; its environ- focused on the economy, and implications for the The initiative also mental impact and the environment, social sustainability of develop- demonstrates the po- consequences for health; justice, culture, and popu- ment, and their relation- tential for achieving the distribution of gains lar participation, ship to other indicators a collaborative part- from development; the During one- and two- or issues. nership approach impact of growth on day facilitated discus- In late 1998, the among diverse groups Penang's values, tradi- sions, each group roundtable findings were and interests. tions, and heritage; and prepared vision state- published in the first 63 ANNEX 7. PARTNERSHIP AND STRATEGIC SELECTIVITY' Since its emergence some three decades ago, the society and with external partners." In a further concept of partnership for development has won broad refinement, the World Bank's Partnership and Develop- acceptance. Donors and beneficiary countries are ex- ment (World Bank 1998i) defines partnership as a pected to participate jointly in analyzing macroeco- collaborative relationship in which parties jointly nomic issues, developing action plans for national develop and agree on objectives and expected out- priorities, and strengthening or building local institu- comes, division of labor and responsibilities, rights and tions. Ultimately, partnership is expected to transform accountabilities, and capacity development to ensure beneficiary countries from aid recipients to aid manag- that weaker members are able to participate fully.' ers. Some governments-Bangladesh, Bolivia, Ethio- These notions are also embodied in the CDF. pia, Ghana, Tanzania, Uganda, and Vietnam-have taken the initiative in setting the agenda for local aid Effective Country-Led Aid Coordination meetings and sharing a lead role in annual or biannual This notion of partnership implies a matrix such as that apex Consultative Group (CG) meetings. They are also shown in table A7.1, with country-driven arrangements participating more intensively in preparatory activities expected to make the greatest contribution to develop- and analytical work, such as country economic memo- ment effectiveness.4 High country commitment and randa and sector reports, that affect aid coordination institutional capacity are likely to be associated with exercises. country-driven aid coordination arrangements, and less The idea of partnership was first articulated in the country commitment and institutional capacity with 1969 report of the Commission on International Devel- donor-driven arrangements. Where capacity is high but opment, Partners in Development, commissioned by commitment is low, the matrix implies joint sponsor- World Bank President Robert MacNamara. The report ship, or intensive donor involvement in tandem with called for a "new partnership" based on reciprocal the government to minimize resource misallocation rights and obligations. Not until 1995, however, was under flagging or missing policy commitment. Where partnership officially made one the of Bank's six commitment is high but capacity is low, country-driven "guiding principles."2 coordination is possible if institutional capacity is Partnership gained additional prominence in strengthened. Institutional capacity includes organiza- DAC's seminal document, Shaping the 21" Century: tional capability and the rules and incentive structures The Contribution of Development Co-operation governing policy formation and resource allocation. (OECD and DAC 1996). It put forward internationally Incentives include both the internal incentive (and agreed goals as part of a new partnership agenda for disincentive) structures of a bureaucracy and the poverty reduction that calls on developing countries to external incentives resulting from market competition and from the various mechanisms for citizen voice or TABLE A7.1 COUNTRY CHARACTERISTICS AND participation (World Bank 1998i: 9-10).s AID COORDINATION Donor Characteristics Instituion II Ciuntrn cotnminent How can a country move to the country-driven quad- cap.:i High I o" rant? Depending on where a country stands, the answer High (-untr -Jmdnri.eii I.u-spow.-r,hip is to strengthen country commitment, strengthen capac- Loi (Lintri -drrL,n D oinor-d ri %n. ity, or do both. But the characteristics of donors and 1Ith 11111ntatiiu. international agencies also affect the ease of moving to strcrI nhnin Ieffective country-led aid coordination arrangements. The most advantageous case for country-led aid coordination commit to an effective policy environment for pro-poor is that of a small number of donors, all with a high growth, and for donors to increase financial support for development orientation (table A7.2). As the number of such policies. The report gave highest priority to donors increases, the environment for aid coordination development strategies grounded in an "open and becomes more challenging, and as nondevelopment collaborative dialogue by local authorities with civil motives for providing aid (such as commercial interests) 65 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS TABLE A7.2 DONOR CHARACTERISTICS AND THE development cooperation, the donor normally imposes AID COORDINATION ENVIRONMENT conditions on the recipient, which must abide by them. Nurhcr -if)~ Dccopment orientation The relationship between the two is far from symmetrical. donor.. and aLencic- High Lo% It is embedded in an institutional culture of donor institutions that views those who offer aid as belonging to a fundamentally different league from those who receive [-. Chlknimiri \cr% [ittII[11 it. Changes in rhetoric have not altered the aid culture, even under the pressure of determined leadership. increase, effective country-led coordination becomes more difficult. While the end of the Cold War has reduced Governance Constraints the dominance of the ideological motive for providing Guided by its Articles of Agreement, the Bank deals aid, geopolitical considerations are still at work.6 primarily with governments. Partnership has meant, for Effective aid coordination involves a process lead- all intents and purposes, an effort to improve relationships ing to an agreement by the recipient country, donors, with government borrowers. This has constrained the and international agencies that all partners will accept ability to form ties with the private sector and civil mutual responsibility for achieving development out- society. In recent years the Bank has expanded the comes, along with distinct accountabilities. For the meaning of partnership to include NGOs, civil society recipient country, accountabilities include making a organizations, and the private sector. But governance commitment to sound policies and effective institu- constraints impose limits on such partnerships, particu- tions. For the donors, accountabilities include adopting larly where NGOs and other civil society bodies operate a development orientation, achieving coherence with under the government's wings or on the edge of legiti- country development priorities and selectivity along macy. True partnership, based on open and free dialogue, the lines of comparative advantage (thus addressing the is not possible in these conditions. As the Bank's General "numbers" issue), providing effective capacity building Counsel pointed out, "the Bank does not have a legal assistance to level the playing field among partners, right to oversee the governance of its borrowing members and supporting country-led efforts to achieve these or to participate in such governance. It is neither a world objectives. policeman, nor a world government." Bilateral donors are far less restricted in their ability to offer advice on The Limits of Partnership political issues. There are limits to the extent to which the realities of partnership match the vision. These include tensions Strategic Selectivity between harmonization and accountability and be- Partnership and selectivity are inseparable, both con- tween conditionality and partnership, as well as issues ceptually and operationally, which makes it difficult to of governance constraints and strategic selectivity. distinguish selectivity from partnership without seri- ously compromising partnership.' Strategic selectivity Harmonization and Accountability may be exercised on the basis of several characteristics, Harmonization of donor policies and procedures does not each with distinct implications: remove the need for recipient accountability for their use of funds to meet the donors' fiduciary responsibility * Need. Donors may disagree with the government toward their governing bodies or parliaments. Account- or each other about which problems, sectors, ability, however, has been one-sided. Donors have not regions, or groups are priorities. Donors may been held accountable to recipients for the methods or focus exclusively on "fashionable sectors," such rationale of the allocation of aid resources or the as health and education, at the expense of predictability of resource availability. The reluctance of complementary investments, such as rural roads. some donors to untie aid illustrates this point. * Comparative advantage of donors. In an ideal world, donors would divide their labor accord- Conditionality and Partnership ing to comparative advantage. In practice there In normal business partnerships, the partners mutually is no ready agreement on what this means. agree to abide by certain conditions and obligations. In Consider the often difficult relationships between 66 Annex 7. Partnership and Strategic Selectivity the Bank and the specialized agencies of the UN, The Road to Partnership: Strategy for Government-Led or among UN agencies themselves. The com- Aid Coordination parative advantage of donor institutions can Partnership requires strong institutions. It also requires conflict with commercial interests. well-developed skills in negotiation, communication, * Country capacity. Resources should be allocated economic and social analysis, information technology, to the most efficient institutions in the recipient and diplomacy. Participants at a workshop for senior country. Pushed too far, this type of selectivity government officials engaged in aid coordination, spon- can undermine the balanced approach called for sored by OED and the Swiss Agency for Development by the CDF, or force a tradeoff with capacity (SDC) in February 1999, concluded that more investment building and ignore dynamic comparative ad- was needed in the skills required for effective aid vantage. Supporting the creation of project man- coordination, particularly government's ability to design agement units, for example, may undermine and carry out economic and social studies and to analyze overall capacity building. donor studies. * Country performance. Selectivity, in the sense of a The Bank provides considerable funding for capac- reward for work well done, may be an alternative ity building through loans, grants, and trust funds, but to conventional ex ante conditionality, which is an most of it is not woven into a national strategy in the incentive to do a good job. In a donor's global way macroeconomic analysis is. Donor efforts to operations, selectivity implies a bounded applica- strengthen country aid management and coordination tion of the partnership principle: where country capacity have tended to be expensive, supply-driven commitment is lacking, recipients must first qualify and ineffective, and aid coordination activities have for partnership by meeting a minimum level of had little positive effect on capacity or partnership performance. Practical difficulties have arisen with (World Bank 1999m). There is a need for real change. selectivity based on performance. The chairman of Much can be learned from the corporate world about the DAC warns against donors trying to invest in partnerships and strategic alliances, particularly about "winners" and withdraw from "losers." This defining and exploiting comparative advantages (box would require, he warns, "better forecasting ability A7.1). The Bank, in consultation with other donors and than anyone claims to possess" (DAC 1999: 4). affected countries, needs to give force to the concept of BOX A7.1 LEARNING FROM BUSINESS ALLIANCES uch can be ances to increase reach, across geographical loca- cultural fit and part- learned mobilize resources, tions and teams. Corpo- nership orientation, rom the source new knowledge, rations are becoming, in collaborative plan- corporate world about lower overhead costs, effect, multiorganiza- ning and learning, partnerships and strate- increase their responsive- tional or networked orga- dedicated inter- gic alliances, particu- ness to clients, and focus nizations. Successful stra- organizational com- larly about defining on core businesses and tegic alliances depend on munication, invest- and exploiting competencies. They are a clear strategic fit and ment in mutually comparative advan- using the Internet for mutual agenda, incre- beneficial goals, rein- tages among partners. collaborative planning mental processes of forcement of partner- Corporations are enter- and facilitating learning increasing involvement ship successes, and a ing into strategic alli- and action on projects and sharing information, sense of "co-destiny." Source: Buono 1997; Hamel and Prahalad 1994; Johnston and Lawrence 1988; Kanter 1994. 67 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX A7.2 CAPACITY BUILDING ASSISTANCE TO KAZAKHSTAN AND VIETNAM FOR AID COORDINATION In 1992 the Bank its numerous reorganiza- agency. The grant was tributing to a "pro- gave an Institu- tions, there was consider- followed by a technical cess" for improved aid tional Develop- able delay in contracting assistance loan to support coordination. ment Fund grant to the long-term adviser. aid coordination and The project evalua- Kazakhstan's aid coor- Once on board, however, management. tion report noted that dination unit, the the adviser established a The UNDP initiated a the project strength- National Agency for strong working relation- three-year technical assis- ened the capacity of Foreign Investment ship with the staff and tance project in Vietnam FERD and helped the (NAFI), to improve its created a database of in 1993, also providing a government prepare system of aid manage- externally financed tech- long-term adviser, short- documentation for the ment and coordination. nical assistance and term consultants, train- first donor meeting The grant was investment projects. The ing, study tours, and and subsequent CG designed to finance the adviser worked with some equipment for the meetings. It did not, costs of a long-term aid NAFI staff to prepare aid coordination unit, the however, achieve its coordination adviser, a documents for the first Foreign Economic ambitious goals for number of short-term and second CG meetings. Relations Department overall improvement consultants, training Both the management (FERD). Although housed of the aid management and study tours for and staff of NAFI cred- administratively in the system. The World agency staff, and ited the adviser with Central Planning Com- Bank has taken over procurement of equip- strong knowledge of the mission, the project aid coordination ment and facilities for subject and success in established ambitious responsibility from the the aid coordination transferring knowledge goals for improving the UNDP. unit. and skills in coordinating overall aid management The similarity of Because of NAFI's foreign aid and establish- system, from line minis- Bank and UNDP expe- lack of familiarity with ing a solid foundation for tries down to the provin- riences in these two Bank procedures and the aid coordination cial level, and for con- cases is striking. Source: World Bank data; UNDP 1996, p 16. country responsibility, putting the country in the than moving along parallel and duplicate trails. Efforts driver's seat. The Bank should work with the govern- by the Bank and the UNDP to strengthen aid coordina- ment and other development partners to formulate a tion capacity in two distinctly different countries multiyear strategy to support countries in assuming underscore the potential for greater success through leadership of CG and consortium meetings and in joint efforts (box A7.2). building aid coordination capacity at the local level. A related issue is the location of CG meetings and The Bank and the UN should jointly make capacity other apex-level aid coordination meetings. Most CG building and aid coordination part of the national meetings take place in donor capitals-especially strategy dialogue with countries at apex aid coordina- Paris-but a growing number are being held in recipi- tion meetings. Both the Bank and the UN call for ent countries, as recommended by the Partnership ownership, harmonization, cost-effectiveness, subsid- Paper and by the 1998 evaluation of the Special iarity (dispersal of authority as close to the grassroots Program of Assistance for Sub-Saharan Africa (World as government allows), and institutional and indi- Bank 1998k: 17). Other recommendations to encourage vidual commitment and discipline. Partnership would broader participation include having the host govern- be stronger and the burdens on government lighter if ment chair aid coordination sessions and frame the the Bank and the UN could follow a single path, rather agenda. Box A7.3 highlights the advantages and 68 Annex 7. Partnership and Strategic Selectivity BOX A7.3 ADVANTAGES AND DISADVANTAGES OF IN-COUNTRY AID COORDINATION MEETINGS Advantages Disadvantages * Participation by the highest levels of govern- * Possible downgrading of the level of donor ment leadership representation (but not necessarily across the * Increased sense of ownership by board) government and subsequent commitment * Government preference in some cases for * Greater public scrutiny and collaboration with meeting outside the country because of the civil society and private sector sensitivity of CG issues * Reduced perception of donor domination * Less free expression of opinions by local donor * Broader participation by government representatives, who also may not have the agencies power to commit their governments * First-hand view of the partner country for * Loss of ability to take advantage of Paris's donor headquarters representatives. central location and Bank facilities designed expressly to meet the needs of CG meetings. disadvantages of in-country aid coordination meetings. resources to assist its members in the building of long- None of this is to suggest that the Bank disengage term capacity. For the country, the challenge of from the aid coordination process or from efforts to responsibility for leadership in aid coordination should strengthen its country programs. On the contrary, by strengthen commitment and ownership-and ulti- encouraging countries to exercise leadership in aid mately development outcomes. coordination, the Bank will be better able to free 69 ANNEX 8. INTERNATIONAL PUBLIC GOODS AND AID EFFECTIVENESS As global integration deepens, the number of develop- and cultural heritage, cross-border (if not global) environ- ment problems that require supranational policy re- mental problems, migrants and refugees, and many forms sponses grows. These cross-border challenges arise of illicit transboundary behavior. Another problem-the from combinations of market, government, and sys- risk of protectionist backlash-could crop up in industrial temic failures. Thus, a new development frontier is countries in response to rising domestic inequality, labor emerging, and with it a new role and complementary insecurity, and disagreements over appropriate environ- rationale for development assistance. A country focus mental standards. will continue to be important, but official development Solutions to these problems are similar to interna- finance will likely be needed to meet a large deficit in tional public goods (box A8.1). Their reach can be international public goods. global (like global public goods), regional (affecting a Global forces (including technological change) are subcontinent, continent, or hemisphere), or local (af- creating not only new, far-reaching opportunities, but also fecting a small number of neighboring countries). a host of potential problems-"international public Solutions may affect industrial countries, sets of devel- bads." These problems include the possibility of financial oping countries (in the case of some tropical diseases), contagion, the spread of disease, the loss of biodiversity or, increasingly, both poor and rich countries. BOX A8.1 PUBLIC GOODS PRIMER public good of externality. An exter- property rights, law and (toll roads), which are is acommod- nality occurs when the order, and, more gener- excludable but ity, service, welfare of an agent ally, functional social nonrival, and com- or resource whose depends directly not and political cohesion mon pool goods or consumption by one only on what that agent are development common property user does not reduce does, but also on what resources with a public (groundwater or its availability to others do or fail to do. good character. Societ- mineral deposits), other users. Public The motivation to ies at different levels of which are nonexclud- goods are nonrival in invest in public goods development distinguish able but rival. Com- consumption and arises from the desire to themselves by their mon property tends to nonexcludable; that bring out positive exter- accumulated wealth of be overused in the is, the provider of the nalities or to correct or public goods (the non- absence of rules. As good cannot prevent compensate for negative physical components with pure public someone from con- ones. Collective (or referred to as social goods, the prudent or suming it, regardless government) action is capital by some sustainable use of of whether the user necessary to produce authors). common property is a pays for the good. public goods because Public goods differ matter of collective Because of this char- private solutions often according to their choice. Government acteristic, public fail. In particular, public benefit on a action (for example, goods-such as clean markets are unable to continuum between in the form of regula- air, national defense, supply nonexcludable pure public goods and tion) may also be and street lighting- goods. pure private goods. needed to ensure equi- tend to be under- Public goods are Private commodities table and competitive supplied. critical to development. and services are rival access to club goods. A close relative of Rules and standards, and excludable. In be- the concept of a pub- infrastructure, institu- tween these polar oppo- lic good is the notion tions in public service, sites are club goods 71 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX A8.2 PROGRESS IN CONTROLLING RIVER BLINDNESS O nchocersiasis, highly successful, with corporations, founda- comparative advan- or river an overall economic rate tions, and NGOs. Inter- tages that each partner blindness, is of return of 20 percent- vention focuses on vector brings and to allocate a painful and debili- 600,000 cases have been control, drug distribu- responsibilities accord- tating disease that prevented, 34 million tion, and capacity build- ingly, with precise infects 20 million people have been ing in national health objectives. Leadership, people in Sub-Saharan protected, 5 million programs. personal relationships, Africa and places 120 years of productive labor The program demon- and trust are crucial. million others at risk have been added, and 25 strates that partnerships All partners must of contracting the dis- million hectares of land are complex and difficult perceive a payoff to ease. A multipartner have been freed of the to establish and main- participation. Credit Onchocersiasis disease. Partners in the tain, particularly given should be shared liber- Control Program has program include govern- the mix of corporate ally and frequently. operated in West ments, local communi- cultures. To replicate the Flexibility and compro- Africa for 25 years. ties, international organi- program's success, it is mise are fundamental. The program has been zations, bilateral donors, important to identify the Aid Effectiveness Linkages The CDF addresses several defects in the (nationally Several strategic and process-related considerations focused) mainstream aid delivery pattern, including: that change the focus and resources of international public goods could change the effectiveness of aid. * A tendency toward top-down and spending- First, if a development problem such as a banking crisis oriented approaches at the expense of local arises from transnational spillover, it cannot be re- capacity building and ownership solved by the traditional approach of an uncoordinated * Fragmented aid delivery with large numbers of set of national overtures. "In contrast to conventional insufficiently coordinated sources of assistance foreign aid that focuses on individual countries, and projects relative to absorption capacity transnational problems demand a multicountry, prob- * Questionable aid allocation patterns. lem-oriented approach to development cooperation" (Gwin 1999). Although intervention at the national The principles by which the framework seeks to level may also be necessary, sector-specific national increase development effectiveness-such as partner- action is usually most effective when embodied in a ship; country ownership and involvement; a results multicountry framework. focus; and a long-term, holistic view-are precisely Second, systemic crises-international conflict; those that make for successful, global, collaborative cross-border environmental, financial, food, or natural programs. resource crises; and disruptions brought about by Two long-standing international public goods ven- excessive social inequality-can destroy foreign aid- tures, the Onchocerciasis Control Program in West financed development achievements. Crises can Africa (box A8.2) and the Consultative Group on quickly spoil the fruits of past economic growth in International Agricultural Research (CGIAR), confirm emerging markets and commodity-exporting develop- the potential effectiveness of highly focused ing countries. Some of this growth may have been multicountry and multiactor partnerships. Both pro- financed by foreign assistance. Such a setback would grams have existed since the early 1970s and show force future official flows to affected countries into less exceptionally high rates of return to investment. The favorable initial conditions. success of these programs demonstrates that, with Third, the aid process can undermine its effectiveness. proper leadership, issue-oriented international cam- 72 Annex 8. International Public Goods and Aid Effectiveness paigns can produce strong results. For example, the More thought needs to be given to the interaction program to eradicate river blindness appears to have among international and national public goods. fostered a disciplined process and motivated partici- Transnational policy should address the synergies pants to persevere. among investments in development resources at the An assessment of the CGIAR reaches similar international and national levels, which could be a new conclusions (Anderson and Dalrymple 1999). (This dimension of aid coordination. The implication is not assessment cautions, however, that "in reporting re- that investments in international public goods should search accomplishments, the basic problems are aggre- wait until conditions are right for their application in gation and attribution.") The CGIAR-a partnership of most countries, but that conditions on the ground must governments, multilateral institutions, and founda- be nurtured so that international development goods tions-has catalyzed international collective action in can be put to use. Current practices, even among such the service of world food security. The program is successful programs as the Onchocerciasis Control devoted to sustainable crop improvement, especially Program and the CGIAR, could do more to promote for staple foods consumed by the poor. The CGIAR has national public goods. Aid coordination partnerships generated impressive global externalities and can be hold the promise that improvements to domestic pro- viewed as a model of transnational standard-setting grams will reveal that norms and standards can be and governance in its field. It has often been suggested superior alternatives to conditionality. that something similar be created to address tropical diseases or, more specifically, the underresearched and Institutional Implications and Outlook underattended "orphan" diseases that account for most The domestic and international dimensions of the of the disease burden in poor countries. A recent move development challenge are becoming increasingly in- in that direction is the Global Forum for Health terrelated. Issue-focused international partnerships for Research, established in 1997 as an independent, development are mushrooming and are likely to con- multiactor foundation hosted by the World Health tinue to grow in number. Both domestic and interna- Organization to correct the "10/90 disequilibrium." tional measures can help address undesirable spillover (Only 10 percent of annual global spending on health and secure ownership for domestic reform and increase research in the private and public sectors is devoted the effectiveness of sectoral interventions at the na- to the health needs of 90 percent of the world's tional level. Reform becomes less dependent on intru- population.) sive conditionality as the policy gap between the Based on examples from the Onchocerciasis Con- regional and global levels closes. It is easier to trol Program and the CGIAR, it appears that mission- encourage voluntary compliance with generally ac- oriented transnational networks that address highly cepted international standards than to introduce top- visible and urgent human priorities can serve as a down conditionality. However, voluntary compliance rallying device for the coordination of contributing requires that developing countries be adequately repre- partners. Selectivity is ensured up-front through the sented in the partnerships that develop the standards choice of public goods to be created. Fragmented and that coordinated national capacity and institution behavior and free-riding are hindered by the visibility building strengthen the ability to implement sectoral of the program and the public support associated with reform. Unfortunately, there is a deficit in both areas. its objectives. Shared learning occurs (or should occur) The CDF provides a formula to address these as a matter of course. Motivation and coordination challenges. The framework's key tenets of inclusiveness among donors and partners are easier to achieve, and and wholeness should be respected when partnerships conditionality and allocation of donor funds are less at the regional and global levels are being built. With contentious than in multiobjective, multisector country guidance from the framework, prioritization at the assistance programs. There is also less scope for national level can help identify areas where interna- politics to interfere with technical integrity. Developing tional programs are needed to supplement national countries should be strengthened by the presence of efforts. Capacity building in national and local institu- these programs in the Networks-a presence that is tions (state and nonstate) is critical to the effective indispensable to finding policy solutions to cross- implementation of coordinated efforts. border externalities within countries. 73 ANNEX 9. RESULTS-BASED MANAGEMENT GLOSSARY This Glossary defines key terms as they are used in this IMPACTS are the long-term effects and changes that report, and builds on definitions in the report "Gover- result from the outcome of an activity. They are the nance in Transition" (OECD 1995). ultimate criterion for development effectiveness. For example, an impact of a microenterprise development ACCOUNTABILITY is the obligation to demonstrate program might be an "increase in the rate of employ- and take responsibility for performance in light of ment." agreed expectations. It can take place in relationships other than the hierarchical, even when there is no INPUTS are the total resources available to carry out actual "conferring" of responsibility. With a move an organization's activities, including the material toward a partnership model of programming and goods, financial resources, and human time and effort. delivery., this new concept allows for mutual account- ability, and thus a more mature relationship between OBJECTIVES usually serve three different functions: to the Bank and the borrower or other codeliverers. In describe the future the organization is trying to achieve order for such a definition to be effective, partners must and give guidelines for the organization's activities; to jointly clarify and set goals and responsibilities; perfor- justify the organization's existence; and to provide the mance expectations that are balanced by the commen- basis for evaluation. Ideally, objectives and strategies surate resources of each party; credible reporting form a shared, consistent, and integrated hierarchical mechanisms to demonstrate performance achieved and system, moving from a general vision to a more what has been learned; and reasonable review and specific and concrete direction for organizational adjustment systems to ensure that feedback on the activities. performance achieved and difficulties encountered can be recognized and corrected as necessary. OUTCOMES are the immediate effects and changes achieved in relation to objectives (for example, using EFFECTIVENESS refers to the extent to which objec- fewer resources compared with plans, previous perfor- tives (of an organization, policy, or program) are mance, or the performance of other organizations). The achieved, or the relationship between the intended and outcome of a health publicity campaign might be a 5 actual effect of outputs in the achievement of objectives percent increase in awareness among those targeted. (for example, the extent to which the condition of hospital patients improves as a result of treatment). OUTPUTS are the direct products of an organization's activities in goods or services (for example, number of EFFICIENCY refers to the relationship between re- training persondays by type of training course). This sources (inputs) used and outputs produced (for ex- says nothing about the actual outcome (such as skills ample, nurse hours per occupied hospital bed day). An absorbed, or whether the skills helped gain long-term efficient activity maximizes output for a given input, or employment). minimizes input for a given output. Efficiency mea- sures take the form of output-input ratios (productivity) ORGANIZATIONAL CULTURE can be described as the and expenditure-output ratios (unit cost). shared understanding and interpretation of the world that emerges in an organization when its members interact EVALUATION is the assessment, in as systematic and with each other and their surroundings. The culture objective a manner as possible, of an ongoing or provides the basis for the informal aspects of organiza- completed project, program, or policy and its design, tional life-values, attitudes, and behaviors. The best implementation, and results. An evaluation should way to energize an organization is to create and provide information that is credible and useful, en- administer culture. What makes a manager a good abling the incorporation of lessons learned into the manager is her or his ability to foster culture. decisionmaking process. 75 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS PERFORMANCE INDICATORS are used to proxy RESULTS-BASED MANAGEMENT (RBM) is an ap- quantitative measures when output or performance is proach to management that has been adopted by many not directly measurable. They do not necessarily cover leading private corporations and government adminis- all aspects of performance, but they can provide trations worldwide, for the purpose of providing a relevant information toward the assessment of perfor- coherent framework for strategic planning and man- mance (for example, qualifications obtained through a agement based on learning and accountability in a training scheme, or hospital admission rates for infec- decentralized framework. All RBM systems are charac- tious diseases). terized by the following features: * Clear corporate goals and objectives PERFORMANCE MEASUREMENT is the compara- * A performance measurement system focusing on tive assessment of policy outcomes, outputs, and results inputs; performance measures are most useful when * A learning culture grounded in evaluation used for comparisons over time or among units per- * Stakeholder participation at all stages of pro- forming similar work. gram design and implementation * Clear accountabilities in a decentralized frame- REACH refers to the process of client orientation. It work means: * Links among results, planning, and resource * Identifying the client of the program or service, allocation and the target group within the client group (such * Client orientation (reach). as older women or poor women) * Identifying results desired in terms of client needs THE RESULTS CHAIN reflects the RBM framework- or problems to be solved from inputs to outputs, through reach (client/benefi- * Designing indicators that can be tested by client ciary orientation), to outcomes, and ultimate impacts. satisfaction or changes in client behavior. 76 ANNEX 10. MANAGING DEVELOPMENT EFFECTIVENESS: AN OVERVIEW FROM THE CODE CHAIRPERSON Despite the aftershocks of the 1997 financial crisis, the flexibility, along with new skills and attitudes, a listening quality of the active portfolio improved significantly and adaptive mode, and far greater strategic selectivity in over the past year, due in part to an increased number cooperation with partners. of poor-performing closures, a significant achievement. Currently, 20 percent of commitments and 19 percent of Improving Performance Management Tensions active projects are considered at risk, by the end of Under the Strategic Compact, considerable progress FY99, compared with 21 percent and 25 percent, has been made in clarifying the Bank's values, reorient- respectively, a year earlier. The large reduction in ing its mission, and realigning its organization and active projects at risk reflects not only a broad-based skills. At the operational level, the existing set of improvement in the estimated prospects of ongoing instruments for portfolio monitoring is proving ad- projects, but also the addition of new projects approved equate. However, the easy gains in portfolio perfor- after the financial crisis and the portfolio cleanup. mance have been realized and further improvements Significant differences remain though across sectors will become increasingly difficult. While it should be and regions. Performance in the EAP and LCR regions feasible for realism and proactivity to hit Strategic have been restored to pre-crisis levels, but ECA perfor- Compact targets, key questions remain on how to mance fell sharply due to the decline of the Russia accelerate the pace of improvements in quality at entry, portfolio. The electric power and energy sector remains supervision, and economic and sector work. Concerns the riskiest in the Bank, while weaknesses in public also exist with respect to the effective management of sector management performance are a matter for safeguard and fiduciary policies, particularly their less concern given the importance of capacity building. stringent application in adjustment lending. Portfolio cleanup in FY98 and FY99 has caused a Managing Operational Risks and Country Management plateauing in quality at exit trends below the Strategic The CDF is an apt response to the need for greater Compact target of 75 percent satisfactory. The ARDE development effectiveness in an environment of stag- reports the proportion of exiting projects with satisfac- nant aid flows and greater public pressure for account- tory outcomes, as evaluated by OED, declined slightly ability and development impact. The changing envi- from a peak of 74 percent in FY97 to 72 percent in ronment has increased the demands on the Bank. In this FY98 and 70 percent for a sample of FY99. context, CDF implementation offers the promise of high Sustainability and institutional development impact for rewards while creating new demands and pressure these projects, while improving, remain far too low. points in an already stretched organization. The ARDE Results for adjustment lending, led by sector-specific highlights the key challenges and tensions, as well as operations, are a notable exception to this stagnant promising approaches that can serve as a starting point performance-reaching 85 percent satisfactory for for addressing them. FY98-99 exits. There is a growing gap between complexity and capacity, confirming the need for Evaluation Agenda increased emphasis on capacity building as proposed Since the new evaluation framework was endorsed by by the President in his Annual Meetings speech. senior management and the Board in 1997, considerable progress has been achieved toward its goals. Completion Individual CDF principles, solidly grounded in develop- reporting has continued to improve, self-evaluation of ment experience, are valid on their own, but it is their analytical and advisory activities has continued to ex- demanding combination that promises to deliver high pand, and evaluation is now better connected to the rewards. The ARDE presents ample evaluation evidence knowledge management system. In contrast, oversight of in support of the individual CDF elements. It also safeguard policy compliance requires continued strength- highlights the challenges inherent in implementing the ening, monitoring and evaluation has made negligible CDF principles in a synergistic way. Promising ap- progress, and program coordination among evaluation proaches to managing these tensions exist, but their and control groups remains weak. Finally, evaluation application will require considerable retooling, increased capacity development in borrowing countries is not being 77 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS given the priority it deserves. In sum, the 1997 evaluation For OED, the priorities are: a) a greater focus on strategy remains relevant, but significant enhancements getting results from performance audits and evaluation in independent and self-evaluation practices are still studies, to be achieved through increased participation needed to fulfill the organizational learning potential of of borrowers and the poor in evaluations; b) promotion the function. of implementation of the ICR reform by the Opera- In particular, four items require priority attention tional Services Board, OCS, and the Regions for in the months ahead: a) review and reform of the increased learning; c) more attention to financial quality assurance system; b) improving monitoring and accountability, social development, institutional as- evaluation through capacity development inside and pects, safeguards, and private sector development; and outside the Bank; c) harmonization of performance d) more effective outreach with respect to evaluation indicators at the project, country, and sector levels; and capacity development, harmonization of evaluation d) improved coordination and user friendliness of methods, coordination of evaluation programs, and evaluation and control programs. evaluation alliances. Jan Piercy Chairperson, CODE 78 ENDNOTES Chapter I provided support since 1985, only 12 had satisfactory perfor- 1. Table A3.2. India and China are excluded because they mance. The East Asia crisis is expected to affect the performance represent exceptional cases; that is, the Bank's performance is ratings in 3 of the 23 countries found to have satisfactory overall likely to have a modest influence on country results (World Bank ratings (Indonesia, Korea, and the Philippines) (Mathieu 1998). 1997g). 2. The donors urged that by June 30, 1991, or at the latest 2. For example, Borrower Ownership of Adjustment Programs before the end of the IDA9 period, environmental action plans be and the Political Economy of Reform (Johnson and Wasty 1993). completed for all IDA recipients, with priority given to countries where major problems have been identified, and that the results be 3. The background papers for the 1999 ARDE are identified by the addition of "BP" following the year of publication. All are listed incorporated into country lending strategies. in the "Background Papers" section of the Bibliography. 3. Other partners perceive a tendency for the Bank to send large missions for too short a time, to produce bulky aide memoires and then disappear, leaving a shell-shocked local government to make sense of the contents and action plans. 1. See Annex 1 for a graphical treatment of the trend Rather than facilitating partnership, this increases the transac- analysis, along with a discussion of outstanding projects. tion costs for clients and partners. 2. Assessments of project characteristics come from OED project evaluations. Demanding refers to the extent to which the Chapter 4 project could be expected to strain the economic, institutional, 1. In size and income level, the group is very close to the and human resources of the government or implementing agency. middle-income countries. But low-income countries and those Complexity refers to such factors as the range of policy and with large populations are overrepresen.ed. The group is fairly institutional improvements contemplated, the number of institu- representative of the Bank Regions, except that Latin America tions involved, the number of project components and their and the Caribbean is underrepresented and South Asia is overrep- geographic dispersion, and the number of cofinanciers. Riskiness resented. refers to the likelihood that the project, as designed, would be expected to fail to meet relevant project objectives efficiently. 2. Investment efficiency is defined as the GDP growth rate 3. Cnsiten wih th trndsin he PPI,outomeperor- divided by the investment-GDP ratio. This is the inverse of the . C t conventional incremental capital-output ratio (ICOR), adjusted mance over the decade is higher when weighted by disbursements, for changes in terms of trade between investment goods and GDP. with more than 80 percent of projects exiting in fiscal 1998 showing satisfactory outcomes. 3. The CAL for Cote d'Ivoire suggests that teachers' wages g s r . were abnormally high relative to per capita GDP. The median 4. This comparison is limited to sectors with at least 10 salary of high school teachers in 1979 was 300,000 CFA francs a projects exiting in FY98-99 and with changes in average outcome month, equivalent to $30,000 a year at 1996 prices. The report of more than 3 percentage points, observed that internal efficiency suffered from high repetition 5. For reference, investment performance levels are largely and dropout rates, and many students were poorly prepared for consistent with Bankwide aggregates. entry into the working world. 6. The Quality Assistance Group's recent quality-at-entry 4. The review examined Bank experience with 83 restructur- assessment suggests similar concerns, with institutional capacity ing operations in 46 countries between fiscal 1980 and 1994 aspects and risk assessment and sustainability receiving two of (Mathieu 1996). the three lowest project subratings (World Bank 1999k). 5. The report concluded that "the twin emphases on policy 7. The sample of countries with more than 10 Bank- reform and privatization, together with a major retreat from supported projects implemented in 1990-99 was isolated and financial intermediation operations (due to their poor results), broken into groups by project performance in institutional devel- also meant that the Bank was left with few instruments (and opment. For the countries in each group a time profile of an often little interest) to deal with institutional issues affecting independent measure of national bureaucratic quality was re- restructuring. This was also the case with issues of technology viewed. The analysis showed a positive time-trend in national development, labor market legislation, tax incentives, investment bureaucratic quality in most countries in the group with high regulations, and trade policy, particularly at the sector and sub- institutional development performance, but no clear pattern in sector levels." countries in the other groups. 6. The review covers 19 poverty assessments and updates (all 8. The sampling strategy provided 50 percent coverage of those done in fiscal 1997 and 1998, and a sample from fiscal 1996). adjustment operations and 33 percent coverage of the larger The 1999 review is a follow-up to OED's first review of poverty group of investment operations. Coverage of both was stratified assessments, conducted in 1996, which examined poverty assess- across sectors through random selection. ments completed through December 1994 (World Bank data). Chapter 3 Chapter 5 1. A 1998 OED review of World Bank assistance to financial 1. For example, "New Lessons from Old Projects: The Workings sector reform found that of 23 countries where the Bank had of Rural Development in Northeast Brazil" (World Bank 1993b). 79 1999 ANNUAL REVIEW Of DEVELOPMENT EFFECTIVENESS Within the framework of flexible program design, the better per- Annex 5 forming activities consistently departed from their original design, 1. As proposed in PREM Note 27 (World Bank 1999j). led by dynamic managers and local involvement (World Bank data). 2. This is supported by PREM Note 25 (World Bank 1999a). 2. That study also identified an inward-focused Bank culture as a source of complexity, a culture that emphasized analytical Annex 6 comprehensiveness, rewarded larger and more visible interven- tions, discouraged risk-taking and making tradeoffs, diffused 1. The Bangalore, India, case cited above is one such accountability, and downplayed the need for external feedback example (Paul 1998). (World Bank data). 3. Some types of social fund projects, such as family plan- Annex 7 ning, are not demanded by the community, although these 1. This annex draws on OED's forthcoming Aid Coordina- activities may have high social returns. The poor in a community tion Study. do nor come forward with proposals because they have limited 2. The other guiding principles were selectivity, client orienta- capacity to propose projects.. tion, results-orientation, cost-effectiveness, and financial integrity. 3. The United Nations articulated a similar framework at about the same time. Also see Robert Picciotto, The Logic of 1. The Bank, IMF, UNDP, and others should exercise caution Partnership. A Development Perspective. (OED, World Bank, in introducing too many planning and programming instruments. September 29, 1998). From a developing country viewpoint, these tools may be top- 4. The 1997 ARDE measured policy performance as a down and confusing foreign concepts and may divert local project-weighted index of three components: inflation, fiscal resources and managerial talents away from building and im- balance, and openness. For institutional quality, it draws on a proving existing local strategy development and planning pro- measure employed by the World Development Report 1997 cesses. Developing countries should have a voice in shaping and (World Bank 1997g), based on a set of responses by foreign harmonizing the design and use of these tools, investors on red tape, the regulatory environment, and the degree 2. For example, Seeing Like a State (Scott 1998). Such of freedom from political pressure (Evans and Battaile 1998: 15, hegemonic planning and social engineering approaches reflect 20). OED evaluative research (Johnson and Wasty 1993) shows little confidence in the skills, initiative, intelligence, and experi- that country commitment and policy performance are closely ence of the beneficiaries. The precision and authority of such related. approaches depended not only on bracketing contingency, but 5. Other characteristics can also have a bearing on the also on standardizing the subjects of development. effectiveness of aid coordination, such as country size and its 3. Some of the recent organizational changes in the Bank may strategic or geopolitical significance. Other things being equal, actually increase the tensions and challenges in implementing the the larger the country and the greater its strategic significance, CDF For example, some budgetary and personnel policies and the the more likely it is that donors will take aid coordination contracting out of technical services may reinforce rather than seriously. But these characteristics are fixed (country size) or alleviate short-term orientation and incentives, promote specializa- nearly impervious to efforts at deliberate modification (strategic tion rather than integration skills, and link budgetary resources more importance). The desire of government officials to maintain the closely to lending rather than recognizing the growing importance of status quo has also been cited as an obstacle to country leadership knowledge, facilitation, and advisory services. (World Bank 1998i: 34). This is part of institutional capacity, broadly defined to include incentive structures. Annex 1 6. Another donor characteristic is diversity of procedural and 1. OED has reviewed all implementation completion reports accountability requirements, which can also impede greater delivered to date by Regional staff, covering 118 of the 277 projects country leadership. However, it is plausible that the greater the exiting the portfolio in fiscal 1999 (43 percent coverage). Comple- development orientation, the greater the willingness of donors to tion reports for the remaining projects exiting in fiscal 1999 are harmonize procedural requirements. The same comment applies expected to be delivered to OED and reviewed by spring 2000. to the internal incentive structures of donor agencies that work against coordination (World Bank 1998i: 34). Several replies to Annex 3 the joint questionnaire sent by UNDP and the Bank to donors mentioned development orientation and procedural diversity as 1. Investment efficiency is defined as the GDP growth rate potential barriers to effective aid coordination. divided by the investment-GDP ratio. This is the inverse of the 7. For a fuller discussion of the subject see ODI, Chapter 2: conventional incremental capital-output ratio (ICOR), but is Partnership and Strategic Selectivity (Maxwell, Simon, Foster, adjusted for changes in terms of trade between investment goods Naschold, and Conway 1999 BP). and overall GDP. 80 BIBLIOGRAPHY This report has drawn on a wide range of World Bank Anderson, J., and D. Dalrymple. 1999. The World documents and numerous outside sources. World Bank Bank, the Grant Program, and the CGIAR: A sources include OED's major studies, Country Assis- Retrospective Review. OED Working Paper Series No. tance Evaluations, and project audits, as well as 1. Washington, D.C.: World Bank. ongoing research. These and other sources are listed Andrews, K. 1987. The Concept of Corporate Strategy alphabetically by author or organization in two Homewood, IL.: Irwin. groups: Background Papers commissioned for this Arboleda, J. 1999. Participatory Country Assistance report and a Selected Bibliography. Strategy in Colombia: A Case Study. Social Develop- ment Papers No. 33. Washington, D.C.: World Bank. Argyris, C., and D.A. Schon 1978. Organizational Background Papers Learning: A Theory of Action Perspective. Reading, The background papers are identified in the text by the MA: Addison-Wesley. addition of "BP" following the year of publication. Aycrigg, M. 1998. "Participation and the World Selected background papers are forthcoming. Bank: Successes, Constraints and Responses: Draft for Discussion." World Bank Development Paper Blackburn, James, Robert Chambers and John No. 29, November. Washington, D.C., World Bank. Gaventa. 1999. "Learning to Take Time and Go Blackburn, J., and Jeremy Holland. 1998. Who Slow: Mainstreaming Participation in Development Changes? Institutionalizing Partnership and Devel- and the Comprehensive Development Framework." opment. London: Intermediate Technologies. Institute of Development Studies, Sussex. Photocopy. Bower, J. L., and Yves Doz. 1979. "Strategy Formula- Branson, William, and Nagy Hanna. 1999. "Policy Re- tion: A Social and Political Process." In Strategic form, Ownership, and Conditionality." Photocopy. Management. Boston, MA: Little-Brown. Crook, Richard C., and James Manor. 1999. "The Po- Buckley, R. 1999. 1998 Annual Review of Develop- tential Contribution of Democratic Decentralization ment Effectiveness. Report No. 19062, OED. Wash- to the Comprehensive Development Framework." ington, D.C.: World Bank. Institute of Development Studies, Sussex. Photocopy. Buono, A. E 1991. "Managing Strategic Alliances: Ellerman, David. 1999. "Helping People Help Them- Organizational and Human Resource Consider- selves: The Challenge of Autonomy-Compatible As- ations." Business and the Contemporary World 3(4): sistance." Photocopy. 92-101. Ferroni, Marco. 1999. "Reforming Foreign Aid: The _. 1997. "Enhancing Strategic Partnerships: Inter- Role of International Public Goods." Photocopy. vening in Network Organizations." Journal of Orga- Fox, James W 1999. "Applying the CDF to USAID nizational Change Management 10(3): 251-66. Experiences." Photocopy. Chambers, R., and G. Conway. 1992. Sustainable Ru- Jones, Stephen, and Andrew Lawson. 1999. "Scaling ral Livelihoods: Practical Concepts for the 21st Up: Issues in Moving from Projects to Programmatic Century. IDS Discussion Paper 296. Brighton, U.K. Aid." Oxford Policy Management. Photocopy. Commission on International Development. 1969. Part- Maxwell, Simon, Mick Foster, Felix Naschold, and ners in Development: Report. New York: Praeger. Tim Conway. 1999. "Operationalizing the Compre- DAC (Development Assistance Committee, OECD). hensive Development Framework: Evidence from 1999. "Overview of the DAC Chair: Staying the Contemporary Research." Overseas Development Course toward Development Results through Partner- Institute. Photocopy. ship." In James H. Michel, ed., Development Coop- eration: Efforts and Policies of the Members of the De- velopment Assistance Committee. Paris: OECD. Selected Bibliography Dasgupta, P. 1997. "Social Capital and Economic Per- Albouey, Y., and R. Bousba. 1998. The Impact of formance." Paper presented at World Bank confer- IPPs in Developing Countries-Out of the Crisis ence: Social Capital: Integrating the Economist's and into the Future: Viewpoint. Washington, D.C.: and the Sociologist's Perspective. World Bank, World Bank. Washington, D.C. 81 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Datta-Mitra, J. 1997. Fiscal Management in Adjust- _. 1999. "Strategic Intent." Harvard Business ment Lending. OED Report. Washington, D.C.: Review 89: 63-76. World Bank. Hanna, N. 1985. Strategic Planning and Manage- DFID. 1998. Sustainable Rural Livelihoods: What Con- ment: A Review of Recent Experience. World Bank tribution Can We Make? Paper presented at the DFID Staff Working Paper No. 751. Washington, D.C. Natural Resources Advisors' Conference, July 1998. . 1991. "The Information Technology Revolu- London. tion and Economic Development." World Bank Dis- 1999. Sustainable Livelihoods Guidance cussion Paper No. 120. Washington, D.C. Sheets Issues. DFID Series. Available from -. 1999. "A Proposed Role for Aid Agencies in http:www.oneworld.org/odi/ Helping Developing Countries to Design National In- Dollar, D. 1999. The Comprehensive Development formation Technology Strategies." Technology Man- Framework and Recent Development Research. agement: Strategies and Applications 4: 299-302. Washington, D.C.: World Bank. Hanna, N., and Sandor Boyson. 1993. Information Dollar, D., and Jakob Svensson. 1998. What Explains Technology in World Bank Lending. World Bank the Success or Failure of Structural Adjustment Pro- Discussion Paper 206. Washington, D.C. grams. World Bank Policy Research Working Paper. Hanna, N., Sandor Boyson, and Shakuntala Washington, D.C. Gunaratne. 1996. The East Asian Miracle and In- Doz, Y., and Gary Hamel. 1998. Alliance Advantage. formation Technology: Strategic Management of Boston: Harvard Business School. Technology Learning. World Bank Discussion Paper Ekbom, A., and J. Bojo. 1997. Mainstreaming Envi- 326. Washington, D.C. ronment in Country Assistance Strategies. Discus- Hirschman, A. 0. 1958. The Strategy of Economic sion Paper 1, World Bank, Environment Group, Af- Development. New Haven: Yale University Press. rica Region. Washington, D.C. . 1970. Exit, Voice, and Loyalty: Resources to Elster, J., C. Offe, and others. 1998. Institutional Design Decline in Firms, Organizations, and States. Cam- in Post-Communist Societies: Rebuilding the Ship at bridge, MA: Harvard University Press. Sea. Cambridge, U.K.: Cambridge University Press. Jayarajah, C., W. Branson, and B. Sen. 1995. Struc- Estrella, M., and John Gaventa. 1998. Who Counts tural and Sectoral Adjustment Operations: World Reality? Participatory Monitoring and Evaluation: Bank Experience, 1980-92. OED Report. Washing- A Literature Review. IDS Working Paper No. 70. ton, D.C.: World Bank. Brighton, U.K.: IDS. _. 1996. Social Dimensions of Adjustment: . World Bank Experience, 1980-93. OED Report. Evans, A., and William Battaile. 1998. 1997 Annual World Bank. Review of Development Effectiveness. Report No. Joh.n, , D.uam W ry 193. 1770, OD. ashngtn, DC.:Word Bnk.Johnson, J., and Sulaiman Wasty. 1993. Borrower 17705ngto, OE,D. Wahing, .C.: Wld B. Ownership of Adjustment Programs and the Politi- Farrington, J., D. Carney, C. Ashley, and C. Turton. clEooyo eom ol akDsuso a 1999. "Sustainable Livelihoods in Practice: Early Applications of Concepts in Rural Areas." Natural per No. 199. Washington, D.C. Johnston, R., and P. R. Lawrence. 1988. "Beyond Ver- Resorce Perpecive No.42.Londn: DI.tical Integration-The Rise of the Value-adding Fukuyama, F. 1995. Trust: The Social Virtues and the Pate." arvardT B usies eViea6()n4 Creaion f Pospeity NewYork Fre Prss.Partnership." Harvard Business Review 66 (4): 94- Creation of Prosperity. New York: Free Press. 101. Gibbs, Christopher, Claudia Fumo, and Thomas Jones, S. 1997. Ethiopia Social Sector Development Kuby. 1998. Nongovernmental Organizations in Programs. Report to ODA. Oxford, U.K.: Oxford World Bank-Supported Projects. OED Report. Policy Management. Washington, D.C.: World Bank. - . 1999. Increasing Aid Effectiveness in Africa? Gwin, C. 1999. "The New Development Cooperation The World Bank and Sector Investment Programs. Paradigm." In ODC Viewpoint. Washington, D.C.: Oxford, U.K.: Oxford Policy Management. Overseas Development Council. Kanter, R. M. 1989. When Giants Learn to Dance: Hamel, G., and C. K. Prahalad. 1994. Competing for Mastering the Challenge of Strategy, Management and the Future. Boston: Harvard Business School Press. Careers in the 1990s. New York: Simon and Schuster. 82 Bibliography 1994. "Collaborative Advantage: The Art of Al- Najmabadi, E, and S. Lall. 1995. Developing Indus- liances." Harvard Business Review 72 (4): 96-108. trial Technology: Lessons for Policy and Practice. Korton, D. 1980. "Community Organization and Ru- OED Report. Washington, D.C.: World Bank. ral Development: A Learning Process Approach." Narayan, D. 1999. "Voices of the Poor." Washington, Public Administration Review 40 (5): 480-511. D.C., World Bank. Photocopy. 1988. "From Bureaucratic to Strategic Orga- OECD. 1995. Governance in Transition: Public Man- nization." In F.F Siy and S.Y. Siy, Jr., eds., Trans- agement Reforms in OECD Countries. Paris. forming a Bureaucracy: the Experience of the Phil- -. 1996. Shaping the 21I Century: The Contri- ippine National Irrigation Administration. West bution of Development Cooperation. Paris. Hartford, CT: Kumarian. _. 1997a. Evaluation of Programs Promoting Kreimer, A., and others. 1998. The World Bank's Ex- Participatory Development and Good Governance: perience with Post-Conflict Reconstruction. Report Synthesis Report. Paris. No. 17752, OED. Washington, D.C.: World Bank. . 1997b. In Search of Results: Performance Kuhn, Thomas S. 1970. The Structure of Scientific Management Practices. Paris. Revolutions. Vol. II. - . 1999. Developing Country Scoping Workshop Lindblom, C. 1990. Inquiry and Change. New Haven: on National Strategies for Sustainable Develop- Yale University Press. ment, November 18-19, 1998. Sunningdale. Mackay, K. (ed). 1998a. Evaluation Capacity Devel- OECD and DAC. 1996. Shaping the 21" Century: The opment in Zimbabwe. ECD Working Paper, OED. Contribution to Development Cooperation. Wash- Washington, D.C. ington, D.C.: OECD Washington Center. 1998b. Public Sector Performance: the Criti- OED. 1997. Lessons of Fiscal Adjustment: Selected cal Role of Evaluation. ECD Working Paper, OED. Proceedings from a World Bank Seminar. Washing- Washington, D.C. ton, D.C.: World Bank. Marcel, M. 1999. "Effectiveness of the State and De- Ohiorheunan, L. 1998. National Long-term Perspec- velopment Lessons from the Chilean Experience." In tive Studies-Lessons from Experience. Prepared for G. Perry and D. Leipziger, eds., Chile: Recent Workshop in Entebbe, Uganda. Policy Lessons and Emerging Challenges. Washing- Paul, S. 1998. Making Voice Work: the Report Card on ton, D.C.: World Bank. Bangalore's Public Service. World Bank Policy Re- March, J.G., and H.A. Simon. 1958. Organizations. search Working Paper No. 1921. Washington, D.C. New York: Wiley. Picciotto, R. 1995. Putting Institutional Economics to Margulis, S., and J. Bernstein. 1995. "National Envi- Work: From Participation to Governance. World ronmental Strategies: Learning from Experience." Bank Discussion Paper 304. Washington, D.C. World Bank Environment Department Dissemina- . 1998. Logic of Partnership. OED Report. tion Notes: 30. Washington, D.C. Washington, D.C.: World Bank. Margulis, S., and T. Vetleseter. 1998. "Environmental _. 1999. Poverty Reduction and Institutional Matters at the World Bank: Annual Review - Fall Change. Inaugural Wolf Lecture. Washington, D.C.: 1998." Report No. 18540. Washington, D.C., World World Bank. Bank. Picciotto, R., and Eduardo Wiesner. 1998. Evaluation Mathieu, N. 1996. Industrial Restructuring: World and Development: The Institutional Dimension. New Bank Experience, Future Challenges. Report No. Brunswick, NJ, and London: Transaction for the 15707, OED. Washington, D.C.: World Bank. World Bank. - . 1998. Financial Sector Reform. Report No. Porter, M. 1980. Competitive Strategy: Techniques for 17454, OED. Washington, D.C.: World Bank. Analyzing Industries and Competitors. New York: Maxwell, S. 1997. "Implementing the World Food Sum- Free Press. mit Plan of Action: Organizational Issues in Multi- _. 1985. Competitive Advantage: Creating and sectoral Planning." Food Policy 22 (6): 515-31. Sustaining Superior Performance. New York: Free Murphy, J. 1997. Mainstreaming Gender in World Press. Bank Lending: an Update. Report No.17169, OED. - . 1996. "What is Strategy?" Harvard Business Washington, D.C.: World Bank. Review, November-December: 61-78. 83 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Purcell, D., and Jock R. Anderson. 1997. Agricultural Subbarao, K., and others. 1997. "Safety Net Pro- Extension and Research: Achievements and Prob- grams and Poverty Reduction: Lessons from Cross- lems in National Systems. Report No. 16851, OED. country Experience." Directions in Development. Washington, D.C.: World Bank. World Bank, Washington, D.C. Rodrik, D. 1999. The New Global Economy and De- Thomas, V., and others. 1999. Quality of Growth. veloping Countries: Making Openness Work. Wash- Background Paper, 1999/2000 World Development ington, D.C.: Overseas Development Council. Report. World Bank, Washington, D.C. Photocopy. Sabel, C. 1994. "Learning by Monitoring: The Institu- UNDP. 1996. Developing Capacities in Aid Coordina- tions of Economic Development." In Lloyd Rodwin tion and Management. Vietnam. and Donald Schon, eds., Rethinking the Develop- _. 1997. Measuring and Managing Results: Les- ment Experience: Essays Provoked by the Work of sons for Development Cooperation. New York. Albert 0. Hirschman. Washington, D.C.: Brookings United Nations. 1998. World Investment Report 1998: Institution. Trends and Determinants. New York. Sachs, J. D., and A. M. Warner. 1995. "Economic Re- Wack, P. 1985. "Scenarios: Uncharted Waters Ahead." form and the Process of Global Integration." Harvard Business Review, September-October 1985: Brookings Papers on Economic Activity 1: 1-118. 73-89. Schumpeter, J. A. 1983. The Theory of Economic De- Williamson, J. 1997. "The Washington Consensus Re- velopment. New Brunswick, NJ: Transaction. visited." In L. Emmerij, ed., Economic and Social Scott, J. C. 1998. Seeing Like a State: How Certain Development into the XXI Century. Washington, Schemes to Improve the Human Condition Have D.C.: Inter-American Development Bank. Failed. New Haven: Yale University Press. Wolf, C., Jr. 1997. Markets or Governments: Choos- Sen, A., and J. D. Wolfensohn. 1999. "Development: ing Between Imperfect Alternatives, 2d ed. Cam- A Coin with Two Sides." May 5. International Her- bridge, MA: MIT Press. ald Tribune. Wolfensohn, J. D. 1997. "Annual Meetings Address: The Challenge of Inclusion." World Bank, Washing- Senge, P. M. 1990. The Fifth Discipline: The Art and Ton C. Practice of the Learning Organization. New York: ton, D.C. Dobe .. 1998. "The Other Crisis: 1998 Annual Meet- DephedA. ings Address." World Bank/IMF Annual Meetings, Shepherd, A. 1999. Evaluation of DFID Support to October 6, 1998. World Bank, Washington, D.C. Poverty Reduction: Lessons from the Evaluation of . 1999a. "A Proposal for a Comprehensive Devel- a Bilateral Program. Working Paper for a World opment Framework." World Bank, Washington, D.C. Bank Conference on Evaluation and Poverty Reduc- _ _. 1999b. "Coalitions for Change." World Bank/ tion, June 14-15, 1999, University of Birmingham. IMF Annual Meetings Address, September 28, 1999. SIDA (Swedish International Development Agency). World Bank, Washington, D.C. 1999. Making Partnership Work on the Ground. World Bank. 1993a. Implementing the World Bank's Workshop Report, Ministry of Foreign Affairs. Strategies to Reduce Poverty: Progress and Chal- Stiglitz, J. 1998. Towards a New Paradigm for Devel- lenge. Report No. 11826. Washington, D.C. opment: Strategies, Policies, and Processes. - . 1993b. New Lessons from Old Projects: The Prebisch Lecture, UNCTAD, Geneva, October 19. Workings of Rural Development in Northeast Bra- . 1999a. Participation and Development: Per- zil. OED Report. Washington, D.C. spectives from the Comprehensive Development . 1994. A Review of World Bank Experience in Paradigm. Speech given February 27, 1999. World Irrigation. Report No. 14908, OED. Washington, Bank, Washington, D.C. D.C. . 1999b. "Whither Reform? Ten Years of Tran- . 1995a. Enhancing the Quality of Life in Ur- sition." Paper prepared for the Annual Bank Confer- ban Indonesia: The Legacy of Kampung Improve- ence on Development Economics, April 29, 1999. ment Program: An Impact Evaluation. Report No. World Bank, Washington, D.C. 14747, OED. Washington, D.C. 84 Biblio graphy - .1995b. Lending for Electric Power in Sub-Sa- . 1998e. Lessons from Experience in HNP. Re- haran Africa. Report No. 14961, OED. Washing- port No. 18642, OED. Washington, D.C. ton, D.C. _. 1998f. Malawi-The World Bank and the Ag- - , 1995c. Poverty Reduction at the World Bank: ricultural Sector. Report No. 17898, OED. Wash- Progress in Fiscal 1994. Report No. 14021. Wash- ington, D.C. ington, D.C. - . 1998g. Mauritania: Second Livestock Project. _. 1996a. Development of Housing, Water Sup- Report No. 18133, OED. Washington, D.C. ply and Sanitation in Nairobi. Report No. 15586, _. 1998h. Paraguay: Community-based Rural OED. Washington, D.C. Water Systems & Development of Village Commit- - . 1996b. Pakistan: On-Farm Water Management tees. Report No. 17923, OED. Washington, D.C. Project and Irrigation Systems and Rehabilitation . 1998i. Partnership and Development: Pro- Project. Report No. 15863, OED. Washington, D.C. posed Actions for the World Bank. World Bank _. 1996c. Poverty Reduction at the World Bank: Partnership Group, Strategy, and Resources Man- Progress and Challenges in the 1990s. Report No. agement Discussion Paper. Washington, D.C. 15950. Washington, D.C. _ . 1998j. Recent Experience with Involuntary . 1996d. Reforming Agriculture: the World Resettlement: An Overview. Report No. 17538, Bank Goes to Market. Report No. 1670, OED. OED. Washington, D.C. Washington, D.C. _ . 1998k. The Special Program of Assistance for _. 1997a. Brazil-Secondary and Feeder Roads Africa (SPA): An Independent Evaluation. OED Re- Project and Second Feeder Roads Project. Report port. Washington, D.C. No. 16738, OED. Washington, D.C. _ . 19981. Sri Lanka-Community Water Supply 1 997b. India-A Comparative Review of and Sanitation Project. Report No. 18113, OED. Health Sector Reform in Four States: An Opera- Washington, D.C. tional Perspective. Report No. 15753, OED. Wash- _ . 1998m. The World Bank and the Agricultural ingtor., D.C. Sector in Kenya: An OED Review. Report No. - . 1997c. Korea-Delayed Development of the 18088, OLD. Washington, D.C. Cholla Region: An Institutional Study. Report No. 1621, OL. Wahingon, .C.. 1998n. World Development Report 199 8/99: 1621,7ED. Povergt , ReDCn aKnowledge for Development. Washington, D.C. _. 1997d. Poverty Reduction at the World Bank: .19oAseinAd:WaWrk,ht Progress in Fiscal 1996 and 1997. Report No. .19o sesn i:Wa ok,Wa 17380. Washington, D.C. Doesn't, and Why. Washington, D.C.: World Bank. .1997e. Sri Lanka-Kurunegala and Second . 1999a. "Assessing Borrower Ownership Using Rural Development Projects. Report No. 16418, Reform Readiness Analysis." PREM Note 25, June. OED. Washington, D.C. Washington, D.C. . 1997f. World Bank Support for Small-Scale . 1999b. Case Study of World Bank Activities Enterprises in Ecuador: An Impact Evaluation. Re- in the Health Sector in India. Report No. 19537, port No. 16790, OED. Washington, D.C. OED. Washington, D.C. . 1997g. World Development Report 1997: The _. 1999c. Civil Service Reform: A Review of State in a Changing World. Report No. 17300. World Bank Assistance. Report No. 19599, OED. Washington, D.C. Washington, D.C. . 1998a. The Brazil Health System. Report . 1999d. Development Effectiveness in Health, 18142, OED. Washington, D.C. Nutrition, and Population: Lessons from World . 1998b. East Asia: The Road to Recovery. Re- Bank Experience. Report No. 19266, OED. Wash- port No. 18457. Washington, D.C. ington, D.C. . 1998c. The Impact of World Bank Support to _. 1999e. Evaluating Health Projects: Lessons the HNP Sector in Zimbabwe. Report No. 18141, from Literature. World Bank Discussion Paper No. OED. Washington, D.C. 356. Washington, D.C. _. 1998d. Jamaica Country Assistance Note. Re- _. 1999f. "Evaluation of Social Fund Projects." port No. 19356, OED. Washington, D.C. OED. Washington, D.C. Photocopy. 85 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS _. 1999g. Health Care in India: Learning From Experience. OED Precis 187. Washington, D.C. _. 1999h. Health Care in Mali: Building on Community Involvement. OED Precis 188. Wash- ington, D.C. . 1999i. Higher-Impact Adjustment Lending (HIAL): Initial Evaluation. Report No. 19797, OED. Washington, D.C. . 1999j. Lessons from Large Adjustment Loans. PREM Note 27, August. Washington, D.C. . 1999k. Poverty Assessments: A Follow-up Re- view. Report No. 19630, OED. Washington, D.C. . 19991. "Poverty Study." OED. Washington, D.C. Photocopy. . 1999m. Review of Aid Coordination and the Role of the World Bank. Report No. 19840, OED. Washington, D.C. . 1999n. Tanzania, Agriculture, and the World Bank: An OED Review. Report No. 18111, OED. Washington, D.C. _. 1999o. World Development Report 1999: Poverty and Development 1999/2000. Washington, D.C. 86 OPERATIONS EVALUATION DEPARTMENT PUBLICATIONS The Operations Evaluation Department (OED), an Ordering World Bank Publications independent evaluation unit reporting to the World Customers in the United States and in territories not Bank's Executive Directors, rates the development served by any of the Bank's publication distributors impact and performance of all the Bank's completed may send publication orders to: lending operations. Results and recommendations are reported to the Executive Directors and fed back into The World Bank the design and implementation of new policies and P.O. Box 960 projects. In addition to the individual operations and Herndon, VA 20172-0960 country assistance programs, OED evaluates the Fax: (703) 661-1501 Bank's policies and processes. Telephone: (703) 661-1580 Summaries of studies and the full text of the Pricis The address for the World Bank publication database and Lessons & Practices can be read on the Internet at on the Internet is: http://www.worldbank.org (select http://www.worldbank.org/html/oed publications/project info). E-mail: pic@worldbank.org How To Order OED Publications Fax number: (202) 522-1500 Operations evaluation studies, World Bank discussion Telephone number: (202) 458-5454 papers, and all other documents are available from the World Bank InfoShop. The World Bank InfoShop serves walk-in customers Documents listed with a stock number and price only. The InfoShop is located at: code may be obtained through the World Bank's mail order service or from its InfoShop in downtown 701 18th Street, NW Washington, D.C. For information on all other docu- Washington, DC 20433, USA ments, contact the World Bank InfoShop. For more information about this study or OED's All other customers must place their orders through other evaluation work, please contact Elizabeth their local distributors. Campbell-Page or the OED Help Desk. Ordering by e-mail Operations Evaluation Department If you have an established account with the World Partnerships & Knowledge Programs (OEDPK) Bank, you may transmit your order by electronic mail E-mail: ecampbellpage@worldbank.org on the Internet to: books@worldbank.org. Please in- E-mail: eline@worldbank.org clude your account number, billing and shipping Telephone: (202) 473-4497 addresses, the title and order number, quantity, and Facsimile: (202) 522-3200 unit price for each item. 87 OED STUDY SERIES 1998 Annual Review of Development Effectiveness Agricultural Extension and Research: Achievements and Problems in National Systems Bangladesh: Progress Through Partnership Developing Towns and Cities: Lessons from Brazil and the Philippines Financial Sector Reform: A Review of World Bank Assistance Fiscal Management in Adjustment Lending India: The Dairy Revolution Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Mainstreaming Gender in World Bank Lending: An Update Nongovernmental Organizations in World Bank-Supported Projects: A Review Paddy Irrigation and Water Management in Southeast Asia Poland Country Assistance Review: Partnership in a Transition Economy Reforming Agriculture: The World Bank Goes to Market The World Bank's Experience with Post-Conflict Reconstruction Zambia Country Assistance Review: Turning an Economy Around Proceedings Lessons of Fiscal Adjustment Lesson from Urban Transport Evaluation and Development: The Institutional Dimension (Transaction Publishers) Monitoring & Evaluation Capacity Development in Africa Public Sector Performance-The Critical Role of Evaluation Multilingual Editions Assessing Development Effectiveness: Evaluation in the World Bank and the International Finance Corporation Appreciation de l'efficacite du developpement: L'dvaluation i la Banque mondiale et a la Socidte financiere internationale Determinar la eficacia de las actividades de desarrollo: La evaluaci6n en el Banco Mundial y la Corporaci6n Financiera Internacional C6te d'Ivoire : Revue de l'aide de la Banque mondiale au pays Philippines: From Crisis to Opportunity Filipinas: Crisis y oportunidades Rebuilding the Mozambique Economy: Assessment of a Development Partnership Reconstruir a Economia de Mogambique www.vworldbank.org/html/oed THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. Telephone: (202) 477-1234 Facsimile: (202) 477-6391 Telex: MCI 64145 WORLDBANK MCI 248423 WORLDBANK Internet : www.worldbank.org E-mail books@worldbank.org ..4 7 0 6 9 780821 347065 ISBN 0-8213-4706-3