keport NO. 1 9905 Annua!Review of Development Effectiveness TOward a ComPrehensive oeve-oPment Strategy lanuary 1 9,2000 operatlons Evaluatlon oePartment @ Document of the Wbrld Bank 日田国国国園国国園国園園自国園国 WORLD BANK OPERATIONS EVALUATION DEPARTMENT QM 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Nagy Hanna with Rangopal Agarwala William Battaile Deepa Chakrapani Ruchira Corcoran Federico Mini 1999 The World Bank www.worldbank.org/htm/oed Washington, D.C. Copyright © 1999 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First edition November 1999 The opinions expressed in this report do not necessarily represent the views of the World Bank or its member governments. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. The World Bank encourages dissemination of its work and will normally grant permission promptly. Permission to photocopy items for internal or personal use, for the internal or personal use of specific clients, or for educational classroom use is granted by the World Bank, provided that the appropriate fee is paid directly to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, U.S.A., telephone 978-750-8400, fax 978-750-4470. Please contact the Copyright Clearance Center before photocopying items. For permission to reprint individual articles or chapters, please fax your request with complete information to the Republication Department, Copyright Clearance Center, fax 978-750-4470. All other queries on rights and licenses should be addressed to the Office of the Publisher, World Bank, at the address above, or faxed to 202-522-2422. Photographer: Still Pictures (Father carrying daughter - Nepal) ISSN 1019-4363 ISBN 0-8213-4706-3 Printed on recycled paper. Contents vii Acknowledgments ix Foreword, Prefacio, Prdface xi Executive Summary, Resumen, Rdsume Analytique xxiii Abbreviations and Acronyms 1 1. Toward a Comprehensive Development Strategy 2 Long-Term, Holistic, Balanced 3 Ownership 3 Partnership and Development Cooperation 3 Results, Learning, Accountability 4 Challenges and Tensions 5 2. Challenges at the Project Level 5 Keeping up with Complexity 6 Varied Performance among Regions, Sectors, and Types of Lending 7 Institutional Development and Sustainability Flagging 8 Bank and Borrower Performance Must Improve 9 Project Performance and CDF Principles 11 3. Challenges in Sectoral and Thematic Programs 11 Short versus Long Term 12 Comprehensiveness versus Selectivity 13 Speed versus Broad-Based Ownership 13 Partnership versus Country Capacity and Transaction Costs 14 Accountability for Results versus Local Capacity 17 4. Challenges at the Country Level 17 Addressing Social Issues 18 Addressing Institutional Capacity and Structural Issues 19 A Long Way to Go 20 Managing the Tensions in the CDF 21 Country Cases 23 5. Promising Approaches 23 Learning Process, not Blueprints 24 Managing Complexity by Sequencing 25 Comprehensive Analysis and Selective Actions 25 Sectorwide Approaches 26 Adaptable Conditionality 26 Time to Build Consensus 27 Broadening Participation 27 Information for Accountability and Learning 28 Capacity Building to Manage for Results 28 From Aid Coordination to Development Partnership 28 Linking Global and Country Strategies 31 6. Implications for Development Effectiveness 32 Implications for the Bank iii 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Annexes 33 Annex 1: Project Performance Trends and Outstanding Projects 43 Annex 2: Challenges in Implementing the CDF Principles at the Sector Level 45 Annex 3: Development Performance in Selected Countries 51 Annex 4: Emphasizing People-Centered Development in a Holistic Framework 53 Annex 5: Ownership, Participation, and Conditionality 57 Annex 6: Managing for Results 61 Annex 7: Partnership and Strategic Selectivity 67 Annex 8: International Public Goods and Aid Effectiveness 71 Endnotes 73 Bibliography 73 Background Papers 74 Selected Bibliography Figures 3 1.1 Interaction of CDF Principles 6 2.1 Demandingness, Complexity, Riskiness, and Outcome, by Exit Fiscal Year 18 4.1 The Bank's Contribution to Implementing CDF Principles 33 A1.1 Aggregate Project Performance Index 34 A1.2 Satisfactory Project Outcomes 35 A1.3 Outcome, Sustainability, and ID Impact, by Region and Exit Fiscal Year Group 35 A1.4 Satisfactory Operations by Sector and Exit Fiscal Year Group 36 A1.5 ID Impact 36 A1.6 Sustainability 37 A1.7 Borrower Performance 37 A1.8 Bank Performance 43 A2.1 Sector Scores on CDF 44 A2.2 Sector Scores on Individual CDF Principles 45 A3.1 Disappointing Performance in Reducing Poverty... 45 A3.2 ...And in Promoting Economic Growth 51 A4.1 Sustainable Livelihood Framework Boxes 8 2.1 Updated Determinants Analysis Puts Spotlight on Bank Performance 24 5.1 Challenges and Promising Approaches 54 AS.1 Conditionality Scenarios 57 A6.1 Barriers to Performance Orientation 58 A6.2 Pooling and Joint Reviews in Ghana's Health Sector Reinforce Accountability 59 A6.3 Malaysia: Sustainable Penang Initiative 63 A7.1 Learning from Business Alliances 64 A7.2 Capacity Building Assistance to Kazakhstan and Vietnam for Aid Coordination 65 A7.3 Advantages and Disadvantages of In-Country Aid Coordination Meetings 67 A8.1 Public Goods Primer 68 A8.2 Progress in Controlling River Blindness iv Contents Tables 2 1.1 Three Development Paradigms 38 A1.1 Outstanding and Poor Performers among Recently Evaluated Projects 39 A1.2 Outcome, Sustainability, ID Impact, and Aggregate by Sector, Network, Lending Type/Source, Region, and WDI Income Group for Exit Fiscal Years 1990-93, 1994-97, and 1998-99 (by projects) 41 A1.3 Outcome, Sustainability, ID Impact, and Aggregate by Sector, Network, Lending Type/Source, Region, and WDI Income Group for Exit Fiscal Years 1990-93, 1994-97, and 1998-99 (by real disbursements, FY96US$) 46 A3.1 Relevance of Sample of Evaluated Countries 47 A3.2 Country Performance: Per Capita Growth, Poverty, and Life Expectancy 49 A3.3 Country Performance: Savings 50 A3.4 Country Performance: Investment Efficiency 61 A7.1 Country Characteristics and Aid Coordination 62 A7.2 Donor Characteristics and the Aid Coordination Environment V ACKNOWLEDGMENTS his report was prepared by a team led by Shafik, William E. Stevenson, and Michael Walton. Nagy Hanna, including Ramgopal Agarwala, Pablo Guerrero also provided valuable comments. William Battaile, Ruchira Corcoran, Deepa The report benefited from comments of the DGO Chakrapani, and Federico Mini. The report benefited and the OED management team: Robert Picciotto, from the contributions of OED staff, as well as Wendy Jarvie, Elizabeth McAllister, Gregory Ingram, background papers from the Institute of Development Ruben Lamdany, and Osvaldo Feinstein. Bruce Ross- Studies, the Overseas Development Institute, and Ox- Larson and Meta de Coquereaumont provided editorial ford Policy Management. These contributors and par- assistance. Parveen Moses provided excellent support. ticipants are listed in the Bibliography. Special thanks The report was published by the Partnerships and to Keith Bezanson and Robert Chamber of IDS, to Knowledge group by the Dissemination and Outreach Simon Maxwell and Mick Foster of ODI, and to unit, directed by Elizabeth Campbell-Pag6 (task team William Branson of Princeton University. leader), including Caroline McEuen (editor), Kathy Valuable guidance was provided by an Advisory Strauss and Lunn Lestina (graphics and layout), and Committee including Robert M. Buckley, Carl Dahlman, Barbara Yale (editorial assistant). Christian Delvoie, Shantayanan Devarajan, Ishac Diwan, David R. Dollar, Marco Ferroni, Alan Harold Gelb, Ian Johnson, Geoffrey B. Lamb, Pierre Landell- Director-General, Operations Evaluation: Robert Picciotto Mills, Ashoka Mody, John Page, Anthony J. Pellegrini, Director, Operations Evaluation Department: Elizabeth McAllister Guillermo Perry, Guy Pfeffermann, Ian C. Porter, Manager, Corporate Evaluations and Methods: Wendy jarvie Wolfgang H. Reinicke, Neil D. Roger, Nemat Talaat Task Manager: Nagy Hanna vii Forew ord FOREWORD PREFACIO PRÉFACE Despite the potential benefits A pesar de los posibles beneficios Malgré les avantages potentiels de of globalization and technological de la globalización y del cambio la mondialisation et du progrès change, world poverty has in- tecnolôgico, la pobreza mundial ha technologique, la pauvreté dans le creased and growth prospects have aumentado y las perspectivas de monde avance et les perspectives de dimmed for developing countries. crecimiento se han oscurecido en los croissance s'assombrissent pour les pays Inequality and instability have in- paises en desarrollo. La desigualdad y la en développement. Les inégalités se creased. Aid flows have stagnated. inestabilidad han aumentado. Los flujos creusent, l'instabilité s'accroît et le vol- Public dissatisfaction with the effi- de ayuda se han estancado. La opinión ume d'aide se tasse. Le manque cacy of development assistance has pública cada vez desconfia más de la d'efficacité de l'aide au développement grown. eficacia de la asistencia para el suscite un mécontentement grandissant. The Comprehensive Develop- desarrollo. Le cadre de développement intégré ment Framework (CDF), launched El Marco Integral de Desarrollo (CDI), lancé par le président de la by the President of the World Bank (MID), puesto en marcha pot el Banque mondiale en janvier 1999, vise in January 1999, is a response to Presidente del Banco Mundial en enero à corriger cette évolution inquiétante. these alarming trends. It is far too de 1999, es la respuesta a esas S'il est encore bien trop tôt pour early to evaluate the initiative. In- tendencias alarmantes. Es todavia évaluer les résultats de cette initiative, stead, this year's Annual Review of demasiado pronto para evaluar esa la présente édition de l'Examen annuel Development Effectiveness exam- iniciativa. Pot ello, en el Examen anual de l'efficacité du développement dresse ines the lessons of development ex- de la eficacia en términos de desarrollo le bilan de l'année écoulée en perience through the lens of the CDF de este afño se analizan las enseñanzas appliquant les principes du CDI. principles. obtenidas de las actividades de Comme par le passé, nous As in past years, the Review desarrollo desde la perspectiva de los mettons en lumière le résultat des tracks the Bank's operational perfor- principios del MID. opérations de la Banque à partir des mance based on the findings of Como en aios anteriores, en el conclusions d'évaluations récemment recent OED evaluations. The over- Examen se consideran los resultados réalisées par l'OED. L'évolution de la all performance trends are positive, operacionales del Banco teniendo en performance globale est positive, alors despite a major increase in the cuenta las comprobaciones de las même que les opérations de la Banque relevance, demandingness, and evaluaciones recientes del sont d'une portée beaucoup plus large complexity of Bank operations-a Departamento de Evaluación de et qu'elles sont devenues plus substantial achievement. But future Operaciones (DEO). Las tendencias exigeantes et plus complexes. C'est là gains will be more difficult to generales observadas son positivas, bo un résultat non négligeable. Les achieve, and quality at exit appears que representa un logro notable si se progrès seront toutefois désormais to have plateaued below the Strate- tiene en cuenta que al mismo tiempo plus difficiles à réaliser, comme le gic Compact target of 75 percent ha aumentado notablemente la montre l'évaluation de la qualité à satisfactory. A growing gap between pertinencia, dificultad y complejidad l'achèvement des projets, qui semble operational complexity and local de las operaciones del Banco. Pero los plafonner au-dessous du niveau fixé capacity confirms the need for the progresos futuros seràn màs dificiles par le Pacte stratégique (75 % enhanced capacity building efforts de conseguir y la calidad final parece d'opérations jugées satisfaisantes). Le highlighted in Mr. Wolfensohn's haberse estabilizado por debajo del fossé qui se creuse entre la complexité Annual Meetings speech. objetivo del Pacto Estratégico, es decir, des opérations et les moyens locaux ix 1999 ANNUAL REVIEW OF DEVELOPM ENT EFFECTIVENESS Implementing the CDF un 75% de proyectos montre bien qu'il est nécessaire principles, both within and calificados como satisfactorios. de redoubler d'efforts pour outside the 13 pilot countries, La diferencia creciente entre la renforcer les capacités, ainsi que raises tough challenges for the complejidad operacional y la l'a rappelé M. Wolfensohn dans Bank and its partners. The capacidad local confirma la son allocution devant Review draws on extensive necesidad de impulsar las l'Assemblée annuelle. evaluation evidence, litera- iniciativas de fortalecimiento de L'application des principes ture reviews and research findings, la capacidad destacada por el Sr. du CDI dans le cadre des opérations country assistance evaluations, Wolfensohn en su discurso con pilotes, et au-delà de celles-ci, est une background papers, and workshops ocasión de las Reuniones Anuales. tâche ardue pour la Banque et ses to identify the tensions and dilem- La aplicación de los principios del partenaires. Pour mettre en évidence mas likely to be involved in CDF MID, dentro y fuera de las iniciativas les tensions et les dilemmes que la mise implementation. It identifies prom- experimentales, plantea dificiles retos en œuvre du CDI risque de faire ising approaches to meet the para el Banco Mundial y sus surgir, le présent examen s'appuie sur challenges implicit in CDF imple- asociados. En el Examen se evalúan de nombreux faits observés dans les mentation. Finally, it draws broad atentamente los testimonios évaluations, sur des études implications for the Bank's future disponibles, se examinan las documentaires et des travaux de re- development effectiveness. publicaciones y los resultados de la cherche, sur des évaluations de l'aide investigación, las evaluaciones de la apportée aux pays, sur des documents asistencia a los paises, los documentos de référence et sur des ateliers. Il de antecedentes y los seminarios présente des formules prometteuses realizados con el fin de descubrir las pour surmonter les difficultés tensiones y dilemas que acompafiarán, inhérentes à cet exercice. Enfin, il tire probablemente, a la aplicación del les grandes conséquences de la situa- MID. Se localizan planteamientos tion du point de vue de l'efficacité que prometedores para responder a los les opérations de la Banque pourront desafios implicitos en la aplicaciôn del avoir sur le développement des pays. MID. Finalmente, se extraen algunas conclusiones generales sobre la eficacia futura del Banco en términos de desarrollo. Robert Picciotto Director-General, Operations Evaluation x Ex c ut1ve Summary EXECUTIVE RESUMEN RÉSUMÉ SUMMARY ANALYTICL_UE Changes in the global envi- Los cambios registrados en el Les changements survenus sur la ronment over the past decade have entorno mundial durante el pasado scène internationale au cours des dix profoundly altered the context of decenio han modificado profundamente dernières années ont profondémcnt Bank operations. Many developing el contexto de las actividades del Banco modifié le contexte dans lequel la countries have undertaken major Mundial. Muchos países en desarrollo Banque opère. De notmbreux pays ont political and economic reforms, han emprendido grandes reformas entrepris d'importantes réformes qui se opening their economies and politi- políticas y económicas, que han sont traduites par une ouverture sur le cal processes. At the same time, representado una apertura de sus plan éconoiqut e et (' lu e . L mIme moment, la mondlsa ~no' ouvs d es perspectives de gains mais faisait aussi naître des risques d'instabilité et creusait - - ___nn__"__ les inégalités sur le plan tant national qu'international. Malgré la montée en flèche de la demande d'aide publique, les flux d'assistance ont fléchi. En dépit d'une amélioration des indicateurs e ~ sociaux partout dans le monde, la pro- gression du développement a été décevante, les statistiques sur la pauvreté se sont aggravées et la plupart des pays à faible revenu sont restés très dépendants de l'aide. Pendant la période comprise entre 1985 et 1995, le revenu par habitant a en fait baissé, de 1,4 % par an dans les pays à faible revenu (sans la Chine, ni l'Inde) et de 0,7 % dans les pays à revenu intermédiaire. globalization has brought new op- economias y de los procesos politicos. Dans ce contexte, le manque portunities for gain-and new risks Ai mismo tiempo, la globalización ha d'efficacité du développement est devenu of instability and rising inequality traido consigo nuevas oportunidades, y un sujet de préoccupation croissante within and among states. Aid flows nuevos riesgos de inestabilidad y tant pour les autorités des pays have stagnated, even as new de- aumento de la desigualdad dentro de développés et des cays ci mands for official resources have cada pais y entre unas naciones y otras. développement que pour le public en escalated. Despite worldwide im- Los flujos de la ayuda se han estancado, général. Face à cette situation, on a provements in social indicators, aun cuando se han multiplicado las recherché différents moyens d'améliorer development progress has been dis- nuevas demandas de recursos oficiales. la façon dont la coopération pour le appointing, poverty trends have A pesar de ls progresos mundiales de développement est abordée, Le Cadre de worsened, and most low-income los indicadores sociales, el avance del développemcnt itg ç(CD reprend les countries remain heavily dependent desarrollo ha sido decepcionante, las grands thènis autour desquels on aid. Per capita incomes actually tendencias de la pobreza se han s'organise le consensuse L fell during 1985-95-by 1.4 percent agravado y la mayoria de los países de communauté du déscloppement. Le a year in low-income countries (ex- ingreso màs bajo continúan cadre stratégique de lutte contre la x i 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS cluding China and India) and dependiendo fuertemente de la pauvreté, qui sera conjointement by 0.7 percent a year in ayuda. Los ingresos per cápita élaboré par la Banque mondiale, middle-income countries. retrocedieron de hecho en 1985- le Fonds monétaire international Inevitably, governments of 7 95, un 1,4% al aio en los paises et les pays emprunteurs, est une developed and developing de ingreso bajo (con exclusiôn de mervention étroitement liée à countries, as well as the public China y la India) y un 0,7% al cette question. at large, have become more aio en los paises de ingreso Depuis son apparition, il y concerned about development effec- mediano. a un an, le CDI suscite un intérêt tiveness. This has led to a wide- Inevitablemente, los gobiernos de considérable. Les éléments sur lesquels ranging search for improved los paises desarrollados y en il se fonde ne sont pas nouveaux. En approaches to development coopera- desarrollo, asi como la sociedad en revanche, la façon dont ils s'articulent tion. The Comprehensive Develop- general, se muestran ahora màs les uns avec les autres pour servir de fil ment Framework (CDF) encapsulates preocupados por la eficacia en conducteur à l'aide au développement major themes around which a consen- términos de desarrollo. Ello ha dado est une donnée nouvelle. sus of the development community is lugar a una búsqueda generalizada de Premièrement, les problèmes auxquels crystallizing. The Poverty Reduction nuevos enfoques que permitan se heurte le développement étant de Strategy Papers-to be prepared orientar mejor la cooperaciôn en mate- nature structurelle et sociale, les jointly by the World Bank, the Inter- ria de desarrollo. El MID compendia mesures de stabilisation économique national Monetary Fund, and bor- los grandes temas en torno a los et le recadrage des politiques ne rowing countries-are a closely cuales se està cristalizando el consenso suffiront pas pour les surmonter related response. de todos los interesados en el l'approche doit être globale et Since its unveiling a year ago, desarrollo. En ese mismo contexto se multisectorielle. Deuxièmement, ni les the CDF has evoked considerable están preparando los documentos de réformes, ni le développement interest. The basic elements of the estrategia para la reducciôn de la institutionnel ne peuvent être importés CDF are not new. What is new is their pobreza, obra conjunta del Banco ou imposés : la pérennité des réformes joint articulation as a guide to devel- Mundial, el Fondo Monetario et des investissements passe par une opment assistance. First, develop- Internacional y los paises prestatarios. adhésion sans réserve au niveau local. ment constraints are structural and Desde que se propuso, hace un Troisièmement, le développement ne social and cannot be overcome afño, el MID ha suscitado considerable peut réussir que s'il fait appel à des through economic stabilization and interés. Sus elementos básicos no son partenariats entre les pouvoirs publics, policy adjustment alone-they re- nuevos. Lo novedoso es su les populations locales, le secteur quire a holistic, broad-based ap- articulaciôn conjunta como guia de la privé, la société civile et les organismes proach. Second, policy reform and asistencia para el desarrollo. Primero, d'aide au développement. Enfin, les ac- institutional development cannot be los obstàculos al desarrollo son tions de développement doivent avoir imported or imposed: without domes- estructurales y sociales y no pueden le souci du résultat. Le CDI est lancé à tic ownership, reforms and invest- superarse únicamente con politicas de titre pilote dans 13 pays et il n'a pas ments are not sustainable. Third, ajuste y de estabilización econàmica, encore revêtu sa forme définitive. Bien successful development requires part- sino que requieren un planteamiento qu'il soit encore trop tôt pour dresser nerships among government, local global, de base amplia. Segundo, la un premier bilan, la présente édition communities, the private sector, civil reforma de las politicas y el desarrollo de l'Examen annuel de l'efficacité du society, and development agencies. institucional no pueden importarse ni développement s'appuie sur les Fourth, development activities must imponerse: si en los propios países no principes du CDI pour apprécier le be guided and judged by results. CDF hay una identificaciôn con las nuevas chemin parcouru et tirer des pilot activity is beginning in 13 coun- propuestas, ni las reformas ni las enseignements qui faciliteront tries, and its dimensions are still inversiones seràn sostenibles. Tercero, l'application des dispositions prévues. evolving. Although it is too early to un desarrollo eficaz requiere el assess the CDF as such, this year's establecimiento de asociaciones entre Évolution générale des projets Annual Review of Development Ef- los gobiernos, las comunidades lo- Les faits observés dans les évaluations fectiveness examines development cales, el sector privado, la sociedad montrent que le résultat des projets est xii Executive Sunimary experience through the lens of civil y los organismos de étroitement lié aux politiques CDF principles, with a view to - desarrollo. Cuarto, las nationales et aux facteurs drawing lessons for CDF imple- actividades de desarrollo deben institutionnels, aspects au cœur mentation. orientarse y juzgarse teniendo du CDI. Les projets étant l'un des en cuenta los resultados. La grands moyens d'assistance de la Project Trends aplicación experimental del Banque, la performance à ce Evaluation evidence confirms MID ha comenzado en 13 niveau continuera à influer de the importance of country policy paises, y sus dimensiones están aún en façon déterminante sur l'efficacité du and institutional factors addressed evolución. Aunque es todavia développement. Sur les dix années by the CDF to project performance. demasiado pronto para evaluar el écoulées, le résultat des évaluations de Projects are important vehicles of MID en cuanto tal, en el Examen projet est en amélioration ce qui, compte Bank assistance, and most perfor- anual de la eficacia en términos de tenu de l'exigence et de la complexité mance at the project level will desarrollo de este aio se examina la toujours croissantes des opérations, continue to be a critical element of realidad del desarrollo desde la constitue un bilan remarquable. Les development effectiveness. Project perspectiva de los principios del MID, prêts à l'ajustement, en particulier, ont evaluations have shown improve- con el fin de extraer enseñ-anzas para donné lieu, pour une assez large part, à ments over the past decade, along su aplicación. des résultats jugés satisfaisants. On a with a considerable increase in toutefois constaté récemment un certain project demandingness and com- Tendencias de los proyectos tassement : 72 % des projets évalués qui plexity-a considerable achieve- Los datos relativos a las evaluaciones sont sortis du portefeuille pendant les ment. Adjustment lending, in confirman la importancia de los factores exercices 98 et 99 ont enregistré des particular, has delivered a rela- normativos e institucionales de los résultats satisfaisants, mais moins de la tively high share of satisfactory paises contemplados en el MID para moitié sont susceptibles de confirmer outcomes. But there has- been a determinar los resultados de los durablement cette performance. recent plateauing in performance: proyectos. Los proyectos son vehiculos Les projets étant devenus plus 72 percent of evaluated projects importantes de asistencia del Banco, por complexes, la performance des exiting in FY98-99 show satisfac- lo que la multiplicaciôn de esos emprunteurs au niveau de l'exécution tory outcomes, while the share of resultados continuará siendo un des opérations est restée stationnaire, exiting projects likely to sustain elemento critico de la eficacia del ce qui contribue à la stagnation des their benefits remains below half. desarrollo. Las evaluaciones de los résultats. Les analyses montrent que As the complexity of exiting proyectos han demostrado los cette performance est un élément qui projects has risen, borrower imple- progresos conseguidos en el pasado vient immédiatement après la supervi- mentation performance has been decenio, a pesar de que ha aumentado sion de la Banque dans les facteurs qui stagnant, which has contributed to también notablemente la dificultad y déterminent le succès d'un projet. Le the plateauing of performance. complejidad de los proyectos, lo que creusement de l'écart entre la Analysis shows borrower imple- representa un importante logro. En par- complexité des opérations et les mentation performance to be a key ticular, el financiamiento para fines de capacités locales souligne l'importance determinant of project success, sec- ajuste ha conseguido una proporción de ces relais et la nécessité d'une ond only to Bank supervision. The relativamente elevada de calificaciones meilleure prise en compte des widening gap between complexity satisfactorias. Pero se ha observado contraintes institutionnelles. Malgré and local capacity highlights the también una estabilización de los une nette amélioration au cours des importance of these inputs, as well resultados: el 72% de los proyectos dernières années, seuls 40 % des as the need for intensified attention evaluados finalizados en los ejercicios de projets sortant du portefeuille ont to institutional constraints. Despite 1998-99 presentan resultados actuellement un impact important sur substantial improvements in recent satisfactorios, mientras que la parte de le développement institutionnel. years, only 40 percent of exiting los proyectos concluidos con projects currently have substantial probabilidades de continuar Une tâche ardue institutional development impact. produciendo beneficios se mantiene por Le CDF repose sur des principes qui debajo de la mitad. tirent parti des synergies. Mais celles-ci 1999 AN NUAl,I Ri\'Il W\ O[ DELVI i PM!-NT FFE CTI-V ENEUSS Tough Chaîlenges Ahead A medida que ha génèrent aussi des antagonismes The CDF principles are syner- aumentado la complejidad de qu'il faudra gérer prudemment. gistic. But their combination los proyectos concluidos, el Le court terme et le long carries with it critical tensions desempeño de los prestatarios terme. La transformation that will require careful man- en la ejecución de los proyectos sociale et le développement agement. se ha estancado, Io que ha institutionnel supposent un ef- Short terni versus long contribuido a la estabilización fort de longue haleine. term. Social transformation and in- de los resultados. Los análisis Pourtant, les mécanismes stitutional development require efectuados revelan que los resultados d'incitation et le mode de long-term and sustained efforts. Yet de la ejecucién de los proyectos son un fonctionnement qui prévalent dans les the incentives and processes in de- determinante fundamental del éxito de pays en développement et dans les veloping countries and aid agencies éstos, cuya importancia sólo es infe- organismes d'aide ont tendance à have tended to focus on the short rior a la supervisión del Banco. La privilégier le court terme. La term. Blueprint planning and the diferencia creciente entre la planification à l'aide de modèles et project approach embody this short- complejidad y la capacidad local pone l'approche-projet s'inscrivent dans terni focus.The abscnce of a long- de manifiesto la importancia de esos cette perspective. Les dimensions term perspectve 'ha been especially aportes, asi como la necesidad de structurelles des réformes detrimental in dealing with the intensa atenciôn a los obstáculos (privatisation, réforme de la fonction structural dimensions of reform, institucionales. A pesar de las publique et déréglementation du such as privatization, civil service considerables mejoras de los últimos secteur financier, par exemple) ont reform, and dereguiation of the aios, sôlo el 40% de los proyectos particulièrement souffert de cette ab- financial sector. concluidos tienen actualmente sence d'optique à long terme. Comprehensiveness versus selec- importantes repercusiones en el Exhaustivité et sélectivité. Du fait tivity. The Bank's expanding agenda desarrollo institucional. de l'expansion du programme de tra- has increased the complexity, de- vail de la Banque, les projets et les mandingness, and risk of projects and Arduos desafios para el futurc programmes se sont fait plus com- programs. In the face of these pres- Los principios en que se fundamenta el plexes, risqués et exigeants. Face à sures, selectivity has not always been MID son sinérgicos, pero la cette situation difficile, on ne procède exercised. Yet evaluation findings combinaciôn de éstos conlleva tensiones pas toujours de façon sélective. highlight the risk of excessive com- críticas que se deberán manejar con Pourtant, les évaluations montrent plexity, especially when ir is built into sumo cuidado. qu'une complexité excessive comporte the design without regard to domestic Corto plazo versus largo plazo. des risques, surtout lorsqu'il n'a pas capacity or ownership. Projects that La transformación social y el été tenu compte des capacités ou de la involve several implementing agen- desarrollo institucional exigen volonté d'appropriation au niveau lo- cies or cross-sectoral assets tend to esfuerzos sostenidos y a largo plazo. cal. Les projets faisant intervenir perform poorly. Sin embargo, los incentivos y procesos plusieurs organismes d'exécution ou Speed versus broad-based own- en los paises en desarrollo y en los des actifs dans plusieurs secteurs ership. Sustainable reform requires organismos de ayuda se han centrado obtiennent généralement des résultats broad-based ownership. Yet part- en el corto plazo, tendencia que se ha médiocres. ners may disagrec about the distinc- caracterizado por una planificación Rapidité et large adhésion. Pour tive roles of the state, the private detallada y un enfoque en los qu'une réforme soit durable, il faut sector, and civil society. Achieving proyectos. La falta de una perspectiva que l'adhésion à l'action menée soit agreement among partners may de- a largo plazo ha sido especialmente totale. Pourtant, les partenaires ne lay reform. Governance reforms perjudicial para abordar las sont pas toujours d'accord sur les may be required to institutionalize dimensiones estructurales de la rôles respectifs de l'État, du secteur participation-a long-term process. reforma, como la privatización, la privé et de la société civile. La réforme 11wnership versus conditionality. reforma de la administración pública y pourra prendre du retard du fait du How should the apparent tension la desreglamentaciôn del sector temps nécessaire aux différents acteurs between country ownership and do- financiero. pour parvenir à un accord. Il faudra xiv Executive Surmnary nor interest in performance (of- Integridad versus parfois réformer le mode de ten enforced through condition- selectividad. El programa de gestion des affaires publiques ality) be resolved? Conditional- actividades del Banco, que va en pour institutionnaliser la par- ity is widely viewed as an aumento, presenta una mayor ticipation, une entreprise de attempt to impose reforms and complejidad, dificultades y longue haleine. blueprint solutions on clients. riesgos de los proyectos y Adhésion au projet et Yet ownership is essential to programas. Ante tales conditionnalité. Comment sustainable reform. New forms of presiones, no siempre se ha l'apparente contradiction entre conditionality-reflecting the CDF aplicado el criterio de la selectividad. l'adhésion du pays à l'action menée et principles-should be instituted. Con todo, en las conclusiones de las les résultats demandés par les bailleurs Partnership versus country ca- evaluaciones practicadas se destaca el de fonds (souvent au moyen de la pacity and ownership. Countries in riesgo de una excesiva complejidad, conditionnalité) peut-elle être résolue ? need of partnership often lack the sobre todo cuando ésta se incluye en el La conditionnalité est très souvent capacity to coordinate aid. Govern- diseio sin tener en cuenta la capacidad perçue comme une tentative non ments, civil society, the private sector, local o la identificación del pais con las déguisée d'imposer des réformes ou un and external donors may have differ- propuestas. Los proyectos que modèle aux pays clients. Et pourtant, ent agendas, requiring a gradual entrafñan la participación de varios il ne peut y avoir de réformes durables approach to consensus building. The organismos de ejecución o activos si le pays ne s'identifie pas à l'action larger and more diverse the partners, intersectoriales tienden a arrojar menée. Il conviendrait de repenser la the higher the transaction costs and resultados deficientes. conditionnalité, sous une forme plus the greater the difficulty of combining Velocidad versus amplia respectueuse des principes du CDI. ownership and partnership. Where identificación. Para que las reformas Partenariat, capacité locale et country commitment is lacking, co- sean sostenibles es preciso que exista adhésion du pays. Les pays devant herent views among external partners una amplia identificación con ellas. No faire appel à des partenaires multiples may be perceived as "ganging up." obstante, los asociados pueden sont rarement à même de coordonner Country-led partnership versus discrepar con respecto a las diferentes l'aide reçue. Les objectifs des donor accountability. AIl develop- funciones del Estado, el sector privado gouvernements, de la société civile, du ment assistance agencies are account- y la sociedad civil. El logro de un secteur privé et des bailleurs de fonds able to their authorities, so placing an consenso entre los asociados puede peuvent en effet être différents, ce qui unresponsive government "in the demorar las reformas. Puede ser obligera les partenaires à rechercher driver's seat" involves risks. The necesario introducir reformas de la progressivement le consensus challenge is to find common ground función de gobierno para nécessaire. Plus les partenaires sont and achieve results over the long term institucionalizar la participación, un nombreux et différents, plus les coûts through patient nurturing of reform proceso a màs largo plazo. de transaction sont importants et plus and capacity building. Identificación versus il est difficile de concilier adhésion et Results-orientation versus local condicionalidad. ¿Cómo debe partenariat. En l'absence, de volonté capacity. The performance of the resolverse la aparente tensión que politique de la part du pays Bank and most developing countries existe entre la identificación del pais y bénéficiaire, une communauté de vues in monitoring and evaluation has el interés de los donantes en el des bailleurs de fonds peut être perçue been weak. Yet the international desempeio (el que a menudo se exige a comme une « coalition ». development goals, the recent atten- través de la condicionalidad)? Existe la Partenariat piloté par le pays et tion to governance, and the move to impresión generalizada de que la responsabilité des bailleurs de fonds programmatic lending reinforce the condicionalidad es un burdo intento envers leur autorité de tutelle. Tous les need for results-based management de imponer reformas y soluciones organismes d'aide au développement and stronger evaluation capacities planificadas a los clientes. Sin em- ayant à répondre de leur gestion and local accountability systems. bargo, la identificación con las devant leur autorité de tutelle, « passer Country-based programs versus reformas es fundamental para que les rênes » à un gouvernement peu global public goods. Development éstas sean duraderas. Deben instituirse motivé comporte des risques. Il s'agira assistance efforts and approaches nuevas formas de condicionalidad, de trouver un terrain d'entente et xv 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS are focused at the project and que reflejen los principios del d'obtenir des résultats à long country levels. With global- - MID. terme en renforçant patiemment ization, development prob- Asociaciones versus les capacités et en laissant mûrir lems require multilateral capacidad e identificaciôn de los les réformes. solutions and stronger links países. Los paises que necesitan Obligation de résultat et between national strategies establecer asociaciones a capacités locales. Le travail de and international policies. menudo carecen de la capacidad suivi et d'évaluation de la para coordinar la ayuda. Los Banque et de la plupart des pays en Promising Approaches gobiernos, la sociedad civil, el sector développement laisse à désirer. Et In order to resolve the above dilem- privado y los donantes externos pourtant l'existence d'objectifs mas and implementation chal- pueden tener distintos programas, Io internationaux de développement, lenges, it will be necessary to em- que hace necesario adoptar un l'accent récemment mis sur le mode ploy creativity and innovation and planteamiento gradual para la d'administration des affaires to heed the lessons of experience. formación de consenso. Mientras màs publiques et le passage à des prêts Several promising practices emerge numerosos y diversos sean los programmatiques rendent d'autant from this review. All are predicated asociados, mayores seràn los costos plus nécessaires des mesures visant à on country commitment to poverty de transacciôn y las dificultades para axer la gestion sur les résultats, à reduction and sustainable growth, combinar la identificaciôn y las améliorer les capacités d'évaluation et and a few are quick to take to scale. asociaciones. Cuando en un pais falta à renforcer les systèmes de contrôle. • An adaptive learning process el sentido de compromiso, la Programmes au niveau des pays starts small and combines top- coherencia de opiniones entre los et biens publics de caractère mondial. down direction with bottom-up asociados externos puede percibirse Les actions et stratégies d'aide au experimentation and learning. como una "alianza". développement sont conçues au niveau This shifts the emphasis from Asociación de esfuerzos des projets et des pays. Du fait de la up-front analysis and detailed encabezada por el pais versus mondialisation, la solution aux design toward developing flex- responsabilidad de los donantes. problèmes de développement passe ible solutions, building local Todos los organismos de asistencia par l'adoption de solutions capacity, and relying on social para el desarrollo son responsables multilatérales et par un renforcement processes and monitoring sys- ante sus autoridades, de manera que des liens existant entre les stratégies tems for adaptation and learn- poner a un gobierno que no reacciona nationales et les politiques ing during implementation. en el " asiento del conductor" plantea internationales. • One way out of the excessive ciertos riesgos. El desafío consiste en complexity of projects and encontrar un terreno común y lograr Des possibilités intéressantes programs is to sequence inter- resultados a largo plazo fomentando Pour sortir des dilemmes mentionnés ventions within a long-term con paciencia la reforma y el plus haut et résoudre les problèmes de strategy that builds on past desarrollo de la capacidad. mise en œuvre qui en découlent, il learning. Such sequencing can Orientaciôn a los resultados ver- faudra faire preuve de créativité et start by piloting comprehensive sus capacidad a nivel local. La d'innovation et savoir tirer les leçons de approaches at the local level, actuación del Banco y de la mayoria de l'action menée. Plusieurs possibilités then scaling them up as part of los paises en desarrollo en lo que intéressantes se dégagent du présent a long-term process of capacity respecta a las actividades de examen. Toutes reposent sur la volonté building and decentralization. seguimiento y evaluación ha sido du pays de faire reculer la pauvreté et de • Tensions between comprehen- deficiente. Con todo, las metas promouvoir une croissante durable, siveness and selectivity can be internacionales de desarrollo, la mais rares sont celles qui peuvent être eased by combining compre- reciente atención a los asuntos rapidement portées à une plus grande hensive analysis with strategic relativos a la función de gobierno y el échelle. actions. Participatory ap- avance hacia el financiamiento para • Tout processus d'apprentissage proaches can enable poor programas refuerzan la necesidad de évolutif doit démarrer à petite people to analyze their realities una gestiôn basada en los resultados, échelle et allier une démarche xvi Executive Summary in a comprehensive way, and así como de una mayor venant d'en haut à une acquisi- then express their priorities capacidad de evaluaciôn y de tion des connaissances et à une and choose interventions se- sistemas para asegurar la expérimentation partant du lectively. An example of a responsabilidad a nivel local. bas. Cela permettra de moins holistic yet selective approach Programas basados en los mettre l'accent sur les analyses is the sustainable livelihoods paises versus bienes públicos initiales et les plans détaillés et approach of the U.K. Depart- mundiales. Las iniciativas y de privilégier la recherche de so- ment for International Develop- enfoques relativos a la asistencia para lutions souples, le renforcement ment, CARE, and the United el desarrollo se centran al nivel de los des capacités locales et le recours Nations Development Program proyectos y de los paises. Con el aux systèmes de suivi et aux (UNDP). proceso de globalizaciôn, los mécanismes encadrant les • Sectorwide and programmatic problemas del desarrollo exigen conduites au sein des collectivités approaches can help reduce the soluciones multilaterales y el pour favoriser l'adaptation et rigidities and burdens associ- establecimiento de vinculos màs l'apprentissage pendant ated with the proliferation of estrechos entre las estrategias l'exécution des activités. donor-financed projects. Learn- nacionales y las politicas • L'un des moyens de remédier à la ing and innovation pilots fol- internacionales. complexité des projets et lowed by adaptable program programmes est d'échelonner les loans can support priority Planteamientos prometedores interventions en appliquant une goals, particularly in sectors Para resolver los dilemas y retos en ma- stratégie qui intègre les acquis sur where flexibility is critical. But teria de aplicación sefialados le long terme. Pour échelonner les moving prematurely from anteriormente, serà necesario recurrir a interventions, on peut commencer projects to full-scale sector ap- la creatividad y la innovación, y poner par expérimenter des méthodes proaches is risky. Capacity atención a las ensefianzas de la globales au niveau local, pour les building must accompany the experiencia. Del presente examen surgen appliquer ensuite à plus grande process of scaling up. More- varias pràcticas prometedoras. Todas se échelle dans le cadre d'un proces- over, sectorwide approaches fundamentan en la determinaciôn de los sus de renforcement des capacités should be tailored to countries paises para reducir la pobreza y lograr el et de décentralisation qui s'inscrit and sectors. desarrollo sostenible, y unas pocas se dans la durée. • Conditionality should be recon- pueden aplicar de inmediato en mayor • En complétant l'analyse globale ciled with country ownership. escala. d'actions stratégiques, on peut Empirical analysis of past • Un proceso de aprendizaje atténuer l'antinomie existant entre country reforms over a long adaptativo comienza en forma exhaustivité et sélectivité. De horizon and of the recent modesta y en él se combina una même, en associant les pauvres à higher-impact adjustment lend- dirección de arriba hacia abajo l'action menée, on peut leur ing in Africa confirms that con- con experimentación y permettre d'analyser la réalité de ditionality is best managed as a aprendizaje de abajo hacia arriba. façon globale, puis de fixer leurs flexible, noncoercive policy Con esto se cambia de énfasis al priorités et de sélectionner les in- compact adapted to different pasar de un anàlisis inicial y un terventions en conséquence. La stages of reform. disefio detallado a la formulación façon dont le Département pour • Broadening ownership across de soluciones flexibles, la creaciôn le développement international many stakeholders with diverse de capacidad a nivel local y el (Royaume-Uni), CARE et le interests and capabilities requires apoyo en los procesos sociales y PNUD abordent la question des time and early mobilization, par- en sistemas de seguimiento que moyens d'existence viables est un ticularly of communities and the permitan la adaptación y el bon exemple d'une approche à la poor. It implies the systematic use aprendizaje durante el proceso de fois globale et sélective. of participatory processes; in- aplicaciôn. • Les approches par secteur et par forming and giving voice to the • Una manera de evitar proyectos y programme peuvent aider à weak partners, particularly programas excesivamente réduire les rigidités et les xvii 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS women and marginalized complejos es establecer un contraintes liées à la populations; and creating an orden de sucesión de las prolifération de projets financés enabling environment for scal- intervenciones dentro de una par des bailleurs de fonds. Des ing up and institutionalizing < estrategia a largo plazo que se opérations pilotes pour le participation. fundamente en las lecciones - développement des • Partnership requires two extraidas del pasado. Tal connaissances et l'innovation, complementary roles for the secuenciamiento puede complétées de prêts à des Bank: supporting country comenzar con la aplicaciôn ex- programmes évolutifs, peuvent leadership and building the ca- perimental de planteamientos aider à la réalisation d'objectifs pacity to exercise that leader- globales al nivel local, que luego se prioritaires, surtout là où la lati- ship, and engaging its develop- pueden ir incrementando como tude d'action joue un rôle ment assistance partners to parte de un proceso a largo plazo déterminant. Il est toutefois risqué promote selectivity, coordinate de desarrollo de capacidades y de transposer prématurément au interventions, and harmonize descentralizaciôn. niveau de l'ensemble d'un secteur procedures. Partnerships may • Las tensiones entre la integridad y les activités réalisées à l'échelon imply up-front investments and la selectividad pueden suavizarse d'un projet. Le passage à l'échelle must be effectively monitored mediante la combinación de un supérieure doit s'accompagner and managed if transaction anàlisis cabal y acciones d'un travail de renforcement des costs to donors and countries estratégicas. Los enfoques capacités. Uapproche devra en are to decline over time. participatorios pueden permitir outre être adaptée au pays et au que la gente pobre analice su secteur considérés. Evaluating Development realidad en forma integral, y luego • Il faut concilier conditionnalité et Effectiveness expresar sus prioridades y adhésion du pays à l'action The changing global economy and escoger selectivamente las menée. L'analyse sur une longue the CDF suggest several principles intervenciones. Un ejemplo de un période des réformes entreprises for evaluating development effec- planteamiento integral pero dans les pays et l'examen des tiveness. selectivo es el enfoque sobre récentes opérations de prêt à Evaluation should focus on te- medios de vida -sostenibles del l'ajustement, qui ont eu des sults, and this depends on accurate Departamento de Desarrollo conséquences importantes en tracking of progress toward devel- Internacional del Reino Unido, Afrique, montrent que la opment goals, with a clear focus on CARE y el PNUD. conditionnalité gagne à prendre la poverty reduction and growth. • Los enfoques sectoriales y forme d'un programme souple, Tracking development outcomes programáticos pueden ayudar a non imposé, qui évolue au fil de should comply with the comprehen- reducir las rigideces y la carga son application. sive development agenda agreed by asociadas a la proliferación de • Pour amener un grand nombre de the government and its partners. proyectos financiados por parties prenantes sans As the focus of the development donantes. Los préstamos communauté de moyens ni effort moves from projects to the experimentales para el aprendizaje d'intérêts à souscrire pleinement à higher plane of country programs, y la innovación seguidos de l'action menée, il faut compter avec so must the evaluation process. Re- préstamos adaptables para le temps et mobiliser rapidement les sources and skills should be invested programas pueden apoyar la intéressés, tout particulièrement la in developing appropriate indica- consecución de metas prioritarias, population et les pauvres. À cette tors and information systems. The sobre todo en sectores en los que fin, il faudra systématiquement faire current preoccupation with project la flexibilidad es fundamental. appel à la participation, s'adresser performance and evaluation should Pero pasar prematuramente de et donner la parole aux partenaires be complemented by a sectoral and los proyectos a enfoques les plus faibles, notamment aux countrywide focus. Public sector re- sectoriales integrales plantea femmes et aux exclus, et créer les form needs to include building the riesgos. El proceso gradual debe ir conditions qui permettront evaluation capacity of countries. acompañado de la creaciôn de d'élargir et d'institutionnaliser cette xviii Executive Summary Development effective- capacidad. Ademàs, los participation. ness should be evaluated in enfoques sectoriales deberían • Pour promouvoir le terms of shared objectives, adaptarse a las circunstancias partenariat, l'action de la joint responsibility for out- de los paises y los sectores. - Banque doit être double. Elle comes, reciprocal obligations e La condicionalidad deberia < doit aider les pays à jouer un to achieve results, and distinct conciliarse con la identificación rôle moteur en contribuant au accountability for perfor- de los paises. El análisis renforcement de leurs capacités mance. In particular, donors empirico de las reformas et elle doit inciter ses partenaires and governments should team up to introducidas en el pasado durante bailleurs de fonds à encourager la involve civil society and the private un prolongado horizonte tempo- sélectivité, à coordonner les inter- sector in monitoring and evaluation, ral y el financiamiento para fines ventions et à harmoniser les and to help all stakeholders acquire de ajuste otorgado recientemente procédures. Les partenariats the needed skills and attitudes. Par- a Âfrica confirman que la mejor impliquent parfois un ticipatory monitoring and evaluation manera de gestionar la investissement initial et doivent hold significant promise for social condicionalidad es mediante un être efficacement suivis et régulés learning and managing for results. pacto de polfticas flexible, no pour que les coûts de transaction Evaluation should be informed coercitivo, adaptado a las à la charge des bailleurs de fonds by the global perspective of the diferentes etapas de la reforma. et des pays diminuent au fil du International Development Goals • Para que la identificación de temps. endorsed by the development com- muchas partes interesadas con munity. Far from implying rigid, diversos intereses y capacidades Évaluation de l'efficacité du top-down global planning, this two- pueda extenderse se requiere développement way link means adapting the inter- tiempo y un proceso temprano de La mondialisation de l'économie et le national goals to country conditions movilización, especialmente de las contenu du CDI donnent à penser que and priorities and enhancing part- comunidades y de los pobres. plusieurs principes doivent s'appliquer à nerships at all levels. Para ello es preciso recurrir l'évaluation de l'efficacité du sistemáticamente a procesos développement. Implications for the Bank participatorios; informar y dar L'évaluation doit être axée sur les The CDF is not a blueprint-the oportunidades de expresión a los résultats. Pour cela, il faut pouvoir hallmark of the planning era and asociados màs débiles, sobre todo suivre avec précision les progrès the adjustment era. Its aim is to a las mujeres y a las poblaciones réalisés par rapport aux objectifs de launch a process that adapts con- marginadas, y crear condiciones développement, en s'intéressant tout tinually to changing circumstances. propicias para incrementar e particulièrement à la croissance et à la Customization should thus replace institucionalizar la participación. lutte contre la pauvreté. La mise en the "one-size-fits-all" mindset. • Para las asociaciones se requieren lumière des fruits du développement To this end, the Bank must do dos funciones complementarias doit correspondre au programme more to learn and listen. The CDF de parte del Banco: el respaldo a d'ensemble arrêté d'un commun ac- points to the pivotal role of knowl- los dirigentes nacionales y el cord par le pays et ses partenaires. edge and learning in development. desarrollo de la capacidad para Le recentrage de l'effort de Too often, it is assumed that the ejercer ese liderazgo, y la développement, qui est passé des projets Bank has the answers, and that the participaciôn de sus organismos aux programmes à l'échelle du pays, only problem is to sell the preferred asociados de asistencia para el doit s'accompagner d'un recadrage ana- solution to clients. But the Bank is desarrollo a fin de promover la logue du processus d'évaluation. Il not a storehouse of universally ap- selectividad, coordinar las convient de mobiliser les ressources et les plicable knowledge just waiting to intervenciones y armonizar los compétences nécessaires pour mettre au be transferred. That is why it should procedimientos. Las asociaciones point les indicateurs et les systèmes empower its clients to tap global pueden entraiar inversiones d'information voulus. Outre l'attention knowledge, connect clients to one desde un comienzo y deben actuellement portée à la performance et another and to other sources of vigilarse y gestionarse de manera à l'évaluation des projets, il faut mettre xix 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS experience, and learn with eficaz a fin de que los costos de l'accent sur les résultats au niveau them about what works. transacciôn para los donantes y de l'ensemble du secteur et du Practicing selectivity los paises puedan disminuir a Io pays. À cette fin, les réformes du through partnership will largo del tiempo. secteur public doivent require continuous change. comprendre le renforcement de la Bank management has initi- Evaluacién de la eficacia en capacité d'évaluation des pays. ated many changes under the términos de desarrollo Des objectifs communs, une Strategic Compact. More change La cambiante economia mundial y el responsabilité conjointe des résultats à will be needed to sharpen the Bank's MID plantean varios principios para obtenir, une obligation de résultat focus and improve the development evaluar la eficacia en términos de réciproque et des impératifs distincts effectiveness of its operations. That desarrollo. en ce qui concerne les comptes à will involve new skills, new ap- La evaluación debe centrarse en los rendre sont les critères à utiliser pour proaches, new incentives, and new resultados, y para ello es necesario un évaluer l'efficacité du développement. attitudes-to make room for genu- seguimiento exacto de los progresos Il faut, en particulier, que les bailleurs me partnerships. Above all, the hacia la consecuciôn de las metas en ma- de fonds et les pouvoirs publics Bank will have to be realistic and teria de desarrollo, con un claro enfoque s'emploient de concert à associer la strategic about its evolving com- en la reducción de la pobreza y en el société civile et le secteur privé au tra- parative advantage in a fast- crecimiento. El seguimiento de los vail de suivi et d'évaluation, et à aider changing environment. resultados en términos de desarrollo toutes les parties prenantes à acquérir debe ajustarse al programa integral de les compétences et l'état d'esprit desarrollo convenido con el gobierno y nécessaires. Ici, l'approche participa- sus asociados. tive laisse bien augurer de l'avenir, car A medida que el énfasis de las elle est instructive pour la société et iniciativas en pro del desarrollo avanzan permet une gestion soumise à des del nivel de los proyectos al plano màs critères de performance. elevado de los programas nacionales, Io Le travail d'évaluation devra propio debe ocurrir con el proceso de s'inscrive dans la perspective mondiale evaluación. Se deben invertir recursos y des Objectifs internationaux de conocimientos para formular développement approuvés par la indicadores y sistemas de información communauté du développement. Loin adecuados. La actual preocupaciôn por de déboucher sur une planification los resultados de los proyectos y la imposée d'en haut et rigide au niveau evaluación debe complementarse con un mondial, ce double lien doit permettre enfoque sectorial y nacional. La reforma d'adapter les objectifs internationaux del sector público debe incluir el à la situation et aux priorités du pays desarrollo de la capacidad de evaluaciôn et de renforcer les partenariats à tous por parte de los paises. les échelons. La eficacia del desarrollo debe evaluarse en términos de los objetivos Répercussions pour la Banque compartidos, la responsabilidad Le CDI n'est pas un modèle et ne vise conjunta por los resultados pas à marquer de son estampille l'âge de obtenidos, las obligaciones reciprocas la planification et de l'ajustement. Il a para alcanzar resultados y la pour objectif de lancer un processus qui distinción de responsabilidades con ne cessera de s'adapter à l'évolution de la respecto al desempefño. En particular, situation. Le « sur mesure »ý est donc los donantes y los gobiernos deben appelé à remplacer la « taille unique ». unirse para hacer participar a la Pour cela, il faut que la Banque sociedad civil y al sector privado en las cherche davantage à apprendre et à actividades de seguimiento y écouter. Le CDI insiste sur le rôle xx Executive Summary evaluación, y para ayudar a déterminant que la diffusion du todas las partes interesadas a savoir et le développement des adquirir y desarrollar los connaissances jouent dans le conocimientos y las actitudes développement. Trop souvent, necesarios. El seguimiento y la - on part du principe que la evaluaciôn con participación Banque détient la réponse et ofrecen grandes esperanzas qu'elle n'a plus qu'à vendre la para el aprendizaje social y la gestiôn meilleure solution aux clients. Mais la de los resultados. Banque ne stocke pas des La evaluacién debe fundamentarse connaissances d'application générale en la perspectiva global de las metas qui attendent d'être transférées. C'est internacionales en materia de desarrollo pourquoi elle doit donner aux pays la aprobadas por la comunidad del possibilité d'exploiter les desarrollo. Muy lejos de implicar una connaissances existant à travers le planificación global rigida, de arriba monde, mettre ses clients en rapport hacia abajo, este vinculo bidireccional les uns avec les autres, leur fournir significa adaptar las metas d'autres sources d'information et internacionales a las condiciones y apprendre, avec eux, les formules qui prioridades de cada pais y mejorar las donnent des résultats concluants. asociaciones en todos los niveles. Pour agir de façon sélective dans le partenariat, il faut être constamment Repercusiones para el Banco prêt à s'adapter. La direction de la El MID no es un plan detallado, bo cual Banque a inscrit de nombreux caracterizô a la era de la planificaciôn y changements dans le Pacte stratégique. la era del ajuste. Su objetivo es poner en Mais l'institution devra continuer à marcha un proceso que se adapte évoluer pour mieux circonscrire son continuamente al cambio de action et améliorer l'impact de ses circunstancias. En consecuencia, la opérations sur le développement. Cela adaptaciôn debe reemplazar a la passe par de nouvelles compétences, mentalidad de "modelo único". de nouvelles approches, de nouvelles Para ello, el Banco debe esforzarse incitations et un nouvel état d'esprit, màs para aprender y escuchar. El MID autant d'éléments qui permettront la destaca el papel fundamental que naissance de véritables partenariats. Et desempeian el conocimiento y el surtout, la Banque devra faire preuve aprendizaje en el desarrollo. Con de réalisme et faire des choix demasiada frecuencia se supone que el stratégiques pour tenir compte de Banco tiene todas las respuestas, y que l'évolution de son avantage el ùnico problema es vender la comparatif dans un environnement en soluciôn preferida a los clientes. Pero pleine mutation. el Banco no es un almacén de conocimientos de aplicaciôn universal en espera de ser transferidos. Por esa razón, el Banco debe dar a sus clientes los medios para aprovechar los conocimientos, poner en contacto a sus clientes entre si y con otras fuentes de experiencia, y sacar conclusiones con ellos acerca de lo que da resultado. xxi 1999 ANNUAL REV[EW OF DEVELOPMENT EFFECTIVENESS Para poner en práctica la - selectividad mediante las asociaciones se requerirán - cambios continuos. La administración del Banco ha emprendido muchos cambios en el marco del Pacto Estratégico. Serán necesarios otros cambios para definir mejor el enfoque del Banco y para aumentar la eficacia de sus operaciones en términos de desarrollo. Para ello se requerirán nuevas aptitudes, nuevos planteamientos, nuevos incentivos y nuevas actitudes, para dar cabida a auténticas asociaciones. Por encima de todo, el Banco deberá ser realista y estratégico con respecto a su cambiante ventaja comparativa en un entorno en rápida evolución. xxii ABBREVIATIONS AND ACRONYMS AFR - Africa Region APL - Adaptable Program Loan APPI - Aggregate Project Performance Index ARDE - Annual Review of Development Effectiveness BP - Background Paper CAE - Country Assistance Evaluation CARE - Cooperative for Assistance and Relief Everywhere, Inc. CAS - Country Assistance Strategy CDF - Comprehensive Development Framework CG - Consultative Group CGIAR - Consultative Group on International Agricultural Research DAC - Development Assistance Committee DANIDA - Danish Cooperation Agency DFID - Department for International Development DGO - Director-General, Operations Evaluation EAP - East Asia and Pacific Region ECA - Europe and Central Asia Region ESSD - Environmentally and Socially Sustainable Development Network FERD - Foreign Economic Relations Department (Vietnam) GDI - Gross Domestic Investment GDP - Gross Domestic Product GDS - Gross Domestic Savings HILAL - Higher-Impact Adjustment Lending ICR - Implementation Completion Report ID - Institutional Development (Impact) IDA - International Development Association IDS - Institute of Development Studies at Sussex University LCR - Latin America and Caribbean Region LL - Learning and Innovation Loan M&E - Monitoring and Evaluation MENA - Middle East and North Africa Region NAFI - National Agency for Foreign Investment (Kazakhstan) NEP - New Economic Program (Bolivia) NGO - Nongovernmental Organization ODI - Overseas Development Institute OECD - Organization for Economic Cooperation and Development OED - Operations Evaluation Department PREM - Poverty Reduction and Economic Management QAG - Quality Assistance Group RBM - Results-Based Management SA - South Asia Region SDC - Swiss Agency for Development SPA - Special Program of Assistance to Africa UN - United Nations UNDP - United Nations Development Program UNICEF - United Nations Children's Fund USAID - United States Agency for International Development WBI - World Bank Institute WDR - World Development Report xxi1 1 TOWARD A COMPREHENSIVE DEVELOPMENT STRATEGY lobal development during the 1990s presents a paradoxical picture. Markets and ,democratic principles continue to extend their reach, and global markets for trade, finance, and information are increasingly integrated. These trends should have helped developing countries to accelerate their growth and reduce poverty. Yet the fight against poverty is being lost, and the efficacy of the development assistance system is being questioned. A necessary condition for poverty reduction- opment Assistance Committee (DAC) has formulated a growth in per capita incomes-is not being achieved by set of international goals for the first decades of the a majority of developing countries: next century that stress poverty reduction, education, health, and the environment. Most bilateral agencies * During 1985-95, the aggregate of low-income have realigned their policies to emphasize these priori- countries (excluding China and India) experi- ties. OED's evaluations (ARDE 1998) have stressed the enced a decline in per capita income of 1.4 importance of looking beyond projects, getting clients percent a year-and middle income countries, a to own and broaden their programs of reform, coordi- decline of 0.7 percent.' nating the Bank's work with that of other donors, and * The number of poor people living on less than injecting more accountability into the development US$1 a day rose from 1,197 million in 1987 to effort-both for countries and for donors. 1,214 million in 1998. Excluding China, there The Comprehensive Development Framework are 100 million more poor people in developing (CDF), proposed by President Wolfensohn in January countries than a decade ago. 1999, pulls together many strands of thought within four overarching themes: These numbers have led to much soul-searching in * Holistic. A development strategy has to go the development community. The United Nations De- beyond macroeconomic management and incor- velopment Programme (UNDP) has adopted human porate governance, human, and social develop- development as its focus. The Organization for Eco- ment objectives. nomic Cooperation and Development (OECD) Devel- * Ownership. The country has to be in the drivers 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS TABLE 1.1 THREE DEVELOPMENT PARADIGMS Com[pr,-hcn!% Dc%lopmcni Plnniniz Adu-tmnent Frane%%ork 111r j 1 1 .[ w .iluir: , * [ .i lIc ' r'rirn a 1ct l !r' iru lim on-dEp:rinJs iH * ..- rn -rr.om nr1,J J,.. .m ir * \.luk l[-Id Jc hC,.pc rn * L >unr%-l c J- l dtrIpmc r rb17.-LhI-" p`.rnerihip, * I Lrlrr ill Jri en, Jd .1i..J * Sho:rt-ternt idii.ienienfl[ * L.r..e-trmrIn lI,-ran. .. .l b lue . l - 01 6- vrir r itr iin, Idlt.1ptC kLra1rp. * .. LTniirI- d .l.. *.*r. . r * Incentive- d i .r.oior n * i Il IC1, in.ri < rd * *c,.ur.ce. * in . ... I'. * Inl(1 ntrEcfr.1rid ,ri-[,[tti,*Fif Inf tl lI.ris c.11Iid ced |I'jnI[, h i - lin il II l L h .-IT. *\i i FP1 in-I.- rd ,i' I) c J. rmionr 0 E,:onoini-i, nd ~Ii3 l :N p-rr- e NhIilidi,Ii. in -rp..i S11713 rl r 1 du ri. 1 1': . i 1 n I p * rf 17el Lr , d ! I% i uc.r-fo rt-.1Of Th ii n.r..LI1 . -. ind uLi itr i. rl inldui fr .i. pilii int. n rrTI- *(ir. C 3 a ,I It' * ** il r: , i. * [1.iri decrrliinL rc,._.urc * Coinir dri%c, Aid ...*rd nin..m a 1i,,mUrL n rli,:, * F~ICWCr, MIsc p'.c"r.~e 0 D006i *rO P.rmi . . i adri[li: .I. i'd ~~~ ~~ 3iI 13iI inL copronr. t.Iil~ rr ~Irsn~ seat in formulating and implementing development The principles of the CDF differ significantly from strategy, and it must involve the private sector and those of the planning and adjustment eras (table. 1.1). The civil society. planning era focused on investment, and neglected Partnership. Donors policies. The adjustment era focused on policies, and and multilateral insti- neglected public investment. Both neglected institutions. tutions should harmo- By contrast, the CDF emphasizes institutions, including CDF differ significantly nize their programs governance, the judicial framework, financial interme- from those of the and practices, concen- diation, and social capital creation. It draws on both old planning and trate on areas of com- and recent lessons of development. adjustment eras. parative advantage, and work with country Long-Term, Holistic, Balanced partners in a frame- A recurrent theme of OED reports has been that work of mutual accountability. institutions matter (Picciotto and Wiesner 1998). Evalu- * Results-orientation. Success has to be judged by ation evidence confirms the need for a broad-based progress on the ground toward equitable and concept of development, including effective public sustainable growth. expenditure management, suitable institutional capa- 2 Toward a Comprehensive Devel opmi nt Stratcgy bilities, and safety nets. Recent research also highlights countries. Recent studies have concluded that there is the value of a comprehensive approach to create an urgent need for a country-led partnership approach virtuous circles (Dollar 1999; Thomas and others to development assistance (World Bank 1998i, 1999m). 1999). Governance and institutional quality are key to An OED study recommends that the Bank "let go" in growth, poverty reduction, and project success (Evans areas where it does not have a comparative advantage, and Battaile 1998; Buckley 1999). and concentrate instead on mainstreaming promising Poverty is multidimensional and location-specific. It programs, relying on other agencies to pilot new can be reduced only through programs tailored to the approaches (World Bank 1999c). Similarly, for local context (Narayan 1999; World Bank 19991). An multicountry, grant-based collaborative programs, an OED case study of health determinants suggests the need OED study concludes that the Bank should team up to go beyond the confines of the health sector-transport, with private foundations and development assistance pollution, communications, the environment, and educa- agencies with a long tradition of awarding grants. tion must all be considered to get results (World Bank Conflicting interests 1999b). Similarly, getting girls into schools and keeping among donors and barri- them there transcends the education sector--concurrent ers to progress within investments in transport, water, sanitation, and domestic countries must be ad- has compromised labor-saving technologies may also be necessary. dressed in order to achieve development reforms. The Bank can effectiveness. Ownership help forge strategic alli- Lack of ownership has compromised the development ances with other lenders effectiveness of many Bank-supported operations (World and donors to overcome these constraints and nurture Bank 1998e, 1998f, 1998m, 1999c). Projects have consensus on policy changes and capacity building tended to perform poorly when they were prepared by requirements (World Bank 1995b). Active involvement by outsiders, did not engage stakeholders and beneficiaries, senior management and well-staffed Resident Missions exceeded local implementation capacity, and did not facilitate cooperation (Kreimer and others 1998). The engender borrower commitment. Success in both invest- OED aid coordination study also highlights the need to ment and adjustment operations has been associated with build local capacity for effective aid management (World ownership and beneficiary participation.2 It is important Bank 1999m). to nurture ownership among senior government officials involved in negotiations, civil servants concerned with Results, Learning, Accountability implementation, and those affected by the intervention, as The aid business has been overly focused on inputs, well as among Bank staff (World Bank 1998m). financial commitments and disbursements, and supply- Decentralization involves risks, but can help im- driven technical assistance. An OED study on resettle- prove ownership (Van Wicklin 1999 BP; World Bank ment suggests that disappointing outcomes were 1999o).3 Over time, decentralized systems create com- largely the product of reliance on "plans" rather than mitment to reform as lower-level officials, accountable "results" as the touchstone of quality management to elected officials, move to higher posts, bringing with them enthusiasm for pluralistic, consensual, and re- sponsive modes of governance (Crook and Manor 1999 FIGURE 1.1 INTERACTION OF CDF PRINCIPLES BP). Decentralization also broadens the scope for partnerships among local government institutions, civil society, and grassroots communities. Partnership and Development Cooperation Spurred by global economic and political change, development cooperation is undergoing fundamental changes in its rationale, strategy, and mode of opera- ' tion. The development assistance system is too frag- a Ownmiup.I mented and onerous, particularly for poor and weak 3 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS (World Bank 1998k). Better performance indicators and Challenges and Tensions learning processes can help manage projects and The CDF principles, a regular staple of research and strategies toward successful outcomes (World Bank evaluation findings, are rarely practiced. We need to 1999d). understand why and find ways to overcome the An extensive evaluation literature provides evidence implementation challenges. There is tension as well as of the crucial role of monitoring for development effec- synergy among the CDF principles. The distinctive tiveness (World Bank contribution of the CDF is to have brought these well- 1999d). Results orienta- established principles together. Each is valid on its The CDF principles, a tion connected to a own, but it is their combination that promises high regular staple of learning process re- rewards, as well as significant risks (figure 1.1). research and quires mechanisms to The challenges arise from current incentives and evaluation findings, are generate and share capacity constraints within developing countries and . knowledge, both within the development assistance system. Leadership, a new a country and among authorizing environment, and broad-based ownership partners. OED's assess- of the CDF principles must be combined with system- ment of public expenditure reviews recommends a focus atic learning. on the goals of expenditure reform from the beginning, both for proper selection of expenditure instruments and for better implementation (Datta-Mitra 1997). 4 2 CHALLENGES AT THE PROJECT LEVEL rend analysis shows that the quality at exit of Bank-supported development projects has improved over the 1990s, but the most recent evaluations suggest a stalling of gains below the Strategic Compact goal of 75 percent satisfactory.' The quality at exit has taken a slight downturn from a peak of 74 percent satisfactory for projects exiting the portfolio in FY97 to 72 percent for FY98-99 exits. Moreover, institutional development impact and sustain- ability of project benefits continue to lag. The demandingness, complexity, and risk of ment, governance). The increased complexity and the projects have also climbed over the past decade, more adverse operating environment suggest that the making the modest performance gains more impres- modest improvement in results during the 1990s repre- sive.2 But stagnant performance in borrower imple- sent a substantial improvement. But the gains in project mentation has led to a growing gap between complex- performance appear stalled, highlighting the need for ity and capacity, contributing to the recent downturn in intensified attention to institutional constraints. project outcomes. Improving project performance will OED evaluators assess project results along three require greater emphasis on developing borrower ca- related dimensions-outcome at the time of evaluation, pacity. Lessons from project experience also point to sustainability of ben- the continuing importance of the Bank's appraisal and efits, and institutional Demandingness, supervisory roles. development impact. complexity, and risk of Last year's ARDE intro- p Keeping up with Complexity duced a new perfor- The changing nature of Bank-financed projects must be mance indicator that over the past decade. kept in mind when considering results trends. The combines these three di- exiting portfolio has become more ambitious relative mensions into an aggregate project performance index to the implementation environment (figure 2.1). Devel- (APPI). The latest evaluation results show a broad- opment work in traditional sectors has become more based improvement in the aggregate performance of complex, and the Bank has expanded its activities in exiting projects. more challenging areas (environment, human develop- Across exit years, the APPI shows a leveling off in 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS performance since the improvement from the decade's downturn in project outcomes combined with the steady nadir in FY94, both by projects and by disbursements (see increase in complexity suggests a possible tradeoff be- figure A1.1). Since FY97 the APPI by projects has tween the two under current conditions. To ensure future remained at 6.7, and by disbursements at roughly 7.1. performance gains, greater project complexity must be The recent results reflect met with improvements in borrower capacity and in- To ensure future stability in overall per- creased vigilance in quality assurance. formance, but also hide Over the short term, outcome performance is performance gains, the dynamics among the expected to remain below Strategic Compact levels. greater project three dimensions of the The Quality Assurance Group (QAG) reports an in- complexity must be met index. A modest down- creased share of projects exiting the portfolio with with improvements in ward shift in the out- unsatisfactory ratings, including most of the projects to comes of recently evalu- be evaluated by OED over the next six months. But borrower capacity and ated projects has been among the remaining active projects, a declining share increased vigilance in offset by an increase in are at risk of failing to achieve their development quality assurance. institutional develop- objectives, which is a promising signal for improve- ment impact (although ments over the long term. from a low base). The percentage of projects with satisfactory outcomes Varied Performance among Regions, Sectors, and at exit-projects that have efficiently met or exceeded Types of Lending their major relevant objectives-has increased over the Regions fall into three groups according to the perfor- 1990s, rising above the 70 percent threshold of the early mance of projects exiting in FY98-99 (see figure A1.3). 1990s (see figure A1.2).1 The share of satisfactory projects The top performers are Latin America and the Caribbean, exiting in FY98-99 is 72 percent, confirming a move Europe and Central Asia, and East Asia and the Pacific- beyond the trend for FY90-96, when outcome perfor- all with more than 80 percent of projects rated satisfactory mance averaged 66 percent satisfactory. But while FY98 in outcome, and more than half as likely to be sustain- is the second consecutive year in which more than 70 able. The Middle East and North Africa and South Asia percent of exiting projects had satisfactory outcomes, the rank in the middle, with roughly two-thirds of projects share is down from 74 percent in FY97, and preliminary having satisfactory outcomes. Sub-Saharan Africa ranks results for FY99 (which may not be statistically signifi- lowest on all three dimensions, despite solid improvement cant) show a further decline to 70 percent. The recent in project outcomes and sustainability. The Region had FIGURE 2.1 DEMANDINGNESS, COMPLEXITY, RISKINESS, AND OUTCOME, BY EXIT FISCAL YEAR Percent 90 - 80 - 70 - _______________ 60 30 30 FY90 FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 [ Demandingness (% substantial) N Complexity (% substantial) 0 Riskiness (% substantial) - Outcome (% satisfactory) Note: Results for FY99 are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. 6 ChalIeng e at the ProTect I QVel the second-largest gain in the share of projects with provement in outcomes since the 1980s and early 1990s- satisfactory outcomes between FY94-97 and 1998-99, satisfactory outcomes rose from 69 percent in FY90-93 to from 55 to 61 percent. 85 percent in FY98-99.s Performance has improved in Performance has improved dramatically in Latin Sub-Saharan Africa (79 percent satisfactory, up from 69 America and the Caribbean over the past decade, and percent in FY94-97) and in the Finance, Private Sector, more recently in Europe and Central Asia. The gains have and Infrastructure Net- put projects in these regions on par with those in East Asia work (94 percent, up and the Pacific, where the financial crisis has led to a from 74 percent). But Fewer than half the deterioration in outcomes and sustainability. The Latin sectoral adjustment evaluated projects America and the Caribbean Region has seen improve- lending has accounted exiting in FY98-99 are ment in all three dimensions. The gains in institutional for the bulk of the gains; development impact are particularly promising for the structural adjustment Region's future performance; 52 percent of projects lending has shown stag- results, a performance exiting in FY98-99 were rated as having substantial nating performance. similar to that impact, the highest share among all the Regions for this Despite the overall throughout the 1990s. period. In Europe and Central Asia, recently exiting improvement in the projects show solid gains in outcomes and sustainability performance of adjust- (83 percent with satisfactory outcomes and 69 percent ment lending, concerns have been expressed about its with likely sustainability in FY98-99). But performance social and environmental content. A 1996 evaluation of in institutional development impact has improved only completed adjustment operations by OED has identi- slightly, and self-evaluations of the active portfolio in the fied a need for greater emphasis on cost-effective Region suggest a diminished development impact in the management of public social expenditures, along with future. Active commitments at risk rose from 25 percent in effective safety nets to protect the extremely poor and FY98 to 47 percent in FY99, making the Europe and those vulnerable to the effects of adjustment Central Asia portfolio the riskiest in the Bank. The (Jayarajah, Branson, and Sen 1996). Management increased riskiness reflects a rapid deterioration in the reviews of the extent of recent progress are under way, Russia portfolio. following QAG and Environmentally and Socially Outcome performance varied significantly among Sustainable Development Network (ESSD) assess- sectors between FY94-97 and 1998-99, deteriorating in ments. Better dissemination of relevant Bank policy six sectors while improving in five (see figure A1.4).4 But directives to increase the congruence between policy several of the improving sectors made tremendous gains, and practice should be considered. lifting Bankwide performance. Gains were particularly strong in three sectors: the share of projects with satisfac- Institutional Development and Sustainability Flagging tory outcomes rose by 41 percentage points in public Sustainability of project benefits and institutional de- sector management (to 93 percent), by 25 points in urban velopment impact remain major concerns (see figures (85 percent), and by 17 points in transportation (93 A1.5 and A1.6).6 Fewer than half the evaluated projects percent). The latest self-evaluation data show that public exiting in FY98-99 are likely to have sustained results, sector management is a key problem area for the Bank, a performance similar to that throughout the 1990s. A however, suggesting much lower performance in the volatile external environment has suppressed improve- future. The two best-performing sectors in the FY94-97 ments in sustainability, particularly in East Asia and exit cohort-social sector and multisector-both experi- the Pacific, because of the recent financial crisis, and in enced deterioration in project performance. Performance Europe and Central Asia. Operations with large dis- declined most for multisectoral projects, with the share bursements perform somewhat better on sustainabil- showing satisfactory outcomes falling from 83 percent for ity-two-thirds of evaluated disbursements for fiscal projects exiting in FY94-97 to 65 percent for those exiting 1998-99 are judged likely to have sustained results. in FY98-99. The decline reflected lower than average An emphasis on institutional development is criti- performance for structural adjustment and technical cal if the Bank is to move beyond simple project assistance loans. financing to long-lasting improvements in developing Overall, adjustment loans have shown strong im- countries and full ownership by borrowers. At the 7 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS project level, the Bank has been making slow progress produce an institutional development impact at the in institutional impact. Still, only 39 percent of the national level.7 Among countries in which more than exiting projects in FY98-99 show substantial institu- half the evaluated projects achieve substantial institu- tional development impact. Better ways to achieve this tional development impact, three-quarters show corre- crucial objective must be found. sponding improvements in institutional quality at the Recent evaluations show marked improvement in the national level. This impact is strongest for projects that institutional impact of projects directed primarily to advance the regulatory framework, streamline public institutional development. Half deliver substantial im- sector operations, and improve the enabling environ- pact, a significantly higher share than the Bankwide ment for private sector activity. average. These focused institutional development inter- ventions, which make up only a quarter of the exiting Bank and Borrower Performance Must Improve portfolio, thus contribute disproportionately to the gains For this Review, OED conducted econometric analyses in the institutional impact of Bank-supported projects. of the key determinants of success for investment and Promising evidence suggests that projects can adjustment lending (box 2.1). The results for invest- BOX 2.1 UPDATED DETERMINANTS ANALYSIS PUTS SPOTLIGHT ON BANK PERFORMANCE his Review tory-as the dependent rower preparation turned Satisfactory quality at revisited variable. out to be insignificant. entry increases the esti- the economet- Investment projects. Compared with the mated likelihood of ric analysis of the The results for the most results for projects exiting success by 12 percent- determinants of perfor- recently exited investment in FY95-97, the most age points, Bank super- mance for investment projects (FY98-99) striking difference is the vision by 13. Political projects using a meth- broadly confirm the greater importance of economy factors also odology employed in conclusion of the earlier Bank supervision; its help predict whether a the 1994 and 1997 analyses: borrower and contribution to the likeli- project will succeed: a ARDEs. And for the Bank performance are the hood of success increased one standard deviation first time, a similar most important determi- fourfold. This finding is increase in the treatment was given to nants of project intuitive given the incumbent's time in adjustment operations, success, with Bank super- increased complexity of power at the start of using a methodology vision and borrower Bank projects and the the operation reduces developed in the Devel- implementation perfor- stagnant performance in the likelihood of opment Research mance both crucial in borrower implementation. success by 13 percent- Group. The main goal determining the final Adjustment operations. age points, and a one was to assess the rela- outcome. Other things Bank and borrower standard deviation tive impact of Bank being equal, improved performance also turned increase in the number and borrower inputs on Bank supervision increases out to be key determi- of average government the success of projects, the likelihood that a nants of the success of crises reduce it by 5 while controlling for project will be successful adjustment projects, percentage points. The country factors such as by 51 percentage points- along with political other political the macroeconomic and satisfactory borrower economy factors. economy variable, a environment and qual- performance by 43. Qual- Borrower performance dummy capturing ity of governance. iry at entry increases a during implementation is whether the incumbent Single-equation probit project's likelihood of critical: when it is satis- was democratically analysis was applied, success by 28 percentage factory, a project's likeli- elected, turned out to with outcome-satis- points, and borrower hood of success increases be insignificant. factory or unsatisfac- compliance by 20. Bor- by 50 percentage points. 8 Challenges at the Project Level ment projects show that Bank supervision has become ensuring the flexibility to adapt to changes in the the most important factor in determining outcomes. implementation environment and in working with and This makes intuitive sense, given the increased com- benefiting from partners. This process role is critical in plexity of Bank projects and the volatility of the developing a learning environment while maintaining operational environment. Bank and borrower inputs a focus on development impact. are both important for adjustment operations, but borrower performance is of greatest significance. This Project Performance and CDF Principles finding confirms the prime importance of country A sample of recent project evaluations was reviewed to ownership to successful policy reform. assess the importance of CDF principles in project For projects exiting in FY98-99, overall borrower performance.' For investment projects, the review cov- performance-based on the average of preparation, ered all four pillars of the framework. A smaller number implementation, and compliance with legal covenants- of adjustment project evaluations was reviewed to assess rose to 73 percent satisfactory, up from 68 percent the balance between attention to social issues and the satisfactory in FY94-97 (see figure A1.7). This improve- macroeconomic components of the reform program. ment in performance, consistent with last year's findings, India's Agricultural Development Project in Tamil keeps borrower inputs on par with Bank inputs. Three of Nadu provides a good example of the way synergies four evaluated projects now show overall satisfactory among policy reforms, insitutional development, and Bank and borrower inputs. public investment can help achieve broad-based, sus- Yet while overall borrower performance is up, tainable rural development. The project shows how a implementation performance continues to hover around committed borrower, with a coherent policy framework 60 percent satisfactory, and remains the weakest aspect of and decentralized project execution, can implement a borrower project processing. This stagnant performance successful, long-term, holistic program. The Tamil reflects the difficulty of improving borrower capacity and Nadu Project now serves as a model for rural develop- underscores the need for sustained emphasis on capacity ment projects in other Indian states. building. The determinants analysis suggests that im- The Ghana Feeder Roads Project shovs the benefits proved borrower implementation would lead to major of ownership and strong partnership among donors. gains in the development impact of projects. The project aimed to help the rural poor by improving Bank performance shows mixed results for evaluated access to markets and agricultural areas and by projects exiting in FY98-99 (see figure A1.8). Compared promoting labor-inten- with results for FY94-97 projects, identification perfor- sive construction meth- mance has declined from 82 percent satisfactory to 78 ods to provide employ- Determinants analysis percent, appraisal performance has stagnated at 62 ment. It also sought to suggests that improved percent satisfactory, and Bank supervision has improved integrate the transport borrower to 76 percent satisfactory. Disaggregating the Bank and agricultural sectors . supervision measure turns even this positive result into a more closely. The De- le mangain wu mixed signal, however: while the share of projects partment of Feeder lead to major gains in reporting satisfactory supervision was 80 percent for Roads, the implement- the development FY98, it dropped to 69 percent for the partial FY99 ing agency, exhibited impact of projects. sample. As the econometric determinants analysis con- strong ownership and firms, this drop is closely associated with the downturn in commitment during all outcomes for evaluated projects exiting in FY99. stages of the project. The project's institutional devel- CDF would have significant implications for Bank opment activities improved the department's work appraisal and supervision. Upstream activities are the programming and resulted in a thriving construction entry point for several key principles of the framework. and consultancy industry in Ghana. Partnership with Project appraisal would include consideration of ben- the Danish Cooperation Agency (DANIDA) and the eficiary participation and local capacity to ensure U.K. Department for International Development was ownership. And it must build in a focus on results from exceptional, with periodic formal meetings enhancing the beginning. Equally important, if not more so, is coordination. Bank supervision, which would assist the borrower in Bank loans often supply the institutional frame- 9 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS work for expanded activities that attract other multilat- Chile's Primary Education Improvement Project eral and bilateral donors. This additional funding can and India's Industrial Technology Development Project help multiply project benefits and impacts, as in highlight the benefits of building flexibility into project Poland's Environmental Management Project. implementation. An adaptable design gave India's Bangladesh's Agricultural Support Services Project and project the flexibility to respond to a changing eco- Poland's Energy Resource Development Project, both nomic environment, contributing strongly to its suc- carried out in partnership with another donor, illustrate cess. And in Chile's education project, adapting instruc- the benefits of donor tional processes in response to the results of monitoring Chile's Primary cooperation. The suc- by the Bank and the borrower ensured highly satisfac- Education Improvement cess of these projects tory outcomes. But adequate resources for effective and can be attributed in permanent monitoring are seldom provided. Many Project and India's part to joint missions, projects, such as Mauritius's Industrial and Vocational Industrial Technology the smooth exchange Training Project, fail to design clear performance Development Project of information, and the targets for project activities-and even when targets highlight the benefits of coordination of techni- are identified, supervision missions often ignore them. cal assistance. Adjustment operations face the challenge of bal- building flexibility into Where coordina- ancing a focus on macroeconomic issues with an equal project implementation. tion is lacking, projects focus on their social implications. Panama's Economic suffer. This was the case Recovery Project shows how this can be done. The for Kenya's Forestry Development Project. Originally project included a poverty reduction program to miti- designed, in collaboration with five donors, to foster gate the employment effects of labor market reforms donor coordination and promote balanced development and to reduce the cost of basic consumption needs, of the forestry sector the project was drastically pared including public transportation. The government also down after donors withdrew. But even the reduced project set up a social emergency fund to protect those most suffered from lack of cooperation, this time between the susceptible to malnutrition and disease. Bank and the borrower. 10 3 CHALLENGES IN SECTORAL AND THEMATIC PROGRAMS he difficulties involved in practicing the CDF principles vary across sectors (see Annex 2). CDF principles have been easier to practice in technology-driven sectors, such as infrastructure, than in the softer sectors, such as human development, or the social and structural dimensions of hard sectors. The challenges involved within and across CDF principles reflect dilemmas the Bank has been grappling with for decades. Five key tensions are at play: short versus long privatization in transition economies, civil service reform, term, comprehensiveness versus selectivity, speed ver- and deregulation of the financial sector (Mathieu 1998; sus broad-based ownership, partnership versus country Stiglitz 1999b; World Bank 1999c). Privatization in- capacity, and accountability for results versus local creases inequality if the appropriate regulatory frame- capacity. Resolving such tensions lies at the core of work and environment quality management in development assistance. for private sector devel- Many factors contribute opment are missing. In to the short-term Short versus Long Term transition economies, orientation of Many factors contribute to the short-term orientation of the rush to mass development efforts: the project approach, financial privatization, without development efforts, crises, political instability, the election cycle, and the establishing the under- yet the most incentive systems of the civil service in developing pinnings of capitalism, fundamental problems countries, as well as the incentives and planning led to corrupt sales, lack processes of donors and the Bank. Yet the most of restructuring, insider- fundamental problems of development, such as institu- dominated transactions, long-term strategies tional development and governance, require long-term and unregulated actions and sustained efforts. strategies and sustained efforts. by investment funds. A long-term perspective is especially important in The more ambitious the reform, the more time and dealing with the structural dimensions of reform. Among resources are needed to prepare the way. A long-term recent examples of failures to take the long view are commitment is essential to success. OED's study of 11 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS financial sector reform found that countries with social fund agencies are particularly vulnerable because unsatisfactory outcomes averaged only 1.5 reform of the programs' weak links to existing government operations dealing with the financial sector over a 12- structures. The Bolivia Social Fund's emphasis on speed year period; countries with satisfactory outcomes aver- and autonomy from line ministries worked against fitting aged 2.3 operations (Effron 1999 BP). Equally, resettle- projects into sectoral plans. The assessments for social ment operations call for involvement of affected funds in Armenia, Ecuador, and Peru highlighted the need communities many years before the infrastructure for complementary actions, such as funding educational investments take place (World Bank 1998j). material, equipment, and other inputs along with infra- structure in school projects or including training for water Comprehensiveness versus Selectivity and sanitation projects. According to the Ecuador benefi- More comprehensive approaches frequently imply ciary assessment, if the social fund does not permit the greater complexity and implementation difficulties for financing of complementary works, serious consideration sectorwide and multisectoral programs. Thematic, should be given to not financing the project. "Inattention cross-sectoral, and structural dimensions are particu- to complementary requirements can . . . put at risk the larly challenging for sector-bound aid agencies and impact of the project, not to mention the satisfaction with, government ministries. For example, it took decades of use, and maintenance of projects" (Carvalho 1999b BP). external pressure and top management leadership to Multisectoral program services have been difficult bring environmental considerations into the Bank's to sustain without good coordination with ongoing operations. Gender, public sector management, and programs of sectoral ministries. In Mali's integrated private sector development issues raise similar chal- Health, Population, and Rural Water Supply Program, lenges (Murphy 1997). Cross-sector themes are often implemented by different line agencies, the population ignored or resisted by sector-bound organizations, both and water supply components were not always well within the Bank and in developing countries. coordinated with the health component. With the Following the debt crisis, the Bank broadened its involvement of many agencies, programs proved diffi- view to address systemic problems. Although now more cult to monitor-and implement. The challenges of relevant, the success rate of the more demanding managing multisectoral, multiagency programs are financial sector operations dropped to 50 percent, even compounded by weak incentives and mechanisms for prior to the 1997 finan- intersectoral coordination, both in countries and within The challenges of cial crisis.' An OED the Bank (Johnston 1999 BP). managing review of recently Integrated programs may also generate tensions closed operations sup- between line agencies and oversight bodies, such as multisectoral, porting financial sector ministries of finance. Activities requiring recurrent multiagency programs reforms shows that suc- funding can create ownership conflicts between central are compounded by cess is often attribut- and local governments, especially in non-revenue- weak incentives and able to government earning operations such as highways. Implementation ownership and com- of a Thailand highways project, an integrated interven- mechanisms for mitment, consensus- tion designed to address cross-sectoral issues, was intersectoral building, a favorable marred by conflicts among agencies. Implementing coordination, both in political climate, and agencies were strongly committed to the physical countries and within good policy dialogue works, but less committed to policy reforms, in part (Mathieu 1998). because policymaking rested elsewhere in government the Bank. A focus on discrete (Levy and Reja 1999 BP). Similarly, urban develop- investments rather than ment projects involving multiple sectors have provoked integrated packages of investments has been a frequent detrimental competition among oversight agencies, feature of multisectoral programs, as noted in a recent making them unmanageable. While the Bank has review of social protection activities based on beneficiary approved no new Integrated Urban Development assessments (Carvalho 1999b BP). Going around minis- Projects since 1986, the performance of some recently tries has advantages, but these are often enjoyed at the completed operations, such as the Brazil Salvador expense of sustainability. Services provided through Metropolitan Development, suffered from needlessly 12 C halienges in Scctoral dI Thc m atic P'rogramin complex designs involving disparate activities imple- priority setting and capacity building, the Bank has mented by too many agencies (Gilbert 1999 BP). helped a number of governments address environmental Sectorwide approaches are necessarily ambitious, issues through National Environmental Action Plans and complex, and demanding of Bank involvement, espe- programs to strengthen national and local environmental cially supervision. In the energy sector, for example, both institutions. But a 1996 OED review revealed that the the sector development model and the Bank's agenda have environmental plans had generally not elicited local grown in complexity. The global move from public ownership. Many of the plans were prepared in haste and monopolies toward privatization and deregulation has driven by deadlines that left little time for participation. required tackling a much broader range of issues: sector Making the plans a requirement for lending further unbundling, private participation, regulation, competi- eroded country support. tion, interregional trade, resettlement, environment, ac- The interests of different ministries (and the priori- cess by the poor, and renewable energy sources, among ties of the center and the districts) can vary, and even others. Operationalizing a sectorwide approach through conflict. Uneven stakeholder commitment and weak programmatic lending has had mixed results. Energy capacity can pose risks. OED evaluators found that the sector loans to Pakistan, the Philippines, and Turkey fell success of health short of their objectives because they were too complex- sector projects was Speed often compromises the FY89 Pakistan energy sector adjustment loan had significantly corre- more than 40 conditions (Barbu 1999 BP). Phased or lated with how well ownership. incremental approaches have been more successful. In program designers China a succession of power sector operations of incre- had assessed ownership by key stakeholders, including mentally increasing policy intensity succeeded because of concrete evidence of commitment (Johnston 1999 BP). a realistic assessment of institutional capacity, judicious In education, too, the number of stakeholders is very use of technical assistance loans to build greater capacity, large, with many agencies and institutions involved in and effective use of economic and sector work. But a executing policies (Dove 1999 BP). Responsibility for gradualist approach may be difficult for the Bank to selecting policy reform measures and deciding on replicate in countries where the assistance strategy can mechanisms to encourage support must come from accommodate only one energy operation every few years, within the country and be grounded in broad-based or where strong government ownership may develop only support for reform. The clear implication is that during brief periods of political opportunity, as in borrowers should be encouraged to take a leadership Argentina and Bolivia (Barbu 1999 BP). role in the preparation of projects and that all stake- holders, including women and the poor, should be fully Speed versus Broad-Based Ownership engaged. The lack of government ownership of reforms or commu- Thematic strategies have a better chance of being nity ownership of local projects has undermined develop- successfully implemented when a range of public and ment efforts. OED evaluations show that ownership is private stakeholders participate. Lessons from The difficult to achieve in sectors that have a broad array of Gambia, Ghana, Madagascar, and Mauritius suggest stakeholders with different interests, such as health and that this holds true for the environment. In Madagascar education (Dove 1999 BP; Johnston 1999 BP); in thematic and Mauritius, national environmental action plans and structural areas, such as environment, rural develop- were able to increase local environmental capacity ment, and civil service reform; and in sectors such as more than in some other countries because of substan- agriculture, where resources provided through state chan- tial local ownership. The more participatory plans nels are under pressure. Partners may have different views were also successful in information gathering and of the roles of the state, the private sector, and civil public education (Liebenthal and Mani 1999 BP). society. Coalition building and media campaigns to overcome vested interests or hold the bureaucracy to Partnership versus Country Capacity and account may not be feasible. Governance reforms may be Transaction Costs required to institutionalize participation, and this may Partnerships of external partners may be essential to take decades to accomplish. coordinate sectorwide programs and reduce demands Speed often compromises ownership. In a spirit of on government capacity. OED evaluators have spot- 13 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS lighted examples of effective coordination with donors * Too little involvement of partners and cofinan- in such sectors as transport, telecommunications, and ciers in project design and supervision energy. In other areas, however, partnerships have a * Divergence, sometimes serious, between exter- long way to go, especially in rural development, nal financiers and the Bank on stated project education, and private sector development. objectives While aid coordination can reduce demands on * Discrepancies in sector policies and procedures government capacity, a lack of implementation capacity for program implementation can still undermine reform. An evaluation of the Ghana * Little interest in contributing to the Implementa- Private Sector Adjustment concluded that "required exper- tion Completion Reports (ICRs), depriving the tise should be on board before the process begins, and reports of the views of cofinanciers potential legal issues, like land transfers, which proved * Problems with cofinanced technical assistance, problematic in Ghana, should be carefully reviewed in such as reluctance of cofinanciers to replace advance" (Galenson 1999 BP). An evaluation of inappropriate technical assistance personnel Jamaica's Private Sector Development Adjustment high- * Dropping out of the project at appraisal after lighted the need to assess borrower capacity to implement expressing support during preparation. reforms, as well as borrower commitment. The borrower's own evaluation was highly critical of unrealis- Accountability for Results versus Local Capacity tic demands by the Bank on Jamaica's weak implementa- In tracking the development impact of projects, the Bank tion capacity (Galenson 1999 BP). has been weak, almost across the board. Monitoring and Partnership and coordination imply high transac- evaluation (M&E) for results has been easier to achieve in tion costs. In a Ghana health operation, the Bank infrastructure lending than in structural lending (institu- adopted a sectorwide approach but provided only tions, public and private sector development), social marginally higher supervision resources than for a lending (health, education), or thematic lending (environ- conventional investment project. This limited the ment, rural development). OED evaluators have consis- Bank's ability to establish local presence, include tently identified weak M&E capacity (including tracking appropriate technical expertise in supervision missions, inputs and outputs) and the need for greater attention to or adequately partici- sectorwide and thematic efforts. Decentralization, pate in coordination privatization, and weak regulation have all compromised meetings (Johnston monitoring and evaluation. Chile's power sector is development impact of 1999 BP). typical: the combination of unbundling, privatization, projects, the Bank has A better alternative and weak regulation led to a deterioration in monitoring been weak, almost may be to promote stra- and evaluation. A clear lesson is that the Bank should across the board. tegic selectivity-that focus on sectorwide M&E as an integral part of its is, to share responsibil- assistance to setting up regulatory frameworks. A broader ity among donors, to M&E issue is that of cross-sectoral links and the dearth of rely more on pooled technical assistance support under data on the social impact of energy policies and sector government control, and to resist the inclination to take reform. on all tasks and cover all bases.' More intense supervision Identification of relevant indicators has been a of increasingly complex Bank-assisted projects will un- challenge, both at the technical level (balancing process doubtedly raise their effectiveness, but complex projects with outcome indicators, and ensuring coverage of key also involve opportunity and transaction costs for govern- issues) and in negotiating a limited list among key ments: the attention of senior officials is directed to Bank stakeholders. A concern expressed in Ghana's health missions at the cost of neglecting other issues that may be sector program was that the indicators placed greater more pressing. Donor pressure also diverts a dispropor- emphasis on donor priorities than on the priorities of tionate share of scarce local budgets and staff to service a national stakeholders or consumers. In addition, institu- bewildering number of donor projects. tional mechanisms have rarely been crafted in order to An OED review of completed infrastructure create incentives for increased performance at the district projects raises a number of issues surrounding the role and facility levels. Excessive emphasis on process indica- of external financiers (Barbu 1999 BP): tors (decentralization, budget allocations) may compro- 14 Challenges in Sectoral and Thematic Programs mise achievement of outcomes (as in Zambia) (World in reducing poverty, what does not, and why. Few rural Bank 1999d). projects have supported those who work with the poor or M&E in thematic programs has also been weak, enhanced monitoring of resource allocations to the poor. particularly in newer dimensions such as gender, Indicators used were generally input measures, such as the informatics, and the cross-cutting areas of environment number of personnel trained or wage expenditures, rather and poverty. The Bank has rarely used its lending than outcomes. Seldom were these gender-disaggregated portfolio to systematically collect evidence on what works (Liebenthal and Mani 1999 BP). 15 4 CHALLENGES AT THE COUNTRY LEVEL his chapter assesses Bank Country Assistance Strategies (CASs) from the perspective of the CDE It focuses on results-growth and poverty reduction over the long term. It also highlights the tensions in implementing principles of the CDF at the country level and identifies examples of good practice. The analysis covers 28 countries that have received OED Country Assistance Evaluations (CAEs). These evaluations were prepared to fit the Bank's CAS schedule, but relevance, regional coverage, and importance to the Bank's portfolio were additional selection criteria. The sample thus includes These findings confirm the view underlying the CDF: developing countries that range widely in size and that the battle against poverty is being lost and that income, and achieves regional balance.' business as usual will not accomplish the objectives of the The facts of growth and poverty in the 28 countries development community. between 1981 and 1997 are sobering (see Annex 3): Addressing Social Issues * In 40 percent of the countries, per capita income The CDF conceives of development as a social transfor- either failed to grow or shrank. mation process. Increased emphasis on social services * In 25 percent, the share of the population in forms part of the new development consensus. The absolute poverty increased. United Nations Children's Fund (UNICEF) and the * In 23 percent, life expectancy declined. International Labor Organization, along with other * In 54 percent, the people experienced stagnating observers, have argued that the inadequate progress in per capita income, rising poverty, declining life reducing poverty can be traced to neglect of social expectancy, or a combination of these events. priorities under adjustment programs (World Bank * In 85 percent, per capita income grew 1 percent a 19991). Yet the country evaluations show no significant year or less in the 1990s. decline in the share of social spending in government * In 59 percent, gross savings as a percentage of GDP budgets during the period under study; countries such were low (less than 10 percent) or declining. as C6te d'Ivoire, Ghana, Morocco, Poland, and Togo * In 67 percent, investment efficiency was less than fully sustained their social spending. 10 percent or declining.' OED's forthcoming study on poverty shows that 17 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS many countries launched social protection and develop- expense of girls. Health sector spending reflects similar ment programs during the 1990s, some with financial biases: more is spent on hospitals than on basic health support from the Bank. An earlier OED report on the centers, on curative than on preventive care, in capital social dimensions of adjustment concluded that in most cities than in rural areas, and on wages than on basic countries that imple- supplies (particularly generic drugs). Social problems stem mented adjustment poli- cies, "real spending per Addressing Institutional Capacity and Structural Issues more from the head on health, educa- Disappointing growth is associated with neglect of inefficiency and anti- tion, and social security structural issues. Many OED reports emphasize a gap poor bias of social and welfare programs between institutional capacity and policy reform. A programs than from either rose during the 1996 review highlighted the importance of institutional adjustment period or re- and capacity development issues over and above bounded soon after .. . macroeconomic stability, price liberalization, trade allocations, few of the Bank's early reform, and wage and interest rate liberalization.4 The adjustment programs report emphasized the need for prudential regulation provided for safety net programs, but such programs have and banking supervision, laws for entry and exit of become more common" (Jayarajah, Branson, and Sen firms (bankruptcy), technology development, technol- 1996: xi). ogy transfers and licensing, arbitration mechanisms to Social problems stem more from the inefficiency settle labor market disputes, labor force training in and anti-poor bias of social programs than from lack of cooperation with the private sector, and improved budgetary allocations. The CAS for C6te d'Ivoire notes information on market opportunities, particularly for that the unfavorable cost-benefit ratio in education was exports.5 produced by four factors-abnormally high wages for Sequencing is an important part of the CDF. teachers, low internal efficiency caused by high drop- Problems arise when financial sector liberalization out rates, low employment potential, and inequitable precedes the development of a regulatory and supervi- access.' Public spending on education typically benefits sory system, or when privatization comes before a the wealthy at the expense of the poor, and boys at the sound framework for regulation and competition. FIGURE 4.1 THE BANK'S CONTRIBUTION TO IMPLEMENTING CDF PRINCIPLES Etlane.m poky dialogue and sec ror wyork____________ BaLuictin lending operationis Hsi 14 1M Btng capac ir) t for go%ernance Holistic___________________ InteT-wctor coordination Long-term view of client potential and constraints Client commitment to reforms Ownership Country-driven strategy formulation/implementation ,A I : Encouraging stAehoAder participation abidig,for country-driven aid coordnation Partnmshilp Encouraging stfategic scklcivity Harmonization of donor procedures Tracking sustainable growth Tracking poverty Results Focus Development of local capacity for M&E Outcomes, impact, and monitorable indicators 0 10 20 30 40 50 60 70 80 90 100 Percent of countries rated modest-negligible 18 Challenges at the Country Level These problems were highlighted in several CAEs. In fell significantly in the 1990s, to less than 2.5 percent of addition, OED's forthcoming study on poverty high- the portfolio in FY99. Still, few CASs explicitly address lights the role of the composition of growth in poverty issues of income distribution, or propose interventions alleviation-rural growth, in particular, was found to that directly address inequality (as distinct from inequi- be critical in reducing poverty. ties in access to services). Despite the stated goal of pro- poor growth, policy prescriptions have deviated little A Long Way to Go from those of the 1980s. The CDF principles imply a different approach to OED's 1999 review of poverty assessments finds managing country assistance programs. For this report, that nearly half do not adequately assess individual a new calibration was tested to determine what the elements of the poverty reduction strategy.6 They fail performance rating of the Bank would have been if the to address the links between poverty and such macro- CDF principles had been the agreed benchmark. To this economic policies as trade and exchange rate policy, or end, the principal authors of CAEs were asked to rate such sectoral issues as food and agricultural policy and the Bank's contribution in helping client countries rural development. They give only limited attention to implement CDF principles on a scale of 1 (poor) to 4 local dimensions of poverty reduction, including in- (excellent) (figure 4.1). These experienced evaluators vestment decisions. And while all assessments recog- rated the Bank's contribution to implementing the nize the importance of labor-intensive growth to pov- principles of the CDF as modest or poor in nearly two- erty reduction, few have analyzed this issue. thirds of cases. This suggests that country assistance Slightly more than a third of the sampled poverty practices will have to change substantially when the assessments achieved a high country-level impact-as CDF is mainstreamed. While the principles are not measured by the influ- new, the Bank's commitment to giving them priority is. ence on the poverty de- Despite the stated goal An upcoming OED report on poverty has reviewed bate, policy design and country assistance strategies, and preliminary findings implementation, and of pro-poor growth, indicate that (World Bank 19991): institutional develop- policy prescriptions ment in the country- have deviated little * More than 70 percent focus on three main policy while slightly under a from those of the areas-trade and tariff reform, macroeconomic third had a low impact. 1980s. balance, and liberalization and deregulation. The impact of poverty * Fewer than 50 percent address other aspects of assessments is strongly the structural agenda, such as regulatory poli- correlated with their quality, with the adequacy of time cies, support for the informal sector, or wage and for their preparation, and with the degree of partner- employment policies. ship, consultation, information sharing, and knowledge * Roughly one in five contains a good discussion of transfer at the country level. A survey of stakeholders distribution or equity issues; one in three covers shows that while six of seven were satisfied overall issues relating to the sectoral composition of with the poverty assessment, they viewed knowledge growth. transfer and local partnership and consultation as the * Fewer than 40 percent emphasize policy support least satisfactory aspects. for the informal sector or wage and employment An OED study of public expenditure reviews finds polices directed at removing labor market con- that, except in the cases of Ghana and Zimbabwe, the straints for the poor. reviews remain primarily internal Bank documents that evince little sense of ownership from the government There has been a significant shift in Bank lending, (Shah and Girishankar 1999 BP). Client governments away from traditional investments in agriculture, and do not believe that they can influence the process or the toward investments in human capital development. output. Some perceive the reviews as an attempt by the Agricultural lending declined from 27 percent in the Bank to micromanage expenditure policies. Even 1980s to 16 percent in the 1990s, while lending for where the government is heavily involved in the human capital development increased from under 7 process, the impact on policy is often weak because of percent to more than 21 percent. Lending to industry inappropriate sequencing or governance constraints. In 19 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS brief, implementation of the CDF principles will call tions costs include: divergent and complex administra- for continuing improvements in country assistance tive procedural requirements and the numerous mis- practices. sions countries must accommodate. For highly aid- dependent countries, these burdens can add up to Managing the Tensions in the CDF thousands of reports and missions each year. The principles of the CDF pose important challenges. Four are highlighted here. (Chapter 5 proposes some * Poor accountability record versus scaling up. promising approaches to address them.) Despite continued emphasis by OED on the importance of M&E for learning and account- Ownership versus conditionality. How should the ability, the record remains far from satisfactory. apparent conflict between country ownership and The international development goals and recent donor interest in performance (often enforced attention to governance reinforce the need for through conditionality) be resolved? How should enhanced accountability for results. the need for ownership be reconciled with the need for policy reform and sound development priorities Past activities to build M&E capacity have focused when country commitment is lacking? at the project level to satisfy donor requirements. The resulting lack of domestic ownership of M&E has Conditionality is widely viewed as a crude attempt to undermined the use of its findings and the acceptance of generate policy reform in exchange for grants or loans. performance measurement for sound governance. The OED studies have shown that when applied as a one- limited capacity created through donor-driven, project- sided, coercive instrument, conditionality can be counter- based M&E has been dissipated at project completion. productive and incom- The CDF raises the bar by emphasizing learning and The CDF raises the bar patible with ownership. accountability for results at all levels-well beyond Some observers have projects. by emphasizing rushed to declare all learning and forms of conditionality * Country focus versus global public goods. With accountability for a failure. This conclu- globalization, development problems increasingly results at all levels- sion is not supported by require multi-country efforts and strengthened links the evidence. But the between national strategies and international poli- principles of ownership cies and programs. How can international efforts and partnership clearly aimed at global public goods complement national call for reconciling the accountability of donors and efforts through the CDF? countries through reciprocal conditionality. As global integration deepens, the number of devel- Country-led partnership versus country capacity opment problems calling for supranational policy re- and commitment. How should donor demands for sponses grows. These cross-border challenges arise from a financial accountability be reconciled with client- combination of market failure, government failure, and led partnership, particularly when countries lack systemic failure. The challenge of overcoming such capacity? How can donors play to their compara- failures creates a new role for development assistance. tive advantage, and thus maximize the effective- The country focus remains critical, but official develop- ness of the development assistance system? ment finance also must help meet the growing deficit in the supply of international public goods. Two enabling conditions for country-led partner- Global forces (including technological change) are ship are often missing: country commitment to sound creating not only far-reaching growth opportunities, policies and development priorities, and institutional but also a host of potential problems-including capital capacity to manage and coordinate aid (World Bank flight, financial contagion, illicit drug trade, cross- 1999m). Most donors and international agencies, in- border environmental problems, the spread of disease, cluding the Bank, continue to impose barriers to waves of migrants and refugees, and loss of bio- country-led coordination. These aid-delivery transac- diversity and cultural heritage. Building purposeful 20 C haII c n gcs at the Country Level and inclusive partnerships is as important in addressing institutions governing financial intermediation and these transboundary development challenges as it is in state-society relations (particularly service delivery). tackling national challenges under the CDF. Albania's experience highlights the need to go beyond traditional monitoring of macro and fiscal trends. Country Cases Development assistance strategies in several countries Bangladesh have gone some way toward implementing the prin- Bangladesh was thought to have little chance of survival ciples of the CDF and resolving some of the inherent after independence in 1971. The Bank, in partnership with tensions. Six such cases offer lessons of good (and other stakeholders, did much to help the new country sometimes bad) practice. grapple with its myriad challenges by following a holistic, long-term approach in providing support. Since Albania then the country has The Albania review highlights some important CDF made remarkable gains Despite-or perhaps lessons. While it stresses the need to be holistic, client- in both macroeconomic because of-its focused, and results-oriented, it reveals that, with a and social development. CDF approach (like most things), the devil is in the The Bank ensured that breadth, the strategy details. Three lessons emerge. the country was not just lacked depth, as well First, a holistic approach requires selectivity. The the owner of the develop- as a focus on the 22 projects supported by the Bank between 1992 and ment program, but also financial sector and 1996 covered the social sectors, infrastructure, power, its author. It played a agriculture, the enterprise and financial sectors, and supportive role in key governance. the environment. Ostensibly the Bank was being sector programs-food holistic. Despite-or perhaps because of-its breadth, production, family planning, and education-within the the strategy lacked depth, as well as a focus on the framework of a comprehensive development program financial sector and governance, which would have prepared by the country. The Bank encouraged broad produced the greatest impact on long-term develop- participation and active NGO involvement in formulat- ment prospects. ing and implementing programs. Finally, it allocated Second, it may not be easy to identify who is in the adequate resources to the country assistance program and driver's seat. During 1992-96, there was strong owner- contributed to improved aid coordination. ship by the government of many key components of the Bank's program, including the initial stabilization Bolivia program, enterprise reform, and agricultural adjust- The case of Bolivia demonstrates the need to go beyond ment. Yet in the year preceding the crisis, the govern- improved macroeconomic management to achieve ment exhibited arbitrary behavior that contributed to poverty reduction. Although the New Economic Pro- polarization. By the time the pyramid schemes col- gram (NEP) launched in 1985 with strong Bank and lapsed in 1997, the country was poised for a conflict donor support was remarkable in achieving stabiliza- that pitted the president and his predominantly north- tion of the economy, it did little to achieve poverty ern supporters against groups from the south. Owner- reduction. Social and structural reforms lagged until ship by government alone is not enough; it must be 1993-97, when a "made in Bolivia" program to broad-based. change the role of the State was launched; it included Third, keeping track of results without assessing health, education, and income-generating programs. In sustainability can be misleading. In the early years of August 1997 the new government initiated a participa- Bank involvement, Albania registered high growth tory process known as the national dialogue, and rates (nearly 9 percent annually). Agricultural produc- crafted a comprehensive framework for development, tion boomed. The government successfully completed a presaging many features of the CDE mass privatization program. By all accounts, Albania The Bolivian experience offers three lessons with was the hope of southeastern Europe. By 1997, it was broad application. First, ownership of the reform apparent that the good news on growth and agricul- program evolved slowly, finally extending to all major tural production obscured serious problems with the segments of civil society after 14 years-after the 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS benefits of the early reforms had become apparent. into agriculture, education, and social and transport Second, establishing partnerships among aid agencies infrastructure. When oil revenues dropped, demand was a huge challenge. Many donor-financed consult- restraint combined with a flexible exchange rate policy ants took line management rather than advisory posi- strengthened incentives for non-oil exports. Over time, tions, which reduced, rather than reinforced, ownership the dependence on oil declined, poverty was reduced, of reforms. Finally, vested interests often stalled the and social services improved. There was also good reforms. The struggle against corruption and toward partnership and strong government ownership; loan reform of the civil service, Customs, and the judiciary conditionality was no longer essential. faced many difficulties that could be resolved only In the years leading to the crisis, complacency set through the continued involvement of all development in, and both the Bank and government ignored gover- participants. nance weaknesses, financial sector dysfunctions, and a fragile social situation. The eventual crisis, rooted in Ethiopia the regional recession, declining export markets, and With a GNP per capita of US$110 and the vast excessive foreign borrowing, exposed cracks in the majority of its 60 million people living on less than Bank's assistance strategy. US$1 a day, Ethiopia is one of the world's poorest countries. Yet since 1991, when the country opted for a Uganda reformist approach to development, it has made im- The CDF emphasis on client ownership, a holistic and pressive economic gains. The Ethiopian experience long-term approach, and stakeholder partnership is not highlights the importance of the commitment of key new to Uganda. The country team focused on many key stakeholders, and their concerns of the CDF as early as 1987. Having The Ethiopian ownership of develop- identified a group of reformers in government, the experience highlights ment programs, as International Development Association (IDA) sup- well as the need for ported the government's rehabilitation program. In the importance of the relationships built on addition to financial support, IDA assisted with policy commitment of key trust. Because of the dialogue, training, and technical assistance, not only to stakeholders, strong relationship be- identify reforms, but also to set out the analytical as well as the need for tween the country's rationale in favor of market reforms and establish r s leadership and the benchmarks for the measurement of progress. relationships built Bank team, the Bank The Bank adopted a holistic approach. Well aware on trust. sometimes accepted that the success of the macroeconomic reform program government positions hinged on the adoption of far-reaching reforms in both (on the size of sector programs, for example), although the financial sector and the key agricultural sector, the not entirely convinced they were the correct choice. An Bank reinforced the government's reform program with important lesson: policy advice is more likely to be a battery of macroeconomic and sector adjustment effective if the client feels that its views are considered, operations and technical assistance credits. The Bank and the tone is neither hectoring nor patronizing. also made extraordinary efforts to keep donors in- volved in reform through co-financing and technical Indonesia assistance. In Indonesia, the Bank's long-standing strategy success- Since FY92, the country's reform program has been fully supported many of the development outcomes transiting from macroeconomic adjustment programs summarized under the CDE Growth was substantial, into a plethora of microeconomic, sector, and thematic comprehensive, and balanced. Economic management strategies. While holistic, these strategies are difficult kept the economy on track through boom and bust. to pursue simultaneously and pose a challenge to Dutch disease was avoided. The use of oil windfall consensus-building. revenues was spread over time, and they were plowed 22 5 PROMISING APPROACHES his chapter outlines some promising approaches to address the challenges and tensions at the project, sector, and country levels (box 5.1). These approaches are synthesized from the lessons of experience and evaluative evidence presented in the background pa- pers prepared for this report (and available on request). More systematic learning is needed to en- rich the tool kit. Learning Process, not Blueprints Inevitably a tradeoff between detailed analysis and The blueprint approach has been a common pitfall. It up-front design and adaptation during implementation is seems to simplify decisionmaking and reduce uncer- involved. The new approach implied by the CDF requires tainties, while it imposes standard solutions in poorly a significant shift of re- understood sociopolitical contexts and varied local sources from program realities and ignores social capital and local institu- design to supervision, Building capacity to tions, sustainability and learning, and the capacity to implementation sup- learn and act adapt during implementation. port, and participatory strategically is as OED lessons indicate that adaptive experimenta- M&E. A fundamental important as preparing tion and sustainable learning through multiple initia- lesson from both devel- tives yield better results than a one-size-fits-all best opment planning and p practice blueprint.' A learning process means starting corporate experience is small, building in a bias for action, avoiding new that strategies emerge from continuous interactions of top- bureaucracies, and supporting multidisciplinary teams. down and bottom-up learning processes (Hanna, It emphasizes flexible, evolutionary, participatory, Agarwala, and Chakrapani 1999 BP). Top-down ap- goals-oriented, and client-driven processes. It calls for proaches are best combined with bottom-up approaches to thinking thematically and managing across sectors, enable local initiatives, identify and scale up successes, without undercutting professional rigor and account- promote stakeholder commitment, and ensure learning ability. Building capacity to learn and act strategically and adaptation over time. is as important as preparing plans. Adaptable Program Loans (APLs) embody some 23 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX 5.1 CHALLENGES AND PROMISING APPROACHES Challenge, and Lten,ionN Apprachei * Short Versus long term * Learning proces., not hlHcprrnr4 * Cornprchensi% enes versus selectiviQ * Managing complexi 'i equncinc * Comprehens! e anaksi. nd selectie crions * Sectorv ide approache' tailored to clunnric, and sectors * Owvner%hip VerIIIN condirionalir% * Adaptable ndinoanli, f r i mutual Commitment procc,s * Speed Nersus broad-based ownership * Buildim congen us * Broadening partikip.i.n * Accountabilic% for results versus local capacity * Infornintion for accun1t.ibihEl and learning * Poor accouruabilt rvc.,rd versus scaling up * Capacitk building to mana-:c for results * Partner.hip iersus country capacity and * From aid coordination c- dcid -.,pnbnt transaction costs partnership and capa,it; buildin4 * Country focus % ersU; flubal pub god * Links bet%%een global aid Count st [13Ec2ic aspects of the learning process approach-agreement on inadequate selectivity. Fiduciary requirements are becom- long-term objectives and broad directions, with design ing more demanding. There is pressure to add compo- limited to start-up processes and institution building. A nents to deal with the Bank's expanding agenda and learning process was embedded in Brazil's Water and survive the internal approval process. Career and budget- Sanitation for low-income settlements (the PROSANEAR ary incentives encourage the design of large projects and Program). Design was demand-based and iterative, discourage priority setting (World Bank 1998a). At the shaped during implementation by beneficiary participa- country level there is often a desire for large resource tion, feedback, and transfers, full-scale national coverage, and "empire build- One way out of crippling learning (Pitman 1999 ing" by the implementing agency. complexity is to take a BP). The program de- One way out of crippling complexity is to take a veloped partnerships long-term view by sequencing a series of projects l among residents for the within a long-term strategy that builds on past learn- selection and manage- ing. The real issue is often premature complexity. ment of water and sanitation systems. Community mobili- Projects that build on past learning and are integrated zation and group decisionmaking were carried out into existing practices can be complex and successful, differently in each community, depending on factors such as the Bangladesh Population Project and the Brazil as levels of social cohesion and organization. In an Health Program show.2 "Repeater" projects have adaptive learning approach, the program encouraged higher rates of success, especially when carried out ongoing evaluation of each community's experience for within a strategic and long-term framework. rapid feedback to the next subproject. Another promising approach is to phase in cover- age, and to expand geographical scope in line with Managing Complexity by Sequencing government capacity to manage policy reform and Project complexity has been rising since the early 1990s implement the program. Accordingly, growth and (Chapter 2). Managing the tradeoffs between comprehen- poverty alleviation interventions should be piloted siveness and selectivity requires an understanding of the regionally, and progressively tailored and broadened. sources of complexity. Bank thinking about development This regional-focus approach is enhanced by explicit has become more complex. Insufficient "upstream" sector attention to capacity building and decentralization, as work before project preparation and appraisal leads to implied in the Bank's Rural Vision to Action Strategy 24 1romniising Approaches (Nelson 1999 BP). Learning and Innovation Loans Sectorwide Approaches (LILs) offer a suitable tool for managing complexity by A sectorwide approach provides an opportunity to shift starting small. Adequate supervision resources should attention from inputs to monitoring against agreed be available to ensure learning during implementation. intermediate and outcome indicators. The Ghana and Bangladesh sectorwide approaches are good examples. Comprehensive Analysis and Selective Actions They have been supported by two programmatic Comprehensive analysis can be combined with strate- investment instruments: sector investment and mainte- gic investments. It is best conducted with key partners nance loans and APLs. The sector investment and with a view to exercising selectivity in line with a maintenance loan is intended to bring sectoral invest- partner's comparative advantage and country capacity. ments, policies, and performance in line with economic Participatory approaches can be used to enable poor priorities and to ensure efficient operation and mainte- people to analyze their realities, express their priorities, nance of investments. The focus is on institutional and make explicit their choices and tradeoffs to improve capacity to plan, implement, and monitor investments selectivity and results. The Participatory Poverty Assess- (Jones and Lawson 1999 BP). ment process holds promise, as evidenced by its recent APLs are especially well suited to the support of application to the Ghana and Uganda programs, where it sectorwide approaches, particularly in sectors where brought the realities and priorities of the poor to the financial flexibility is needed. A recent review con- attention of national policymakers (Van Wicklin 1999 cluded that several operations would probably not have BP). For participatory approaches to lead to superior been brought to the approval stage without this new selectivity and results, participants must have access to instrument, because of difficulties in accurately predict- information, options, and learning experience, as social ing activities, costs, implementation arrangements, fund programs have shown.' and results beyond three or four years. An example of this approach is the sustainable Moving from projects to a full-scale sectorwide livelihood approach, which emphasizes people-cen- approach (with pooling of donor finances) is risky if tered development in a holistic framework. Sustainable done prematurely (Jones and Lawson 1999 BP). Such a livelihoods provides an analytical structure for under- change takes time and systematic capacity building. Its standing the factors that influence poverty and to pace should vary ac- identify where interventions might best be made. This cording to the quality The fiduciary risks are approach has been adopted by the U.K. Department for of macroeconomic and International Development, the United Nations Devel- public expenditure higher for sectorwide or opment Program (UNDP), and CARE, among other management, sector- programmatic lending groups. It proposes an integrated and dynamic way of level policies and re- than for project lending, understanding poverty and thinking about poor sources, quality of sec- since more fungible people's livelihoods-the capabilities, assets (human, tor-level management, forms of financing are natural, financial, social, and physical), and activities degree of aid depen- required to earn a living (see Annex 4). The approach dence, and other coun- included. But these risks builds on what people have and how they live their try- and sector-specific should be balanced lives to add to their accumulation of assets and remove factors. Because sec- against the costs of barriers to the realization of their own livelihood torwide approaches business as usual. choices. add to program com- Central to this approach is a recognition of people's plexity for donors, diverse livelihood goals-such as health, income, or more resources are required for supervision and imple- reduced vulnerability-and the complex household mentation assistance. Risks should be managed by strategies adopted to meet them. Strategies are driven supporting capacity building, setting clear perfor- by preferences and priorities shaped by vulnerability to mance targets and safeguards, strengthening financial shocks and seasonal variations. Options are also accountability, and emphasizing M&E. influenced by structures (the form and organization of The fiduciary risks are higher for sectorwide or government and the private sector) and processes programmatic lending than for project lending, since (policies, laws, institutions). more fungible forms of financing are included (SIDA 25 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS 1999). But these risks should be balanced against the tomers: as long as the customer projects a credible path costs of business as usual. The proliferation of projects of earnings, lending continues. puts an enormous burden on weak administrations, Assessing ownership should lead to selectivity. often undermining local capacity building and Energy lending to reforming states in India in the 1990s sustainability. is a promising example of selectivity used to reinforce reform. Taking advantage of India's decentralized Adaptable Conditionality decisionmaking to demonstrate the benefits of reform, Conditionality should be understood as a credible the Bank shifted its policy dialogue from the federal to indicator of commitment by the Bank and its partners, the state level and engaged only reforming states. not as an attempt to force externally designed policy Ownership and partnership were strengthened by wait- changes on unwilling governments (see Annex 5). The ing until the climate was right. The Bank halted Bank and external partners act as enabling agencies to lending to the power sector in India for three years, support the country's motivation for a reform process until it found evidence of real ownership of policy that is guided by genuine learning from successes and reform in selected states. With other partners, it then failures (Branson and Hanna 1999 BP). This kind of engaged in capacity building to solidify local owner- conditionality is agreed and consensual. It represents a ship. Subsequent lending produced more sustainable policy compact based on mutual commitment. results than the earlier approach. Support for conditionality as a commitment pro- cess comes from a reevaluation of the Dollar-Svensson Time to Build Consensus data (in World Bank 1998o), using the country as the Mobilizing the support of beneficiaries cannot start unit of observation, early enough. Pilot projects do not always proceed Conditionality is part of and a recent study on smoothly, even when rooted in strong community a policy reform higher-impact adjust- support. Flexibility is essential, along with a willing- ment lending (HIAL) ness to listen and develop a program incrementally, compact: the Bank and (Dollar and Svensson informed by lessons learned. the borrower first 1998; World Bank Recent irrigation operations in India, the Philip- develop and then 1999i). When the data pines, and Turkey show the importance of allowing mutual are re-analyzed using time for interventions to take effect on a socially the country as the unit appropriate scale: trust and commitment of analysis, past suc- as reform proceeds. cess becomes a highly * In Andhra Pradesh, India, in the early 1990s, significant predictor of irrigator groups were formed around pipe commit- future success. This result supports the view of condi- tees of 20-100 farmers. This group size allowed the tionality as a process of mutual commitment, since the local elite to continue to dominate and led to water Bank can refer to lending history in formulating future allocation disputes among pipe committees. The conditionality. Such conditionality is adapted as a democratic election of much larger groups in the country increases its ownership of reform, assisted by late 1990s overcame this problem. capacity building to achieve parity in the relationship. * In the Philippines, large national irrigation Ongoing reformers can be offered the option of ex post schemes were effectively no more than fee- conditionality, while credible new reformers might collection groups for the government agency and choose to adopt floating tranche loans, as in the HIAL had limited responsibility for operations and approach in Africa. maintenance. Water-user groups were more suc- In this context, conditionality is part of a policy cessful in small communal irrigation projects reform compact: the Bank and the borrower first that had more autonomy. develop and then nurture mutual trust and commitment * In Turkey these lessons were taken into account. as reform proceeds. Conditionality is the Bank's side of Efforts were made to build a consensus among a continuing relationship, while ownership is the stakeholders-a process greatly facilitated by the country's side. A model for this approach is the World Bank Institute-before irrigation systems relationship between a commercial bank and its cus- were turned over to water-user groups. Larger 26 Promising Approaches groups were also more likely to be financially change among all actors and at all levels viable and could be built around existing institu- (Blackburn and Holland 1998). The bottom line in tions, such as municipalities. participatory change is individual and personal. Broadening Participation The CDF expands the ability of the Bank to better How should participation be broadened across interest deal with the difficult governance issues that often groups and scaled-up to the national level? Extensive involve stakeholders other than government. For ex- evaluation and research findings point to several ample, in the area of clean government and improved lessons (see Annex 5): public financial accountability, the Bank's concerns extend well beyond * Integrate a learning process. A well-known government, toward . p success in broadening participation in a govern- other elements of the ment bureaucracy is the Philippines National state, such as civil so- approaches to work, Irrigation Administration, which adopted a step- ciety, the media, and attitudes and behavior by-step approach to building the capacity of user the watchdog agencies must change among all associations to manage local irrigation systems. charged with monitor- actors and at all levels. * Beware of procedural inflexibility. Changing ing and protecting the organizational systems and procedures to facili- public interest under tate participatory development is a complex the laws of the country. The judiciary and legislature undertaking. But the costs of not doing so can be also take on new emphasis in capacity building (Sahgal heavy, as in the government of Uttar Pradesh's and Chakrapani 1999 BP). Doon Valley Project in the Himalayas (Shepherd A recent OED study finds mixed results for efforts 1999). to broaden NGO interventions (Gibbs, Fumo, and * Avoid rushing. Participation cannot be rushed- Kuby 1998). Many NGOs are reluctant to increase the and the larger the scale, the greater the risk. scope of their operations or to enter large-scale partner- When the Indonesian government tried to insti- ships. Scaling-up can pose a challenge to an NGO's tute nationwide village development planning in management capacity and create obligations to mem- less than a year in 1995-96, there was little or no bers that are difficult to sustain. Any scaling-up ownership of the process at the village level. Too initiative must be preceded by an assessment of little time was spent building partnerships with capacities and commitment. A critical step is to involve NGOs, whose skills might have enabled the all stakeholders in developing performance indicators, government to do a better job. a process that clarifies expectations and priorities. * Persevere. The district-level Rural Integrated Project Support Program in the Lindi and Information for Accountability and Learning Mtwara regions of Tanzania has evolved into a Targets for monitoring progress are effective tools for holistic program involving local government, guiding decisionmakers during implementation. Recent agriculture, natural resource management, education projects have invested heavily in setting and transport and marketing, education, savings and monitoring targets. The Mozambique Second Educa- credit, health, and water. This broad participa- tion Program used indicators to help sustain the tion emerged from the accumulated experience operation after the credit closed. It used targets set at of repeated mistakes and learning over 25 years. appraisal and added new ones to establish a five-year * Identify a champion. A pronounced shift toward plan agreed with the borrower. It also identified participation by government or NGOs can often outstanding policy and implementation issues, leading be traced to a single leader or strong alliance of to agreement on remedial actions. supporters. Taking the lead, these champions Experience in health, nutrition, and population have often battled institutional inertia and politi- projects also shows the importance of effective M&E cal pressures from wealthy elites. design. Selecting a limited number of well-chosen * Change attitudes and behavior. For participatory indicators and attending to capacity for data collection approaches to work, attitudes and behavior must and analysis increase the focus on results and the 27 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS likelihood of achieving development impact. Strength- From Aid Coordination to Development Partnership ening borrower systems for the collection, analysis, Effective aid coordination guides countries and donors and use of health information in policymaking takes toward agreement that all partners will accept mutual time. It requires attention and resources during pro- responsibility and distinct accountability for develop- gram design and implementation and strengthened ment outcomes. For countries, this requires commit- incentives to achieve results and use the information. ment to developing sound policies and effective institu- tions. For donors, it calls for adoption of a development Capacity Building to Manage for Results orientation, the exercise of selectivity that reflects Developing a results-oriented public sector is a key comparative advantage, accommodation of country- challenge in many developing countries (Annex 6). An led efforts to achieve coherence and selectivity, and the initial focus on performance monitoring in selected provision of effective capacity-building assistance to sectors or ministries can create a demonstration effect, create a level playing field among partners. easing the way to broader application to program Although many countries have expressed a strong evaluation as opportunities arise (Marcel 1999). desire to take the lead in aid coordination, only a few, The emphasis on managing performance for such as the Republic of Korea, Malaysia, and Thai- projects and programs is being complemented by a land, can be said to have fully assumed the role. Many wider focus on gover- countries lack the capacity to take the lead, and some a Mnance and intersectoral still lack the commitment and resolve to do so. The Participatory M&E IS synergies. Dissemina- time has come for a real change-rapid, but deliberate one step toward tion of M&E informa- and finely executed. A logical and critical step is for the building capacity to tion on government Bank-in consultation with other donors and coun- learn and manage for performance can sup- tries-to make country responsibility a reality wher- resulIts. port the active involve- ever this is feasible. The Bank's chief role would be to ment of civil society in support country leadership and to help build the the assessment of that capacity needed to exercise that leadership effectively. performance. Civil society also has a role to play in By giving the country a chance to exert leadership, the influencing the evaluation agenda. Bank would be better positioned to assist in building Key stakeholders such as NGOs, the media, and long-term capacity, in concert with the principles of the parliaments-particularly those representing and empow- CDF and the OECD/DAC (see Annex 7). ering the poor-gradually learn how to use performance The costs of poor partnership and inefficient aid concepts and tools and to understand their limitations and coordination-in decisionmakers' time, gaps in assis- weaknesses. Donors and governments can help build such tance, and distortions in country priorities-are borne capacities among these stakeholders. primarily by developing countries. For donors and the Participatory M&E is one step toward building Bank, the tensions between practicing partnership and capacity to learn and manage for results and applying reducing the transaction costs involved may be high in the CDF principles (Estrella and Gaventa 1998). As the short term. There are up-front costs, but they should multiple stakeholders work together to develop indica- be viewed as long-term investments in building the tors of success, their differing expectations and priori- infrastructure of partnership and creating skills, trust, ties are brought into the open. Stakeholders must then and learning. And the costs are likely to decline sharply negotiate to develop a more generally applicable if partnerships are pursued selectively and strategi- framework, thereby building ownership in the out- cally. The M&E of partnership and coordination can comes and reflecting partnership in the assessment be strengthened to assess the cost-effectiveness of itself. Participatory M&E also allows the tracking of different approaches. Much can be learned from the holistic goals, both tangible and intangible. Develop- experience of private firms in selecting and building ing large-scale participatory M&E requires skills un- strategic alliances (Doz and Hamel 1998; Hanna and like those needed for traditional evaluation. Stakehold- Raphaeli 1999 BP; Picciotto 1998). ers need help to acquire skills in indicator development, monitoring, facilitation, and conflict Linking Global and Country Strategies resolution (Van Wicklin 1999 BP). Mission-oriented transnational networks addressing 28 Promising Approachcs highly visible and urgent human priorities-such as the The CDF provides guidance. Efforts to build Onchocerciasis Control Program and the Consultative regional and global partnerships should follow its key Group on International Agricultural Research-can tenets of inclusiveness and comprehensiveness. Setting serve as a rallying device for contributing partners. priorities at the national level under the CDF can help Selectivity is ensured up-front by the choice of public identify areas where in- goods to be created, shared learning occurs as a matter ternational programs Setting priorities at the of course, and motivation and coordination among are needed to supple- national level under donors and partners may be achieved more easily than ment national efforts. in multisectoral national programs (see Annex 8). And building capacity the CF can help Attention needs to be given to the interface between in national and local identify areas where international and national public goods-a new dimen- institutions (state and international programs sion of aid coordination. The implication is not that nonstate) will be criti- are needed to investments in international public goods should wait cal in achieving the de- supplement national until conditions are right in most countries, but that velopment goals of glo- conditions need to be nurtured through transnational bal programs. efforts. collaboration programs. 29 6 IMPLICATIONS FOR DEVELOPMENT EFFECTIVENESS ncreased relevance implies greater complexity. Managing complexity effectively calls for new forms of partnership that facilitate strategic selectivity in line with comparative advantage. Sus- tainability implies ownership of reforms, empowerment of domestic actors, and adequate domes- tic capacity. Thus institutional development-which provides the foundations for balanced develop- ment, domestic ownership, participation, and partnership-emerges as the linchpin of the CDE Development effectiveness also depends on a re- First, a results-based approach should be designed up sults orientation, a critical aspect of the CDF. The front. That is, programs should be "evaluable" and the framework is a compass, not a blueprint. Managing it tracking of development outcomes and leading indicators effectively will therefore require accurate monitoring should be carried out jointly with the government and its of progress toward development goals. partners, with a clear The results orientation of the CDF reflects hard- focus on poverty reduc- Programs should be won lessons of Bank lending. OED's independent tion and growth. Pro- assessment of operational performance trends shows cess indicators for key "evaluable" and the that quality management must be strengthened further: principles, such as own- tracking of development ership and partnership, outcomes and leading * A plateau seems to have been reached in the should also be moni- indicators should be share of projects with satisfactory outcomes. tored. * Institutional development continues to improve, Second, as the de- but from a very low base. velopment effort shifts the government and its * Sustainability remains low, and the operating its focus to the higher partners, with a clear environment is becoming riskier. plane of country pro- focus on poverty * The Bank is being called on to deal with more grams, so must the reduction and growth. complex and demanding development problems. evaluation process. Resources and skills The changing global economy and the CDF suggest need to be invested in developing appropriate indica- four principles for evaluating development effectiveness. tors and information systems. The current preoccupa- 31 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS tion with project performance and evaluation should be And the framework will have to be continually adapted. complemented by a sectoral and countrywide focus. Rigidity is a real implementation risk.' Third, evaluation must become participatory. Devel- Learning will complement dissemination. In the opment effectiveness must be considered in the context of planning and adjustment eras, donors and central shared objectives, joint responsibility for outcomes, recip- governments too often adopted a hegemonic planning rocal obligations to achieve results, and distinct account- mentality that excluded vital local knowledge and ability for performance. Donors and governments should practical know-how.2 Similarly, in the adjustment era team up to link M&E it was assumed that the Bank had all the answers-the Every effort will be systems with civil soci- only problem was to sell them to its clients through ety. Building country ca- conditionality. Under the CDF, Bank staff can no longer pacity for M&E is im- pretend that the Bank is a storehouse of universally CDF does not become perative. applicable knowledge. Instead, they will be induced to another blueprint, Fourth, the growing empower their country counterparts and learn with repeating the pattern of interdependence of the them how to achieve development results effectively. world's economies calls The CDF is certain to increase the demand for for evaluating country- nonlending tools and advisory services, to engender adjustment eras. based development pro- ownership, partnership, and long-term holistic think- grams from a global ing. The use of these tools will either empower clients perspective, taking account of international development or lead to cognitive dependency-the end result will goals. Country assistance strategies should link country depend on the skills and attitudes of users. Tendencies goals, progress indicators, and national strategies with toward bureaucratization and excessive documenta- these goals. But these links should not imply top-down, tion-the pitfalls of the planning era-will need to be target-based planning. Instead, the international goals overcome by participatory and creative approaches to must be adapted to country circumstances and priorities strategy development. Donor-led economic work and (Maxwell, Foster, Naschold, and Conway 1999 BP). A policy prescriptions-the hallmarks of the adjustment global perspective also calls for greater harmonization of era-should give way to country-led approaches that evaluation across development agencies. would build on local processes and experiences and Finally, given the emphasis on the learning and develop a strong commitment to policy reforms. process aspects of the development programs, Bank Bank processes have been attuned to a different performance indicators should emphasize professionalism paradigm, and will have to change for implemention of and partnership, creativity and innovation, prudence and the CDF (Hanna, Agarwala, and Chakrapani 1999 BP). probity, and flexibility and responsiveness. Sharing the Bank management has taken steps to adapt the organi- knowledge emerging from country-based programs and zation under the Strategic Compact, but a survey of pilots should leverage their learning costs and accelerate Bank staff (and local donor representatives) found that the diffusion of lessons and promising approaches within more than half did not consider an active government and among countries. role a prerequisite for in-country aid coordination (World Bank 1999m). Further internal changes, both Implications for the Bank subtle and demanding, will be necessary to fulfill the While the Bank has experience in implementing indi- potential of the CDE vidual CDF principles, it has yet to mainstream their joint Finally, a new development architecture is neede to application across the board. In effect, the Bank is moving address the crisis of global poverty and mainstream into uncharted territory. To equip itself to implement the practice of the CDF principles throughout the develop- framework effectively, the Bank will have to continuously ment system. The Bank should promote the development examine the results of ongoing experiments. of this architecture, in partnership with other multilateral The CDF thus implies continued change in the Bank. institutions and developing counrtries. Among the first A "one-size-fits-all" mentality will be replaced by a steps is to harmonize donor procedures, build an informa- "customization" mindset. Every effort will be made to tion-sharing network, and innovate ways to strengthen ensure that the CDF does not become another blueprint, client voice and build local capacity for partnership, repeating the pattern of the planning and adjustment eras. broad-based participation, and learning from results. 32 ANNEXES ANNEX 1. PROJECT PERFORMANCE TRENDS AND OUTSTANDING PROJECTS OED evaluates all closed projects. These evaluations divide. A project with an unsatisfactory outcome will are the basis for the estimated trends in project perfor- never score higher than 6, no matter what ratings it mance and lessons from outstanding projects presented receives along the other two dimensions. here. The number of projects exiting the portfolio has risen significantly over the past two fiscal years, and the Outcome number of independent project evaluations has increased Outcome assessments are based on a determination of nearly 20 percent since the last ARDE, greatly adding to whether the project achieved most of its major relevant the evaluation knowledge base. goals efficiently and with few shortcomings. An The newly evaluated cohort consists mainly of evaluator's judgement about outcome essentially boils investment projects approved in the early 1990s and down to answering the question: Did the project fast-disbursing adjustment operations approved in achieve satisfactory development results, considering FY96-98. This yields a complete picture of the FY98 the importance and relevance of its major stated cohort of exiting projects and a partial look at the FY99 objectives and the associated costs and benefits? The cohort.' The composition of the evaluated portfolio outcome rating takes into account relevance (to check has evolved over time, with the share of exiting whether the project's objectives were consistent with projects in Europe and Central Asia and in the human the country's development strategy), efficacy (to exam- development sectors more than doubling during the ine whether the operation achieved its stated goals), 1990s. The strong performance of adjustment opera- and efficiency (to assess results relative to inputs by tions detailed in tables A1.2 and A1.3 is especially costs, implementation times, and economic and finan- notable and will require further analysis. cial returns). Outcome is rated on a six-point ordinal scale: highly satisfactory, satisfactory, marginally sat- Performance Trends isfactory, marginally unsatisfactory, unsatisfactory, OED evaluators assess project results along three and highly unsatisfactory. related dimensions-outcome (which combines rel- evance, efficacy, and efficiency), sustainability of Significant Variation among Regions and Sectors project benefits, and institutional development (ID) Figures A1.3 and A1.4 show the wide variation in impact. Last year's ARDE introduced another perfor- project results by region and sector. The three main mance indicator for quality at exit that combined these OED dimensions of outcome, sustainability, and ID key dimensions: the Aggregate Project Performance Index (APPI). In parallel, OED assesses Bank and FIGURE A1.1 AGGREGATE PROJECT PERFORMANCE borrower inputs during key stages of the project cycle. INDEX Figures A1.1 and A1.2 present the aggregate and APPI average outcome trends by year of exit from the portfolio. 9 Aggregate Project Performance Index The APPI is defined by assigning cardinal weights to 8 - the ratings of each of the three results-oriented dimen- sions (outcome, sustainability, and ID impact), then summing them in a simple way. Thus, the APPI ranges Bse from 2 (for projects with a highly unsatisfactory outcome, benefits that are unlikely to be sustainable, 6 - By projects and negligible institutional development impact) to 10 (for projects that performed strongly on all three measures). It is noteworthy that of the three dimen- 5 - -FY90 91 92 93 94 95 96 97 98 99 sions, outcome is the key force behind the index. The index draws a line between satisfactory and unsatisfac- Note: By exit fiscal year. Results for FY99 are preliminary (43 tory outcomes: an index measure of 6 represents the percent coverage). Source: World Bank, Operations Evaluation Department data. 33 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS FIGURE A1.2 SATISFACTORY PROJECT OUTCOMES By Exit FY (percent satisfactory) By Approval FY (percent satisfactory) 90 - 90 80 By disbursements 80 - By disbursements 70 70- By projects .60 60 By projects 50 I I I I I I1l50 FY90 91 92 93 94 95 96 97 98 99 FY82 84 86 88 90 92 Note: By exit and approval fiscal year. Results are preliminary for FY99 exits (43 percent coverage) and FY92 approvals (57 percent coverage). Source: World Bank, Operations Evaluation Department data. impact are presented in three separate graphs, with the into account a wide variety of factors, including regional information segregated into the three exit country conditions (for example, the government's fiscal year periods of 1990-93, 1994-97, and 1998-99. commitment to the future of the project, as documented Of the 16 sectors in the Bank, the sectoral figure in the operational plan); government economic and highlights 13-these are the sectors with at least 10 financial policies, including policies on cost recovery; projects exiting in FY98-99. Like the regional repre- availability of funds for operation and maintenance; sentation, each sectoral performance is segregated into the political situation; sector conditions; and conditions the three exit fiscal year periods of 1990-93, 1994-97, specific to the operation (particularly the quality of and 1998-99. project management and the capacity of project institu- tions). Sustainability is rated on a three-point scale: Institutional Development likely, uncertain, and unlikely. Aggregate results are Emphasis on institutional development is critical to portrayed in figure A1.6. enabling the Bank to move beyond project financing and to achieve long-lasting improvements in develop- Borrower Performance ing countries, with ownership by borrowers. OED Borrower performance is defined as the assumption of measures ID impact as the extent of the improvement in ownership rights and responsibilities and delivery of the ability of an agency or a country to use its human the inputs needed to prepare and implement the project. and financial resources effectively and to efficiently OED assesses borrower performance along three di- organize economic and social activities prompted by mensions-preparation, implementation, and compli- the project. ID impact is rated on a three-point scale: ance with legal covenants. Of particular importance substantial, modest, and negligible. The most recent are the assignment of qualified personnel, the provision trend in ID impact is presented in figure A1.5. of the required financial resources, and compliance with the conditions of the loan agreement, including Sustainability reporting and auditing requirements and specific pro- Conceptually, sustainability differs from outcome, be- curement rules. The performance of implementing cause it focuses on the features that determine whether agencies is rated on such elements as the quality of the operation will last over its intended useful life. management and staff associated with the operation; Thus, sustainability is defined as the likelihood, at the the use and effectiveness of technical assistance, includ- time of evaluation, that the project will maintain its ing training, advisers, and contractual services; the results in the future. Assessments of sustainability take adequacy of M&E systems; and the extent and quality 34 Annex 1, Project P'erfornianc-e Trends and utistianding Projects FIGURE A1.3 OUTCOME, SUSTAINABILITY, AND ID FIGURE A1.4 SATISFACTORY OPERATIONS BY SECTOR IMPACT, BY REGION AND EXIT FISCAL YEAR GROUP AND EXIT FISCAL YEAR GROUP F Exit FY90-93 O Exit FY94-97 U Exit FY98-99 M Exit FY90-93 O Exit FY94-97 N Exit FY98-99 Percent Outcome (percent satisfactory) By Projects (percent satisfactory) 90 Public sector 80 - management II 70 -Transportation Urban 60 development 50 - Social sector 40 - Education 30 - Finance 20 - Oil & gas 0 L Multisector AFR EAP ECA LCR MNA SAR Population, Region health & nutrition Agriculture Percent Sustainability (percent likely) Eericuporet 90 - Electric power 90 & other energy 80 - Water supply & sanitation 70 - 60 - 50 Percent 0 20 40 60 80 100 40 - By Disbursements (percent satisfactory) 30 - Public sector 20 -management 10 Transportation Urban AFR EAP ECA LCR MNA SAR development Region Social sector Percent Institutional Development Impact Education 90 (percent substantial) Finance 80 - Oil & gas 70- 60 - Multisector 50 _ Population, health & nutrition 40 Agriculture 30 - Electric power 20 - & other energy Water supply 10 & sanitation 0 Industry AFR EAP ECA LCR MNA SAR Region Percent 0 20 40 60 80 100 Note: The figures for FY99 exits are preliminary Note: Includes only sectors with at least 10 projects exiting in (43 percent coverage). FY98-99. Source: World Bank, Operations Evaluation Department data. Source: World Bank, Operations Evaluation Department data. 35 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS FIGURE A1.5 ID IMPACT By Projects By Disbursements Percent Percent 70 - 70 - 60 - 60 - 50 - oet50 -Modest 40 ~.u 40 30 Sb ia 30 Substantial 20 20 Negligible 10 10 ~ Neggliblgible 10 10 0 I II I l l 0 FY90 91 92 93 94 95 96 97 98 99 FY90 91 92 93 94 95 96 97 98 99 Note: By exit fiscal year. Results for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. of participation by the intended beneficiaries, including appropriate advice as an integral part of the appraisal their contribution to the project's outcome. Perfor- and supervision process, the ability to adapt to chang- mance by fiscal year groups is shown in figure A1.7. ing circumstances (if necessary, by restructuring a project), and the oversight of loan/credit conditionality. Bank Performance Performance by fiscal year group is shown in figure Bank performance is defined as the quality of service A1.8. delivered by the Bank, especially in those tasks for which it has primary responsibility, such as appraisal Lessons from Outstanding Projects and supervision. OED assesses Bank performance Of the 289 operations evaluated since last year's ARDE, along three dimensions-identification, appraisal, and OED assessed 21 (7 percent) as outstanding and 8 (3 supervision. These three important considerations in percent) as having particularly poor performance (table judging Bank performance encompass the provision of A1.1). The selection criteria for outstanding projects FIGURE A1.6 SUSTAINABILITY By Projects By Disbursements Percent Percent 70 -70 60 60 50 -. * 50 -Likely 40 40 40Uncertain a a 4 Uncertain 30 30 20 20 Unlikely 10 10 1 Unlikely 0 0 FY90 91 92 93 94 95 996 97 98 99 FY90 91 92 93 94 95 96 97 98 99 Note: By exit fiscal year. Results for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. 36 Annex I Project Perform ance Trends and Outstanding Projects FIGURE A1.7 BORROWER PERFORMANCE By Projects (percent satisfactory) By Disbursements (percent satisfactory) Percent Percent 100 100 - 80 80 - 60- 60 - 40- 40 20- 20 0 e 0 Preparation Implementation Compliance Preparation Implementation Compliance O Exit FY90-93 0 Exit FY94-97 E] Exit FY98-99 Note: By exit fiscal year. The figures for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. include a highly satisfactory outcome, likely sustainabil- Several of these characteristics are amply demon- ity, and substantial ID impact. These projects are judged strated by Argentina's Provincial Reform Loan, the first to have met or exceeded all their major goals, and many Bank adjustment operation targeting subnational enti- had innovative designs or are judged to be potentially ties. The project's main objective was to support the replicable in other countries or sectors. Their success can federal government's effort to promote the reform of be traced to borrower ownership, consistent monitoring, provincial finances. The loan's contributions went good supervision, an innovative design allowing flexibil- beyond the conditions for each tranche by providing a ity in responding to changing conditions, and continuity forum for the provinces to agree on a reform strategy. in the Bank staff assigned to them. The operation also set in place support mechanisms FIGURE A1.8 BANK PERFORMANCE By Projects (percent satisfactory) By Disbursements (percent satisfactory) Percent Percent 100 100 - 80 80 - 60 60 - 40 40 20 20 40 0L Identification Appraisal Supervision Identification Appraisal Supervision : Exit FY90-93 0 Exit FY94-97 E Exit FY98-99 Note: By exit fiscal year. The figures for FY99 exits are preliminary (43 percent coverage). Source: World Bank, Operations Evaluation Department data. 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS that not only advanced reform among the provincial elsewhere. A telecommunications project in Sri Lanka governments, but also helped to curb the contagion demonstrates the importance of staff continuity and effects of the Mexican crisis at the provincial level. The strong borrower commitment in the difficult area of operation's success is directly related to the consistency institutional development. The project sought public and commitment of both the national government and reforms and restructuring in telecommunications, and the Bank. The government's strategy was transparent regulation and promotion of private investments in the and uniform across provinces. And the Bank's lending sector. Initially classified as a problem project because portfolio was fully consistent with the government's of slow disbursement, the project eventually exceeded strategy, with common rules for all provincial lending. all its stated objectives, thanks to a strong commitment This project offers lessons directly applicable to the to dialogue on both sides. The Bank team responsible increasingly important area of subnational lending and for the project remained unchanged from identification serves as a model for operations in Latin America and to completion. TABLE A1.1 OUTSTANDING AND POOR PERFORMERS AMONG RECENTLY EVALUATED PROJECTS Country Project Loan or credit Argentina Pr% mcial Recrn L3836 Argentina Protincial Penion Reh-rim Alju*tmeir L4116 Bolivia (-.ipir.ili;ation Progiam C2761 Bolivia Hvdrocarbon Secror Ret-rin C2762 China Shuikou Hvdr-rlc,rric 2 L3515 Georgia Tran'porr Rehabilianon C2809 Ghana Feeder Rnad C2319 Hungary Enrerpri;c and Financial S-ctor Adjusriirnr L4141 India Iniductri.al Technolo"'N [)Devel-pnicn C2064 India Techniman Edu,irion C2130 India NIah.ir.%htra Encrgencv Earrhquakc C2594 Indonesia \illage Infrastriicturc L3888 Korea, Rep. Enytronmental Re;arch and Edue.umn L3612 Latvia Structural Adiustment L4126 Macedonia, FYR Tran-ir Facilitation L3868 Morocco Emergency Drouphr Recovers L3935 Mozambique Economi Recoter% 3 CN010 Panama Economic Recovery L3438 Poland Environmental \[anagenicrt L3190 Romania Transportation L3593 Sri Lanka Telecommunication; 2 C2249 Bulgaria Agricultural Development L Bulgaria Private Investment and Export Finance 1. 011 Indonesia Financial Sector Development L 3 2 Morocco Niunal Agricultural Credit L '' S Morocco National Rural Finance L >n' Nigeria National Population < Philippines Rural Electrification - 4 Vanuatu Housing i "r-' Note: Covers only the 289 projects evaluated since last year's ARDE. Source: World Bank, Operations Evaluation Department data. 38 TABLE Al.2 OUTCOME, SUSTAINABILITY, ID IMPACT, AND AGGREGATE BY SECTOR, NETWORK, LENDING TYPE/SOURCE, REGION, AND WDI INCOME GROUP FOR EXIT FISCAL YEAR GROUPS 1990-93, 1994-97, AND 1998-99 (BY PROJECTS) I-uI I-Y )-' 3 Exa F I4-9' I: FYI-[95-L\u jtr n f)e Sir.r. ID i.p. r (1np , I i 'ut O ron1 su..i.IL marr Prole,i. %bare . . - g ca Pr'.cei Share ' '.. -%.. \ .ar PrIicer% share ", " fl " l,Ii L-,1 -b. .idl fl ", ir. dilah sub bi 3* ''1 La. Ik,I1 ..ub. 'J' J Sector Agriculture 2.52 31 5' 4 - h.1 21 24 t 42 -b 19 1> 4 e s4 ni Education 6¯ 4 s2 (4 e -~ 8 - 44 f- n4 1 - i" 33i Electric Power & Othw Firc. 9 i., r9 l > "4 t,. 84 4 i> 54 2s I 24 6 I i 'i ' ' E n v ir o n m e n t 0 -- - 9 1 4 4 1 .S S 4 S l i 'n Finance 5. e S' 43 21 Sl i S' 4' S i - - -4 55 - Industry 5 6 S 4 24 n2 41 4 SI ¼ 32 . 10 1 4 ; s.4 M ining 12 1 S SI Si . - -- ,- - iI-. Multisector 62 - -1 S2 ;S n 100 Ii ss sa 2- '.Æ 31 5 tA 45 i Oil&Gas - -S 12 n -. 24 2 -q Ss 42 13 3 h' 4 q 31 . Population, Health 2 Nrrir.r 2! 2 o-- 43 l4 n2 42 4 - s l 0.4 8 - 04 5< '5 t ; PublicSectorMana5riin U 4 50 4' 'i i.i - n jI 44 2 5 s 29 N - - 5' - SocialSector l 1>> ii 100 . 16 2 NS , 5,, i 20 i 5' 9> 4 i - Telecommunicatior 11 1 (4 -1 o - 22 2 S2 -3 3" < 4 I 10 l>> l<> I.<. Transportation 95 l1 -4 55 34 o.' 108 11 - S> 1 .- 2 S 9f - en - Urban Developmerr 44 5 -- 41 t- n 5 5u 5 &0 3r I . 21 SS 45 45 r.» Water Supply & Sai-r iii.i ~' 4 1' 4>' - s . .< 4 i 5 2 l - 23 o 45 2o 3' '. Network Environmentally & 5. i11. Sustainable Develpmi 252 10 51 34 1 n 244 2< h.s 42 iS r.) h6 2< 0>s 1 s-44-· fDarapoint 1 Daapoint 2 Daapoin 3 1)atapi in L 4 Daåpom i Earh) 1 , n< Alb:en 2.5S 35.5 196) ~ - Azer i' i ·¯.?C 61.0 14 i1 BarJ , J l i1.hs 1.4¯ t 40.9 ;S 11 .¯ 41 3Jm 42 t, 'p11 - ' >0 Bol -, i l.M4 ht.. 189 t , L (" Burk . i F:. o.44 i.2 l 45 4-4 Camh. .I, .l i Côt, d ..-i i -1.4¯ l.. .i ] I. ISS) 1 ¯.' 1ss ss 1 2 ',1. .. 4~ Cr r i 1.14 ¯l ¯, Ec a-a-<.23 fl? .''V. 35.> 194> ¯"s Eth -ri -0.41) l.#" -"' 2? S. i 42 43 Gh lj0.0 i. 1 3¯ t".v (88) 4 l.x 1'%" . Ind.au , L .1 1.44 221.6 1'41 1¯.4 1.ý¯ 1i b,4 11 Jan.:,: .l -1.¯ ''Ss 29.8 \SX 14.2 ? 12I 5 Ken. i U.33 3.4 47.y " X' SI> 46.4 '< 1 - Mali. i0.0 .4> " 'U 42 4. Mali I ¯s M or. 1.. 1.9 s .1 i."l In.5 S> - e 191 6 Mob.'i k 1.6 . ".1"44 Ne:I o.82 . '.'A ;6.2 >¯W¯) 41.4 S6' 44.h 1,eh, 4( Papii i Nif- (:llill. -o.04 I'> i452" Phibpp.i,, -0.¯1 (1 ' 'ii 34 190) 3 194' Pol:nd u.91 2 1' 1.i4 6.0 88 2i.i 114 Sri i nk . .09 l.G 1.24 2-. <9> 22.4 1 I'411 Th.:l i ndi 2.54 's. 2 22.2 Ks> ¯.u li01 i n? b 65 11421 (table continues on following page) 「기 - A ii n e x e v e o p iii e ii t P e r fo i ni a ii c e i i) S e I e (- e e d C o 11 11 t r e TABLE A3.3 COUNTRY PERFORMANCE: SAVINGS Saiines rate 1'n, GDP) 1-incir trend täkinus 11-clunirN l gsus T-Imäi Ä Kl 11 4 ej ni b, -J ý.4 r j r, nj 1 4,1 4 K j j n.,N 4. 4.ý 5 1,317.11.1 NC.. P. 5 I., L j k j . l j nj ti T. 4 1 r I,r,L Ilie ii z j Per,ent ot cowirriý, ith. S!gr!ifi,.irirl\ I,z,:iiniiii; nitc: ak rate .. lu~.. r, lekull, 4, 11 1.2 -1 1! 111 n ý c. r: i ii d 1, 1,-s ýa 11 1 n c, rj ik ý,4 i le.!2 -1 a. For the period 1990-97 for which data are available. b. Data available only since 1991 and 1993 for Azerbaijan and Croatia. e. For period 1988-97. Note: Savings for 1980s and 1990s are calculated as a simple average for this period (GDS as % of GDP). Rate of savings is calcu- lated as a linear trend-Savings (% GDP) = a + (Rate * Time) for the period 1981-97. 49 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS TABLE A3.4 COUNTRY PERFORMANCE: INVESTMENT EFFICIENCY GDPgro% t I.,1 GD/GDPloo Inescnmenr cficincy. Country 11-SO 10I1-9- 19l-S50 19h1-4- 19 t-o 11-0 Alb ni. '1 2.1 .I Srb1 1.45i Rark I de-b 14 S h1.83 23.42 15. - I t j-h - -13 1. < .- .J ;.4c, 1$ 1.4.44 (.ved i:r '.4 1 - 24.4w 1 .1 21.7 12.'" t t.*.i[l3 Ecusi 9.15 2..24.21 2.1.. .9 12 Erhpi.i ; 1. ;4 thn 7[.5..12 9.3 1375i ~ V Indoln': - N2.45 I.N 35.$2. Ksni $18 3.24 24.11 2 11.94 14 MlIs. in.5 1.1 .14 18ih4 i3. N1.:mbu Il!bIIl 2.4t 10.41 1 N4p 2.11 - 13.11 21. 16.1 1 4 Sr L3nkt 4.4 1 4 4 -S.4.S:11 2 .4 I i-.4 Th.IanJ 6.N - -r. 2b.1 14.S9 2i. 1 2 T*:... 4.41 1 29.4i 1-.1 14.9- 4 ZarbiA 1.45 1.1 19.-1 14-1 . 4. 1 r. Percent of countrit ah: Dechmintz efficienc,% of inc4rnenc 42 .. 1 1'241 Low efficiercN of mn% eq; nt r lil.. I 211. 16/24) Decingiloit ettwicnco 1t In estment 6 . 16.'241 Note: The Investment Efficiency Ratio is defined as the GDP growth rate divided by the investment GDP ratio. This is the inverse of the conventional Incremental Capital Output Ratio (ICOR), but is adjusted for changes in terms of trade between investment goods and overall GDP. Simple averages are used. 50 ANNEX 4. EMPHASIZING PEOPLE-CENTERED DEVELOPMENT IN A HOLISTIC FRAMEWORK The sustainable livelihood approach to develop- The approach identifies five kinds of capital assets ment adopted by the U.K. Department for International on which people can build or draw: human, natural, Development, CARE, and the UNDP proposes a holis- financial, social, and physical. It aims to build on what tic understanding of poverty and an integrated and people have and how they live their lives, to add to the dynamic way of thinking about poor people's liveli- accumulation of assets, and to remove the barriers to hoods. This approach defines a livelihood as "the people's ability to advance their livelihood choices. capabilities, assets (including both material and social Early experience with the sustainable livelihood resources), and activities required for a means of approach has shown that its common framework living. A livelihood is sustainable when it can cope facilitates cross-sectoral and multidisciplinary think- with and recover from stresses and shocks, and main- ing. It makes explicit the choices and tradeoffs in tain or enhance its capabilities and assets both now and planning and execution. It emphasizes a small number in the future, while not undermining the natural of entry points, with multisectoral links evolving over resource base" (Chambers and Conway 1992). time. The iterative approach demands learning and The sustainable livelihood approach provides an sequencing, and thus requires that funding partners be analytical structure to guide the understanding of flexible and responsive. factors that influence poverty and to identify where But the experience has also raised new challenges. interventions might best be made (figure A4.1). It The approach can result in a huge agenda, and thus recognizes people's diverse livelihood goals-better risks becoming too multifaceted and time-intensive. health, more income, reduced vulnerability-and the Institutional issues-such as developing closer opera- complex strategies that households adopt to meet these tional links with the social sectors-need more atten- goals. These strategies are shaped by preferences and tion. And while the approach may improve the under- priorities that arise in a context of vulnerability standing of the problems faced by the poor, effecting resulting from shocks, changing trends, and seasonal change in a world of entrenched power structures variations. They are also affected by structures (such as remains a huge challenge. Another challenge is to the form and organization of government and the develop effective tools for managing tradeoffs and private sector) and processes (policies, laws, and sequencing-and for measuring change in people's institutions). livelihoods and security. FIGURE A4.1 SUSTAINABLE LIVELIHOOD FRAMEWORK Key H = Human capital S = Social capital N = Natural capital P = Physical capital F = Financial capital VULNER4XILFTY PROFS '00 In * More income CONTEX I H H-- STRUCTURES * Increased * SHOCKS S N e * Levels ofwell-being * TRENDS :c acces government * Reduced Private vulnerability * SEASONALUTY P F #K sector .Policies - - F*Culturete to * Improved food Cultue . security * Institutions achieve PROCESSES -- * More sustainable use of natural resources 51 ANNEX 5. OWNERSHIP, PARTICIPATION, AND CONDITIONALITY Ownership at its broadest involves commitment by difficulties that the Bank and its clients are likely to the public sector, the private sector, and civil society to face in trying to scale up participation (and promote a national development effort. At its narrowest it consultation) in national development and country involves beneficiaries, local government, and commu- assistance strategies, or even in advisory and analyti- nity organizations in project design and implementa- cal activities. Not all NGOs want to increase the scale tion. Ownership can apply to a national development of their operations to work on Bank-supported projects. strategy, country assistance strategy, sectoral reform Scaling up can also strain an NGOs management programs, or a project. capacity and create financial obligations to staff and clients that are difficult to sustain. Ownership and Participation Any scaling up initiative must be preceded by a Participation is important because it increases the careful assessment of the capacities and commitment of likelihood of a project's relevance and effectiveness. all parties. A critical step is to involve all stakeholders OED findings suggest that participation promotes in developing success indicators, a process that can ownership and broad, long-term attention to human expose differences in expectations and priorities. Nego- issues. It improves sustainability of development pro- tiating to develop a common framework helps to build grams and promotes learning and results-based orienta- ownership of outcomes. tion-key goals of the CDE OED findings suggest several lessons for success in scaling up participation: Benefits of Participation At the project level there are clear benefits to ownership * Integrate a learning process. Perhaps the best and participation. Participation promotes a more holis- known success in scaling up participation is the tic approach. For instance, in Bangladesh and India, Philippines National Irrigation Administration, participation quickly uncovered the necessary adapta- which adopted a step-by-step approach to build- tions that turned projects in roads, lighting, reliable ing user associations' capacity to manage local power, and better services into good investments. This irrigation systems. The use of a learning process in turn facilitated a longer-term view. And the more approach and the introduction of a number of projects that are controlled by community clients managerial innovations allowed local priorities rather than by the Bank or government, the more likely and processes to guide the activities of the the projects are to have a long-term perspective. bureaucracy-rather than the other way around Experience also shows that participation increases (Korten 1980, 1988). A working group was transparency and accountability-and reduces corrup- formed to act as a catalyst for change. It brought tion. When civil works belong to the people who use together frontline workers and higher-level staff them, there is no tolerance for siphoning off project to define a common vision of change, and then funds in bribes. Contractors are grateful for community worked through a learning cycle to achieve that support and protection from extortion. While close change over a number of years. community supervision might lead to more complaints, * Beware of procedural inflexibility. Changing contractors were able to fix problems more quickly and organizational systems and procedures to facili- cheaply, reducing complaints on completion and im- tate participatory development is a complex proving relationships with communities. undertaking. But the costs of not doing so can be heavy, as in the Government of Uttar Pradesh's Scaling-up Participation Doon Valley Project in the Himalayas (Shepherd It takes time and effort to foster participation, particu- 1999). Examples across the world show that larly to scale up participation from a project to simply adding participatory rural assessments programs, sectorwide reform, multisectoral efforts, or onto existing programs will not lead to partici- a national strategy. There have been few successes in pation. scaling up NGO involvement from projects to wider * Avoid rushing to scale up. Participation cannot programs. Bank evaluations and research name the be rushed, and it is especially dangerous to try to 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS do so on a large scale. The Indonesian can often be traced to an alliance of individuals government's attempt to implement nationwide or a single person. These champions have often village development planning in less than a year led the battle against institutional inertia, ndi- in 1995-96 illustrates the pitfalls of forced vidual hostility, and political pressures from the participation. There was little or no ownership of wealthy elite. For example, the regional commis- the process at the village level, and too little time sioners of the Mtwara and Lindi regions in was devoted to building partnerships with Tanzania allied themselves with the Rural Inte- NGOs, whose skills might have enabled the grated Project Support Program to introduce government to do a better job. participatory approaches. Persevere. The Rural Integrated Project Support * Change attitudes and behavior. For participatory Program in the Lindi and Mtwara regions of approaches to work, actors at all levels will need Tanzania is one of the best documented cases of a to change attitudes and behavior (Blackburn and government scaling-up program. This district- Holland 1998). Otherwise a new authoritarian level holistic program involved participation by manager can come into an organization and set stakeholders in local government, agriculture, the clock back, ruining a participatory culture natural resource management, transport and and the practices patiently nurtured by a prede- marketing, education, savings and credit, health, cessor. The bottom line in participatory change and water. The successful collaboration and is individual and personal. results visible today are an outgrowth of the accumulated experience of repeated mistakes Ensuring attitude and behavior change also poses a and learning over some 25 years. major challenge for the Bank. The Bank may be asked * Identify a champion. A strong shift toward to help the government develop participation-for participation in government or NGO practice example, through World Bank Institute seminars for a BOX A5.1 CONDITIONALITY SCENARIOS At least four * A higher-impact tranches that could be contact, and readi- conditionality structural adjustment converted to floating ness to commit scenarios can lending model of tranches as ownership support if circum- be imagined, ranging floating tranches. develops or experi- stances change. from the most flexible This model might mental single-tranche This model could to the most restrictive. apply to a country operations. This apply to a country seen to be a credible model could apply to with a government Ex post conditions reformer with strong countries with not yet ready for for lending to an ownership, but no nascent or developing reform. ongoing reformer real track record. The ownership, but with with a track record government chooses recent changes in All the scenarios are of success. Here the the sequence and regime or other consistent with adjust- process of mutual timing of the condi- sources of credibility able conditionality and commitment is tions and tranche in reform. The single- tranches. As countries fully under way. release. Four coun- tranche operation progress, they would The country is tries with floating could apply to a move up the condition- making progress in tranches-Cameroon, potential reformer, ality ladder. The possi- reforms and the C6te d'Ivoire, with the promise of bility of this kind of Bank is lending. Malawi, and Tanza- additional lending to sequencing should An example may nia-outperformed follow if reform actu- strengthen the be Indonesia during others. ally comes about. movement toward 1985-95. * Regular multiple * Advice, continuing mutual commitment. 54 Annex 5. Ownership, Participation, and Conditionality wide range of participants, including members of tal tranche, or with normal conditions that could be parliament and the press. Cooperation with local converted to floating tranches as the process develops. research organizations can also help develop policy The government would choose the sequence and timing knowledge and contribute to ownership through the of sector reforms as external support is calibrated to the participation of local researchers and officials.' quality of the program. A model might be the relation- Broad participation in program development, with ship between a commercial bank and its customers. As cooperation between the Bank and the government, long as the customer can project a credible path of should strengthen ownership on both sides. Broad earnings, lending continues. Thus conditionality is the support can lend greater credibility to government Bank's side of a continuing relationship, and ownership ownership, which in turn can result in greater Bank is the country's side. flexibility regarding conditionality. It is difficult to establish criteria for ownership.2 The frameworks commonly used to assess ownership- Ownership and Conditionality leadership analysis and stakeholder analysis-are help- Conditionality, through its relationship to ownership, ful, but insufficient. Reform-readiness analysis is often can foster mutual commitment to ongoing reform. proposed as an alternative, but it does not eliminate Traditionally the Bank has attached conditionality to uncertainty. Broad participation in program develop- tranche releases on specific, single loans. The CDF ment, with cooperation between the Bank and the suggests a broader view of conditionality as the government, strengthens ownership on both sides. outcome of an ongoing process that allows the bor- Broad support can lend credibility to government rower and the Bank to develop and nurture mutual trust ownership, leading, in turn, to greater flexibility on and commitment, a process that adjusts as country conditionality. ownership grows. Empirical evidence on successful adjustment lending suggests different forms of condi- Conditionality as a Commitment Process tionality for countries at varying phases of the reform Two recent studies support the view of conditionality as a process. It proposes a new view of flexible conditional- commitment process: a study of HIAL and a reevaluation ity, combined with capacity building, to level the of the 1998 Dollar-Svensson data using the country as the playing field for all parties in the relationship. unit of observation (Dollar and Svensson 1998; World Bank 1999i). Both studies support the view of condition- Reconciling Conditionality and Ownership ality as a process of mutual commitment. Under the CDF, conditionality can play a key role in The HIAL study found better policy outcomes and the Bank's dialogue with reforming borrowers. Rather economic impacts for countries receiving HIAL than than an attempt to force externally designed policy for other groups of IDA countries. This performance changes on unwilling governments, conditionality gain was attributed to greater selectivity in lending and should be used as a credible indicator of commitment more flexibility in disbursement arrangements. HIAL by the Bank and its partners. Under the CDF, condition- introduced floating tranches, with the government ality should represent a transparent and explicit under- deciding when to meet conditions, which were gener- standing of sustained Bank commitment in support of a ally sectoral, and when to draw on the corresponding new or ongoing policy reform program formulated by tranche. Several loans had an initial tranche based on the country, with wide participation by the private selection criteria (generally macroeconomic stabiliza- sector and civil society in cooperation with external tion), followed by floating tranches. Analysis for this partners. The program would be owned by the country, report of the relationship between conditionality and and conditionality would define the parameters of tranche arrangements and outcomes found that coun- external support. tries with higher-impact adjustment loans performed Viewed in this way, conditionality could be applied better during the loan period than before, with the to actions already taken-with further support expected largest average increase in growth of 3.5 percent. on the assumption that the reform process will continue While the samples are small, this provides encouraging (box A5.1). As the reform process and ownership support for flexibility in conditionality and for condi- deepen, conditions would become more flexible. A new tionality as a process of mutual commitment. reformer might begin with a single up-front experimen- The 1998 study by Dollar and Svensson of the 55 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS importance of country political economy variables cess. This result supports the view of conditionality as a compared with variables under Bank control in predict- process of mutual commitment, since the Bank can use ing success or failure (as measured by OED ratings) lending history to formulate future conditionality. concluded that "the results have clear implications for This view of flexible conditionality differs from the the management of policy-based lending. They suggest view of conditionality as coercion-an attempt to that the role of adjustment lending is to identify induce the borrower to take action against its will. reformers, not to create them." The study uses the loan Participation, by involving other stakeholders, reduces as the unit of analysis. But structural adjustment the risks of coercion implicit in a secret negotiation lending tends to be repetitive: 27 percent of countries between a hard-pressed borrower and a powerful receiving adjustment loans since 1979 received more lender. Equally, transparency and participation should than five loans. Past successes could be predictors of minimize the risks of shirking or policy reversals. future successes. When the data are re-analyzed using Under the CDF, this form of conditionality is comple- the country as the unit of analysis, past success mentary to ownership. becomes a highly significant predictor of future suc- 56 ANNEX 6. MANAGING FOR RESULTS There are growing pressures on both governments to evaluate CDF processes, and thus to help govern- and donors to make and track progress in achieving ments and donors learn what works, what does not, and development objectives. These pressures include: why, from the CDF pilots and their approaches. CDF countries, which will adopt different approaches built * Citizen demands for more accountability and around the four core CDF themes, can share lessons greater value for the money about the strengths and weaknesses of these ap- * Scarce donor funds and growing competition proaches. among countries Experience shows that countries must demand a * Market perceptions of national economic com- focus on results (UNDP 1997); donor-driven efforts are petitiveness not enough. Donors can "sell" a greater performance * Greater willingness of civil servants to take a orientation to countries, but an infrastructure that systematic approach to learning from experience. measures performance-through systematic M&E- will founder unless the government accepts the donor's Developing countries are placing more emphasis on arguments or reaches its own conclusions about the good governance and are adapting approaches and merits of such an approach (box A6.1). models from developed countries. One popular model is Countries and donors often disagree on ways to results-based management (RBM), which focuses on track implementation and impact, and few local goals, objectives, and the results chain-the linkage of institutions can bridge the gap between the technical resources, processes, outputs, client reach, outcomes, and task of research and evaluation and the political task of impact. It emphasizes the perspectives of clients and other policymaking. The experience of the U.S. Agency for stakeholders, and can help build a client-oriented service International Development (USAID) suggests that coun- culture. At the core of RBM is a performance measure- tries are more likely to monitor policy implementation ment system focusing on results, not just inputs, processes, and systematically use the findings in decisionmaking or narrowly defined outputs. A number of governments when: have implemented elements of RBM, and their experi- ence-particularly in measuring government perfor- * Think tanks and other credible, nongovernment mance-provides key lessons for a more performance- parties help monitor. oriented approach to government (Mackay 1998b; * Explicit responsibility is assigned for translating OECD 1995, 1997a). research and evaluation findings into policy implications. RBM and the CDF * Evaluation findings are discussed in public forums. The CDF implies an RBM approach: keep the focus on achieving development effectiveness. RBM is a means It may take a decade or more to build a strong, systematic approach to measuring government perfor- mance. The challenge goes beyond building skills and BOX A6.1 BARRIERS TO PERFORMANCE civil service infrastructure. It also encompasses build- ORIENTATION ing sustained government and civil service commit- * Poor demand and ownership in countries ment to performance orientation. * Lack of a culture of accountability (often re- Incentives are crucial to ensuring a performance lated to ethics or corruption) orientation (Picciotto and Weisner 1998). To under- * Absence of evaluation, accounting, or audit- stand a country's incentive frameworks, one must ing skills diagnose its institutions and consult closely with its * Poor quality of performance information, and of government (as in Zimbabwe) (Mackay 1998a, 1998b) accounting and auditing standards and systems and, where agreeable to government, with civil society * Little integration of M&E findings and and the private sector. M&E findings can enhance mechanisms into decisionmaking. accountability, improve manager performance, and help guide budget allocations (including budget cuts). 57 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX A6.2 POOLING AND JOINT REVIEWS IN GHANA'S HEALTH SECTOR REINFORCE ACCOUNTABILITY hana's health and allows an emphasis tions. In-depth evalu- on the outcomes or -.-sector demon- on joint accountability ations of selected impact of health strates an for the donors and the problem areas-pub- ministry programs. innovative approach to government. lic health, institu- * Inability of the donor funding and The government and tional care, and review to take full reviews of sector the donors conduct a systems of manage- account of the performance. Donors joint review of the health ment and support- activities and have agreed to pool a sector every year by had been planned as services provided significant proportion assessing 20 performance part of the 1998 by bodies external of their funds for this indicators. They also review, but were not to the health minis- sector, giving up the identify priorities for the completed. try-such as other right to select which coming year. Two such * Deficiencies in the ministries, NGOs, projects to finance in joint reviews have been health ministry's and the for-profit exchange for a voice in conducted since 1997. capacity to collect, private sector. developing sectoral Although close govern- analyze, use, and * A need for greater strategy and allocating ment-donor collaboration report performance dialogue with civil resources. This pooling has been effective, there monitoring data, society, including approach means that is still work to be done. relating not only to the district level. the cost-effectiveness of The joint review faced the ministry but also specific donor funds problems such as: to the sector. cannot be evaluated * Incomplete evalua- * Lack of evaluations Where government performance is poor, M&E can with leading ministries. But experience also suggests help to improve accountability. Measuring govern- that a holistic approach to RBM will not be easy to ment performance increases the pressure for a more trigger across all ministries. It might be more realistic responsive public sector. In Bangalore, India, for to start with performance monitoring in selected sectors example, an NGO surveys citizens on the quality of or ministries, and then extend the approach to others, municipal government services and the extent of and add measurement tools such as project and corruption in delivery (Paul 1998). program evaluation as opportunities allow. RBM has synergies with many types of public Chile took such an incremental approach (Marcel sector reform. These include public expenditure man- 1999: 265-325). Initial monitoring identified some agement (medium-term expenditure frameworks and problem programs, and this led to questions about performance budgeting), civil service reform (perfor- program outcomes and impact that have since been mance contracts for senior civil servants), intergovern- answered by in-depth evaluations. mental fiscal decentralization, commercialization and An emphasis on performance management for private sector delivery of public services, service projects and programs should be complemented by a delivery or beneficiary surveys, participation and the focus on governance and intersectoral synergies. Do- "voice" of civil society, and anticorruption efforts that nors have attempted to foster good governance and a strengthen financial management and oversight bodies. performance culture at the national and ministry Efforts to foster RBM are thus likely to benefit from levels. But attempts to strengthen M&E have focused close links with related public sector reforms. on the project level, typically only to satisfy donor A holistic approach to governance encourages requirements. The lack of domestic ownership of M&E greater performance orientation. A cross-sectoral per- has undermined acceptance of performance measure- spective puts pressure on lagging ministries to keep up ment for sound governance. Moreover, the limited 58 Annex 6 Managing for Results M&E capacity created through donor-driven, project- and reviews. It can voice views of ordinary citizens, based activities was often lost once the project was NGOs, and the private sector-the consumers of completed. Ghana is starting to work with donors, government goods and services. The beneficiary assess- including the Bank, to build more robust national and ments now conducted in many countries provide such sectoral approaches to performance management and opportunities. Civil society can also influence the measurement. Donor adoption of a sectorwide ap- evaluation agenda by identifying government activities proach helps this process along (box A6.2). in need of assessment. Box A6.3 demonstrates another The CDF provides an opportunity to involve civil type of civil society involvement. society in assessing government performance and in Donors and governments both have a role to play in discussions about national and sector priorities and building the capacities of key stakeholders such as NGOs, progress in achieving development goals. Civil society the media, and parliaments. These stakeholders need to has a number of potentially important roles to play. It have some familiarity with performance concepts and can provide a deeper understanding of the constraints M&E tools to make full use of information on govern- and limitations on government activities. It can pres- ment performance; they also need to understand its sure governments and civil servants to improve man- limitations and weaknesses. In South Africa, for example, agement of inefficient or ineffective government activi- an NGO works directly with parliament on a fee-for- ties.' It can use and learn from information on service basis to help it understand and digest information performance indicators and the findings of evaluations on the performance of government ministries. BOX A6.3 MALAYSIA: SUSTAINABLE PENANG INITIATIVE n late 1997, the heritage; and the extent ments and made a list of Penang, and reviewed Malaysian State of to which citizens feel they key performance indica- by the state assembly. Penang initiated have a voice in the tors for its topic. The The intention is that the the Sustainable Penang changes affecting their groups chose indicators roundtable groups will Initiative, with support neighborhoods. based on their rigor, reconvene each year to from the Canadian The initiative was relevance, and availabil- review improvements International Develop- designed as a pilot ity. In community health and identify problem ment Agency, UNDP, project for community care, for example, indi- areas. and the UN Economic indicators. State assembly cators included cancer The initiative and Social Commission members, state govern- rates, infant mortality, appears to have been for Asia and the ment officials, business quality of health care, successful in generat- Pacific. The initiative representatives, NGOs, levels of patient satisfac- ing stakeholder is a long-term pilot community-based organi- tion, and the ratio of ownership of evalua- response to community zations, academics, and health care facilities and tion, showing that concerns about the concerned individuals professionals to the popu- RBM is possible at costs of Penang's rapid organized into five lation. The groups used the community level. development: the roundtable groups of 50 data to identify trends The initiative also sustainability of people each. The groups and implications for the demonstrates the po- growth; its environ- focused on the economy, sustainability of develop- tential for achieving mental impact and the environment, social ment, and their relation- a collaborative part- consequences for justice, culture, and popu- ship to other indicators nership approach health; the distribution lar participation. or issues. among diverse groups of gains from develop- During one- and two- In late 1998, the and interests. ment; the impact of day facilitated discus- roundtable findings were growth on Penang's sions, each group published in the first values, traditions, and prepared vision state- People's Report on 59 ANNEX 7. PARTNERSHIP AND STRATEGIC SELECTIVITY' Since its emergence some three decades ago, the society and with external partners." In a further concept of partnership for development has won broad refinement, the World Bank's Partnership and Develop- acceptance. Donors and beneficiary countries are ex- ment (World Bank 1998i) defines partnership as a pected to participate jointly in analyzing macroeco- collaborative relationship in which parties jointly nomic issues, developing action plans for national develop and agree on objectives and expected out- priorities, and strengthening or building local institu- comes, division of labor and responsibilities, rights and tions. Ultimately, partnership is expected to transform accountabilities, and capacity development to ensure beneficiary countries from aid recipients to aid manag- that weaker members are able to participate fully.' ers. Some governments-Bangladesh, Bolivia, Ethio- These notions are also embodied in the CDF. pia, Ghana, Tanzania, Uganda, and Vietnam-have taken the initiative in setting the agenda for local aid Effective Country-Led Aid Coordination meetings and sharing a lead role in annual or biannual This notion of partnership implies a matrix such as that apex Consultative Group (CG) meetings. They are also shown in table A7.1, with country-driven arrangements participating more intensively in preparatory activities expected to make the greatest contribution to develop- and analytical work, such as country economic memo- ment effectiveness.4 High country commitment and randa and sector reports, that affect aid coordination institutional capacity are likely to be associated with exercises. country-driven aid coordination arrangements, and less The idea of partnership was first articulated in the country commitment and institutional capacity with 1969 report of the Commission on International Devel- donor-driven arrangements. Where capacity is high but opment, Partners in Development, commissioned by commitment is low, the matrix implies joint sponsor- World Bank President Robert MacNamara. The report ship, or intensive donor involvement in tandem with called for a "new partnership" based on reciprocal the government to minimize resource misallocation rights and obligations. Not until 1995, however, was under flagging or missing policy commitment. Where partnership officially made one the of Bank's six commitment is high but capacity is low, country-driven "guiding principles."2 coordination is possible if institutional capacity is Partnership gained additional prominence in strengthened. Institutional capacity includes organiza- DAC's seminal document, Shaping the 211t Century: tional capability and the rules and incentive structures The Contribution of Development Co-operation governing policy formation and resource allocation. (OECD and DAC 1996). It put forward internationally Incentives include both the internal incentive (and agreed goals as part of a new partnership agenda for disincentive) structures of a bureaucracy and the poverty reduction that calls on developing countries to external incentives resulting from market competition and from the various mechanisms for citizen voice or TABLE A7.1 COUNTRY CHARACTERISTICS AND participation (World Bank 1999i: 9-10).1 AID COORDINATION Donor Characteristics Institutional Countn commitment How can a country move to the country-driven quad- capacity High Low% rant? Depending on where a country stands, the answer High Couiir% -driven lJonc-sponor4hip is to strengthen country commitment, strengthen capac- Low Countr%-dri%en Donor-drren ity, or do both. But the characteristics of donors and (wih InuratIonal international agencies also affect the ease of moving to atrrngthenmng effective country-led aid coordination arrangements. The most advantageous case for country-led aid coordination commit to an effective policy environment for pro-poor is that of a small number of donors, all with a high growth, and for donors to increase financial support for development orientation (table A7.2). As the number of such policies. The report gave highest priority to donors increases, the environment for aid coordination development strategies grounded in an "open and becomes more challenging, and as nondevelopment collaborative dialogue by local authorities with civil motives for providing aid (such as commercial interests) 61 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS TABLE A7.2 DONOR CHARACTERISTICS AND THE AID CO- development cooperation, the donor normally imposes ORDINATION ENVIRONMENT conditions on the recipient, which must abide by them. Number of De%iopmeni orienta.ion The relationship between the two is far from symmetrical. donors High Lo%% It is embedded in an institutional culture of donor High Fa%otable Rem%c% D 1 [institutions that views those who offer aid as belonging to a fundamentally different league from those who receive Low C hllengin- VCr Dtt t it. Changes in rhetoric have not altered the aid culture, even under the pressure of determined leadership. increase, effective country-led coordination becomes more difficult. While the end of the Cold War has reduced Governance Constraints the dominance of the ideological motive for providing Guided by its Articles of Agreement, the Bank deals aid, geopolitical considerations are still at work.' primarily with governments. Partnership has meant, for Effective aid coordination involves a process lead- all intents and purposes, an effort to improve relationships ing to an agreement by the recipient country, donors, with government borrowers. This has constrained the and international agencies that all partners will accept ability to form ties with the private sector and civil mutual responsibility for achieving development out- society. In recent years the Bank has expanded the comes, along with distinct accountabilities. For the meaning of partnership to include NGOs, civil society recipient country, accountabilities include making a organizations, and the private sector. But governance commitment to sound policies and effective institu- constraints impose limits on such partnerships, particu- tions. For the donors, accountabilities include adopting larly where NGOs and other civil society bodies operate a development orientation, achieving coherence with under the government's wings or on the edge of legiti- country development priorities and selectivity along macy. True partnership, based on open and free dialogue, the lines of comparative advantage (thus addressing the is not possible in these conditions. As the Bank's General "numbers" issue), providing effective capacity building Counsel pointed out, "the Bank does not have a legal assistance to level the playing field among partners, right to oversee the governance of its borrowing members and supporting country-led efforts to achieve these or to participate in such governance. It is neither a world objectives. policeman, nor a world government." Bilateral donors are far less restricted in their ability to offer advice on The Limits of Partnership political issues. There are limits to the extent to which the realities of partnership match the vision. These include tensions Strategic Selectivity between harmonization and accountability and be- Partnership and selectivity are inseparable, both con- tween conditionality and partnership, as well as issues ceptually and operationally, which makes it difficult to of governance constraints and strategic selectivity. distinguish selectivity from partnership without seri- ously compromising partnership.' Strategic selectivity Harmonization and Accountability may be exercised on the basis of several characteristics, Harmonization of donor policies and procedures does not each with distinct implications: remove the need for recipient accountability for their use of funds to meet the donors' fiduciary responsibility * Need. Donors may disagree with the government toward their governing bodies or parliaments. Account- or each other about which problems, sectors, ability, however, has been one-sided. Donors have not regions, or groups are priorities. Donors may been held accountable to recipients for the methods or focus exclusively on "fashionable sectors," such rationale of the allocation of aid resources or the as health and education, at the expense of predictability of resource availability. The reluctance of complementary investments, such as rural roads. some donors to untie aid illustrates this point. * Comparative advantage of donors. In an ideal world, donors would divide their labor accord- Conditionality and Partnership ing to comparative advantage. In practice there In normal business partnerships, the partners mutually is no ready agreement on what this means. agree to abide by certain conditions and obligations. In Consider the often difficult relationships between 62 Annex 7. Partnership and Stra tegic Selectivity the Bank and the specialized agencies of the UN, The Road to Partnership: Strategy for Government-Led or among UN agencies themselves. The com- Aid Coordination parative advantage of donor institutions can Partnership requires strong institutions. It also requires conflict with commercial interests. well-developed skills in negotiation, communication, * Country capacity. Resources should be allocated economic and social analysis, information technology, to the most efficient institutions in the recipient and diplomacy. Participants at a workshop for senior country. Pushed too far, this type of selectivity government officials engaged in aid coordination, spon- can undermine the balanced approach called for sored by OED and the Swiss Agency for Development by the CDF, or force a tradeoff with capacity (SDC) in February 1999, concluded that more investment building and ignore dynamic comparative ad- was needed in the skills required for effective aid vantage. Supporting the creation of project man- coordination, particularly government's ability to design agement units, for example, may undermine and carry out economic and social studies and to analyze overall capacity building. donor studies. * Country performance. Selectivity, in the sense of a The Bank provides considerable funding for capac- reward for work well done, may be an alternative ity building through loans, grants, and trust funds, but to conventional ex ante conditionality, which is an most of it is not woven into a national strategy in the incentive to do a good job. In a donor's global way macroeconomic analysis is. Donor efforts to operations, selectivity implies a bounded applica- strengthen country aid management and coordination tion of the partnership principle: where country capacity have tended to be expensive, supply-driven commitment is lacking, recipients must first qualify and ineffective, and aid coordination activities have for partnership by meeting a minimum level of had little positive effect on capacity or partnership performance. Practical difficulties have arisen with (World Bank 1999m). There is a need for real change. selectivity based on performance. The chairman of Much can be learned from the corporate world about the DAC warns against donors trying to invest in partnerships and strategic alliances, particularly about "winners" and withdraw from "losers." This defining and exploiting comparative advantages (box would require, he warns, "better forecasting ability A7. 1). The Bank, in consultation with other donors and than anyone claims to possess" (DAC 1999: 4). affected countries, needs to give force to the concept of BOX A7.1 LEARNING FROM BUSINESS ALLIANCES uch can be ances to increase reach, across geographical loca- cultural fit and part- learned mobilize resources, tions and teams. Corpo- nership orientation, rom the source new knowledge, rations are becoming, in collaborative plan- corporate world about lower overhead costs, effect, multiorganiza- ning and learning, partnerships and strate- increase their responsive- tional or networked orga- dedicated inter- gic alliances, particu- ness to clients, and focus nizations. Successful stra- organizational com- larly about defining on core businesses and tegic alliances depend on munication, invest- and exploiting competencies. They are a clear strategic fit and ment in mutually comparative advan- using the Internet for mutual agenda, incre- beneficial goals, rein- tages among partners. collaborative planning mental processes of forcement of partner- Corporations are enter- and facilitating learning increasing involvement ship successes, and a ing into strategic alli- and action on projects and sharing information, sense of "co-destiny." Source: Buono 1997; Hamel and Prahalad 1994; Johnston and Lawrence 1988; Kanter 1994. 63 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX A7.2 CAPACITY BUILDING ASSISTANCE TO KAZAKHSTAN AND VIETNAM FOR AID COORDINATION n 1992 the Bank its numerous reorganiza- agency. The grant was tributing to a "pro- gave an Institu- tions, there was consider- followed by a technical cess" for improved aid - tional Develop- able delay in contracting assistance loan to support coordination. ment Fund grant to the long-term adviser, aid coordination and The project evalua- Kazakhstan's aid coor- Once on board, however, management. tion report noted that dination unit, the the adviser established a The UNDP initiated a the project strength- National Agency for strong working relation- three-year technical assis- ened the capacity of Foreign Investment ship with the staff and tance project in Vietnam FERD and helped the (NAFI), to improve its created a database of in 1993, also providing a government prepare system of aid manage- externally financed tech- long-term adviser, short- documentation for the ment and coordination. nical assistance and term consultants, train- first donor meeting The grant was investment projects. The ing, study tours, and and subsequent CG designed to finance the adviser worked with some equipment for the meetings. It did not, costs of a long-term aid NAFI staff to prepare aid coordination unit, the however, achieve its coordination adviser, a documents for the first Foreign Economic ambitious goals for number of short-term and second CG meetings. Relations Department overall improvement consultants, training Both the management (FERD). Although housed of the aid management and study tours for and staff of NAFI cred- administratively in the system. The World agency staff, and ited the adviser with Central Planning Com- Bank has taken over procurement of equip- strong knowledge of the mission, the project aid coordination ment and facilities for subject and success in established ambitious responsibility from the the aid coordination transferring knowledge goals for impr'oving the UNDP. unit. and skills in coordinating overall aid management The similarity of Because of NAFI's foreign aid and establish- system, from line minis- Bank and UNDP expe- lack of familiarity with ing a solid foundation for tries down to the provin- riences in these two Bank procedures and the aid coordination cial level, and for con- cases is striking. Source: World Bank data; and UNDP 1996, p 16. country responsibility, putting the country in the than moving along parallel and duplicate trails. Efforts driver's seat. The Bank should work with the govern- by the Bank and the UNDP to strengthen aid coordina- ment and other development partners to formulate a tion capacity in two distinctly different countries multiyear strategy to support countries in assuming underscore the potential for greater success through leadership of CG and consortium meetings and in joint efforts (box A7.2). building aid coordination capacity at the local level. A related issue is the location of CG meetings and The Bank and the UN should jointly make capacity other apex-level aid coordination meetings. Most CG building and aid coordination part of the national meetings take place in donor capitals-especially strategy dialogue with countries at apex aid coordina- Paris-but a growing number are being held in recipi- tion meetings. Both the Bank and the UN call for ent countries, as recommended by the Partnership ownership, harmonization, cost-effectiveness, subsid- Paper and by the 1998 evaluation of the Special iarity (dispersal of authority as close to the grassroots Program of Assistance for Sub-Saharan Africa (World as government allows), and institutional and indi- Bank 1998k: 17). Other recommendations to encourage vidual commitment and discipline. Partnership would broader participation include having the host govern- be stronger and the burdens on government lighter if ment chair aid coordination sessions and frame the the Bank and the UN could follow a single path, rather agenda. Box A7.3 highlights the advantages and 64 Annex 7 Partnership and Strategic Selectivity BOX A7.3 ADVANTAGES AND DISADVANTAGES OF IN-COUNTRY AID COORDINATION MEETINGS Advantages Disadvantages *Participation by the highest levels of govern- * Possible downgrading of the level of donor ment leadership representation (but not necessarily across the *Increased sense of ownership by board) government and subsequent commitment * Government preference in some cases for * Greater public scrutiny and collaboration with meeting outside the country because of the civil society and private sector sensitivity of CG issues * Reduced perception of donor domination * Less free expression of opinions by local donor * Broader participation by government representatives, who also may not have the agencies power to commit their governments * First-hand view of the partner country for * Loss of ability to take advantage of Paris's donor headquarters representatives. central location and Bank facilities designed expressly to meet the needs of CG meetings. Ultimately, the venue for any coordination forum should be determined by practical considerations. disadvantages of in-country aid coordination meetings. resources to assist its members in the building of long- None of this is to suggest that the Bank disengage term capacity. For the country, the challenge of from the aid coordination process or from efforts to responsibility for leadership in aid coordination should strengthen its country programs. On the contrary, by strengthen commitment and ownership-and ulti- encouraging countries to exercise leadership in aid mately development outcomes. coordination, the Bank will be better able to free 65 ANNEX 8. INTERNATIONAL PUBLIC GOODS AND AID EFFECTIVENESS As global integration deepens, the number of develop- and cultural heritage, cross-border (if not global) environ- ment problems that require supranational policy re- mental problems, migrants and refugees, and many forms sponses grows. These cross-border challenges arise of illicit transboundary behavior. Another problem-the from combinations of market, government, and sys- risk of protectionist backlash--could crop up in industrial temic failures. Thus, a new development frontier is countries in response to rising domestic inequality, labor emerging, and with it a new role and complementary insecurity, and disagreements over appropriate environ- rationale for development assistance. A country focus mental standards. will continue to be important, but official development Solutions to these problems are similar to interna- finance will likely be needed to meet a large deficit in tional public goods (box A8.1). Their reach can be international public goods. global (like global public goods), regional (affecting a Global forces (including technological change) are subcontinent, continent, or hemisphere), or local (af- creating not only new, far-reaching opportunities, but also fecting a small number of neighboring countries). a host of potential problems-"international public Solutions may affect industrial countries, sets of devel- bads." These problems include the possibility of financial oping countries (in the case of some tropical diseases), contagion, the spread of disease, the loss of biodiversity or, increasingly, both poor and rich countries. BOX A8.1 PUBLIC GOODS PRIMER public good of externality. An exter- property rights, law and (toll roads), which are is a commod- nality occurs when the order, and, more gener- excludable but .ity, service, welfare of an agent ally, functional social nonrival, and com- or resource whose depends directly not and political cohesion mon pool goods or consumption by one only on what that agent are development common property user does not reduce does, but also on what resources with a public (groundwater or its availability to others do or fail to do. good character. Societ- mineral deposits), other users. Public The motivation to ies at different levels of which are nonexclud- goods are nonrival in invest in public goods development distinguish able but rival. Com- consumption and arises from the desire to themselves by their mon property tends to nonexcludable; that bring out positive exter- accumulated wealth of be overused in the is, the provider of the nalities or to correct or public goods (the non- absence of rules. As good cannot prevent compensate for negative physical components with pure public someone from con- ones. Collective (or referred to as social goods, the prudent or suming it, regardless government) action is capital by some sustainable use of of whether the user necessary to produce authors). common property is a pays for the good. public goods because Public goods differ matter of collective Because of this char- private solutions often according to their choice. Government acteristic, public fail. In particular, public benefit on a action (for example, goods-such as clean markets are unable to continuum between in the form of regula- air, national defense, supply nonexcludable pure public goods and tion) may also be and street lighting- goods. pure private goods. needed to ensure equi- tend to be under- Public goods are Private commodities table and competitive supplied. critical to development. and services are rival access to club goods. A close relative of Rules and standards, and excludable. In be- the concept of a pub- infrastructure, institu- tween these polar oppo- lic good is the notion tions in public service, sites are club goods 67 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS BOX A8.2 PROGRESS IN CONTROLLING RIVER BLINDNESS nchocersiasis, highly successful, with corporations, founda- comparative advan- or river an overall economic rate tions, and NGOs. Inter- tages that each partner blindness, is of return of 20 percent- vention focuses on vector brings and to allocate a painful and debili- 600,000 cases have been control, drug distribu- responsibilities accord- tating disease that prevented, 34 million tion, and capacity build- ingly, with precise infects 20 million people have been ing in national health objectives. Leadership, people in Sub-Saharan protected, 5 million programs. personal relationships, Africa and places 120 years of productive labor The program demon- and trust are crucial. million others at risk have been added, and 25 strates that partnerships All partners must of contracting the dis- million hectares of land are complex and difficult perceive a payoff to ease. A multipartner have been freed of the to establish and main- participation. Credit Onchocersiasis disease. Partners in the tain, particularly given should be shared liber- Control Program has program include govern- the mix of corporate ally and frequently. operated in West ments, local communi- cultures. To replicate the Flexibility and compro- Africa for 25 years. ties, international organi- program's success, it is mise are fundamental. The program has been zations, bilateral donors, important to identify the Aid Effectiveness Linkages The CDF addresses several defects in the (nationally Several strategic and process-related considerations focused) mainstream aid delivery pattern, including: that change the focus and resources of international public goods could change the effectiveness of aid. * A tendency toward top-down and spending- First, if a development problem such as a banking crisis oriented approaches at the expense of local arises from transnational spillover, it cannot be re- capacity building and ownership solved by the traditional approach of an uncoordinated * Fragmented aid delivery with large numbers of set of national overtures. "In contrast to conventional insufficiently coordinated sources of assistance foreign aid that focuses on individual countries, and projects relative to absorption capacity transnational problems demand a multicountry, prob- * Questionable aid allocation patterns. lem-oriented approach to development cooperation" (Gwin 1999). Although intervention at the national The principles by which the framework seeks to level may also be necessary, sector-specific national increase development effectiveness-such as partner- action is usually most effective when embodied in a ship; country ownership and involvement; a results multicountry framework. focus; and a long-term, holistic view-are precisely Second, systemic crises-international conflict; those that make for successful, global, collaborative cross-border environmental, financial, food, or natural programs. resource crises; and disruptions brought about by Two long-standing international public goods ven- excessive social inequality-can destroy foreign aid- tures, the Onchocerciasis Control Program in West financed development achievements. Crises can Africa (box A8.2) and the Consultative Group on quickly spoil the fruits of past economic growth in International Agricultural Research (CGIAR), confirm emerging markets and commodity-exporting develop- the potential effectiveness of highly focused ing countries. Some of this growth may have been multicountry and multiactor partnerships. Both pro- financed by foreign assistance. Such a setback would grams have existed since the early 1970s and show force future official flows to affected countries into less exceptionally high rates of return to investment. The favorable initial conditions. success of these programs demonstrates that, with Third, the aid process can undermine its effectiveness. proper leadership, issue-oriented international cam- 68 Annex 8. International Public Goods and Aid Effectivencs paigns can produce strong results. For example, the More thought needs to be given to the interaction program to eradicate river blindness appears to have among international and national public goods. fostered a disciplined process and motivated partici- Transnational policy should address the synergies pants to persevere. among investments in development resources at the An assessment of the CGIAR reaches similar international and national levels, which could be a new conclusions (Anderson and Dalrymple 1999). (This dimension of aid coordination. The implication is not assessment cautions, however, that "in reporting re- that investments in international public goods should search accomplishments, the basic problems are aggre- wait until conditions are right for their application in gation and attribution.") The CGIAR-a partnership of most countries, but that conditions on the ground must governments, multilateral institutions, and founda- be nurtured so that international development goods tions-has catalyzed international collective action in can be put to use. Current practices, even among such the service of world food security. The program is successful programs as the Onchocerciasis Control devoted to sustainable crop improvement, especially Program and the CGIAR, could do more to promote for staple foods consumed by the poor. The CGIAR has national public goods. Aid coordination partnerships generated impressive global externalities and can be hold the promise that improvements to domestic pro- viewed as a model of transnational standard-setting grams will reveal that norms and standards can be and governance in its field. It has often been suggested superior alternatives to conditionality. that something similar be created to address tropical diseases or, more specifically, the underresearched and Institutional Implications and Outlook underattended "orphan" diseases that account for most The domestic and international dimensions of the of the disease burden in poor countries. A recent move development challenge are becoming increasingly in- in that direction is the Global Forum for Health terrelated. Issue-focused international partnerships for Research, established in 1997 as an independent, development are mushrooming and are likely to con- multiactor foundation hosted by the World Health tinue to grow in number. Both domestic and interna- Organization to correct the "10/90 disequilibrium." tional measures can help address undesirable spillover (Only 10 percent of annual global spending on health and secure ownership for domestic reform and increase research in the private and public sectors is devoted the effectiveness of sectoral interventions at the na- to the health needs of 90 percent of the world's tional level. Reform becomes less dependent on intru- population.) sive conditionality as the policy gap between the Based on examples from the Onchocerciasis Con- regional and global levels closes. It is easier to trol Program and the CGIAR, it appears that mission- encourage voluntary compliance with generally ac- oriented transnational networks that address highly cepted international standards than to introduce top- visible and urgent human priorities can serve as a down conditionality. However, voluntary compliance rallying device for the coordination of contributing requires that developing countries be adequately repre- partners. Selectivity is ensured up-front through the sented in the partnerships that develop the standards choice of public goods to be created. Fragmented and that coordinated national capacity and institution behavior and free-riding are hindered by the visibility building strengthen the ability to implement sectoral of the program and the public support associated with reform. Unfortunately, there is a deficit in both areas. its objectives. Shared learning occurs (or should occur) The CDF provides a formula to address these as a matter of course. Motivation and coordination challenges. The framework's key tenets of inclusiveness among donors and partners are easier to achieve, and and wholeness should be respected when partnerships conditionality and allocation of donor funds are less at the regional and global levels are being built. With contentious than in multiobjective, multisector country guidance from the framework, prioritization at the assistance programs. There is also less scope for national level can help identify areas where interna- politics to interfere with technical integrity. Developing tional programs are needed to supplement national countries should be strengthened by the presence of efforts. Capacity building in national and local institu- these programs in the Networks-a presence that is tions (state and nonstate) is critical to the effective indispensable to finding policy solutions to cross- implementation of coordinated efforts. border externalities within countries. ENDNOTES Chapter 1 Chapter 3 1. Table A3.2. India and China are excluded because they 1. A 1998 OED review of World Bank assistance to financial represent exceptional cases; that is, the Bank's performance is sector reform found that of 23 countries where the Bank had likely to have a modest influence on country results (World Bank provided support since 1985, only 12 had satisfactory perfor- 1997g). mance. The East Asia crisis is expected to affect the performance 2. For example, Borrower Ownership of Adjustment Programs ratings in 3 of the 23 countries found to have satisfactory overall and the Political Economy of Reform (Johnson and Wasty 1993). ratings (Indonesia, Korea, and the Philippines) (Mathieu 1998). 3. The Background Papers for the 1999 ARDE are identified by 2. The donors urged that by June 30, 1991, or at the latest the addition of "BP" following the year of publication. All are listed before the end of the IDA9 period, environmental action plans be in the "Background Papers" section of the Bibliography. completed for all IDA recipients, with priority given to countries where major problems have been identified, and that the results be Chapter 2 incorporated into country lending strategies. 3. Other partners perceive a tendency for the Bank to send 1. .se ane 1 f aigrphic of th trn large missions for too short a time, to produce bulky aide memoires and then disappear, leaving a shell-shocked local 2. Assessments of project characteristics come from OED government to make sense of the contents and action plans. project evaluations. Demanding refers to the extent to which the Rather than facilitating partnership, this increases the transac- project could be expected to strain the economic, institutional, tion costs for clients and partners. and human resources of the government or implementing agency. Complexity refers to such factors as the range of policy and institutional improvements contemplated, the number of institu- tions involved, the number of project components and their 1. In size and income level, the group is very close to the geographic dispersion, and the number of cofinanciers. Riskiness middle-income countries. But low-income countries and those refers to the likelihood that the project, as designed, would be with large populations are overrepresented. The group is fairly expected to fail to meet relevant project objectives efficiently. representative of the Bank Regions, except that Latin America 3. Consistent with the trends in the APPI, outcome perfor- and the Caribbean is underrepresented and South Asia is overrep- mance over the decade is higher when weighted by disbursements, resented. with more than 80 percent of projects exiting in fiscal 1998 2. Investment efficiency is defined as the GDP growth rate showing satisfactory outcomes. divided by the investment-GDP ratio. This is the inverse of the 4. This comparison is limited to sectors with at least 10 conventional incremental capital-output ratio (ICOR), adjusted projects exiting in FY98-99 and with changes in average outcome for changes in terms of trade between investment goods and GDP. of more than 3 percentage points. 3. The CAE for C6te d'Ivoire suggests that teachers' wages 5. For reference, investment performance levels are largely were abnormally high relative to per capita GDP. The median consistent with Bankwide aggregates. salary of high school teachers in 1979 was 300,000 CFA francs a month, equivalent to $30,000 a year at 1996 prices. The report 6. The Quality Assistance Group's recent quality-at-entry observed that internal efficiency suffered from high repetition assessment suggests similar concerns, with institutional capacity an d o ut rate,a many sutere por prepedifor aspects and risk assessment and sustainability receiving two of . the three lowest project subratings (World Bank 1999k). entry into the working world. 7. The sample of countries with more than 10 Bank- 4. The review examined Bank experience with 83 restructur- supported projects implemented in 1990-99 was isolated and ing operations in 46 countries between fiscal 1980 and 1994 broken into groups by project performance in institutional devel- (Mathieu 1996). opment. For the countries in each group a time profile of an 5. The report concluded that "the twin emphases on policy independent measure of national bureaucratic quality was re- reform and privatization, together with a major retreat from viewed. The analysis showed a positive time-trend in national financial intermediation operations (due to their poor results), bureaucratic quality in most countries in the group with high also meant that the Bank was left with few instruments (and institutional development performance, but no clear pattern in often little interest) to deal with institutional issues affecting countries in the other groups. restructuring. This was also the case with issues of technology development, labor market legislation, tax incentives, investment 8. The sampling strategy provided 50 percent coverage of regulations, and trade policy, particularly at the sector and sub- adjustment operations and 33 percent coverage of the larger setrlvs" group of investment operations. Coverage of both was stratified sector levels." across sectors through random selection. 6. The review covers 19 poverty assessments and updates (all those done in fiscal 1997 and 1998, and a sample from fiscal 1996). The 1999 review is a follow-up to OED's first review of poverty assessments, conducted in 1996, which examined poverty assess- ments completed through December 1994 (World Bank data). 71 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Chapter 5 Annex 5 1. For example, "New Lessons from Old Projects: The Workings 1. As proposed in PREM Note 27 (World Bank 1999j). of Rural Development in Northeast Brazil" (World Bank 1993b). 2. This is supported by PREM Note 25 (World Bank 1999a). Within the framework of flexible program design, the better per- forming activities consistently departed from their original design, Annex 6 led by dynamic managers and local involvement (World Bank data). 2. That study also identified an inward-focused Bank culture 1. The Bangalore, India, case cited above is one such as a source of complexity, a culture that emphasized analytical example (Paul 1998). comprehensiveness, rewarded larger and more visible interven- tions, discouraged risk-taking and making tradeoffs, diffused Annex 7 accountability, and downplayed the need for external feedback 1. This annex draws on OED's forthcoming Aid Coordina- (World Bank data). tion Study. 3. Some types of social fund projects, such as family planning 2. The other guiding principles were selectivity, client orienta- ,are not demanded by the community, although these activities tion, results-orientation, cost-effectiveness, and financial integrity. may have high social returns. The poor in a community do not 3. The United Nations articulated a similar framework at come forward with proposals because they have limited capacity about the same time. Also see Robert Picciotto, The Logic of to propose projects (Carvalho 1999b BP). Partnership. A Development Perspective. (OED, World Bank, September 29, 1998). Chapter 6 4. The 1997 ARDE measured policy performance as a 1. The Bank, IMF, UNDP, and others should exercise caution project-weighted index of three components: inflation, fiscal in introducing too many planning and programming instruments. balance, and openness. For institutional quality, it draws on a From a developing country viewpoint, these tools may be top- measure employed by the World Development Report 1997 down and confusing foreign concepts and may divert local (World Bank 1997g), based on a set of responses by foreign resources and managerial talents away from building and im- investors on red tape, the regulatory environment, and the degree proving existing local strategy development and planning pro- of freedom from political pressure (Evans and Battaile 1998: 15, cesses. Developing countries should have a voice in shaping and 20). OED evaluative research (Johnson and Wasty 1993) shows harmonizing the design and use of these tools. that country commitment and policy performance are closely 2. For example, Seeing Like a State (Scott 1998). Such related. hegemonic planning and social engineering approaches reflect 5. Other characteristics can also have a bearing on the little confidence in the skills, initiative, intelligence, and experi- effectiveness of aid coordination, such as country size and its ence of the beneficiaries. The precision and authority of such strategic or geopolitical significance. Other things being equal, approaches depended not only on bracketing contingency, but the larger the country and the greater its strategic significance, also on standardizing the subjects of development. the more likely it is that donors will take aid coordination 3. Some of the recent organizational changes in the Bank may seriously. But these characteristics are fixed (country size) or actually increase the tensions and challenges in implementing the nearly impervious to efforts at deliberate modification (strategic CDE For example, some budgetary and personnel policies and the importance). The desire of government officials to maintain the contracting out of technical services may reinforce rather than status quo has also been cited as an obstacle to country leadership alleviate short-term orientation and incentives, promote specializa- (World Bank 1998i: 34). This is part of institutional capacity, tion rather than integration skills, and link budgetary resources more broadly defined to include incentive structures. closely to lending rather than recognizing the growing importance of 6. Another donor characteristic is diversity of procedural and knowledge, facilitation, and advisory services. accountability requirements, which can also impede greater country leadership. However, it is plausible that the greater the Annex 1 development orientation, the greater the willingness of donors to . OD harmonize procedural requirements. The same comment applies . o as Reidal implemenin completion rets to the internal incentive structures of donor agencies that work deivered to date by Regional staff, covering 118 of the 277 projects against coordination (World Bank 1998i: 34). Several replies to exiting the portfolio in fiscal 1999 (43 percent coverage). Comple- the joint questionnaire sent by UNDP and the Bank to donors tion reports for the remaining projects exiting in fiscal 1999 are metoddvlpenornainadpocualiestys expetedto b deiveed t OE an revewe bysprig 200.mentioned development orientation and procedural diversity as expected to be delivered to OED and reviewed by spring 2000. poeta.arest fetv ai.oriain potential barriers to effective aid coordination. 7. For a fuller discussion of the subject see ODI, Chapter 2: Annex 3 Partnership and Strategic Selectivity (Maxwell, Simon, Foster, 1. Investment efficiency is defined as the GDP growth rate Naschold, and Conway 1999 BP). divided by the investment-GDP ratio. This is the inverse of the conventional incremental capital-output ratio (ICOR), but is adjusted for changes in terms of trade between investment goods and overall GDP. 72 BIBLIOGRAPHY This report has drawn on a wide range of World Bank Ferroni, Marco. 1999. "Reforming Foreign Aid: The documents and numerous outside sources. World Bank Role of International Public Goods." Photocopy. sources include OED's major studies, Country Assis- Fox, James W. 1999. "Applying the CDF to USAID tance Evaluations, and project audits, as well as Experiences." Photocopy. ongoing research. These and other sources are listed Fumo, Claudia. 1999. "Lessons of Experience from the alphabetically by author or organization in two World Bank's Work with Nongovernmental Organi- groups: Background Papers commissioned for this zations (NGOs) and Implications for the CDE" report and a Selected Bibliography. Photocopy. Galenson, Alice. 1999. "Private Sector Development Background Papers and the CDE" Photocopy. The background papers are identified in the text by the Gilbert, Roy. 1999. "Urban Development and the addition of "BP" following the year of publication. CDE" Photocopy. Selected background papers are forthcoming. Grasso, Patrick. 1999. "Bank Grant Programs and Development." Photocopy. Barbu, Alain. 1999. "Energy, Mining, and Telecom Hanna, Nagy. 1999. "Evaluating CDF Pilots." Photo- Sectors: Lessons of Experience for the CDF." Photo- copy. copy. Hanna, Nagy, and Nimrod Raphaeli. 1999. "Partner- Battaile, William, and Federico Mini. 1999. "Determi- ship and Strategic Selectivity." Photocopy. nants of Project Success." Photocopy. Hanna, Nagy, Ramgopal Agarwala, and Deepa Blackburn, James, Robert Chambers and John Chakrapani. 1999. "Toward a Comprehensive De- Gaventa. 1999. "Learning to Take Time and Go velopment Strategy: Lessons of Experience." Photo- Slow: Mainstreaming Participation in Development copy. and the Comprehensive Development Framework." Hassan, Fareed M. A. 1999. "Higher-Impact Adjust- Institute of Development Studies, Sussex. Photocopy. ment Lending Study: Implications for the Compre- Branson, William, and Nagy Hanna. 1999. "Policy hensive Development Framework." Photocopy. Reform, Ownership, and Conditionality." Photo- Jones, Stephen, and Andrew Lawson. 1999. "Scaling copy. Up: Issues in Moving from Projects to Programmatic Carvalho, Sonia. 1999a. "Poverty Assessments." Pho- Aid." Oxford Policy Management. Photocopy. tocopy. Jonhston, Timothy. 1999. "Health, Nutrition, and 1999b "Social Funds and Implications for the Population." Photocopy. CDF." Photocopy. Levy, Hernan, and Binyam Reja. 1999. "Transport and Cook, Thomas J., and Lawrence Cooley. 1999. "Re- the CDE" Photocopy. sults-Based Management in Developing Countries: Liebenthal, Andres, and S. Mani. 1999. "Environment Some Lessons Learned." Photocopy. and the CDE" Photocopy. Crook, Richard C., and James Manor. 1999. "The Maxwell, Simon, Mick Foster, Felix Naschold, and Potential Contribution of Democratic Decentraliza- Tim Conway. 1999. "Operationalizing the Compre- tion to the Comprehensive Development Frame- hensive Development Framework: Evidence from work." Institute of Development Studies, Sussex. Contemporary Research." Overseas Development Photocopy. Institute. Photocopy. Dove, Linda. 1999. "CDF and the Education Sector." McKay, Keith. 1999. "Managing for Results." Photo- Photocopy. copy. Effron, Laurie. 1999. "Financial Sector and the CDE" Nelson, Ridley. 1999. "Rural Development and the Photocopy. CDE" Photocopy. Ellerman, David. 1999. "Helping People Help Them- Pitman, Keith. 1999. "Water in the Bank: Lessons for selves: The Challenge of Autonomy-Compatible As- the Comprehensive Development Framework." Pho- sistance." Photocopy. tocopy. Evans, Alison. 1999. "OED Poverty Evaluation: Find- Sahgal, Vinod, and Deepa Chakrapani. 1999. "Clean ings and Lessons for the Comprehensive Develop- Government & Financial Accountability - Implica- ment Framework." Photocopy. tions of the CDE" Photocopy. 73 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Shah, Anwar, and Navin Girishankar. 1999. "Implica- Chambers, R., and G. Conway. 1992. Sustainable tions of the CDF for Governance Reforms." Photo- Rural Livelihoods: Practical Concepts for the 21st copy. Century. IDS Discussion Paper 296. Brighton, U.K. Van Wicklin, Warren. 1999. "Participation: Implica- Commission on International Development. 1969. tions for the Comprehensive Development Frame- Partners in Development: Report. New York: work." Photocopy. Praeger. Wasty, Sulaiman S. 1999. "Evaluation Experience of DAC (Development Assistance Committee, OECD). Country Relations and Assistance Strategies: Impli- 1999. "Overview of the DAC Chair: Staying the cations for CDF Implementation." Photocopy. Course toward Development Results through Partner- ship." In James H. Michel, ed., Development Coop- Selected Bibliography eration: Efforts and Policies of the Members of the Albouey, Y., and R. Bousba. 1998. The Impact of IPPs Development Assistance Committee. Paris: OECD. in Developing Countries-Out of the Crisis and into Dasgupta, P. 1997. "Social Capital and Economic the Future: Viewpoint. Washington, D.C.: World Performance." Paper presented at World Bank confer- Bank. ence: Social Capital: Integrating the Economist's and Anderson, J., and D. Dalrymple. 1999. The World the Sociologist's Perspective. World Bank, Washing- Bank, the Grant Program, and the CGIAR: A ton, D.C. Retrospective Review. OED Working Paper Series Datta-Mitra, J. 1997. Fiscal Management in Adjust- No. 1. Washington, D.C.: World Bank. ment Lending. OED Report. Washington, D.C.: Andrews, K. 1987. The Concept of Corporate Strategy World Bank. Homewood, IL.: Irwin. DFID. 1998. Sustainable Rural Livelihoods: What Arboleda, J. 1999. Participatory Country Assistance Contribution Can We Make? Paper presented at the Strategy in Colombia: A Case Study. Social Develop- DFID Natural Resources Advisors' Conference, July ment Papers No. 33. Washington, D.C.: World Bank. 1998. London. Argyris, C., and D.A. Schon 1978. Organizational _. 1999. Sustainable Livelihoods Guidance Learning: A Theory of Action Perspective. Reading, Sheets Issues. DFID Series. Available from MA: Addison-Wesley. http:www.oneworld.org/odi/ Aycrigg, M. 1998. "Participation and the World Bank: Dollar, D. 1999. The Comprehensive Development Successes, Constraints and Responses: Draft for Framework and Recent Development Research. Discussion." World Bank Development Paper No. Washington, D.C.: World Bank. 29, November. Washington, D.C., World Bank. Dollar, D., and Jakob Svensson. 1998. What Explains Blackburn, J., and Jeremy Holland. 1998. Who the Success or Failure of Structural Adjustment Changes? Institutionalizing Partnership and Devel- Programs. World Bank Policy Research Working opment. London: Intermediate Technologies. Paper. Washington, D.C. Bower, J. L., and Yves Doz. 1979. "Strategy Formula- Doz, Y., and Gary Hamel. 1998. Alliance Advantage. tion: A Social and Political Process." In Strategic Boston: Harvard Business School. Management. Boston, MA: Little-Brown. Ekbom, A., and J. Bojo. 1997. Mainstreaming Environ- Buckley, R. 1999. 1998 Annual Review of Develop- ment in Country Assistance Strategies. Discussion ment Effectiveness. Report No. 19062, OED. Wash- Paper 1, World Bank, Environment Group, Africa ington, D.C.: World Bank. Region. Washington, D.C. Buono, A. F 1991. "Managing Strategic Alliances: Elster, J., C. Offe, and others. 1998. Institutional Organizational and Human Resource Consider- Design in Post-Communist Societies: Rebuilding the ations." Business and the Contemporary World 3(4): Ship at Sea. Cambridge, U.K.: Cambridge University 92-101. Press. 1997. "Enhancing Strategic Partnerships: Inter- Estrella, M., and John Gaventa. 1998. Who Counts vening in Network Organizations." Journal of Or- Reality? Participatory Monitoring and Evaluation: A ganizational Change Management 10(3): 251-66. Literature Review. IDS Working Paper No. 70. Brighton, U.K.: IDS. 74 Bibliography Evans, A., and William Battaile. 1998. 1997 Annual . 1996. Social Dimensions of Adjustment: World Review of Development Effectiveness. Report No. Bank Experience, 1980-93. OED Report. Washing- 17705, OED. Washington, D.C.: World Bank. ton, D.C.: World Bank. Farrington, J., D. Carney, C. Ashley, and C. Turton. Johnson, J., and Sulaiman Wasty. 1993. Borrower 1999. "Sustainable Livelihoods in Practice: Early Ownership of Adjustment Programs and the Political Applications of Concepts in Rural Areas." Natural Economy of Reform. World Bank Discussion Paper Resources Perspectives No. 42. London: ODI. No. 199. Washington, D.C. Fukuyama, F. 1995. Trust: The Social Virtues and the Johnston, R., and P. R. Lawrence. 1988. "Beyond Creation of Prosperity. New York: Free Press. Vertical Integration-The Rise of the Value-adding Gibbs, Christopher, Claudia Fumo, and Thomas Kuby. Partnership." Harvard Business Review 66 (4): 94- 1998. Nongovernmental Organizations in World 101. Bank-Supported Projects. OED Report. Washington, Jones, S. 1997. Ethiopia Social Sector Development D.C.: World Bank. Programs. Report to ODA. Oxford, U.K.: Oxford Gwin, C. 1999. "The New Development Cooperation Policy Management. Paradigm." In ODC Viewpoint. Washington, D.C.: -. 1999. Increasing Aid Effectiveness in Africa? Overseas Development Council. The World Bank and Sector Investment Programs. Hamel, G., and C. K. Prahalad. 1994. Competing for Oxford, U.K.: Oxford Policy Management. the Future. Boston: Harvard Business School Press. Kanter, R. M. 1989. When Giants Learn to Dance: _. 1999. "Strategic Intent." Harvard Business Mastering the Challenge of Strategy, Management Review 89: 63-76. and Careers in the 1990s. New York: Simon and Hanna, N. 1985. Strategic Planning and Management: Schuster. A Review of Recent Experience. World Bank Staff -. 1994. "Collaborative Advantage: The Art of Working Paper No. 751. Washington, D.C. Alliances." Harvard Business Review 72 (4): 96- _. 1991. "The Information Technology Revolu- 108. tion and Economic Development." World Bank Korton, D. 1980. "Community Organization and Rural Discussion Paper No. 120. Washington, D.C. Development: A Learning Process Approach." Public _. 1999. "A Proposed Role for Aid Agencies in Administration Review 40 (5): 480-511. Helping Developing Countries to Design National _. 1988. "From Bureaucratic to Strategic Organi- Information Technology Strategies." Technology zation." In F. Siy and S.Y. Siy, Jr., eds., Transform- Management: Strategies and Applications 4: 299- ing a Bureaucracy: the Experience of the Philippine 302. National Irrigation Administration. West Hartford, Hanna, N., and Sandor Boyson. 1993. Information CT: Kumarian. Technology in World Bank Lending. World Bank Kreimer, A., and others. 1998. The World Bank's Discussion Paper 206. Washington, D.C. Experience with Post-Conflict Reconstruction. Re- Hanna, N., Sandor Boyson, and Shakuntala port No. 17752, OED. Washington, D.C.: World Gunaratne. 1996. The East Asian Miracle and Bank. Information Technology: Strategic Management of Kuhn, Thomas S. 1970. The Structure of Scientific Technology Learning. World Bank Discussion Paper Revolutions. Vol. II. 326. Washington, D.C. Lindblom, C. 1990. Inquiry and Change. New Haven: Hirschman, A. 0. 1958. The Strategy of Economic Yale University Press. Development. New Haven: Yale University Press. Mackay, K. (ed). 1998a. Evaluation Capacity Devel- . 1970. Exit, Voice, and Loyalty: Resources to opment in Zimbabwe. ECD Working Paper, OED. Decline in Firms, Organizations, and States. Cam- Washington, D.C. bridge, MA: Harvard University Press. -. 1998b. Public Sector Performance: the Critical Jayarajah, C., W. Branson, and B. Sen. 1995. Struc- Role of Evaluation. ECD Working Paper, OED. tural and Sectoral Adjustment Operations: World Washington, D.C. Bank Experience, 1980-92. OED Report. Washing- ton, D.C.: World Bank. 75 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Marcel, M. 1999. "Effectiveness of the State and OED. 1997. Lessons of Fiscal Adjustment: Selected Development Lessons from the Chilean Experience." Proceedings from a World Bank Seminar. Washing- In G. Perry and D. Leipziger, eds., Chile: Recent ton, D.C.: World Bank. Policy Lessons and Emerging Challenges. Washing- Ohiorheunan, L. 1998. National Long-term Perspec- ton, D.C.: World Bank. tive Studies-Lessons from Experience. Prepared for March, J.G., and H.A. Simon. 1958. Organizations. Workshop in Entebbe, Uganda. New York: Wiley. Paul, S. 1998. Making Voice Work: the Report Card on Margulis, S., and J. Bernstein. 1995. "National Envi- Bangalore's Public Service. World Bank Policy Re- ronmental Strategies: Learning from Experience." search Working Paper No. 1921. Washington, D.C. World Bank Environment Department Dissemination Picciotto, R. 1995. Putting Institutional Economics to Notes: 30. Washington, D.C. Work: From Participation to Governance. World Margulis, S., and T. Vetleseter. 1998. "Environmental Bank Discussion Paper 304. Washington, D.C. Matters at the World Bank: Annual Review - Fall _. 1998. Logic of Partnership. OED Report. 1998." Report No. 18540. Washington, D.C., World Washington, D.C.: World Bank. Bank. . 1999. Poverty Reduction and Institutional Mathieu, N. 1996. Industrial Restructuring: World Change. Inaugural Wolf Lecture. Washington, D.C.: Bank Experience, Future Challenges. Report No. World Bank. 15707, OED. Washington, D.C.: World Bank. Picciotto, R., and Eduardo Wiesner. 1998. Evaluation _. 1998. Financial Sector Reform. Report No. and Development: The Institutional Dimension. 17454, OED. Washington, D.C.: World Bank. New Brunswick, NJ, and London: Transaction for Maxwell, S. 1997. "Implementing the World Food the World Bank. Summit Plan of Action: Organizational Issues in Porter, M. 1980. Competitive Strategy: Techniques for Multi-sectoral Planning." Food Policy 22 (6): 515- Analyzing Industries and Competitors. New York: 31. Free Press. Murphy, J. 1997. Mainstreaming Gender in World _. 1985. Competitive Advantage: Creating and Bank Lending: an Update. Report No.17169, OED. Sustaining Superior Performance. New York: Free Washington, D.C.: World Bank. Press. Najmabadi, F., and S. Lall. 1995. Developing Indus- _. 1996. "What is Strategy?" Harvard Business trial Technology: Lessons for Policy and Practice. Review, November-December: 61-78. OED Report. Washington, D.C.: World Bank. Purcell, D., and Jock R. Anderson. 1997. Agricultural Narayan, D. 1999. "Voices of the Poor." Washington, Extension and Research: Achievements and Prob- D.C., World Bank. Photocopy. lems in National Systems. Report No. 16851, OED. OECD. 1995. Governance in Transition: Public Man- Washington, D.C.: World Bank. agement Reforms in OECD Countries. Paris. Rodrik, D. 1999. The New Global Economy and -. 1996. Shaping the 21st Century: The Contribu- Developing Countries: Making Openness Work. tion of Development Cooperation. Paris. Washington, D.C.: Overseas Development Council. _. 1997a. Evaluation of Programs Promoting Sabel, C. 1994. "Learning by Monitoring: The Institu- Participatory Development and Good Governance: tions of Economic Development." In Lloyd Rodwin Synthesis Report. Paris. and Donald Schon, eds., Rethinking the Develop- - . 1997b. In Search of Results: Performance ment Experience: Essays Provoked by the Work of Management Practices. Paris. Albert 0. Hirschman. Washington, D.C.: Brookings _. 1999. Developing Country Scoping Workshop Institution. on National Strategies for Sustainable Development, Sachs, J. D., and A. M. Warner. 1995. "Economic November 18-19, 1998. Sunningdale. Reform and the Process of Global Integration." OECD and DAC. 1996. Shaping the 21s' Century: The Brookings Papers on Economic Activity 1: 1-118. Contribution to Development Cooperation. Wash- Schumpeter, J. A. 1983. The Theory of Economic ington, D.C.: OECD Washington Center. Development. New Brunswick, NJ: Transaction. 76 Bibliography Scott, J. C. 1998. Seeing Like a State: How Certain Wolf, C., Jr. 1997. Markets or Governments: Choosing Schemes to Improve the Human Condition Have Between Imperfect Alternatives, 2d ed. Cambridge, Failed. New Haven: Yale University Press. MA: MIT Press. Sen, A., and J. D. Wolfensohn. 1999. "Development: A Wolfensohn, J. D. 1997. "Annual Meetings Address: Coin with Two Sides." May 5. International Herald The Challenge of Inclusion." World Bank, Washing- Tribune. ton, D.C. Senge, P. M. 1990. The Fifth Discipline: The Art and -. 1998. "The Other Crisis: 1998 Annual Meet- Practice of the Learning Organization. New York: ings Address." World Bank/IMF Annual Meetings, Doubleday. October 6, 1998. World Bank, Washington, D.C. Shepherd, A. 1999. Evaluation of DFID Support to _. 1999a. "A Proposal for a Comprehensive Poverty Reduction: Lessons from the Evaluation of a Development Framework." World Bank, Washing- Bilateral Program. Working Paper for a World Bank ton, D.C. Conference on Evaluation and Poverty Reduction, . 1999b. "Coalitions for Change." World Bank/ June 14-15, 1999, University of Birmingham. IMF Annual Meetings Address, September 28, 1999. SIDA (Swedish International Development Agency). World Bank, Washington, D.C. 1999. Making Partnership Work on the Ground. World Bank. 1993a. Implementing the World Bank's Workshop Report, Ministry of Foreign Affairs. Strategies to Reduce Poverty: Progress and Chal- Stiglitz, J. 1998. Towards a New Paradigm for Devel- lenge. Report No. 11826. Washington, D.C. opment: Strategies, Policies, and Processes. Prebisch _. 1993b. New Lessons from Old Projects: The Lecture, UNCTAD, Geneva, October 19. Workings of Rural Development in Northeast Brazil. . 1999a. Participation and Development: Per- OED Report. Washington, D.C. spectives from the Comprehensive Development -. 1994. A Review of World Bank Experience in Paradigm. Speech given February 27, 1999. World Irrigation. Report No. 14908, OED. Washington, Bank, Washington, D.C. D.C. . 1999b. "Whither Reform? Ten Years of Transi- _. 1995a. Enhancing the Quality of Life in Urban tion." Paper prepared for the Annual Bank Confer- Indonesia: The Legacy of Kampung Improvement ence on Development Economics, April 29, 1999. Program: An Impact Evaluation. Report No. 14747, World Bank, Washington, D.C. OED. Washington, D.C. Subbarao, K., and others. 1997. "Safety Net Programs - .1995b. Lending for Electric Power in Sub- and Poverty Reduction: Lessons from Cross-country Saharan Africa. Report No. 14961, OED. Washing- Experience." Directions in Development. World ton, D.C. Bank, Washington, D.C. -. 1995c. Poverty Reduction at the World Bank: Thomas, V., and others. 1999. Quality of Growth. Progress in Fiscal 1994. Report No. 14021. Washing- Background Paper, 1999/2000 World Development ton, D.C. Report. World Bank, Washington, D.C. Photocopy. -. 1996a. Development of Housing, Water Sup- UNDP. 1996. Developing Capacities in Aid Coordina- ply and Sanitation in Nairobi. Report No. 15586, tion and Management. Vietnam. OED. Washington, D.C. . 1997. Measuring and Managing Results: Les- _. 1996b. Pakistan: On-Farm Water Manage- sons for Development Cooperation. New York. ment Project and Irrigation Systems and Rehabilita- United Nations. 1998. World Investment Report 1998: tion Project. Report No. 15863, OED. Washington, Trends and Determinants. New York. D.C. Wack, P. 1985. "Scenarios: Uncharted Waters Ahead." _. 1996c. Poverty Reduction at the World Bank: Harvard Business Review, September-October 1985: Progress and Challenges in the 1990s. Report No. 73-89. 15950. Washington, D.C. Williamson, J. 1997. "The Washington Consensus _. 1996d. Reforming Agriculture: the World Bank Revisited." In L. Emmerij, ed., Economic and Social Goes to Market. Report No. 1670, OED. Washing- Development into the XXI Century. Washington, ton, D.C. D.C.: Inter-American Development Bank. 77 1999 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS - . 1997a. Brazil-Secondary and Feeder Roads -. 1998k. The Special Program of Assistance for Project and Second Feeder Roads Project. Report Africa (SPA): An Independent Evaluation. OED No. 16738, OED. Washington, D.C. Report. Washington, D.C. . 1997b. India-A Comparative Review of -. 19981. Sri Lanka-Community Water Supply Health Sector Reform in Four States: An Operational and Sanitation Project. Report No. 18113, OED. Perspective. Report No. 15753, OED. Washington, Washington, D.C. D.C. _ . 1998m. The World Bank and the Agricultural -. 1997c. Korea-Delayed Development of the Sector in Kenya: An OED Review. Report No. Cholla Region: An Institutional Study. Report No. 18088, OED. Washington, D.C. 16211, OED. Washington, D.C. . 1998n. World Development Report 1998/99: -. 1997d. Poverty Reduction at the World Bank: Knowledge for Development. Washington, D.C. Progress in Fiscal 1996 and 1997. Report No. 17380. -. 1998o. Assessing Aid: What Works, What Washington, D.C. Doesn't, and Why. Washington, D.C.: World Bank. -. 1997e. Sri Lanka-Kurunegala and Second _. 1999a. "Assessing Borrower Ownership Using Rural Development Projects. Report No. 16418, Reform Readiness Analysis." PREM Note 25, June. OED. Washington, D.C. Washington, D.C. -. 1997f. World Bank Support for Small-Scale _. 1999b. Case Study of World Bank Activities in Enterprises in Ecuador: An Impact Evaluation. the Health Sector in India. Report No. 19537, OED. Report No. 16790, OED. Washington, D.C. Washington, D.C. -. 1997g. World Development Report 1997: The -. 1999c. Civil Service Reform: A Review of State in a Changing World. Report No. 17300. World Bank Assistance. Report No. 19599, OED. Washington, D.C. Washington, D.C. -. 1998a. The Brazil Health System. Report -. 1999d. Development Effectiveness in Health, 18142, OED. Washington, D.C. Nutrition, and Population: Lessons from World Bank -. 1998b. East Asia: The Road to Recovery. Experience. Report No. 19266, OED. Washington, Report No. 18457. Washington, D.C. D.C. -. 1998c. The Impact of World Bank Support to _. 1999e. Evaluating Health Projects: Lessons the HNP Sector in Zimbabwe. Report No. 18141, from Literature. World Bank Discussion Paper No. OED. Washington, D.C. 356. Washington, D.C. 1998d. Jamaica Country Assistance Note. . 1999f. "Evaluation of Social Fund Projects." Report No. 19356, OED. Washington, D.C. OED. Washington, D.C. Photocopy. -. 1998e. Lessons from Experience in HNP. _. 1999g. Health Care in India: Learning From Report No. 18642, OED. Washington, D.C. Experience. OED Precis 187. Washington, D.C. -. 1998f. Malawi-The World Bank and the -. 1999h. Health Care in Mali: Building on Agricultural Sector. Report No. 17898, OED. Wash- Community Involvement. OED Precis 188. Washing- ington, D.C. ton, D.C. -. 1998g. Mauritania: Second Livestock Project. -. 1999i. Higher-Impact Adjustment Lending Report No. 18133, OED. Washington, D.C. (HIAL): Initial Evaluation. Report No. 19797, OED. -. 1998h. Paraguay: Community-based Rural Washington, D.C. Water Systems & Development of Village Commit- _. 1999j. Lessons from Large Adjustment Loans. tees. Report No. 17923, OED. Washington, D.C. PREM Note 27, August. Washington, D.C. -. 1998i. Partnership and Development: Pro- -. 1999k. Poverty Assessments: A Follow-up posed Actions for the World Bank. World Bank Review. Report No. 19630, OED. Washington, D.C. Partnership Group, Strategy, and Resources Manage- -. 19991. "Poverty Study." OED. Washington, ment Discussion Paper. Washington, D.C. D.C. Photocopy. -. 1998j. Recent Experience with Involuntary _. 1999m. Review of Aid Coordination and the Resettlement: An Overview. Report No. 17538, Role of the World Bank. Report No. 19840, OED. OED. Washington, D.C. Washington, D.C. 78 Bibliography . 1999n. Tanzania, Agriculture, and the World Bank: An OED Review. Report No. 18111, OED. Washington, D.C. . 1999o. World Development Report 1999: Poverty and Development 1999/2000. Washington, D.C. 79