t T t N sE VA L U A T O N E P A R T E N¯ LB A NK O PE RA ON A 21392 July 1999 Investinlg in lealth D evelop,nent Effectiveness Developmenl in the Hjealth, Nutrition, and Population Sector FP ~twl 'ymå ~~71 rP F~1ECOn OPERATIONS EVALUATION DEPARTMENT ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. WORLD BANK OPERATIONS EVALUATION DEPARTMENT (m Investing in Health Development Effectiveness in the Health, Nutrition, and Population Sector Timothy Johnston Susan Stout 1999 The World Bank www.worldbank.org/htmlVoed Washington, D.C. Copyright © 1999 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First edition July 1999 The opinions expressed in this report do not necessarily represent the views of the World Bank or its member governments. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. The World Bank encourages dissemination of its work and will normally grant permission promptly. Permission to photocopy items for internal or personal use, for the internal or personal use of specific clients, or for educational classroom use is granted by the World Bank, provided that the appropriate fee is paid directly to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, U.S.A., telephone 978-750-8400, fax 978-750- 4470. Please contact the Copyright Clearance Center before photocopying items. For permission to reprint individual articles or chapters, please fax your request with complete information to the Republication Department, Copyright Clearance Center, fax 978-750-4470. All other queries on rights and licenses should be addressed to the Office of the Publisher, World Bank, at the address above, or faxed to 202-522-2422. Photo credits: Cover, Curt Carnemark; page 13, UNICEF/HQ81-105/Joh Isaac; page 15, UNICEF/ HQ92-031 1/Sean Sprague; page 21, UNICEF/HQ91-031/Jorgen Schytte; page 24, Curt Carnemark; page 27, Jeff Rosen. ISBN 0-8213-4310-6 Library of Congress Cataloging-in-Publication Data Stout, Susan, 1949- Investing in health : development effectiveness in the health, nutrition, and population sector / Susan Stout, Timothy A. Johnston p. cm. - (Operations evaluation study) Includes bibliographical references. ISBN 0-8213-4310-6 1. Public health administration-Developing countries-Evaluation. 2. Nutrition policy-Develop- ing countries-Evaluation. 3. Public health-International cooperation. 4. Developing countries- Population policy-Evaluation. 5. World Bank. Population, Health and Nutrition Dept.- Evaluation. I. Johnston, Timothy A. II. Title. III. World Bank operations evaluation study. RA441.5.S74 1999 362.1'09172'4-dc2l 99-052688 O Printed on recycled paper. Contents v Acknowledgments vii Foreword, Prefacio, Priface ix Executive Summary, Resumen, Rdsum6 Analytique xix Abbreviations and Acronyms 1 1. Introduction and Evaluative Framework 1 Health Outcomes and the Health System 2 Evaluating the Bank's Performance 5 2. Evolution of Bank HNP Strategy and Lending 6 1970s: Early Population Projects 6 1980s: Direct Lending for Primary Health Care 6 1990s: Health Financing and Health System Reform 9 3. Project Performance and Determinants 9 Project Performance Trends 10 Borrower Performance and Country Context 10 Bank Performance 16 Institutional Factors Influencing Performance 19 4. Development Effectiveness of HNP Investments and Policy Dialogue 19 Strengthening the HNP Service Delivery Structure 24 Enhancing Health System Performance 26 Health Financing 27 Instruments and Strategies for Reform 29 5. Recommendations 29 Increase Strategic Selectivity 30 Strengthen Quality Assurance and Results-Orientation 30 Enhance Learning and Increase Institutional Development Impact 31 Enhance Partnerships 33 Annexes 33 Annex A. Pr6cis of Case Studies 33 Health Care in Brazil: Addressing Complexity 39 Health Care in India: Learning from Experience 45 Health Care in Mali: Building on Community Involvement 51 Meeting the Health Care Challenge in Zimbabwe 57 Annex B. Project Outcome-Sources of Information 59 Annex C. Ledger of OED Recommendations and Management Response 63 Annex D. Report from the Committee on Development Effectiveness (CODE) 65 Endnotes 67 Bibliography iii Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Figures 2 Figure 1.1 Achieving Change in HNP 9 Figure 3.1 Outcome and Sustainability Improving, but Institutional Performance is Weak 13 Figure 3.2 Supervision Intensity and Project Size 14 Figure 3.3 Greater Complexity in Difficult Settings Boxes 3 Box 1.1 HNP Evaluation Literature 7 Box 2.1 Bank HNP Policy Statements in the Past Decade 11 Box 3.1 Lessons from Successful Institutional Development 12 Box 3.2 Lessons from Successful M&E 20 Box 4.1 Bank Experience in Nutrition 22 Box 4.2 Bank Support for Population and Reproductive Health iv Acknowledgments This study is the result of an effort by the Operations This report was prepared by the HNP Cluster of the Evaluation Department (OED) to assess close to 30 years Sector and Thematic Evaluations Group of the Opera- of World Bank experience in the population, health, and tions Evaluation Department. The task leader of the nutrition sector. We would like to thank the entire overall OED study was Susan Stout. This report, co- management group of OED for the opportunity to work authored by Timothy Johnston and Susan Stout, is based on this effort and for their continuing guidance and on a comparative assessment of performance in the HNP support. We would especially like to thank Roger Slade. Lending Portfolio, Lessons from Experience in HNP, This study would not have been possible without his completed by a team including the authors and Laura foresight and patience, and, not least, his good humor. Raney, Varun Gauri, Janet Nassim, and Diana Qualls. Gregory Ingram ably guided the report through its final The four country case studies (published separately) were phase, and Jacintha Wijesinghe helped to manage the authored by Varun Gauri (Brazil); Timothy Johnston resources necessary to complete the effort. Several exter- (Zimbabwe); Timothy Johnston, Sheila Dohoo Faure, nal advisers contributed to the study at various stages, and Laura Raney (Mali); and Ronald G. Ridker and including Jose-Luis Bobadilla, Rodolpho Bulatao, Henry Philip Musgrove (India). William Hurlbut edited the Mosley, Joan Nelson, and Amy Tsui. Many Bank staff original report and prepared the graphs and manuscript. contributed as reviewers and sources of frank feedback Marcia Bailey provided administrative support. and reaction at various stages of the study. We would like This study was published in the Partnerships and to acknowledge, in particular, the efforts of Anwar Bach Knowledge Group (OEDPK) by the Outreach and Dis- Baoub, Caroline Cederlof, Xavier Coll, Ed Elmendorf, semination Unit. The task team includes Elizabeth Alison Evans, Navin Girishankar, Salim Habayeb, Keith Campbell-Pag6 (task team leader), Caroline McEuen Hansen, Evangeline Javier, Cornelius Kostermans, Rama (editor), Kathy Strauss and Lunn Lestina (graphic design- Lakshminarayanan, Maureen Lewis, Christopher ers), and Juicy Qureishi-Huq (administrative assistant). Lovelace, Jane Nassim, Tawhid Nawaz, Hope Phillips, Alex Preker, Wendy Roseberry, Jean-Jacque de St. Antoine, Richard Skolnick, Fatoumata Traore-Nafo, and Christopher Walker. Finally, we would like to thank the Director-General, Operations Evaluation: Robert Picciotto many task team leaders and managers who took time Director, Operations Evaluation Department: Elizabeth McAllister from busy schedules to participate in the focus group discussions of the study and its recommendations. Manager, Sector and Thematic Evaluations: Gregory Ingram Task Managers: Susan Stout and Timothy Johnston v F o r e word FOREWORD PREFACIO PREFACE The World Bank began lend- El Banco Mundial comenzo a La Banque mondiale a procédé ing for population in the 1970s and otorgar préstamos para el sector de à ses premières opérations de prêts au to health in 1980, and has since población en los afios setenta, y para titre de projets ayant trait à la committed over $14 billion to sup- el de salud en la década de 1980, y population dans les années 70 et ses port health, nutrition, and popula- desde entonces ha comprometido más premiers projets dans le domaine de la tion (HNP) activities in 92 coun- de US$14.000 millones en respaldo de santé en 1980 ; depuis lors, elle a tries. The growth in lending has actividades de salud, nutrición y engagé plus de 14 milliards de dollars accelerated, and the Bank's empha- población en 92 paises. El ritmo de pour financer des activités dans le sis bas evolved from expanding crecimiento del financiamiento se ha secteur santé, nutrition et population HNP service delivery capacity to acelerado y el Banco, que en un dans 92 pays. Le rythme de ses encouraging systemic reform. The principio centró la atención en opérations s'est accéléré et la Banque Bank is now the major source of aumentar la capacidad para prestar qui, au départ, s'efforçait d'accroître external finance for the sector in the esos servicios, ahora fomenta la les capacités disponibles pour fournir developing world, and the policy reforma sistémica del sector. En la des services de santé, nutrition et influence of its nonlending services actualidad el Banco es la principal population vise maintenant à is potentially significant. fuente de financiamiento externo para promouvoir une réforme des systèmes. This volume is based on a review este sector en el mundo en desarrollo, Elle est la principale source de of the HNP evaluation literature, a y la influencia en materia de politicas financement extérieur de ce secteur desk review of the World Bank's HNP de sus servicios no crediticios puede dans le monde en développement, et portfolio, four country case studies llegar a ser significativa. ses services hors prêts peuvent avoir (Brazil, India, Mali, and Zimbabwe), El presente estudio està basado en un impact sensible sur les politiques de and consultations with Bank staff, un examen de los análisis sobre la la santé. borrowers, NGOs, and donors. The evaluación de los proyectos, un L'étude présentée dans ce volume HNP Sector Board has broadly en- estudio técnico de la cartera de repose sur un examen des analyses dorsed the findings and recommenda- proyectos de salud, nutrición y consacrées à l'évaluation des projets, tions of the review. población, cuatro estudios de casos un examen sur dossier du portefeuille The overarching recommenda- prácticos (Brasil, India, Mali y Zimba- de la Banque dans le secteur santé, tion of the review is that the Bank bwe), y en consultas con funcionarios nutrition et population, quatre étude should seek to do better-not more- del Banco, prestatarios, ONG y de cas (Brésil, Inde, Mali et in the HNP sector. The rapid growth donantes. La Junta Sectorial de Salud, Zimbabwe), et des consultations avec of the portfolio, together with com- Nutrición y Población ha aprobado les services de la Banque, des plex challenges posed by health sys- ampliamente las observaciones y emprunteurs, des ONG et des bailleurs tem reform, require consolidation recomendaciones del examen. de fonds. Le groupe technique pour le with a focus on selectivity and qual- La principal recomendación que se secteur santé, nutrition et population ity. OED recommends substantial formula en el examen es que el Banco s'est prononcé en faveur des improvement in monitoring and debería tratar de mejorar sus conclusions et recommandations de evaluation of project and sector per- operaciones, y no de aumentar su cet examen. formance and increased attention to número. El acelerado crecimiento de la L'examen recommande avant tout institutional development in project cartera, sumado a los complejos que la Banque s'efforce d'accroître la design and supervision. It also recom- desafios que plantea la reforma del qualité - et non le volume - de ses mends strengthened efforts in health sistema de salud, hacen necesaria su opérations. Étant donné la rapide promotion and intersectoral interven- consolidación con miras a la selectividad expansion de son portefeuille, vii Investing in Health: Development Effectivenesï in the Health, Nutrition, and Population Sector tions; a renewed emphasis on y la calidad. El DEO recomienda conjuguée aux difficiles economic and sector work; mejorar considerablemente el problèmes soulevés par la greater understanding of stake- seguimiento y la evaluación de los réforme du système de santé, la holder interests, and the forg- resultados de los proyectos y del Banque doit recentrer ses ing of strategic alliances with sector, y prestar más atenciôn al activités en mettant l'accent sur development partners at the desarrollo institucional a la hora la sélectivité et la qualité. L'OED regional and global levels. de disefiar y supervisar los recommande d'améliorer proyectos. Recomienda asimismo sensiblement le suivi et l'évaluation des intensificar los esfuerzos para promover projets et les résultats du secteur, et de la salud y las intervenciones prêter davantage attention au intersectoriales; volver a poner el acento développement institutionnel au en los estudios económicos y sectoriales; niveau de la conception et de la comprender mejor los intereses de las supervision des projets. Il partes involucradas, y establecer recommande également d'intensifier alianzas estratégicas con los asociados les efforts de promotion de la santé et en el desarrollo a nivel tanto regional d'accroître les interventions como mundial. intersectorielles ; de mettre encore plus l'accent sur les études économiques et sectorielles ; de mieux comprendre les intérêts des parties prenantes et de forger des alliances stratégiques avec les partenaires au développement au niveau régional et mondial. Robert Picciotto Director-General, Operations Evaluation Department viii Executive Sunmmary EXECUTIVE RESUMEN RESUME SUMMARY ANALYTIQUE World Bank lending to the El financiamiento del Banco La Banque mondiale s'est lancée health, nutrition, and population Mundial para el sector de salud, dans des opérations de prêt dans le (HNP) sector, begun in the 1970s, nutrición y población se inició en los secteur santé, nutrition et population has grown from a modest start to a afños setenta y, tras un comienzo dans les années 70, et le montant total total 1998 portfolio of US$14 bil- moderado, se ha convertido en una des projets réalisés à ce titre dans son lion. The Bank is now the world's cartera que en 1998 llegô a un total de portefeuille, limité au départ, a atteint 14 largest international financier of US$14.000 millones. El Banco es en la milliards de dollars en 1998. La Banque HNP, with average annual com- actualidad la principal fuente est maintenant la plus importante mitments of $1.3 billion, and has internacional de financiamiento para source de financement internationale de - _ ce secteur, puisqu'elle engage en moyenne 1,3 milliard de dollars par an et joue un rôle croissant dans les débats consacrés à la politique de la santé par J- les instances internationales et les pays emprunteurs. Depuis le début des années 90, la Banque recentre son action, qui consistait jusque-là à élargir Ai_ la portée des services de base assurés par le secteur public, pour privilégier l'amélioration des politiques de la santé et encourager une réforme du secteur. Dans le cadre des projets actuels, la Banque et ses partenaires s'efforcent de remédier aux obstacles fondamentaux sans oublier, toutefois, qu'il est difficile d'améliorer l'efficacité et l'efficience du played an increasingly important este sector, con una cifra anual media de secteur de la santé, même dans les pays role in international and borrower compromisos de US$1.300 millones, y développés. Le présent volume fait la health policy debates. Since the su participación en los debates sobre la synthèse de la première évaluation early 1990s, the Bank's opera- politica de salud a nivel internacional y globale des activités poursuivies par la tional emphasis has shifted from con los prestatarios ha adquirido cada Banque dans ce secteur. L'étude de l'OED expanding the public provision of vez màs importancia. Desde principios a donné lieu à un examen sur dossier du basic services toward improving de los afños noventa, el énfasis de las portefeuille d'opérations dans le secteur health policies and promoting operaciones del Banco se ha desplazado santé, nutrition et population de la health sector reforms. With the de la ampliaciôn del suministro de Banque et de quatre études de cas current generation of projects, the servicios bàsicos pot el sector público a (Brésil, Inde, Mali et Zimbabwe), et Bank and its partners are attempt- la mejora de las politicas de salud y la couvre des projets achevés et en cours. ing to address underlying con- promoción de las reformas del sector. straints to sector performance, En la generación actual de proyectos, el Performance des opérations while remaining cognizant of the Banco y sus asociados estàn intentando L'OED, qui a évalué les 107 projets difficulty of improving health sec- abordar las limitaciones bàsicas que réalisés dans le secteur santé, nutrition et tor effectiveness and efficiency, influyen en los resultados del sector, al population entre les exercices 75 et 98, a ix Investing in Health: Development Effcctiveness in the Health, Nutrition, and Population Sector even in developed countries. tiempo que reconocen las jugé que 64 % d'entre eux This volume offers a synthe- dificultades que plantea aumentar avaient donné des résultats sis of the first comprehen- la eficacia y eficiencia en el sector satisfaisants, contre 79 % pour sive assessment of the Bank's de salud, incluso en los paises les projets menés dans d'autres experience in the sector. The desarrollados. En esta secteurs. Il se pourrait toutefois OED study included a desk publicación se presenta una que les efforts déployés par les review of the Bank's HNP sintesis de la primera evaluación services de la Banque et les portfolio and four country case integral de la experiencia del secteurs pour améliorer les résultats studies (Brazil, India, Mali, and Banco en este sector. El estudio del DEO commencent à porter leurs fruits : 79 % Zimbabwe), and encompasses comprendió un estudio técnico de la des projets achevés durant l'exercice 97/ both completed and ongoing cartera de proyectos de salud, nutrición 98 ont atteint leurs objectifs de projects. y población del Banco y cuatro estudios développement, soit un pourcentage de casos prácticos (Brasil, India, Mali y proche de la moyenne de 77 % observée Project Performance Zimbabwe), y abarca los proyectos pour la Banque. Bien que la moitié Of the 107 HNP projects completed terminados y los que estàn en ejecución. seulement de tous les projets dans le between FY75 and FY98, OED secteur santé, nutrition et population rated 64 percent satisfactory, com- Resuitados de los proyectos menés à terme aient été jugée pouvoir pared with 79 percent for non-HNP De los 107 proyectos de salud, produire des résultats durables, cette projects. But efforts by the Bank and nutricién y población terminados proportion est passée à deux tiers sector staff to improve performance entre los ejercicios de 1975 y 1998, el durant l'exercice 97/98. Il est vrai que le may be showing results: 79 percent DEO calificó como satisfactorios al pourcentage élevé de réalisation des of projects completed in FY97/98 64%, en comparación con el 79% de objectifs physiques masque les difficultés satisfactorily achieved their devel- los proyectos correspondientes a qu'a rencontré la Banque lorsqu'elle opment objectives, close to the Bank otros sectores. Con todo, los s'est efforcée d'induire des modifications average of 77 percent. Although esfuerzos del Banco y del personal que institutionnelles et politiques dans le only half of all completed HNP trabaja en ese sector orientados a secteur. L'OED a jugé que le projects were rated as likely to be mejorar los resultados comienzan a développement institutionnel n'était sustainable, this figure rose to two- surtir efecto: el 79% de los proyectos substantiel que pour 22 % des projets thirds in FY97/98. Yet high rates of terminados en el ejercicio de 1997/98 achevés dans le secteur santé, nutrition completion of physical objectives lograron alcanzar satisfactoriamente et population ; or, ce pourcentage n'était disguise difficulties the Bank has sus objetivos de desarrollo, porcentaje toujours que de 25 % pour l'exercice encountered in achieving policy and cercano al 77%, que es el promedio 97/98, soit un niveau bien inférieur à la institutional change in HNP. OED del Banco. Se estimó que tan sólo la moyenne observée pour la Banque rated institutional development as mitad de todos los proyectos de salud, (38 %) pour la même période. Le secteur substantial in only 22 percent of nutrición y población terminados santé, nutrition et population de la completed HNP projects, a figure tenían probabilidades de ser Banque doit donc chercher en priorité à that rose to only 25 percent in FY97/ sostenibles, pero esa cifra aumentó a améliorer les résultats en matière de 98, well below the Bank average of dos tercios en el ejercicio de 1997/98. développement institutionnel. 38 percent for the same period. Sin embargo, las elevadas tasas de Improving institutional develop- consecución de los objetivos fisicos Principales conclusions de ment performance is therefore a ocultan las dificultades con que ha l'évaluation priority for the Bank's HNP sector. tropezado el Banco para lograr La Banque mondiale a largement cambios institucionales y en materia contribué à renforcer les politiques et les Major Evaluative Findings de politicas en ese sector. El grado de services dans le secteur santé, nutrition The World Bank has made important desarrollo institucional fue calificado et population dans le monde entier. contributions to strengthening health, como considerable por el DEO en tan Grace à ses financements, elle a aidé à nutrition, and population policies and sólo el 22% de los proyectos élargir la portée géographique des services worldwide. Through its fi- terminados, cifra que aumentó apenas services de santé de base, à promouvoir nancing, the Bank has helped expand al 25% en el ejercicio de 1997/98 y la fourniture d'une formation utile aux x Executive Summary geographical access to basic que està muy por debajo del prestataires de services et à health services, sponsored promedio del Banco, de 38%, fournir d'autres intrants valuable training for service correspondiente al mismo importants aux services de santé providers, and supplied other periodo. En consecuencia, una de base du secteur public. La important inputs to basic gov- prioridad de los proyectos del Banque a eu également recours à ernment health services. The Banco en el sector de salud, ses opérations de prêts et à ses Bank also has used its lending nutricién y población es la services hors prêts pour and nonlending services to promote mejora de los resultados en encourager le dialogue et des réformes dialogue and policy change on a materia de desarrollo institucional. dans divers domaines importants, et variety of key issues, including fam- notamment les stratégies en matière de ily planning, health financing, and Principales conclusiones de la planning familial, de financement de la nutrition strategies. Clients appreci- evaluacién santé et de nutrition. Les clients ate the Bank's broad strategic per- La contribucién del Banco Mundial al accueillent favorablement le cadre spective on the sector, and the Bank fortalecimiento de las politicas y général dans lequel la Banque replace la has taken a growing role in donor servicios de salud, nutricién y stratégie pour le secteur, et l'institution coordination. Despite an initial focus población en todo el mundo ha sido participe dans une mesure croissante à on government health services, the importante. El financiamiento la coordination des opérations des Bank is increasingly focusing on is- otorgado por el Banco ha permitido bailleurs de fonds. Bien qu'elle ait, au sues of private and nongovernmental ampliar la cobertura geográfica de los départ, mis l'accent sur les services de organization (NGO) service delivery, servicios bàsicos de salud, patrocinar santé du secteur public, la Banque insurance, and regulation. In recent valiosas actividades de capacitacién s'intéresse désormais de plus en plus au years, the Bank also has placed para los proveedores de servicios y problème de la prestation des services greater emphasis on client ownership proporcionar otros insumos par le secteur privé et les organisations and beneficiary assessments in necesarios a los servicios básicos de non gouvernementales (ONG), project design and supervision. salud del sector público. El Banco l'assurance de ces services et leur Several broad concerns emerge también ha utilizado sus servicios réglementation. Depuis quelques années, regarding the Bank's performance to crediticios y de otro tipo para elle déploie de plus amples efforts pour date. First, the Bank has been more promover el diálogo y la reforma de obtenir l'adhésion des clients aux successful in expanding health service las politicas con respecto a diversas projets et l'opinion des bénéficiaires au delivery systems than in improving cuestiones fundamentales, como la niveau de la conception et de la service quality and efficiency or pro- planificacién de la familia, el supervision des projets. moting institutional change. Al- financiamiento de la salud y las Plusieurs commentaires généraux though the quality of institutional estrategias de nutricién. Los clientes peuvent être formulés à l'égard de la analysis has improved in recent valoran la amplia visién estratégica del performance de la Banque à ce jour. years, the Bank is often better at Banco con respecto al sector, y el Premièrement, l'institution a mieux specifying what practices need to Banco ha asumido un papel más réussi à étendre la portée des systèmes change than how to change them or preponderante en la coordinacién de de prestation de services de santé qu'à why change is difficult. Paradoxi- las operaciones de los donantes. En un améliorer la qualité et l'efficacité des cally, Bank project designs are usu- comienzo el Banco se concentré en los services ou à encourager des réformes ally the most complex-with a servicios de salud del sector péblico, institutionnelles. Bien que la qualité de greater number of components and pero en medida creciente ahora estâ ses analyses institutionnelles se soit organizational units-in countries centrando la atencién en la prestacién récemment améliorée, la Banque sait with weak institutional capacity. The de servicios por el sector privado y las souvent mieux indiquer quelles sont les Bank is adopting increasingly sophis- ONG, los seguros y la reglamentacién. pratiques qu'il importe de modifier ticated approaches to the promotion En ados recientes, el Banco también ha qu'expliquer comment les modifier ou of sector reform, but the institutional puesto mayor énfasis en la pourquoi il est difficile de les modifier. problems being addressed are grow- identificacién de los clientes con los Paradoxalement, les projets de la ing in difficulty. Yet experience has proyectos y las evaluaciones de los Banque sont généralement plus demonstrated that realistic objec- beneficiarios en la etapa de diseño y complexes - en ce sens qu'ils ont xi nvesting in Health: Developnent Effectiveness in the Health, Nutrition, and Population Sector tives, together with increased supervisiôn de los proyectos. davantage de composantes et attention to whys and hows, Se pueden formular varias font intervenir un plus grand increases the likelihood of observaciones generales en Io que nombre d'entités achieving institutional objec- respecta a la actuación del Banco administratives - dans les pays tives. hasta la fecha. En primer lugar, el dont les capacités Second, during project Banco ha logrado resultados màs institutionnelles sont limitées. La implementation, the Bank satisfactorios en la ampliación de Banque poursuit des méthodes typically focuses on providing in- los sistemas de prestación de de plus en plus élaborées pour puts rather than on clearly defining servicios de salud que en el promouvoir la réforme sectorielle, mais and monitoring progress toward mejoramiento de la calidad y eficacia de les problèmes institutionnels auxquels HNP development objectives. Be- los servicios o la promoción de cambios elle est confrontée sont de plus en plus cause of weak incentives and unde- institucionales. Si bien la calidad del délicats à résoudre. Or, on a vu à veloped systems for monitoring and anàlisis de las instituciones ha mejorado l'expérience que la probabilité evaluation (M&E) within both the en los últimos afios, el Banco suele ser d'atteindre les objectifs institutionnels Bank and borrower governments, màs eficaz en identificar cuàles son las est d'autant plus élevée que les objectifs there is little evidence regarding the pràcticas que deben modificarse, y no adoptés sont réalistes et que l'on impact of Bank investments on sys- tanto en decidir cômo deben cambiarse s'efforce de trouver des réponses aux tem performance or health out- o en determinar por qué son dificiles de pourquoi et comment. comes. The Bank therefore has not cambiar. Curiosamente, el diseñio de los Deuxièmement, pendant la phase used its lending portfolio to system- proyectos del Banco suele ser más d'exécution d'un projet, la Banque atically collect evidence on what complejo --en el sentido de que s'efforce généralement de fournir des works, what does not, and why. comprenden muchos componentes y intrants plutôt que de définir clairement Methodological challenges can requieren la intervención de un gran les objectifs de développement pour le make it difficult to conclusively link número de unidades administrativas- secteur santé, nutrition et population, et project interventions with changes en los paises en que la capacidad de suivre les progrès accomplis en ce in HNP outcomes or system perfor- institucional es más deficiente. El Banco domaine. Étant donné l'insuffisance des mance. But experience shows that está adoptando enfoques cada vez más incitations et les carences des systèmes de effective M&E design-including complejos para promover la reforma del suivi et d'évaluation tant à la Banque the selection of a limited number of sector, pero la dificultad de los que dans les pays emprunteurs, on ne appropriate indicators, attention to problemas institucionales que debe dispose guère d'informations sur responsibilities, and capacity for abordar va en aumento. Con todo, la l'impact des investissements de data collection and analysis-en- experiencia ha demostrado que cuando l'institution sur la performance du hances the focus on results and los objetivos son realistas, y cuando se système ou les résultats obtenus dans le increases the likelihood of achiev- presta más atenciôn al cómo y al domaine de la santé. La Banque ne tire ing development impact. porqué, hay más probabilidades de donc pas de son portefeuille de prêts des Third, with some notable excep- alcanzar los objetivos institucionales. informations sur les activités qui tions, the Bank has not placed suffi- En segundo lugar, durante la donnent de bons résultats, celles qui cient emphasis on addressing deter- ejecuciôn de los proyectos el Banco n'en donnent pas et les raisons d'être de minants of health that lie outside the normalmente se ocupa de proporcionar cet état de fait. Pour des raisons medical care system, including be- insumos y no de definir claramente y méthodologiques, il peut être difficile havioral change and cross-sectoral vigilar los progresos para alcanzar los d'établir une relation probante entre les interventions. The incentives and objetivos de desarrollo en materia de interventions poursuivies et l'évolution mechanisms for intersectoral ap- salud, nutriciôn y población. Como no des résultats enregistrés dans le secteur proaches are currently weak, both hay suficientes incentivos y los sistemas ou de la performance du système. On a within the Bank and in borrower de seguimiento y evaluación tanto del toutefois pu observer que l'adoption governments, so priorities for inter- Banco como de los gobiernos d'une structure efficace de suivi et sectoral work must be chosen care- prestatarios no están plenamente d'évaluation -grâce, notamment, au fully. The Bank has a fundamental desarrollados, las pruebas del impacto choix d'un petit nombre d'indicateurs responsibility, however, to more ef- de las inversiones del Banco en el pertinents et à l'importance accordée xii Executive Summary fectively link its macroeco- desempefño de los sistemas o en aux fonctions et aux capacités de nomic dialogue with sector los resultados en materia de collecte et d'analyse des dialogue, particularly on is- salud son escasas. Por bo tanto, el données - permet de mieux sues of health financing, the Banco no se ha valido de la cibler les résultats et d'accroître la health work force, and civil cartera de préstamos para probabilité que les opérations service reform. recopilar sistemáticamente aient un impact sur le Finally, promoting health pruebas de bo que da resultado, développement. reform requires strategic and flexible bo que no funciona y las razones Troisièmement, à quelques approaches to support the develop- correspondientes. Los problemas importantes exceptions près, la ment of the intellectual consensus and metodolôgicos pueden dificultar la tarea Banque ne porte pas une attention broad-based coalitions necessary for de vincular categóricamente las suffisante aux facteurs déterminants change, but the Bank is still in the intervenciones en el marco de los de la santé qui ne relèvent pas du early stages of adapting its instru- proyectos con los cambios en los domaine médical, et notamment la ments to emphasize learning and resultados relativos a la salud, nutrición modification des comportements et les knowledge transfer. System reform is y poblaciôn o el desempeîo del sistema. interventions intersectorielles. Les difficult and time-consuming, and No obstante, la experiencia indica que incitations et mécanismes applicables stakeholders outside ministries of con un seguimiento y evaluación eficaces aux approches intersectorielles sont health can determine whether reforms -que incluya la selecciôn de un número très limités pour l'instant, tant à la succeed or fail. This highlights the limitado de indicadores adecuados y la Banque que dans les pays importance of realism in setting atención a las responsabilidades y a la emprunteurs, de sorte qu'il importe project objectives, strong country capacidad de recopilación y análisis de d'établir avec soin l'ordre des priorités presence, stakeholder analysis, and a los datos-es posible centrar más la pour ces opérations. Il est toutefois more strategic use of the Bank's atención en los resultados y aumentar la crucial que la Banque relie mieux son convening role. While incremental probabilidad de que las operaciones dialogue de politique approaches are not always more ap- tengan un impacto en el desarrollo. macroéconomique et son dialogue propriate, the Bank may have been En tercer lugar, salvo algunas sectoriel, notamment dans les excessive in its encouragement of excepciones importantes, el Banco no ha domaines du financement de la santé, overly ambitious reforms. puesto suficiente atención a los factores du personnel sanitaire et de la réforme determinantes de la salud que escapan al de la fonction publique. Implications for Current and Future àmbito del sistema de atención médica, Enfin, pour promouvoir une Work como las modificaciones del réforme de la santé, il est nécessaire de In 1997 the Bank released its Healtb, comportamiento y las intervenciones poursuivre une approche stratégique et Nutrition, and Population Sector intersectoriales. Tanto el Banco como los souple de manière à parvenir à un Strategy Paper, which will guide the gobiernos prestatarios no tienen los consensus sur les questions de fonds et Bank's work in the sector over the incentivos y los mecanismos adecuados bâtir les vastes alliances nécessaires au next decade. The strategy identifies para adoptar enfoques intersectoriales, changement ; toutefois, la Banque three objectives for the Bank: (i) de manera que las prioridades de la commence seulement à adapter ses improve the health, nutrition, and labor intersectorial deben establecerse instruments pour mettre l'accent sur les population outcomes of the poor; (ii) con mucho cuidado. No obstante, el transferts de savoir et de connaissances. enhance the performance of health Banco tiene la responsabilidad Les réformes systémiques sont care systems; and (iii) secure sustain- fundamental de vincular más l'aboutissement d'un processus difficile able health care financing. The strat- eficazmente su diâlogo macroeconómico et long, dont le succès ou l'échec peut egy paper incorporated preliminary con el diálogo sectorial, sobre todo en lo être déterminé par les parties prenantes findings from the OED review, and que respecta al financiamiento de la hors du ministère de la Santé. Il est donc sector leadership has already initi- salud, los trabajadores de la salud y la manifestement crucial de fixer des ated a number of activities to address reforma de la administración pública. objectifs réalistes pour les projets, de the issues raised. Por último, para promover la maintenir une présence importante sur Several concerns will need to be reforma del sector de salud es preciso le terrain, d'analyser les informations addressed, however, if the strategy aplicar enfoques estratégicos y flexibles des parties prenantes et de faire un xiii Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector is to meet its goals. First, que permitan formar el consenso although the need for im- intelectual y las coaliciones de uge luslst atiu d pouvoir de mobilisation de la proved system performance amplia base que se requieren para Banque. S'il ne vaut pas toujours and reforms is manifest in el cambio; sin embargo, el Banco mieux procéder de manière many client countries, the recién està empezando a adaptar progressive, il se peut en revanche Bank is increasingly engaged sus instrumentos para in areas-such as public regu- intensificar la adquisiciôn y la que la Banque ait par trop lation of private insurers-where it transferencia de conocimientos. encouragé des réformes excessivement ambitieuses. bas little experience and where no La reforma de los sistemas es una tarea clear right models exist. Second, dificil y prolongada, y su éxito o fracaso Conséquences pour les opérations current approaches may be success- Io pueden determinar las partes actuelles et futures ful, but the emphasis on institu- interesadas distintas de los ministerios tional reform means that the Bank is de salud. Lo anterior subraya la En 1997, la Banque a publié un doing more of what it has done least importancia de fijar objetivos realistas dunt dtratéie poulesectur well in the past. An aggressive para los proyectos, de mantener una santé, trion e an , qu program to develop appropriate sólida presencia en el pais, de analizar la guidera ses travaux dans ce secteur au standards, instruments, and staff informacion de los interesados y de cors deie prochais annes La training for HNP institutional aprovechar màs estratégicamente el strt ite le triqe q analysis is necessary, together with poder de movilizaciôn que tiene el .it porsre la aq e encouragement of realism in setting Banco. Aunque no siempre es institutional objectives. conveniente proceder en forma les pauvres dans le secteur de la santé, nutrition et population ; ii) renforcer Third, the rapid growth in the progresiva, es posible que el Banco se size and ambition of the HNP port- haya excedido en promover reformas la performance des systèmes de soins folio has coincided with only mod- demasiado ambiciosas. de santé ; et iii) mettre en place un est growth in the number of staff, système de financement viable des stagnation in supervision resources, Consecuencias para las soins de santé. Le document de and declining funding for analytic operaciones actuales y futuras stratégie sectorielle faisait état des and advisory work. Staff are En 1997 el Banco emitiô el Documento conclusions préliminaires de l'examen overprogrammed, particularly in de estrategia para el sector de salud, de l'OED, et les responsables sectoriels ont déjà entrepris diverses activités relation to the time-intensive de- nutriciôn y población, que orientará su , mands of participatory approaches, labor en dicho sector en el próximo pour s'attaquer aux problèmes posés. partnerships, and consensus build- decenio. En la estrategia se establecen Pour que la stratégie puisse . . atteindre son objectif, il faudra toutefois ing. This imbalance calls for a more tres objetivos para el Banco: i) mejorar agir s plses pla rmèeent flexible allocation of administrative los resultados obtenidos para los agir sur plusieurs plans. Premièrement, . , bien qu'il faille manifestement améliorer resources and greater selectivity by pobres en el sector de salud, nutricion y la perfance dunsystmeet procéder management regarding priority poblacion; ii) mejorar los sistemas de ... des reformes dans de nombreux pays countries, sector activities, and in- atención de salud, y iii) establecer un des m anse nombreux pays struments. sistema de financiamiento sostenible clint lapBaneous n nmr Fourth, to achieve the sector goal para la atención de salud. En el coist 'opér ation of improving HNP outcomes for the documento de estrategia se tuvieron en dai ' es ueula réeenan poor, the Bank will need to place cuenta las observaciones preliminares l'aues assurersepriés - dn stronger emphasis on poverty target- del examen preparado por el DEO, y los . ing, measuring IINP outcomes, and responsables del sector ya han iniciado il n'existe pas de modèles établis. assessing the poverty impact of its diversas actividades para abordar los Deuximement, meme sites initiatives investments and policy advice. Inten- problemas planteados. entreprises donnent de bons résultats, la invetmens ad poicyplace importante accordée aux réformes sive experimentation, learning, and Sm embargo, para que se puedan insittn e signifie u la fanqe sharing of experiences within the alcanzar las metas que establece la institutionnelles signifie que la Banque Bank and with clients and partners estrategia habrà que actuar en diversos intervient de plus en plus dans le must receive higher priority. ámbitos. Primero, si bien es evidente que domaine où elle a obtenu les moins bons résultats dans le passé. Il est nécessaire xiv Executive Summary Moving Forward en muchos paises clientes hay que de poursuivre un programme To improve the effectiveness of mejorar los sistemas e introducir intensif pour formuler des future Bank efforts, OED sug- reformas, el Banco interviene normes appropriées, créer des gests that the Bank give prior- cada vez màs en áreas -como la instruments et former le ity to the following, in both its regulación estatal de los personnel pour procéder à des internal processes and its inter- aseguradores privados- en las analyses institutionnelles du actions with borrowers: que su experiencia es escasa y no secteur santé, nutrition et existen modelos establecidos. population, tout en encourageant la • Enhance quality assurance and Segundo, incluso si los enfoques formulation d'objectifs institutionnels results orientation. To improve actuales surten efecto, el acento que se réalistes. HNP portfolio quality, the pone en las reformas institucionales Troisièmement, la rapide expansion HNP Sector Board and te- significa que el Banco està interviniendo du volume et de l'envergure du gional technical managers màs en àmbitos en los que ha sido portefeuille d'opérations dans le secteur should strengthen their role in menos eficaz en el pasado. Hace falta santé, nutrition et population s'est monitoring portfolio quality, aplicar un programa enérgico para accompagnée d'un faible accroissement project results, and quality as- establecer normas y crear instrumentos des effectifs, d'une stagnation des surance. Routine quality assur- adecuados, e impartir capacitación al ressources de supervision et d'une ance mechanisms should be personal para realizar análisis diminution du financement des travaux enhanced to provide timely institucionales del sector de salud, analytiques et des services de conseil. Le support to task teams in project nutriciôn y población. Además, es personnel est surchargé, notamment si design and supervision. To necesario ser màs realistas a la hora de l'on considère le temps qui doit être strengthen results orientation, fijar los objetivos institucionales. investi dans les approches participatives, the Bank should continue ef- Tercero, el acelerado aumento del les partenariats et la formation de forts to develop standards and tamafño y la envergadura de la carrera de consensus. Pour réduire ce déséquilibre, good practice examples for proyectos de salud, nutriciôn y la direction devra assurer une allocation M&E, and increase staff train- población ha estado acompafñada de un plus flexible des ressources ing. But strengthening incen- incremento moderado del número de administratives et déterminer de manière tives to achieve results and use funcionarios, el estancamiento de los plus sélective les pays, activités information, both within the recursos destinados a la supervisión y la sectorielles et instruments auxquels il Bank and in client countries, is disminución de los recursos financieros convient de donner la priorité. critical to enhancing borrower para estudios analiticos y actividades de Quatrièmement, pour pouvoir M&E capacity. Increased ex- asesoria. El personal enfrenta una atteindre l'objectif qu'elle s'est fixé, à perimentation with and learn- sobrecarga de trabajo, sobre todo si se savoir améliorer les résultats obtenus ing from performance-based tiene en cuenta el tiempo que se debe pour les pauvres dans le secteur santé, budgeting mechanisms in Bank dedicar a los enfoques participatorios, nutrition et population, la Banque devra projects would be an important las asociaciones de esfuerzos y la accorder une importance accrue au step. formación de consenso. Este ciblage de la pauvreté, à la détermination • Intensify learning from lend- desequilibrio hace necesaria una mayor des résultats et à l'évaluation de l'impact ing and nonlending services. flexibilidad en la asignación de los de ses investissements et de ses conseils In light of the institutional recursos administrativos y una mayor sur la pauvreté. Il lui faudra accorder challenges facing the health selectividad por parte de la une plus haute priorité à la poursuite sector and weak institutional administración en bo que respecta a los d'expérimentations intensives, ainsi qu'à performance, the Bank should paises, sectores, actividades e l'acquisition et au partage de seek to establish appropriate instrumentos que revisten màs connaissances tant dans ses services tools, guidelines, and training prioridad. qu'avec ses clients et partenaires. programs for institutional and Cuarto, para alcanzar los objetivos stakeholder analysis in HNP. sectoriales que se ha fijado, a saber, Orientation pour l'avenir This should include strength- mejores resultados para los pobres en L'OED suggère que, pour accroître ening analytic work on major materia de salud, nutrición y población, l'efficacité de ses efforts à l'avenir, la xv 1nvesting in Health: Developnient Effectiveness in the Health, Nutrition, and Population Sector institutional challenges and el Banco deberá intensificar sus Banque s'efforce en priorité, providing flexible support to esfuerzos por Ilegar a los pobres, aussi bien dans le cadre de ses task teams facing difficult medir los resultados y evaluar el opérations internes que dans institutional problems. To impacto de sus inversiones y celui de ses interactions avec les strengthen the analytic base asesoria sobre politicas en la emprunteurs, de : for Bank advice and lending, pobreza. Se debe dar mayor * Renforcer l'assurance de la management should increase , prioridad a la experimentación qualité et améliorer l'orientation funding for advisory and ana- intensiva, la adquisicién de des résultats. Pour améliorer la lytic services and shift some conocimientos y el intercambio de qualité du portefeuille d'opérations of the budget for those ser- experiencias dentro del Banco y con los dans le secteur santé, nutrition et vices from country depart- clientes y asociados. population, la Commission tech- ments to regional technical nique du secteur et les directeurs managers. The HNP Sector Recomendaciones para el futuro techniques régionaux devraient Board and technical manag- A fin de aumentar la eficacia de los accroître leur contribution au suivi ers should strengthen their esfuerzos que emprenda el Banco en el de la qualité du portefeuille, aux role in enhancing the quality futuro, el DEO recomienda dar résultats des projets et à of advisory and analytic ser- prioridad a los siguientes aspectos, l'assurance de la qualité. Il vices, and encourage more tanto en sus procedimientos internos conviendrait de renforcer les intensive use of project expe- como en su interacción con los mécanismes normalement utilisés rience as a source of both prestatarios: pour s'assurer de la qualité des questions and answers. opérations de manière à fournir un Enhance partnerships and selec- * Intensificar la garantia de calidad y appui rapide aux équipes chargées tivity. To strengthen strategic la orientación hacia los resultados. de la conception et de la supervi- selectivity, Bank management Para mejorar la calidad de la sion des projets. Pour améliorer and the HNP Sector Board carrera de proyectos de salud, l'orientation des résultats, la should undertake a review of nutrición y población, la Banque devrait continuer de current staffing, lending, and correspondiente Junta Sectorial y s'efforcer de formuler des normes administrative resources in light los directivos técnicos regionales et de produire des exemples de of the 1997 sector strategy and deberian intensificar su función de bonnes pratiques dans le domaine the recommendations above. vigilancia de la calidad de la du suivi et de l'évaluation, et aussi The goal should be to establish carrera, los resultados de los d'intensifier les efforts de forma- priorities, assess resource impli- proyectos y la garantia de calidad. tion. Il ne sera toutefois possible de cations, and reduce conflicting Habria que mejorar los renforcer les capacités de suivi et mandates. Selectivity also te- mecanismos habituales de garantia d'évaluation des emprunteurs quires effective partnerships. de calidad, a fin de apoyar qu'en incitant mieux les services de The Bank should select a few oportunamente a los equipos a la Banque et les pays clients à strategic areas for enhanced sec- cargo del diseio y la supervisión obtenir des résultats et à exploiter toral coordination, with particu- de los proyectos. Para intensificar les informations disponibles. Il sera lar focus on macroeconomic dia- la orientación hacia los resultados, important, à cet égard, de recourir logue and health work force el Banco deberia proseguir su labor dans une mesure accrue à des issues. In client countries, the de promoción de los ejemplos de mécanismes de budgétisation Bank could use its prestige and normas y prácticas recomendadas assortis de critères de performance convening role to encourage en materia de seguimiento y et d'en tirer les leçons qui communication and collabora- evaluación, y aumentar la s'imposent dans le cadre des tion among government minis- capacitación que imparte al pet- projets de la Banque. tries, and between government sonal. No obstante, si se ha de * Tirer de plus amples and other parmers. At the inter- mejorar la capacidad de enseignements des opérations de national level, the Bank could seguimiento y evaluación de los prêts et des services hors prêts. strengthen its partnership with prestatarios, es indispensable Étant donné les carences et les xvi Executive Summary the World Health Organiza- ofrecer mayores incentivos para problèmes institutionnels dont tion (WHO) and other inter- obtener resultados y aprovechar souffre le secteur de la santé, la ested agencies to address such la información disponible no Banque devrait s'efforcer de priorities as strengthening sólo en el propio Banco sino mettre en place les instruments, M&E and performance-based también en los paises clientes. directives et programmes de for- health management systems in Una medida importante seria mation nécessaires à la client countries. probar en mayor medida poursuite d'une analyse des in- mecanismos de presupuestacién stitutions et des besoins des par- basados en los resultados de los ties prenantes dans le secteur proyectos del Banco, y obtener santé, nutrition et population. Il ensefñanzas de dicha experiencia. faudrait, pour ce faire, renforcer • Adquirir más conocimientos a les travaux analytiques consacrés partir de los servicios crediticios y aux principaux problèmes no crediticios. En vista de los d'ordre institutionnel et offrir un problemas y deficiencias appui flexible aux équipes de institucionales que enfrenta el projet qui se heurtent à de graves sector de salud, el Banco deberia problèmes en ce domaine. Pour procurar establecer herramientas, permettre à la Banque de baser directrices y programas de ses conseils et ses opérations de capacitacién adecuados para prêts sur des analyses plus realizar anàlisis de las solides, la direction de instituciones y de las necesidades l'institution devrait accroître les de las partes interesadas en el sec- ressources affectées aux services tor de salud, nutrición y de conseil et d'études et réaffecter poblacién. Esta labor deberia une partie du budget consacré à incluir el fortalecimiento de los ces services des départements- estudios analiticos sobre los pays aux directions techniques principales desafios de tipo régionales. La Commission tech- institucional, y un respaldo flex- nique du secteur santé, nutrition ible a los equipos responsables et population et les directeurs que enfrentan dificultades en ese techniques devraient contribuer àmbito. A fin de fortalecer la base dans une mesure accrue à analitica de la asesoria y el améliorer la qualité des services financiamiento que proporciona de conseil et d'analyse et encour- el Banco, la administracién ager les services de la Banque à deberia aumentar los recursos utiliser l'expérience accumulée financieros para la prestación de dans le cadre des projets pour ex- servicios de asesoria y análisis y aminer diverses questions et ten- traspasar una parte del ter d'y répondre. presupuesto para esos servicios • Renforcer les partenariats et desde los departamentos a cargo procéder de manière plus de paises a los directivos técnicos sélective. Pour procéder de regionales. La Junta Sectorial de manière plus sélective, la direction Salud, Nutrición y Población y de la Banque et la Commission los directivos técnicos deberian technique du secteur santé, nutri- tener una mayor participacién en tion et population devraient la mejora de la calidad de los entreprendre un examen de l'état servicios de asesoria y análisis, y des effectifs, des opérations de promover un aprovechamiento prêt et des ressources xvii Investing ¡i Health: I)evelopnienrt Effectiveness in the Health, Nutrition, and Population Sector más intensivo de la experiencia administratives eu égard á la derivada de los proyectos que stratégie définie pour le secteur les permita examinar diversas en 1997 et aux interrogantes e intentar darles recommandations présentées respuesta. ci dessus. Ce faisant, elles • Incrementar las asociaciones devraient chercher á définir des y la selectividad. A fin de priorités, évaluer les objectifs incrementar la selectividad qui peuvent étre poursuivis avec estratégica, la administración del les ressources disponibles et Banco y la Junta Sectorial de réduire le nombre des missions Salud, Nutrición y Población inconciliables. Pour procéder de deberían emprender un examen maniére plus sélective, il leur de la actual dotación de personal, faudra aussi constituer des el financiamiento y los recursos partenariats efficaces. La Banque administrativos teniendo en devrait choisir quelques cuenta la estrategia sectorial de domaines stratégiques dans 1997 y las recomendaciones an- lesquels elle renforcera la coordi- tes enunciadas. El objetivo sería nation sectorielle et mettra plus establecer prioridades, evaluar las particuliérement l'accent sur le repercusiones en lo que respecta dialogue de politique a los recursos, y reducir los macroéconomique et les ques- mandatos incompatibles. La tions ayant trait au personnel de selectividad también exige la santé. Dans les pays qui sont ses formación de asociaciones clients, la Banque pourrait utiliser eficaces. El Banco debería son prestige et son pouvoir de seleccionar unas pocas áreas mobilisation pour encourager la estratégicas en las cuales sea communication et la collabora- necesario mejorar la tion entre les ministéres coordinación sectorial, poniendo nationaux, et entre le especial énfasis en el diálogo gouvernement et d'autres macroeconómico y los asuntos partenaires. Au plan interna- relativos a los trabajadores de la tional, la Banque pourrait salud. En los países clientes, el renforcer son partenariat avec Banco podría aprovechar su l'Organisation mondiale de la prestigio y poder de movilización santé (OMS) et d'autres institu- para alentar la comunicación y la tions intéressées pour poursuivre colaboración entre los certains objectifs prioritaires tels ministerios, y entre el gobierno y que le renforcement des activités otros asociados. En el plano de suivi et d'évaluation et la mise internacional, podría fortalecer en place de systémes de gestion de su alianza con la Organización la santé assortis de critéres de Mundial de la Salud (OMS) y réalisation dans les pays clients. otros organismos interesados para abordar temas prioritarios como el reforzamiento de la supervisión y la evaluación, y los sistemas de administración de la salud basados en los resultados en los países clientes. xviii ABBREVIATIONS AND ACRONYMS AFR - Africa Region APL - Adaptable Program Loan ARDE - Annual Review of Development Effectiveness CAS - Country Assistance Strategy CDF - Comprehensive Development Framework CPR - Contraceptive prevalence rates DAC - Development Assistance Committee DEC - Development Economics and Chief Economist EAP - East Asia and Pacific Region ECA - Europe and Central Asia Region ESAP - Economic Structural Adjustment Program ESW - Economic and sector work FHP2 - Second Family Health Project (Zimbabwe) FW - Family Welfare Program (India) HIV - Human immunodeficiency virus HNP - Health, nutrition, and population IBRD - International Bank for Reconstruction and Development (World Bank) ICB - International competitive bidding ICDS - Integrated Child Development Services (India) ICR - Implementation Completion Report IDA - International Development Association IEC - Information, Education, and Communication IMR - Infant mortality rate LAC - Latin America and the Caribbean Region LIL - Learning and Innovation Loans MCH - Maternal and Child Health M&E - Monitoring and evaluation MNA - Middle East and North Africa Region MOH - Ministry of Health MOHFW - Ministry of Health and Family Welfare (Zimbabwe) NGO - Nongovernmental organization OED - Operations Evaluation Department PCU - Project coordinating unit PDS - Health Development Project (Projet de Developpement Sanitaire) (Mali) PMU - Project management unit PPAR - Project Performance Audit Report PPM - Pharmacie Populaire du Mali PSPHR - Health, Population, and Rural Water Project (Mali) QAG - Quality Assurance Group RCH - Reproductive and Child Health Project (India) SAR - Staff Appraisal Report SAS - South Asia Region SIP - Sector Investment Program SUS - Sistema Unica da Saide (Brazil) STI - Sexually transmitted infection SWAP - Sector-wide Approach TA - Technical assistance TINP - Tamil Nadu Integrated Nutrition Project (India) UNICEF - United Nations Children's Fund WHO - World Health Organization xix Introduction and Evaluative Framework he multiplicity of interactions among the determinants of health status outcome and the performance of health systems make assessing the Bank's impact on the health, nutrition, and population (HNP) sector a challenge. This study seeks to evaluate the relevance, effectiveness, efficiency, institutional impact, and sustainability of nearly 30 years of Bank lending and nonlending services in HNP. The past 200 years have witnessed remarkable In 1996, OED initiated a study to assess the changes in human demographic patterns, including a effectiveness of this body of work and to distill lessons shift from high, relatively uncontrolled fertility to low, for future strategy in the sector. This volume, the third controlled fertility throughout large parts of the world. and final report of the exercise, synthesizes the findings Mortality also declined rapidly, and the rate of decline and conclusions of these studies and recommends steps has accelerated in the past 30 years. Fertility rates fell the Bank might take to strengthen its performance in by over 40 percent globally and by close to 50 percent the sector.' The report is deliberately summary in throughout much of the developing world, although nature. (Readers seeking details on the problems of they remain high in some areas. Despite this remark- evaluation in the sector or on OED's evaluation of the able progress, much remains to be done. This is Bank's performance are re- implicit in the focus that the Development Assistance ferred to the earlier reports.) The Bank is now Committee (DAC) has placed on several key health outcomes (such as reductions in infant, child, and Health Outcomes and the the major source of maternal mortality and improved access to reproduc- Health System external finance for tive health services) in its effort to build consensus on As shown in figure 1.1, mor- the sector in the goals and targets for the twenty-first century. bidity, mortality, nutritional The World Bank has been active in the HNP sector status, and fertility are deter- since 1970; by the close of fiscal 1997, it had committed mined by an array of factors more than $14 billion to lending in the health sector and in addition to health services. The most important are had initiated activities in 92 countries. The pace of growth income, education, and the quality of the environment- in sector activities has accelerated significantly in the past including access to safe housing, clean water, and seven years. The Bank is now the major source of external sanitation. The next most important are individual and finance for the sector in the developing world, and its community practices related to nutrition, sanitation, advice and research influence policies at many levels. reproduction, alcohol and tobacco use, and other health- 1 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector related behaviors, which are in turn related to social and is to follow activities in such areas as finance or economic status and culture (Lerer and others 1998). telecommunications (Israel 1987). Third, interactions Finally, HNP interventions can reduce the burden of among consumers and providers are shaped by informa- disease or shorten disease duration, either through preven- tion asymmetries and ritual: health practitioners often tive services and encouraging healthy behavior or by assume the role of "expert," while consumers have more providing curative care. In- information about their actual condition and health Prevention is creased understanding of the practices. Fourth, because of information failures, con- causes of disease and im- sumers are typically more willing to pay for curative often-although proved interventions for both services than for prevention and health promotion, and not always-more preventive and curative ser- demand for health care usually increases with income, cost-effective than vices-such as antibiotics creating a bias in health markets and public spending treatment. and vaccination-have im- toward urban curative care. Important preventive services proved HNP outcomes are often underfinanced as a result. Finally, the perfor- throughout the world. Pre- mance of health systems is strongly influenced by external vention is often-although not always-more cost-effec- factors such as macroeconomic performance and civil tive than treatment, but strong demand for curative service procedures. Improving policy and outcomes in this services can lead to a disproportionate emphasis on the sector is far from a simple matter of applying administra- medical care system, in both public policy and the market tive and technological expertise to stable and predictable for health care. epidemiological and medical challenges. All health systems share a number of institutional characteristics. First, they include multiple players, often Evaluating the Bank's Performance with conflicting interests and priorities. Second, because Figure 1.1 shows two major pathways of World Bank health outcomes are complex, it is more difficult to influence on the health sector and health outcomes in a monitor the performance of health facilities or staff than it country. The first is lending and policy advice directed FIGURE 1.1. ACHIEVING CHANGE IN HNP Behavior of Individuals/Households Social & Economic Well-being Performance of Health System Health Status Outcomes * Clinical Effectiveness * Fertility * Accessibility and Equity * Mortality * Quality and Consumer Satisfaction * Morbidity * Economic Efficiency * Nutritional Status Macroeconomic Environment HEALTH CARE SYSTEM Delivery Structure Institutional Capacity * Facilities (public and private) * Regulatory and Legal Framework * Staff (public and private) * Expenditure and Finance * Information, Education, and * Planning and Budgeting Systems Communication * Client and Service Information/Accountability * Incentives Governance Bank Projects and Policy Advice Introduction and Evaluative Framework BOX 1.1. HNP EVALUATION LITERATURE ED's review example, programs to of economic analysis to mance and efficiency of the evalua- improve the treatment of setting health priorities in developed and de- tion literature a particular disease), (World Bank 1993a). Yet veloping countries, (Stout and others 1997) rather than by evalua- these methods are best health officials and in- found that the litera- tions of the performance suited for analysis of se- ternational partners are ture on the evaluation of health systems. Evalu- lected interventions in increasingly grappling of family planning pro- ation in the health sector specific settings, and pro- with the challenges of grams is wider and is further constrained by vide little guidance on monitoring and evalu- more tractable than the paucity of reliable lo- linking the findings of ating clinical effective- that for health, nutri- cal estimates of health technically sound eco- ness, economic effi- tion, and disease con- outcomes, and, more fun- nomic analysis to par- ciency, and consumer trol programs. But in damentally, by the con- ticular institutional and satisfaction with health all three areas, the ceptual and philosophi- political contexts. system performance evaluation literature is cal problems of valuing Nevertheless, with (Durch, Bailey, and dominated by studies life. A growing body of the growing interest in Stoto 1997; McPake of the efficacy of spe- work is developing tools and emphasis on enhanc- and Kutzin 1997; Aday cific interventions (for to enable the application ing HNP system perfor- and others 1993). at the health system, which in turn may affect the approach of this study, and of the supporting case studies, system's delivery structure and institutional capacity. has been to make plausible judgments about the relevance Through these activities, the Bank (or other sources of and effectiveness of the Bank's work at various stages of finance or knowledge) can attempt to improve the the causal chain shown in figure 1.1. The study team accessibility, quality, and efficiency of health services, conducted extensive interviews and focus groups with or to influence behavior through health-promotion relevant Bank and borrower activities. Alternatively, the Bank can influence the officials, health workers, and health sector and health outcomes indirectly through its consumers, and sponsored macroeconomic policy advice and influence on na- background research papers HNP projects and tional governance, as well as through the mix and on key topics. Where data policy advice been effectiveness of investments in other sectors. Although allowed, the case studies relevant, effective, this review focuses primarily on the Bank's HNP compiled household or facil- activities, it also touches on the influence of Bank ity-based data to assess the macroeconomic advice on health sector performance. impact of selected Bank inter- An evaluation of the development effectiveness of the ventions on health outcomes or health system perfor- Bank's HNP work faces two major challenges. First, the mance, examining changes over time or among project multiple determinants of health outcomes make it diffi- and nonproject districts. cult-at both the individual and the population levels-to This review seeks to address three fundamental link a specific health intervention with changes in health, questions. First, have World Bank HNP projects and nutrition, or fertility status. Second, the Bank is only one policy advice been relevant to promoting improved of a multitude of actors involved in HNP in any given outcomes and health system performance (did the Bank country, and its lending and advice are invariably "do the right things")? Second, have Bank-supported mediated through national or local institutions and interventions been effective and efficient in achieving governments. Neither of these challenges to attribution is their stated objectives (did the Bank "do things right")? easily resolved (see box 1.1). Third, has the Bank been effective in strengthening But at the same time, this does not excuse the Bank or health care institutions, and have Bank interventions policymakers from attempting rigorous evaluation. The been financially and institutionally sustainable? 3 AM Evolution of Bank HNP Strategy and Lending he Bank's HNP strategy has evolved from relatively modest investments in population and family planning in the 1970s, to direct lending for primary health care in the 1980s, to health system reform in the 1990s. Bank lending to HNP has expanded dramatically during this period. Through its current emphasis on health sector reforms and health financing, the Bank is attempting to address some of the underlying constraints that limited the effectiveness of earlier efforts. Since 1970 the Bank has lent $14 billion in support India, and Indonesia) account for half of the lending of HNP operations-three-quarters of this sum since since 1970. The performance of projects in these 1990. Total commitments in HNP grew from about countries therefore strongly influences the average $500 million in the first decade of experience to about performance of their respective regions, and of the $1 billion from 1981 to 1987. More than $11 billion HNP portfolio as a whole. has been approved since 1990. The HNP portfolio is The content of specific HNP investments re- thus relatively young; by fiscal 1997 only a third of flects, with a substantial time lag, the Bank's evolv- projects were complete and evaluated. The average ing policy perspective. Investments in HNP evolved size of Bank projects has grown from less than $20 from single-purpose efforts million in the 1970s to about $75 million today. More to improve family plan- than half of HNP lending has been in the form of ning programs in the The Bank's HNP International Development Association (IDA) credits, 1970s, to efforts to expand portfolio is highly the highest percentage among the social sectors. health system capacity and concentrated in a HNP activities are integrated in many projects, expand delivery of pri- few countries. which makes it difficult to determine the amount of mary health care in the lending to each subsector. An estimated one-third of 1980s, and, more recently, HNP lending supported population and reproductive to encourage broad sector reforms. Less than 20 health, while Bank projects helped mobilize about $2 percent of all projects in the early 1980s focused on billion for nutrition programs. South Asia has received health system reforms, compared with nearly half of the greatest volume of lending (27 percent of the total), the projects approved since FY95. The Bank also but Africa has the largest number of projects. The sponsored a number of stand-alone population, nu- Bank's HNP portfolio is highly concentrated in a few trition, and disease control projects in the 1980s and countries: five countries (Bangladesh, Brazil, China, 1990s. 5 I nvesting in Health )evelopm ent Effectivenes in the Health, Nutrition, and Population Sector 1970s: Early Population Projects loan, usually with a geographic focus, although by the The World Bank began lending for population and family late 1980s the Bank increasingly sought to improve the planning activities in the early 1970s. The rationale was coherence of donor efforts by financing "umbrella" demographic. Rapid rates of population growth were projects that included financing from several donors. considered a major threat to development progress in The Bank strategy addressed a genuine need in many many developing countries; narrow, focused population borrower countries for improved access to basic health and family planning programs were believed to be services, particularly in rural areas. The interventions necessary to slow population supported in these projects-maternal and child health, In early loans, the growth. In keeping with this family planning, and nutrition education-addressed a view, in the 1975 Health significant portion of the burden of disease for the poor. Bank essentially Sector Policy Paper, the The projects also usefully promoted the integration of engaged in Bank's first formal HNP basic health services-both with family planning and conventional bank policy statement, it was with other health programs, such as immunization. lending. stated that the Bank would These family health or basic health projects, lend only for family planning however, were remarkably similar in design and and population, and not di- approach across regional and country settings.' The rectly for health, although health activities could be part large number of project components contributed to of population or other development efforts.' In these early project complexity, a particular challenge when many loans, the Bank essentially engaged in conventional bank health ministries were administratively weak and lending, providing finance to expand government pro- borrowing from the Bank for the first time. The Bank grams. Where these programs were relatively effective in responded by including "capacity building" compo- addressing consumer demand (as in Bangladesh or nents in most projects, and by relying increasingly on Indonesia), Bank projects were relatively successful, but project management units to facilitate project imple- where the government programs were weak (as in India), mentation, sometimes isolating the "project" from the the projects had less impact (OED 1992). rest of the ministry. These projects, like the earlier The Bank's approach in the 1970s sought to expand population projects, usually failed to effectively ad- the supply of, and increase access to, publicly provided dress underlying quality issues in government health family planning services. Despite clear evidence of the services, and typically provided for little interaction relatively significant role of private (nongovernmental with private, nongovernmental, or traditional health and commercial) providers in many countries, Bank providers (except for traditional midwife training). project designs tended to support public provision of services, relying on its policy analyses and nonlending 1990s: Health Financing and Health System Reform work to alert borrowers to the potential role of private A number of factors in the late 1980s and the 1990s providers. The Bank typically asserted an "unmet need" brought about a shift in Bank strategy, and led to an for family planning services, based on various surveys of increased emphasis on health financing and health women, and assumed that increased geographical access service reform (see box 2.1). First, efforts to expand to services would lead to increased use. This approach infrastructure and staffing for primary health care had three major failings: it did not assess the actual services in the 1980s coincided with the onset of demand for the services to be provided; it did not consider economic crisis in much of the developing world. whether increased public provision would result in a net Budget pressures, together with continued inefficiencies increase in health service availability (or merely displace in government health spending, threatened the quality consumers from other facilities); and it did not address the and sustainability of expanded government primary underlying constraints affecting the quality of public health care systems. Second, disappointment with the service provision. progress of specific investment projects in bringing about systemic change-together with a rising trend 1980s: Direct Lending for Primary Health Care toward health sector reforms in industrial countries- Its 1980 Health Sector Policy Paper committed the led to a growing international consensus on the need Bank to direct lending in the health sector. The primary for health sector reform in developing countries. Third, instrument for executing this policy was the investment the HIV/AIDS epidemic and the demographic transition 6 Evolution of Bank HNP Strategy and Lending in middle-income countries created new challenges for attention on the constraints to providing efficient and disease control. Finally, the challenges facing health effective health services in client countries. The Bank's systems increasingly diverged among regions. In much efforts have helped to raise awareness regarding health of Africa and Asia, communicable diseases and access financing issues in borrower countries and internation- to services are still problems, while in Central Europe ally; contributed to international debates on the cost- and Latin America, issues of cost-escalation and the effectiveness of various interventions; encouraged the burden of an aging population and high-cost chronic development of strategies for, and adoption of, health diseases have come to the fore (Prescott 1997). system reforms; and, in many aid-dependent countries, The Bank's increased emphasis on health financing helped improve donor coordination in the sector. The and health system reform is consistent with its com- major shortcoming of Bank strategy has been inad- parative advantage in the sector, and has focused equate focus on how to effectively improve HNP BOX 2.1. BANK HNP POLICY STATEMENTS IN THE PAST DECADE he Bank's sored sector studies on health programs and also incorporated 1987 study health financing and essential clinical preliminary findings Financing raised health financing services. Third, diversity from the OED review Health Services in issues in policy dialogue, and competition in the regarding overall port- Developing Countries and occasionally condi- provision of health folio performance. As a placed health financing tioned its loans on making services and insurance supplement to the HNP at the center of its health financing reforms. should be promoted. The strategy, in 1999 the policy dialogue with The World Develop- Bank's 1993 publication Bank released a Popu- borrowers. Even in the ment Report 1993: Disease Control Priori- lation and Reproduc- absence of severe Investing in Health ties in Developing Coun- tive Health Strategy budget constraints, the looked at the role of tries was an important paper (a nutrition strat- study argued, new government and the contribution to interna- egy paper is planned approaches to health market in health and tional discussions, and for fiscal 2000). The care financing were examined the most led to increased Bank paper describes how the required to improve appropriate ownership support for project lend- Bank is responding to both the efficiency and and financing arrange- ing for disease control. the integrated approach the equity of health ments to improve health In 1997 the Bank to population and care. The paper outcomes, reach the poor- released its Health, Nutri- reproductive health proposed four reforms: est, and contain costs. tion, and Population agreed upon at the implement user charges The report stressed a Sector Strategy paper, 1994 Cairo conference. at government health three-pronged approach. which will guide the These strategy papers facilities; introduce First, governments Bank's work in the sector articulate an appropri- insurance or other risk should foster an environ- over the next decade. The ate vision for Bank coverage; use nongov- ment that enables house- strategy identifies three engagement in the ernmental resources holds to improve health. objectives: (i) improve the sector. The challenge more effectively; and Second, government health, nutrition, and will be to make strate- introduce decentralized health spending should population outcomes of gic decisions on areas planning, budgeting, be made more effective the poor; (ii) enhance the of emphasis, and to and purchasing for by reducing expenditures performance of health allocate lending, staff government health on the less cost-effective care systems; and (iii) time, and administra- services. At the country interventions and secure sustainable health tive resources accord- level, the Bank spon- expanding basic public care financing. The paper ingly. 7 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector efficiency, effectiveness, and equity-including the in- stitutional and political factors that influence cost- effectiveness and the pace and feasibility of system reforms. The institutional sophistication of the Bank's approaches has increased, but so has the complexity of the challenges addressed. Given the lack of agreement on the right configuration of an effective health system and wide variation in country context, the Bank's increased focus on knowledge management is appropri- ate, but further emphasis on flexible instruments and analytical and advisory services may be needed. 8 Proj ect Performance and Determinants he percentage of HNP projects rated as satisfactory by OED is below the Bank average, but performance has improved in recent years. Institutional development impact remains the weak point of the portfolio-less than a quarter of projects substantially achieve their insti- tutional objectives. Borrower performance and country context are the two most influential factors in project performance. Of the factors the Bank can control, three stand out: assessment of borrower institutions and capacity, strong supervision, and attention to monitoring and evaluation. Although the performance of the Bank's HINP portfo- HNP projects were rated as likely to be sustainable, lio has improved recently, it has historically been below this figure rose to two-thirds in FY97/98. Improvements the Bank average. Many factors behind this record are in sustainability appear to flow from a combination of beyond the Bank's control. But on the operational level, a improved economic situations in many borrower coun- closer analysis of the record shows that weaknesses in the tries (prior to the recent Asia crisis), increased Bank design and supervision of Bank projects have contributed attention to the recurrent cost implications of invest- to disappointing performance.' ments, and greater emphasis by the Bank and borrow- ers on client ownership. Project Performance Trends Of the 107 HNP projects completed between FY75 and FIGURE 3.1. OUTCOME AND SUSTAINABILITY FY98, OED rated 64 percent satisfactory, compared with IMPROVING, BUT INSTITUTIONAL PERFORMANCE 79 percent for non-HNP projects (figure 3.1). Recent IS WEAK efforts by the Bank and sector staff to improve perfor- Percent mance appear to be showing results, however-79 per- 90 cent of projects completed in FY97/98 satisfactorily 80 - Outcome (% satisfactory) achieved their development objectives, close to the Bank 70 - average of 77 percent. Few of the more recent sector 60 - reform projects have been completed, so the performance 50 - consequences of the current strategy remain uncertain. 40 Sustainability (% likely) But in mid-1999, the Bank's portfolio monitoring system 30 Institutional 'ft t o s 20 development listed a third of HNP projects as at risk, above the Bank 10 (% substantial) average of 24 percent, which suggests that recent im- 0 II I II provements should not be a cause for complacency. FY80-84 FY85-89 FY90-94 FY95-96 FY97-98 Although only half (52 percent) of all completed Source: OED data. 9 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Yet high rates of completion of physical objectives levels of female literacy create a formidable challenge to disguise difficulties the Bank has had in achieving increasing the use of family planning in rural Mali, and policy and institutional change in HNP. OED has the complexities of the Brazilian political system compli- consistently rated institutional development as substan- cate health reform efforts. Although national institutions tial in only about a quarter of completed HNP projects; evolve slowly, the results suggest that the Bank needs to for FY97/98 this is well below the Bank average of 38 understand the institutional context, and make choices percent. Institutional impact thus remains the Achilles' regarding appropriate instruments and objectives. This heel of the HNP portfolio. includes choosing not to lend when governance is particu- larly bad-as when the Bank stopped lending to Nigeria Borrower Performance and Country Context in the mid-1990s after several failed HNP projects. All Bank projects are financed through loans to governments, and responsibility for implementation Bank Performance rests primarily with those governments. Not surpris- Although borrower performance and country context ingly, evidence from the case studies, Implementation are the most important determinants of project out- Completion Reports (ICRs), and an econometric model come, a number of important factors-including of determinants of project performance' indicates that project design and supervision-are under Bank con- borrower performance is the most important determi- trol. According to OED assessments of completed nant of project outcome.4 Yet borrower performance is projects, Bank performance in HNP project appraisal not exogenous: it is also influenced by the Bank's and supervision has improved from only about 60 assessment (and encouragement) of borrower project percent satisfactory in the early 1990s (10 to 15 percent ownership, the fit between project design and borrower below the Bank averages) to over 75 percent satisfac- capacity, and the effectiveness of supervision. OED tory for projects exiting in FY97/98-equal to the Bank ratings of borrower performance in project preparation average for supervision, and above the Bank average improved from 63 percent satisfactory in the early (66 percent) for project appraisal. This still falls short, 1990s to 92 percent satisfactory in FY97/98 (roughly however, of the Strategic Compact goal of 100 percent comparable to Bank aver- satisfactory Bank performance by 2001. OED's review Experience ages). In contrast, borrower found that the quality of project preparation and illustrates the implementation perfor- supervision in the HNP sector has improved over the mance fell from 71 percent past decade in a number of respects, but weaknesses importance of to 58 percent satisfactory remain in several key areas. political and social over the same period. Al- institutions in the though the reasons for the Institutional Analysis achievement of decline in borrower imple- The Bank confronts a number of inherently difficult mentation performance are institutional challenges in the HNP sector, including project and policy not entirely clear, they many that have not been adequately resolved in devel- objectives, could be partly the result of oped countries. In addition, ministries of health are often inadequate assessment of administratively weak, particularly in areas such as implementation capacity and shortcomings in Bank financial management. Yet these difficulties alone do not supervision. explain the Bank's disappointing performance in institu- Country context is the second most important influ- tional development. Other factors are at work: ence on performance, particularly the overall quality of borrower institutions.s In countries where high levels of * The Bank often does not adequately assess corruption prevail or legal mechanisms for contract borrower capacity to implement planned project enforcement are weak, resources from Bank investments activities. This was the factor most commonly can be diverted, or the project must put in place cited in ICRs as contributing to poor project cumbersome procedures to increase accountability. Coun- performance, including 69 percent of projects try experience also illustrates the importance of political rated unsatisfactory (see Annex B). and social institutions in the achievement of project and * In seeking to promote institutional change and policy objectives. For example, cultural beliefs and low build borrower capacity, the Bank often does not 10 Project Performance and Determinants adequately analyze the constraints underlying analysis is therefore likely to yield better outcomes. current performance. Although institutional Although some institutional issues require sophisticated analysis has improved since the mid-1990s, it analysis, the criteria used by OED merely asked remains weak, particularly in relation to the whether project designers appeared to have thought much more daunting systemic reforms the Bank through relevant institutional issues (Stout and others is now promoting. 1997). Experience from * Weak analysis contributes to a lack of clarity in the HNP projects that success- We know little articulation of institutional development objec- fully achieved institutional tives, including whether the instruments chosen are objectives could be more about what the the best to bring about change. Bank projects have widely replicated (box 3.1). Bank has "bought" traditionally addressed capacity constraints This suggests that the with its through the provision of training and additional Bank's institutional devel- investments. resources, although a growing number of projects opment performance in (particularly in Latin America and the Caribbean HNP could be raised to a and Europe and Central Asia) are focusing on level that equals or exceeds current Bank averages if improving incentives or regulations. staff and management commitment to achieving insti- * The absence, until recently, of appropriate indi- tutional goals were strengthened and standards and cators for institutional goals has contributed to tools for institutional analysis were developed, and the tendency to assert that "capacity was built" staff trained in their use. because training or technical assistance was provided, reducing the focus on the ultimate Monitoring and Evaluation of HNP Outcomes objectives. Although nearly all World Bank project design docu- ments assert that the project will improve HNP out- The econometric model found that the quality of comes, system performance, or health service access institutional analysis during project preparation has a for the poor, few ICRs provide evidence that these significant influence on project outcome-improved development objectives were actually achieved. Not BOX 3.1. LESSONS FROM SUCCESSFUL INSTITUTIONAL DEVELOPMENT f the 73 HNP stakeholders regard- stakeholders. Design- substantially proj ects ing priorities and ers then developed achieved institu- completed approaches, and, if realistic strategies to tional goals were between FY91 and necessary, developing address these significantly modi- FY98, only 13 were strategies to antici- constraints, including fied during imple- rated by OED as pate and soften attention to the mentation. having substantially resistance. proper sequencing of * A governance achieved their institu- * Project designs based interventions. and macroeco- tional objectives. These on a solid analysis of * Flexible project nomic context that projects shared several the underlying implementation, with was supportive of characteristics: constraints to regular reviews of institutional and improved perfor- progress toward insti- organizational * Consistent commit- mance-through tutional objectives, development. If ment to achieving some combination of and proactive atten- this was not institutional objec- sector work, evalua- tion to problems by present, the above tives, including the tion of previous Bank staff and factors were promotion of experience, and borrowers. About half particularly impor- consensus among dialogue with key the projects that tant. 11 Investing in Health: Development Effectiveness in the Health, Nutrition. and Population Sector only do we know little about what the Bank has problem for HNP. Project designs often give primary "bought" with its investments, but when progress responsibility for implementing M&E to the borrower, toward objectives is not measured, they are less likely but do not adequately consider how data will be to be achieved.' collected or analyzed, the incentives and capacity of Most HNP project designs currently identify key borrowers to do so, or the appropriate balance between performance indicators, and intentions for monitoring the use of internal monitoring systems and external and evaluation (M&E) have improved in recent years. (including rapid assessment) evaluations. But the overwhelming problem noted in ICRs is that the A number of projects have sought to improve data required were not collected or analyzed, at least borrower M&E capacity-some successfully. But the not in a manner that allowed assessment of impact. Bank has tended to place excessive emphasis on providing Two-thirds of unsatisfactory projects reported that the equipment and training, and has underestimated the time Bank gave inadequate attention to M&E during project required to achieve agreement among various bureau- design and implementation. cratic stakeholders on indicators, to clarify roles and Both OED and the Quality Assurance Group tesponsibilities for data collection and analysis, and to (QAG) have found that monitoring and evaluation is strengthen incentives to use evaluative information in weak throughout the Bank, but the gap between M&E decisionmaking.' The challenges of M&E are more intentions and actual implementation is a particular difficult for system reform than for targeted interventions, BOX 3.2. LESSONS FROM SUCCESSFUL M&E Ithough the early treatment, and impact. Ghana, govern- Bank's record case management, * In Mali, the Health ment and donors in M&E is this effort contributed and Rural Water (including the disappointing, a num- to a decline in Supply project (1991- Bank) agreed- ber of projects have malaria incidence 98) eventually helped after lengthy nego- demonstrated successful and fatality rates. to establish a nation- tiations-on a lim- approaches to assessing * The Tamil Nadu wide health ited number of the effectiveness of Integrated Nutrition information system, national indicators project interventions, project in India estab- although data were that will serve as strengthening borrower lished a community- not available until benchmarks for health information and based system for the final years of the joint annual disease surveillance regularly monitoring project. This illus- reviews of sector systems, or monitoring the growth and trates the importance performance. progress toward sector- weight of children of balancing long- Remaining chal- wide objectives. under age 3, with term efforts to lenges include a The Brazil Amazon targeted feeding (and strengthen borrower better linking of Basin Malaria education in feeding monitoring capacity system perfor- Control project practices for mothers) with provisions for mance indicators to helped to train for children found to periodic external HNP outcomes and malaria field- be malnourished. The qualitative or quanti- ensuring that workers and project significantly tative assessments, in- national indicators strengthen disease reduced severe cluding rapid assess- create incentives surveillance malnutrition in the ments (WHO 1993). for performance at systems. Together target group. The * In the current lower levels of the with a shift in strat- monitoring system sector-wide health system (Adams egy from eradica- both contributed to reform programs in 1998). tion to control, and documented the Bangladesh and 12 Project Performance and Determinants FIGURE 3.2. SUPERVISION INTENSITY AND PROJECT SIZE clearly necessary. But discussions with staff confirm a Supervision staffweeks Average loan/credit widespread feeling that supervision budgets have de- per project amount (US$ million) cined in real terms in recent years, and that pressures 35 - - 80 to "do more with less" are 30 - 70 having a negative impact 25 - Supervision intensity - 60 on quality. Staff cite reduc- The sector faces a 20- 50 tions in the number of tech- serious problem of 15- 40 nical specialists included on multiplying tasks 10 - supervision missions and in and mandates. - 20 the time and budget avail- 5 Average loan/credit amount 10 able for priorities such as 1975 77 79 81 83 85 87 89 91 93 9596 stakeholder consultation or advisory and analytical Fiscal year services. Senior management has asserted that overall Source: OED data. HNP supervision budgets have been constant, and, at the aggregate level, average supervision budgets in the but lessons from HNP projects with successful M&E are Bank's Human Development Network averaged close broadly applicable (see box 3.2). to $52,000 per project in FY94-98. OED was unable to resolve the apparent conflict Quality of Supervision between staff perceptions and aggregate trends. But it is Although responsibility for project implementation clear that the sector faces a serious problem of rests with borrowers, the quality of Bank supervision has an important influence on project outcome (see Annex B). ICRs for 69 percent of unsatisfactory projects reported that supervision was inadequate, compared with only a third of satisfactory projects. The case studies and ICRs suggest that effective supervision requires a team with an appropriate skill mix; continu- ity among team members; strong managerial skills and client orientation; proactive recognition and solution of problems; and an appropriate balance among high- level policy dialogue, attention to implementation issues, field supervision, and consensus-building among stakeholders. Recent QAG supervision reviews found that supervision ratings for HNP project imple- mentation performance tended to be excessively opti- mistic, and that few of the HNP projects reviewed based supervision assessments on progress toward the achievement of development objectives. The HNP sector is also slow to restructure problem projects. Yet an apparent factor behind recent increases in HNP project outcome ratings was the restructuring of a number of problem projects that had languished during the early 1990s. The impact of resource constraints on the quality of supervision has been a source of growing tension between Bank HNP staff and Bank management. QAG and OED analyses found no simple link between supervision quality and quantity (measured by total staffweeks), although a minimum level of resources is Mother nursing baby in India. 13 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector FIGURE 3.3. GREATER COMPLEXITY IN complexity "right," combining assessment of the ca- DIFFICULT SETTINGS pacity of implementing organizations with greater effort to prioritize and sequence interventions, and to reduce the burden of Bank procedures, particularly complex when capacity is limited. 4.2 4.1- Economic Analysis 4.0 - Although the econometric model found that institutional analysis has a greater influence on project outcome, economic analysis remains an important foundation for Low effective project design and policy advice. The Bank's 3.6- LHNP sector leadership has established guidelines for 3.5- IMR economic analysis that call for analysis of alternative Low Hinterventions, the justification for public sector involve- Institutional Quality n=75 countries ment, cost-effectiveness or cost-benefit analysis, assess- ment of recurrent cost impact, risk analysis, and assess- Source: OED data and World Bank 1997d. ment of poverty impact, among other measures (Preker, Brenzel, and Ratta 1997). OED's assessment of HNP project appraisal docu- multiplying tasks and mandates, with inadequate guid- ments found that economic analysis has improved in the ance from management regarding priorities or selectiv- past several years, but that it remains below the standards ity, particularly with growing project size and the established by the sector. This is consistent with QAG increased challenges of promoting sector reforms, findings on HNP quality at entry. Recently designed institutional development, and stakeholder consulta- projects are more likely to assess the cost-effectiveness of tion (see figure 3.2). the intervention, possible alternatives, or reasons for public sector involvement. Risk analysis has improved Complexity of Project Designs somewhat, but is still discussed very generally, with As discussed in Chapter 1, health systems include a wide limited sensitivity analysis or consideration of exit strate- array of public and private stakeholders, and achieving gies. Project documents commonly suggest that borrower improvements in HNP outcomes may require multiple institutional capacity and commitment to project goals interventions. If project designs are too simple-for are the major risks, to be addressed by training and example, financing only infrastructure and training, or technical assistance. Assessment of health care markets ignoring key ministries or agents-they risk not meeting could be strengthened to better assess the relative roles of their development objectives. public and private providers and consumer demand Notably lacking in Although the econometric (Hammer 1996). model found no linear corre- lation between complexity Consumer Responsiveness is an adequate and project outcome, about The majority of HNP projects include objectives that can assessment of half the ICRs for projects only be met through client-responsive services (Heaver demand for health rated unsatisfactory con- 1988). Yet notably lacking in most Bank analysis is an services. cluded that the project design adequate assessment of demand for health services. The was too complex. Bank has increased its attention to health consumers in the The number of project past few years, but a minority of project design and components is an important aspect of complexity, completion documents provide basic data on current although the number of agencies involved may matter levels of service utilization (in both the public and private more. OED found that HNP project designs tend to be sectors) or consumer satisfaction. Overall, only 40 per- more complex in countries with low institutional cent of all project design documents provided evidence on capacity, and where the pace of change in infant consumer demand, and only 2 percent estimated con- mortality is slow (figure 3.3).8 The challenge is to get sumer response to the proposed interventions. Although 14 Project Performance and Determinants significant improvements in health outcomes will be achieved. Design documents, however, seldom present a coherent analysis of how project interventions will trans- late into improved health outcomes for the poor. Conse- quently, the Bank usually presents overly optimistic projections of health impact and, more important, often rI - does not consider whether alternative approaches would effect a greater impact on the disease burden of the poor.' r ~ All four country case studies and the portfolio review found that Bank investments and policy advice tend to focus on the medical care system, while greater aggregate Phealth improvements may be achieved through health education and behavior Mother and baby, Pernambuco Mother and Child Hospital change initiatives, or through in Recife, Brazil. intersectoral interventions such as water and sanitation ill-health among beneficiary surveys and consultations have become more (Lerer and others 1998). Inter- the poor, selecting common, only 4 of 224 projects documented the presence sectoral interventions can be interventions that of beneficiary decisionmaking power in project design. difficult to implement, how- are likely to ever, and incentives for inter- Sustainability sectoral coordination are achieve the Almost 40 percent of ICRs reported that shortages of weak within the Bank and maximum impact recurrent expenditure adversely affected project imple- most borrower governments. on their overall mentation and prospects for sustainability. Despite They must be chosen care- disease burden. recent improvements in project sustainability ratings, fully, and adequate time must project appraisal documents often do not realistically be allocated for supervision. estimate the recurrent cost burden of Bank project In addition, prevention is not always more cost-effective investments. The Bank, therefore, sometimes contrib- than curative approaches, as demonstrated by the Ama- uted to the recurrent cost problems it emphasized in its zon Basin Malaria Control project (see box 3.2). policy dialogue. The realism of recurrent cost assess- ments of investments has improved in the past five Ownership and Stakeholder Analysis years, but the assessments still tend to be optimistic. Recent project documents are more likely to include The move toward sector-wide programs in low-income evidence of borrower ownership in project identification countries increases the likelihood that the recurrent and design, but many still simply assert that the project is costs of Bank lending will be considered in relation to consistent with borrower priorities, or discuss central other donor or government investments (Peters and government commitment, but not that of local govern- Chao 1998). The Bank has also shown greater flexibil- ment or the beneficiaries. Few Implementation Comple- ity in allowing loans to be used for recurrent costs, such tion Reports or design documents discuss or elaborate on as drugs, but caution remains advisable to prevent Bank efforts to stimulate ownership. Yet projects that can Bank loans from being treated as grants. point to specific evidence of borrower ownership are more likely to achieve their objectives (Johnson and Wasty Linking Inputs to HNP Outcomes for the Poor 1993). Greater participation and work with a range of Although the Bank usually focuses on poor regions, or stakeholders is characteristic of more recent projects, and diseases that most affect the poor, it has been weaker in some task teams undertake stakeholder analysis, but do analyzing the factors that lead to ill-health among the not document findings to avoid jeopardizing reformers. poor and in selecting interventions that are likely to Projects tend to be more successful in achieving achieve the maximum impact on their overall disease their institutional development goals when they are burden. Project design documents typically describe the strongly supported by relevant borrower agencies and disease burden, list project activities, and then assert that key stakeholders. Differences of policy opinion across 15 Investing in Health Developinent Effectiveness in the Health, Nutrition, and Population Sector government can seriously undermine institutional de- close attention to the design and nurturing of participa- velopment goals. Such differences are common in tory structures. Yet even in communities where local sector reform efforts (Reich 1997). committees managed the local health center, women The challenges of health reform require strategic were usually underrepresented, and community mem- and flexible approaches to support the development of bers did not always think the committees represented the intellectual consensus and broad-based coalitions their interests. The Bank is not always well-placed to necessary for change. System reform is difficult and directly foster local participation or strengthen mecha- time-consuming, and stakeholders outside ministries of nisms for consumer voice, but it can emphasize these health can determine whether reforms succeed or fail. issues in policy dialogue or encourage direct or parallel In practice, the Bank has often focused primarily on financing for enhanced participation through interna- dialogue with government officials, particularly in tional partners or local nongovernmental organizations ministries of health, without taking advantage of its (NGOs).12 convening role to build consensus among stakeholders. Some recent efforts give Coordination Weak coordination greater emphasis to these Because health is a popular sector with many donors, priorities. In addition, the ministries of health-particularly in aid-dependent and turf baffles Bank has frequently failed countries-frequently have been undermined by a among ministries to develop sufficient under- proliferation of projects and vertical programs. In the and ministerial standing to anticipate re- 1980s, the Bank began to encourage other donors to subunits are sponses to reforms, includ- finance projects jointly, and in the 1990s it has taken ing which measures are the lead in building consensus among donors for sector- c0mm0n in likely to be adopted, which wide approaches, and in helping governments develop developed and may be resisted, and pos- the policy framework for sector programs (Peters and developing sible changes in content that Chao 1998). Despite these efforts, some partners ex- countries alike. may be made in the course press concern that inadequate field presence can hinder of policy debate and imple- the Bank's ability to coordinate effectively. mentation.0 Experienced In addition to the challenge of donor coordination, task managers are well aware of the political dimen- weak coordination and turf battles among ministries sions of reform, but staff have limited skills and tools and ministerial subunits are common in developed and for stakeholder and political analysis." developing countries alike. Weak linkages among ministries of finance and planning and sectoral minis- Participation tries, in particular, can hinder both internal planning Developing mechanisms to increase the voice of com- and budgeting and the external coordination of part- munities in the management of health services is key to ners (van de Walle and Johnston 1996). Such problems improving service quality and consumer responsive- are not easily resolved, but the case studies suggest that ness. Despite the interest in community participation the Bank can facilitate improved relationships among and mobilization during the primary health care focus government units, particularly through its convening of the 1980s, subsequent Bank project documents tend power and its relationships with both ministries of to confuse participation with charging consumers for finance and sectoral ministries." But improving coordi- services. Community-based management and financing nation within governments or among donors can be of health services-as advocated in the 1988 Bamako difficult and time-consuming. The Bank and its part- Initiative-is one means to improve service quality and ners need to establish appropriate strategies, priorities, client responsiveness. But community mobilization is and divisions of responsibilities in this area. time-intensive and context-specific. Even among projects that plan to encourage local participation, descriptions of local and traditional participatory Institutional Factors Influencing Performance structures are rare. Few observers inside or outside the Bank dispute the The Mali case study found that successfully estab- importance of defining and monitoring objectives, careful lishing community-managed health services requires institutional assessment, or the political nature of health 16 Project Performance and Determinants reform. The vast majority of Bank HNP staff are * Low priority is given to M&E by Bank manage- knowledgeable and dedicated. What, then, are the under- ment, and there is little incentive for staff to lying sources of these difficulties, and why do they persist? become involved. Many staff report that their We focus here on three major areas: quality assurance, managers rarely express interest in reviewing M&E, and learning and intellectual leadership.14 development progress. * The Bank's core business processes and incen- Quality Assurance tives remain focused on lending money rather In recent years, the Bank's HNP sector has focused than on achieving impact. Until incentives are attention on issues of quality in the portfolio by adjusted, progress will remain sporadic. The establishing an HNP quality committee and lead Comprehensive Development Framework pilots quality adviser and by sponsoring training programs currently under way offer an opportunity to shift on quality at entry and supervision. Yet routine quality Bank processes and procedures toward achieving assurance and monitoring mechanisms remain weak, development results. particularly under the current matrix management * Forums for staff to discuss and review progress system. Reasons appear to include: toward development objectives, or to recognize and reward evidence of HNP development im- * Lack of clear lines of responsibility and account- pact, are lacking. Staff still perceived that ability for quality. Nominal responsibility rests rewards were linked primarily to project ap- with the HNP Sector Board and lead technical proval and disbursement. specialists, but budget and staff allocation decisions * In most client countries, health monitoring sys- are made by the country departments. The HNP tems are either weak or rarely used in policy Board's recent decision to strengthen its role in decisionmaking, and national or local budgets quality assurance, including the establishment of a are seldom linked to monitoring data. Conse- benchmarking system to regularly monitor portfo- quently, there is little demand for information, lio performance, will be an important step. and few incentives for its collection. * Regional technical advisers are overburdened, and * Few Bank client countries have the "information are often unable to give sustained time and infrastructure" necessary to routinely and reliably attention to reviewing the quality of project design, measure health status outcomes through vital supervision, and economic and sector work. registration systems and up-to-date censuses. De- * Managerial accountability mechanisms such as spite significant Bank and donor investments in QAG are not balanced by mechanisms to pro- household surveys, the Bank and its partners have vide early and collegial support to task teams given little attention to these routine systems. during project design and supervision, or to build quality assurance into the overall project cycle. Learning and Intellectual Leadership The project design peer review process is not The Bank's Human Development Network and HNP functioning effectively. Sector Board are ahead of some other parts of the Bank * Staff are reluctant to restructure projects because in attempting to respond to President Wolfensohn's call of continued high transaction costs, and a per- for a "Knowledge Bank." ception among HNP staff that management The Network has estab- Regional research views restructuring as an admission of failure. lished systems for knowl- is difficult to fund. Restructuring occurs, but often not until a new edge management, and fo- task manager has been appointed. cused on strengthening staff skills in key areas. The Bank's ability to provide Monitoring and Evaluation intellectual leadership, however-both internationally M&E has been weak throughout the Bank, despite and at the country level-is being compromised by repeated exhortations from OED (OED 1994). In several trends: addition to the methodological issues discussed earlier, several factors have constrained Bank and borrower * Few staff have explicit training in organizational, M&E performance: institutional, or stakeholder analysis, and they 17 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector have few tools to undertake such analysis during * Staff now rely heavily on external grant facilities project design. Developing operationally relevant to fund project preparation and analytical work, training and tools is an important challenge, as is but application for such funds is time-consuming balancing staff analytical skills with the need for and the grants carry a number of restrictions in practical implementation experience." the use of their funding. Since all budgets now * Resources for economic and sector work (ESW) reside in country departments, regional research have declined in the past five years relative to the is difficult to fund, although many issues would lending portfolio. This has placed the Bank in the be more effectively and efficiently addressed position of embarking on ambitious sector re- through regional studies, particularly in Europe forms in many countries without first establish- and Central Asia (ECA) and Africa. ing a strong empirical foundation to guide the * HNP staff have not made wide use of new process. Reviews by QAG and DEC (Develop- lending instruments such as Adaptable Program ment Economics and Chief Economist) have Loans (APLs) and Learning and Innovation raised concerns regarding the uneven quality and Loans (LILs). Despite growing interest, staff impact of sector work, and inconsistent links report that while such instruments often require between these efforts and the lending program. extra staff time, additional resources are not * Borrowers are often reluctant to use loan money for provided, and administrative requirements have technical assistance and the Bank has few grant reduced their intended flexibility. In addition, resources available outside of project preparation some borrowers are reluctant to accept small funds. Staff (particularly in IBRD countries) report loans, or are concerned that flexible instruments that this limits their ability to engage in policy could give the Bank discretion to reduce future dialogue and stakeholder consultation. funding. 18 Development Effectiveness of HNP Investments and Policy Dialogue ank investments have provided valuable support in expanding and strengthening the building blocks of borrower health systems, including facilities and staff. The impact of these investments, however, has often been reduced by continued problems with service quality and underfinancing of basic HNP services. The Bank tended to conceptualize projects as injections of capital and technical expertise, and provided policy advice based on normative prescriptions, without fully appreciating the incentives, institutions, and external constraints governing sector performance. Recent approaches are thus on areas where improvement is needed-the more sophisticated, with greater attention to the institu- Bank's performance in HNP must be considered in the tional dimensions of reform, but the challenges being context of the challenges posed by any effort to addressed are also more difficult. improve performance in the social sectors. Completed projects and the OED country case studies are rich sources of information about the extent Strengthening the HNP Service Delivery Structure to which Bank-sponsored HNP projects have achieved Expanding Access to Health Services HNP development objectives. Throughout the 1970s Expanding physical access to health services has been and 1980s, a major goal of Bank HNP projects was to a major goal of Bank HNP lending for the past two support the expansion and strengthening of health decades.' Even recent health reform projects typically service delivery systems. This remains an important devote substantial portions of expenditure to construc- goal today in many countries. In recent years the Bank tion, and geographical access remains a challenge in has turned more attention to promoting enhanced some poor countries. Client governments generally health system performance, and it has gained experi- place high priority on infrastructure development when ence with measures to improve HNP service delivery requesting Bank assistance, and these investments are and to secure sustainable health system financing. usually appreciated by consumers and providers. The discussion here applies broadly to the Bank's OED found that Bank projects usually are success- work in HNP. Boxes 4.1 and 4.2 provide further detail ful in expanding geographical access to government on the Bank's experience in nutrition and reproductive health and family planning services, but often experi- health, respectively. Although the discussion focuses ence delays (two years on average) or problems with primarily on shortcomings in Bank effectiveness-and uneven quality. Ministries of health often are inexperi- 19 Investing in Health Development Effectiveness in the Health, Nutrition, and Population Sector enced in construction, and ministries of construction, constructed with Bank support in Brazil, India, and which frequently implement civil works, typically have elsewhere were often underutilized because of poor limited experience with health facilities, and are service quality. Second, if new or upgraded facilities vulnerable to inefficiencies and graft. Careful attention are not located near catchment populations, or are to design, site selection, and supervision-and the placed near competing facilities (such as mission funding of architectural consultants-can improve the hospitals), their net impact on utilization is reduced (as quality and cost-effectiveness of civil works. In the in Zimbabwe). Inadequate planning or political influ- longer term, enhancing the capacity of ministries of ences can lead to inappropriate site selection, while health to evaluate facility design and supervise con- establishing transparent technical criteria can reduce struction can enhance the quality of health infrastruc- distorting political influences. If poor consumers prefer ture investments, although in some contexts, third- private or NGO services, then encouraging improved party contracting may be preferable. access to, and quality of, private service delivery or Increased physical access to government health health insurance may be appropriate. Bank efforts in services often does not lead to increased use of services, this area are recent, however, and generally confined to however, at least not at the levels anticipated during Latin America and the Caribbean (LAC) and Eastern project design. Several factors explain why. First, and Central Europe (ECA). service quality-including the availability of trained providers, provider attitudes, and drug availability-is Strengthening Referral Systems a key determinant of consumer demand. Family plan- A functioning referral system helps to mitigate the risks ning and maternal and child health (MCH) clinics associated with ill-health and can increase service BOX 4.1. BANK EXPERIENCE IN NUTRITION utritional tion. Changing from food a consequence of the the attention given to status is an entitlements to more tar- Bank's relatively small nutrition in policy dia- important geted nutrition programs nutrition staff; the diffi- logue, and the Bank's determinant of health can be politically difficult, culty of integrating the success in introducing status, and several however. The OED case intersectoral aspects of targeting and nutritional decades of experience studies demonstrate a food policy and a nutrition impact monitoring into have demonstrated that wide variation in Bank program into standard government programs. targeted nutrition nutrition activities, from a health investments; and, This suggests that a programs can measur- relative neglect of nutri- sometimes, concern that strengthened strategic ably improve proteins- tion issues in Brazil and adding a nutrition compo- approach to nutrition energy malnutrition and Mali, to significant contri- nent will create excessive could help increase the micronutrient deficien- butions in India and Zim- project complexity. Bank's impact. The cies for children and babwe (see Annex A). Where the Bank has HNP sector is in the mothers. But nutrition The Bank's nutrition given priority to nutrition first stages of develop- programs in many efforts have been con- issues, it has helped to ing a nutrition strategy, borrower countries are strained both by inconsis- raise the profile of nutri- beginning in 1999 with hindered by a number tent attention to nutrition tion within ministries of a comprehensive assess- of factors, including an across portfolios at the health and ministries of fi- ment of lessons of expe- overreliance on country level and by the nance (as in Zimbabwe). rience from past nutri- expensive and poorly uncertainty of Bank staff Nutrition subcomponents tion efforts. This targeted feeding and governments as to of projects are often co-fi- evaluation is being un- programs and weak how to intervene effec- nanced by other donors, dertaken jointly with mechanisms for tively and efficiently to but the source of financing UNICEE intersectoral coordina- improve nutrition. This is seems to matter less than 20 Development Effectiveness of HNP Investments and Policy Dialogue efficiency. Reducing maternal mortality, for example, depends on the ability of providers at lower levels to identify and refer high-risk pregnancies to higher-level - - facilities. Improving service efficiency, in contrast, W calls for treating basic ailments in primary care - facilities. Strengthening a referral system requires attention to several factors, including service quality at each level, provider training in referral protocols, and consumer demand patterns. Bank-sponsored projects I use several approaches to strengthen referral systems, including investments in facilities, staff training, and equipment at primary and first-level referral facilities; provision of transportation and communications equip- ment; and policy advice to encourage progressively Nurse/healhworker preparing to inoculate child, higher fees at each level, unless a patient is referred. Harare, Zimbabwe. The case studies and other project experience memorization, inadequate hands-on practice, and diffi- demonstrate that Bank investments in facilities, equip- culties in attracting and retaining trainers, often be- ment, transport, and communications equipment have cause of inadequate pay and poor career prospects. enhanced referral effectiveness and efficiency in many Training systems do not change overnight, but the Bank countries. The major constraints, not surprisingly, tend needs to address such issues during design and imple- to be on the "software" side, including inadequate mentation, and to give greater attention to assessing training in referral at the primary level (India), short- the quality and impact of training. ages of qualified physicians at the first or second Bank training investments also have been consis- referral levels (Zimbabwe, Mali), and continued con- tently undermined by inadequate attention to the health sumer preferences for higher-level care, which leads to labor market and performance incentives for providers continued bypassing (Brazil). Changing price signals in both the public and private sectors. Each of the can enhance referral efficiency, but only if the service country studies concluded that health work force issues quality is adequate at the primary levels, yet the Bank are perhaps the most pressing challenge facing the has tended to emphasize changes in fee structures respective health systems. Where a large private sector before a focus on service quality. The Bank is only exists, civil service pay often is not competitive, beginning to address the difficult issues of private particularly for doctors and sector referral and public/private linkages. staff with technical or spe- Inadequate cialized training, which attention to the Health Staffing and the Health Work Force contributes to attrition or The Bank has invested heavily in the training of health moonlighting. Providers of- health labor market staff and health managers, both through direct financ- ten have few incentives to and performance ing of training and by helping to establish or upgrade work in rural areas. In the incentives for borrower training programs. The goals of training public sector, low morale pr0viders in both often include improving provider technical skills; in- and lack of client-orienta- provi inb creasing the number of trained providers, particularly tion often results from dis- the public and in underserved areas; and enhancing overall provider satisfaction with terms of private sectors. motivation and performance. Bank training invest- service, inadequate supervi- ments are usually appreciated by providers, and in sion, or the limited ability of consumers to demand many countries have contributed to improvements in improved service. Private providers often overprescribe technical proficiency, the introduction of new ap- and have few incentives to provide preventive care. In proaches, and enhanced quality of care. But the quality such contexts, training investments often produce little and relevance of training-which is often provided change in the number of providers serving poor through government institutions-can vary consider- populations, and have only a marginal impact on the ably. Typical problems include excessive emphasis on quality of care. The Bank has neglected these issues in 21 Investing in Health Development Effectiveness in the Health, Nutrition, and Population Sector its projects and policy dialogue, however, at least until national competitive bidding (ICB) for major drug very recently (see below). procurements can produce substantial cost savings for governments, and generally lead to improved drug Essential Drugs and Equipment supplies. Yet the rapid design and implementation of Drug availability is an important determinant of these programs, often without adequate training for service utilization rates. The Bank has sought improve- local staff in Bank procurement procedures and ICB, ment through several means, including direct procure- often led to bottlenecks and delays in the drug procure- ment of drugs, establishment of revolving drug funds at ment and distribution process. Bank clients frequently both national and community levels, and pharmaceuti- complain of the complexity and rigidity of Bank cal reform efforts. Drug procurement was a common procurement procedures, particularly for the procure- sub-component of Bank health investment projects in ment of medical equipment and pharmaceuticals. the 1980s, but the onset of the HIV/AIDS epidemic led Procurement issues often take up a substantial portion the Bank to finance several large drug procurement of the time of Bank HNP specialists, time that could be projects, primarily to finance the purchase of drugs to spent on policy dialogue or implementation issues. combat sexually transmitted infections (as in Kenya, Procurement delays can also appear when the Bank Uganda, and Zimbabwe). Bank requirements for inter- rejects improperly prepared bids, which may stem from BOX 4.2. BANK SUPPORT FOR POPULATION AND REPRODUCTIVE HEALTH ver the past reduction. report, including a prevalence is how to 25 years, the Bank-sponsored relative neglect, until reconcile the emphasis Bank's repro- population projects and recently, of private and on non-barrier contra- ductive health focus policy dialogue have NGO channels for ceptive methods has shifted from an contributed to the devel- service delivery. advocated by family exclusive emphasis on opment of population Implementation of an planning programs fertility reduction to policy, supported the integrated reproductive with the prevention of an integrated approach expansion of family plan- health approach faces HIV and sexually (see box 2.1). Several ning service delivery, both bureaucratic and transmitted infections decades of interna- and encouraged integra- programmatic chal- (STIs). Improving tional experience have tion of family planning lenges. Many developing reproductive health shown that while with health services (as countries-with donor may require strength- socioeconomic in Bangladesh and support-initially estab- ening overall health factors-particularly Zimbabwe). The Bank's lished organizations system performance income and female emphasis until the early independent of ministries (for example, improv- education-are key 1990s was primarily on of health to deliver ing referral systems to determinants of the expansion of existing family planning services, reduce maternal demand for family government programs; and later established mortality). Yet popula- planning services, the thus the impact varied additional organizations tion and reproductive provision of quality considerably depending to promote HIV preven- health specialists inside and client-responsive on the quality and client- tion. Bangladesh is and outside the Bank reproductive health responsiveness of these attempting to reintegrate have expressed concern services, together with programs (OED 1992). these programs as part of that the current empha- effective information The weaknesses in Bank a sector-wide program, sis on health system programs, can contrib- population investments with support from the reform should not lead ute to increased are similar to those iden- Bank and other donors. to neglect of reproduc- contraceptive preva- tified for the overall An unresolved issue in tive health programs lence and fertility HNP portfolio in this countries with high HIV (PAI 1998). 22 Development Effectiveness of HNP Investments and Policy Dialogue corrupt practices, suggesting the importance of both Bank only recently began to finance and promote social training in procurement and ensuring adherence to marketing through the private or non-profit sectors. rules for transparency. Yet when the Bank has made behavior change a The Bank has also supported the establishment of priority in its interventions, the results have been encour- community-managed revolving drug funds to help allevi- aging. Some of the Bank's more recent HIV/AIDS projects ate the persistent shortage of essential drugs in local highlighted behavior change, including successful efforts clinics and to increase the participation of communities in to encourage NGOs to direct and manage educational the management and provision of health services. Experi- efforts in Brazil and India. The Bank's recently launched ence in Mali and elsewhere has demonstrated that the anti-tobacco campaign (with WHO and other partners) success of community initiatives depends on progress in emphasizes changes in taxation and other national national pharmaceutical reform, particularly in increas- policies. Since these efforts are relatively new, the Bank ing the availability of affordable essential drugs. Unless should give priority to evaluating the effectiveness and affordable drugs are available, low-income communities impact of the various health promotion approaches. cannot sustain the drug funds. Using Bank loans to capitalize national or regional revolving drug funds can Private Sector Quality and Equity be an effective way to improve drug availability, but only Until the early 1990s, the Bank paid relatively little if such investments are preceded by sufficient institutional attention to private (nonprofit, for-profit, and tradi- reforms to ensure efficiency and accountability. Commu- tional) providers of health or family planning services. nities also require ongoing training and support to Yet private spending often constitutes more than half of establish effective management systems, and oversight to all health spending, and ensure that prices remain equitable and revenues are used may represent a majority of Behavior change appropriately (UNICEF 1998). the health spending by the and intersectoral poor. The nature and extent Health Promotion and Behavior Change of the private sector varies health promotion As noted in Chapter 3, although behavior change and considerably among coun- efforts are intersectoral health promotion efforts are essential to tries and regions, however, essential to improved HNP outcomes, Bank-financed interventions as does the extent of govern- improved HNP have generally given inadequate emphasis to these ment financing or regula- priorities. Many of the Bank's basic health and family tion of private provision. outcomes. planning projects included information, education, and Qualified private practitio- communication (IEC) or health promotion subcompo- ners generally prefer urban areas, and focus on middle- nents designed to change consumer beliefs and behav- or upper-income consumers, or those with health iors. These initiatives have provided valuable equip- insurance. Although consumers (including the poor) ment and training, and often heightened the priority of often prefer private providers because service is per- health education. But Bank health promotion efforts ceived to be of better quality, clinical quality can vary frequently have been constrained by several factors. substantially in many countries (there has been a First, project designs have emphasized IEC without proliferation of clinicians with unknown or question- adequate attention to the broader policy and regulatory able clinical skills). Even qualified providers often feel changes-often outside the health sector-that are substantial pressure to prescribe drugs and have few frequently necessary for success. Second, the design of incentives to provide preventive services. IEC efforts has consistently emphasized the I (informa- Regulatory or quality assurance mechanisms for tion) aspect, with less attention to health education and private provision are weak or nonexistent in many counseling, which usually have a greater impact on developing countries, but professional associations behavior (Nutbeam and Harris 1998). Third, IEC may strongly resist a broadened government regulatory campaigns sponsored by the Bank have often been mandate. The Latin America Region is increasingly poorly executed-with little or no field testing of engaged in such issues, but few projects have been messages or materials, or targeting of specific groups- completed or evaluated. Recent Bank policy statements and implemented by units in health ministries with have called for a better balance of public and private little experience in marketing or behavior change. The roles in health (box 2.1). The challenge now is to build 23 Investing in Health )evelopment Effectiveness in the Health, Nutrition, and Population Sector .. - V Ar7 Health clinic in Mali. a solid empirical foundation on the optimal balance in services less efficiently and effectively than the private different country contexts, and the processes by which or nonprofit sector. But the actual impact on perfor- changes can be achieved. mance depends on the details of design and implemen- tation, and the wider institutional and political context. Enhancing Health System Performance State and local governments may be more responsive to The Bank has taken a variety of approaches to enhancing local populations, but they may also be more sensitive the quality and efficiency of health services, although, to the demands of local elites, and prefer expenditures until recently, the general focus was on improving the for hospitals rather than for primary health care. performance of government health services. Recent OED The case studies found that the Bank-at times-has research supports the argument that the Bank has not promoted decentralization without sufficient regard for considered a sufficiently wide menu of institutional the administrative or political implications, or without approaches to enhance performance (Girishankar 1999). giving the necessary attention to determining what responsibilities should be devolved to which levels of the Decentralization and Devolution health ministry or local government. Even in successful The Bank has widely recommended decentralization, examples of decentralization, the Bank has tended to contracting out, and separating the purchaser from the underestimate the training and technical support needed provider as institutional arrangements that will im- to help districts undertake their new responsibilities (as in prove system performance and the efficiency of govern- Mali). Similarly, the case studies found that separation of ment health services (World Bank 1993). Many health purchase and provision alone does not guarantee im- delivery systems are overly centralized and unrespon- proved efficiency (Brazil, India). In sum, decentralization, sive to local needs, and governments often provide devolution, and contracting out services may enhance 24 Development Effectiveness of HNP Investments and Policy Dialogue system performance, but the Bank needs to take a more does not suggest that all such Bank efforts are similarly nuanced approach to help borrowers determine the flawed, but that the level of institutional, political, and appropriate levels for various services, the appropriate technical sophistication necessary to achieve results is sequencing of reforms, and training requirements. considerable, and has exceeded the prevailing practice of the Bank. On the positive side, in several countries (India Strengthening Organizational Capacity and Indonesia) the Bank leveraged improvements in If policies and institutions are the rules of the game, district-level performance by working with central or then organizations and individuals are the players, and regional government to establish performance-based cri- their success depends on their ability to work within the teria for financing district health plans and programs, rules (or by circumventing them; North 1990). Bank sometimes on a competitive basis. HNP capacity building initiatives tend to focus on symptoms rather than on the root constraints on Health Work Force Reform performance, although this is unique neither to the As noted earlier, all of the country studies concluded Bank nor to the HNP sector (Grindle 1997). The Bank's that health work force issues were perhaps the domi- efforts to improve organizational performance have nant constraint facing the HNP sector. But with a few invariably fallen under the category of "capacity exceptions, the Bank has conducted very little sector building," which typically involves the provision of research on those issues. The standard Bank response training, technical assistance, and other resources. has been either to proceed with capacity building and These may be needed, and if inadequate skills and training, even though the fundamental capacity and technical capacity are the major performance con- performance problems relate to staff incentives or high straints, improved performance can result. Not surpris- turnover, or to try to strengthen health work force ingly, when inadequate skills are not the major con- planning capacity within ministries of health. Bank straints, these efforts have little impact. efforts to accomplish the latter have often met with The lack of clarity regarding objectives has contrib- limited success, largely because conditions of service uted to a complete absence-until very recently-of any are determined by finance ministries or public service indicators of organizational capacity or performance. commissions. ICRs tend to assert that capacity was built in a given Bank macroeconomic dialogue and HNP strategy organization because workshops were attended, staff were and investments have often been poorly coordinated with trained, and computers were provided. This focus on civil service reform and health work force issues. In both inputs also contributes to a lack of attention to proper Zimbabwe and Mali, the sequencing, which is often essential to achieving results. health sectors were ad- versely affected by the civil State and local Rules for Resource Transfers service reduction programs governments may be The rules that govern the transfer of resources between that accompanied eco- more responsive to levels of government, from government to private provid- nomic adjustment; at the local populations, ers, or from insurance companies to providers, fundamen- same time, Bank projects but they may also be tally structure the incentives for health care delivery. Until required additional staff. recently, few project design documents or sector studies HNP staff are increasingly more sensitive to the gave attention to understanding how these rules and aware of the importance of demands of local processes operate in a given setting (as distinct from health work force con- elites. normative assessments of how they should operate), or to straints, but often do not changing them. The Reforsus project in Brazil attempted have adequate mechanisms to improve health system efficiency by paying more to or sufficient sector analysis of the issues to elevate their hospitals or providers who provide cost-effective services. concerns into the macroeconomic dialogue. OED analysis of this case, however, suggests that this Recent Bank policy reform projects have more strategy is unlikely to work, because it does not account explicitly recognized the constraints inherent in the civil for political influences on the rate-setting process or the service, and have sometimes supported health ministries mechanisms by which payment to hospitals would influ- in proceeding ahead of government on key reforms. In ence the treatment decisions of individual providers. This some cases, this has included devolving health staff to 25 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector local government, or taking health staff out of the civil address the political dynamics that underlie inequities and service (as in Zambia). These efforts may help address a resource misallocations. In Brazil, for example, the key constraint to improved government services, but they politics of the Brazilian budget process create consistent also create other problems, including staff resistance and underfinancing of basic health services, and key preven- attrition because of reduced job security or lack of career tive programs often receive their allocations only in the mobility, late or nonpayment of salaries by local govern- final months of the fiscal year. As an external agency, and ments (often in the hope that they will be bailed out by a relatively small player in large countries such as Brazil, central government), or increased use of ethnic or political the Bank alone certainly cannot influence such dynamics, criteria in hiring. The HNP sector has identified health but could better account for them in its strategy, sector work force issues as a priority for further research, but a work, and lending program. concerted effort will be necessary to develop consistent and effective approaches to these difficult matters Cost Recovery (Martinez and Martineau 1998). The Bank's advocacy of increased cost recovery for HNP services has generated considerable controversy Health Financing and a substantial literature (Nolan and Turnbat 1995). The Bank's health financing efforts have sought to About 40 percent of all projects-nearly 75 percent in improve the quality, efficiency, and sustainability of Africa-included some provision to establish or public and private health services. Overall, the case strengthen the user fee system. The Bank has argued studies and portfolio review found that the Bank has that cost recovery can improve the quality and sustain- played an important role in raising awareness in ability of services (particularly if fees are retained), and borrower countries regarding the equity and efficiency that the poor are often willing to pay more for implications of health ex- improved services (World Bank 1994). The initial penditure and resource implementation experience followed one of two some- HNP sector studies of mobilization patterns, what distinct paths, depending on whether the country health financing, and Bank diagnosis of the adopted Bamako Initiative-style community manage- public expenditure health financing chal- ment of fee revenues, as in parts of West Africa, or if reviews, and related lenges has generally been fees were implemented through the existing govern- good. HNP sector studies ment system, as in much of East and Southern Africa documents are often of health financing, public (Gilson 1997). key sources of expenditure reviews, and The Mali and Zimbabwe case studies mirror this information for related documents are of- experience. In Zimbabwe, fees were increased at the officials. ten key sources of infor- primary and secondary levels during the early 1990s, but mation for officials, both because the Ministry of Finance did not approve fee inside and outside the retention until 1998, the overall impact on service quality government. The Bank's recommendations for address- and utilization by the poor was negative. In Mali, the ing the problems, however, often have given inad- financing of community-managed health centers signifi- equate attention to the institutional challenges inherent cantly expanded drug availability and access to rural in implementation. As a result, the Bank's record in health services. Utilization rates remain low overall, achieving effective change has been mixed. however, and cost recovery has not proved to be an effective means to fund preventive services or individual Allocative Efficiency disease interventions (UNICEF 1998). Although asserting The Bank has consistently pointed out the high percentage that the poor should be protected from fee increases, the of government health expenditures allocated to urban Bank has often failed to propose administratively feasible tertiary care in many developing countries, and has called methods to protect the poor. Bank advice on user fees has for a shift toward more cost-effective primary care generally become more nuanced in recent years-for interventions and for services likely to benefit the poor example, arguing that essential preventive services should (World Bank 1993). But while acknowledging that spend- be provided free of charge-but the Bank is still widely ing patterns are often the result of pressure by urban elites, perceived to be an unabashed advocate for increased cost the Bank has not always developed effective strategies to recovery (Watkins 1997). 26 Development Effectiveness of HNP Investments and Policy Dialogue equately captured in Country Assistance Strategies or macroeconomic discussions. Hospital Financing and Reform While tertiary hospitals and the billing of private insurance companies are often a major source of lost revenue, the Bank has had limited success in encourag- ing improved cost recovery at these levels. Initially, the Bank merely raised the issue of hospital or insurance billing in the context of sector work or dialogue, but gave little attention to supporting or encouraging actual changes. The Bank has advocated reductions in tertiary hospital expenditures-sometimes successfully, particularly in aid-dependent countries-but without providing the advice and support to help hospitals become financially sustainable.2 Hospital reform has now come onto the Bank's HNP agenda, but it is - - -proving to be institutionally and politically challeng- ing. The HNP sector cur- rently is developing im- The Bank has proved indicators and analytical tools for hospital experienced some reforms, which will need to success in be complemented by inten- encouraging sive sharing of experiences increased (Over and Watanabe 1999; Children in Brazil Preker and Harding 1999). allocations to Links to Macroeconomic Dialogue health. The Bank has not always effectively linked its health Risk Pooling and Insurance financing dialogue with macroeconomic dialogues. Resource mobilization and risk pooling through strength- The onset of economic crisis in much of the developing ened social or private insurance arrangements has world led to the adoption of Bank-sponsored economic emerged as a major focus of Bank HNP efforts in middle- stabilization and adjustment programs in many coun- income countries, particularly in LAC and ECA (World tries. Initially, the Bank and IMF did not give adequate Bank 1997c). Because most of these projects have been attention to protecting expenditures for social services, approved in the past few years, there is limited evaluation and some countries experienced reductions in health of their relevance and effectiveness. This will need to be a budgets. By the late 1980s, however, partially in priority for the HNP sector and OED in the coming years. response to global criticism, the Bank and IMF became more proactive in attempting to protect social expendi- Instruments and Strategies for Reform tures and staff in the design of adjustment programs. Strengthening the Policy Framework Since the early 1990s, the Bank has included the The Bank has contributed to improving the coherence protection or expansion of the social expenditure-as of health policy in many countries, and many borrow- appropriate-as an important component of its macro- ers consider the Bank's broad strategic view of the economic and sector policy dialogue. The Bank has sector a major asset. The Bank played a particularly experienced some success, particularly in smaller coun- important role in encouraging borrower governments tries, in encouraging increased allocations to health or to adopt formal, written policies on key sector issues in shifting public resources toward basic and preven- (such as population), and in the development of sector- tive care. Issues of equity and efficiency and govern- wide health policies in the 1990s. The Bank sometimes ment health expenditures, however, are still not ad- undermined local ownership by allowing staff or 27 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector consultants to dominate the policy development pro- nearly all the health investment budget and a substantial cess, but in recent years it has placed greater priority portion of recurrent spending. In large countries, the Bank on ensuring that government officials take the lead. In must use its lending program and policy advice to small, aid-dependent countries, however, the risk re- investigate, demonstrate, and advocate more effective mains that the development of national policies can approaches to health financing. In small, poor countries, become excessively linked to Bank project deadlines. the Bank needs to give careful attention to recurrent implications of investment programs, and take care to be Focusing on the "How" of Reform consultative and not to push for policy changes without Bank policy advice and reform strategy are too often sufficient consideration to their likely impact in a low- insufficiently grounded in empirical evidence or institu- capacity context. tional analysis of the country context. The Bank has been better at specifying what needs to be done than Linking Instruments to Objectives why problems persist and The case studies and portfolio review found that the how to address them. As a Bank has relied excessively on investment loans, even The Bank has result, the Bank has a ten- though they are often ineffective vehicles for promoting perhaps dency to promote standard complex policy reforms. A number of other instruments encouraged overly solutions to health system are available, each with strengths and drawbacks. Two ambitious reforms. problems, without giving recently developed loan instruments-Learning and sufficient attention to local Innovation Loans (LILs) and Adaptable Program Loans institutions or details of (APLs)-allow the flexibility and learning necessary implementation. Conversely, when the Bank begins to for effective health system reform, but a relatively tackle the how questions, through its policy advice, small (although growing) number of HNP projects research, or lending-as in India since the 1990s-its have made use of these instruments. The Bank also has influence and impact increase. approved a number of large sector adjustment loans in past few years. These can help tip the balance in favor Incremental versus "Big Bang" Approaches of a particular reform effort if conditions for tranche The Bank is increasingly engaged in reform issues that release are transparent and negotiated among a variety have no commonly agreed solutions or universal of stakeholders. But experience with structural adjust- models, limited evidence about what works, and are ment lending suggests that conditionality alone is areas of limited Bank experience (Nelson, in press). ineffective in promoting broad-based systemic reform These include health insurance reform, regulation of (Nelson and Eglington 1993). the private sector, pharmaceutical policy, health work The rapid growth of sector-wide approach (SWAp) force reform, and the appropriate balance between programs to encourage policy reform and increased donor public and private roles in health service financing and coherence is an important innovation. SWAps are charac- delivery. Incremental approaches may therefore be terized by government-led partnerships, where govern- more appropriate, built on solid research, pilots, and ment and donor investments fall under a sector-wide focused efforts to learn from experience. The Bank, policy and expenditure framework (Peters and Chao however, has perhaps encouraged overly ambitious 1998). Although they can bring much-needed coherence to reforms, although the choice ultimately depends on a sector, SWAps require a strong government policy borrower judgments of country circumstances. vision, a high degree of confidence among government and partners, and can entail higher risk than single Large versus Small Countries investments (Walt and others 1999). The emphasis on There is considerable difference in the Bank's role in strengthening national policies is consistent with the health system reform in larger or middle-income coun- Bank's comparative advantage, but constraints to perfor- tries, and the role it assumes with smaller, aid-dependent mance may lie elsewhere. The recent enthusiasm for borrowers. In the former, Bank assistance usually repre- SWAps therefore should not obscure the need for thorough sents only a fraction of government health spending; in examination of the appropriate instrument, or mix of the latter, the Bank and donors collectively may represent instruments, for the context and objectives. 28 Recommendations chieving institutional change and improving health system performance are inherently difficult, both technically and politically. Although the Bank is only one of many players, it can increase its impact in the HNP sector by strengthening portfolio quality assurance and results orientation, intensifying learning from lending and nonlending services, enhancing strategic selectivity within the Bank, and strengthening partnerships with other donors, client governments, and civil society organizations. The overarching recommendation to emerge from Recommendations: the study is: Do better, not more. After a decade of By the end of calendar year 1999, Bank management rapid growth in HNP lending and a widening health and the HNP Sector Board should establish priorities policy reform agenda, this review suggests that the and guidelines for staffing, lending, and administrative Bank needs to focus on consolidation-both of lessons resources (including project supervision and ESW) in learned and portfolio quality-and making strategic light of the overall objectives of the 1997 sector choices about where to engage and how to allocate strategy. Particular atten- scarce resources, particularly staff time. The HNP tion should be given to Do better, not more. sector is engaged in a number of activities to address (a) how the issues raised issues raised in this report. The following are recom- in this OED assessment mendations that OED believes can further strengthen will be addressed, including the budgetary implica- the development effectiveness of the Bank's work in the tions; (b) how the sector plans to focus activities and HNP sector. budgets to sustain quality in light of staff overprogramming and pending declines in administra- Increase Strategic Selectivity tive budgets under the Strategic Compact; and (c) how The 1997 HNP Sector Strategy provides an effective country directors will be brought on board with the vision for the sector, but there is a growing disconnect recommendations and guidelines. among HNP strategies, mandates, and available re- sources. 29 [nvestng in Health Developnent Effectiveness in the Health, Nutrition, and Population Sector Strengthen Ouality Assurance and Results-Orientation The Bank needs to give greater attention to assessing Despite improvements in many areas, HNP lags borrower capacity and incentives for evaluating health behind the Bank average on several key quality system performance, and to building borrower M&E indicators, and it is not currently on target to meet capacity. Strategic Compact goals. Recommendations: Recommendations: To strengthen borrower capacity and the incentives for The HNP Sector Board, in conjunction with regional M&E in the HNP sector, sector strategies and project sector leaders, should strengthen its role in monitoring designs should include (a) assessments of borrower and bolstering portfolio quality and results orientation: (a) incentives and capacity for M&E and (b) where establish a regular system to review portfolio quality appropriate, recommendations and measures to better indicators, including identification of priorities for reme- enable borrowers to monitor and report results, includ- dial action; (b) establish supportive mechanisms to help ing strengthening health information and vital registra- task teams improve performance; (c) in conjunction with tion systems, and a description of the role of the Bank Bank management, identify steps to strengthen routine relative to other partners in this process. quality assurance mechanisms; and (d) in annual reports on the HNP sector strategy, present more evidence Strengthening borrower incentives for analysis and use regarding progress toward sector goals. of evaluative data in health policy and budget deci- sions is essential to improved M&E performance. Assessing the impact of health interventions can be challenging, but excessive Bank focus on inputs and the Recommendations: low priority given to M&E are also to blame. The Bank should seek ways to strengthen the incentives for monitoring, evaluation, and results-orientation Recommendations: within client countries by (a) promoting wider experi- To strengthen Bank performance in M&E, management mentation with and use of performance-based budget- should (a) identify a core group of HNP staff and ing systems in its lending and policy dialogue, particu- consultants with experience in implementing HNP M&E larly in the CDF pilot countries; (b) producing a who could be available to preliminary "lessons learned" paper on experience in The Bank needs to assist other staff during performance-based budgeting in HNP, in conjunction g tproject design and supervi- with partner organizations, including implications for give greater attention sion; (b) develop a "good the CDF by the end of fiscal 2000; and (c) the Bank to borrower capacity practices" manual of should increasingly engage independent evaluative and incentives for M&E design for use in organizations, preferably based in borrower countries evaluating health decisionmaking, both at or regions, to provide periodic assessments of Bank- the project and systemic financed activities. levels, including lessons from partner organiza- Enhance Learning and Increase Institutional tions; (c) in collaboration with the World Bank Institute, Development Impact develop M&E case studies and training modules; (d) Although promoting institutional change in the HNP periodically give recognition to task teams that can sector can be difficult, Bank performance in achieving demonstrate measurable results from Bank-supported HNP institutional development objectives has been activities; and (e) in parallel with the Comprehensive disappointing. Development Framework (CDF) pilots, report by the end of fiscal 2000 on how Bank business practices and Recommendations: procedures could be modified to allow greater results- To strengthen the institutional development effective- orientation in Bank lending, and to increase internal ness of the Bank's work in HNP, management should incentives for monitoring and reporting on results. (a) develop appropriate tools, guidelines, and training 30 Recommendations programs for institutional and stakeholder analysis in programs that require intensive stakeholder consulta- HNP, both for targeted interventions and systemic tion, country directors and sector managers should reforms, in coordination with PSM and other internal ensure that these time requirements are reflected in and external partners; (b) clarify the requirements for project preparation and supervision budgets. institutional analysis in project appraisal documents; and (c) establish a core of HNP staff and consultants Bank-supported programs have not placed adequate with experience in institutional design and stakeholder emphasis on health promotion and the intersectoral analysis who would be available to assist other staff. dimensions of health. The Bank must establish a strong analytic and empiri- Recommendations: cal base to provide effective guidance on how to To strengthen the Bank's effectiveness in health promo- enhance health system performance. tion and in addressing the intersectoral dimensions of health: (a) the Bank's Human Development Network Recommendations: and Regional vice presi- To strengthen the analytic base for Bank advice and dents should identify several Systemic change lending: (a) management should increase funding for key areas for improving HNP sector work; (b) the Sector Board should sponsor intersectoral collaboration requires operational research and provide good practice guide- within the Bank, including understanding lines on improving the effectiveness and efficiency of coordination of macroeco- stakeholder ESW and other Bank advisory and analytic services; nomic and sectoral dialogue interests and and (c) management should shift some the ESW budgets on social sector work force building coalitions from country departments to regional technical manag- issues; HIV/AIDS preven- ers to encourage regional research on priority issues. tion and mitigation; and for reform. key health promotion ac- Enhance Partnerships tivities (defined on a regional basis); and (b) the HNP Promoting systemic change in HNP requires understand- Network should strengthen staff skills in health promo- ing stakeholder interests and building coalitions for tion and establish "good practice" guidelines and reform. Although borrowers necessarily take the lead, the examples for task managers. Bank needs to play these roles more effectively. Effective partnerships are necessary to address several Recommendations: of the above issues. To increase the Bank's ability to sustain a continued presence in borrower country health policy debates, Recommendations: and to develop long-term partnerships with various The Bank should strengthen work with HNP develop- stakeholders in client countries, (a) the Bank should ment partners (such as WHO, UNICEF, and bilateral continue its current efforts to base sector specialists in donors) in several key areas, including strengthening countries or regions, with a clear mandate for collabo- HNP M&E systems and incentives and assessing rative policy dialogue with stakeholders inside and progress and strategies on the current generation of outside government; and (b) for projects and reform health sector reforms. 31 ANNEXES ANNEX A: PRECIS OF CASE STUDIES HEALTH CARE IN BRAZIL: ADDRESSING COMPLEXITY World Bank health programs in Brazil have been relevant, but results have been uneven, according to a recent study by the Operations Evaluation Department (OED). Bank-supported programs have helped to control the spread of serious tropical diseases, improved access to health services in poor areas, and contributed to the construction of a system for epidemiological surveillance. But while projects in disease control and basic health services have targeted important and relevant concerns, critical challenges remain to be addressed, including persistent inequities and inefficiencies in the financing of health services. In order to be a more effective partner in tackling the considerable challenges of the Brazilian health sector, the Bank should work to establish the kind of strong and consistent presence that is required to build-gradually, but persistently-a broad-based coalition for reform. The Strategy of consumers. Second, it is not clear whether the Bank's The Bank has financed 10 projects in Brazil's health, research and efforts to transform the health care delivery nutrition, and population sector; carried out major system in Brazil sufficiently took into account the system's field research; and served as a policy interlocutor for complex politics, institutions, and political economy. This the government. Its health strategy has focused on three may have made the Bank's ambitions broader, and main concerns: providing resources to expand the effectiveness more transitory, than they would have been accessibility of basic medical services in poor or with a clearer picture of the country context in view. And marginal areas; offering policy advice and studies on third, the Bank's focus on endemic diseases neglected methods to improve the efficiency and efficacy of the other, noncurative, health needs of the aging and increas- health care system; and financing projects to control ingly urbanized population. endemic diseases such as malaria, schistosomiasis, and AIDS. The strategy appears appropriate in the Brazil- Basic Health Care ian context-that of a middle-income country with a Child Health and Nutrition relatively high degree of poverty, a health care system Brazilian children have become much healthier in the known for inefficiency and inequity, and a population past two decades. Infant mortality rates and childhood that is exposed to a variety of endemic diseases. height-for-age charts, two good indicators of the gen- The Bank's strategy has evolved over time- eral health of children, show significant improvements expanding access to basic services was a primary in recent years. The changes have been brought about emphasis in the 1980s and early 1990s, but by the mid- by a complex interaction of improvements in purchas- 1990s Bank lending and policy dialogue increasingly ing power; maternal education; access to health care, focused on improving system efficiency and efficacy. including oral rehydration therapy; community infra- Because Bank lending represents less than 1 percent of structure and water supply; and individual behavior, annual health expenditures in Brazil, the Bank's effec- such as increased breastfeeding and fewer short-inter- tiveness depends on its use of lending and policy advice val births. to leverage wider changes in Brazil's health system. But despite the recent improvements, serious Several shortcomings are evident in Bank strategy, regional inequalities persist: children in the Northeast, however. First, many of the health posts constructed to the poorest, most rural, and most traditional region of improve health care access for the poor are underutilized, the country, are much less healthy, and their health is short of qualified staff, and lack the facilities needed to improving more slowly, than children in other areas. serve their increasingly demanding and urban population While more than 10 percent of children nationwide still 33 Invest1ng in Health: Development Effectiveness in the Health, Nutrition, and Population Sector suffer severe growth retardation, or "stunting," in the for the Northeast region. The projects built health care Northeast the figure rises to almost 18 percent, and in facilities, encouraged management improvements at the rural Northeast, one in four children is affected. the federal and state levels, and provided technical Despite significant attention from the Brazilian skills training for the development of new basic health government and international donors, including the care modules and programs of comprehensive care for Bank, the Northeast has not kept pace with health women and children. improvements in the other regions. This cannot be Partly as a result of a difficult political and explained by differences in income; it is the legacy of macroeconomic context, these projects evolved into greater improvements in access to health care for facilities construction and medical equipment pro- children and mothers, maternal education, and repro- grams. They succeeded in expanding access to basic ductive practices in the more urban areas. health services but did not transform the mode of basic health care delivery within that system. Child health Fertility and Women's Reproductive Health improved during the life of the projects, and access to Brazil's fertility decline has been dramatic. The num- health care played a significant role in that improve- ber of births per woman fell from 5.8 in 1970 to 2.3 in ment. But because the projects did not include an 1996, despite the near absence of a government adequate monitoring and evaluation system, it is population policy. The speed of the decline in fertility difficult to assess their impact on these trends. has been more rapid than in India; Bangladesh; The evidence suggests that the projects' contribution Mexico; and, by some measures, Indonesia, a country may have been limited-they disbursed slowly until 1994, with an active population policy. Recent declines in and by this time most of the improvement in child health total fertility have been particularly remarkable in the had already occurred. Many of the clinics remain Northeast. Although government interest in family underutilized, and focus group sessions suggest that planning and reproductive health has increased consid- consumers are dissatisfied with service quality, and erably in recent years, previous indifference limited the increasingly prefer to visit doctors rather than nurses at Bank's role. clinics. The projects were negatively affected by the poor Lower fertility rates have contributed to the recent labor market for health care providers, and hampered by improvements in childhood health by reducing the risks the Bank's inadequate understanding of the political associated with short birth intervals and high parity, and forces in the sector and their institutional context. have reduced demand for immunization, prenatal care, and birth attendance, ultimately lessening the pressure on Infectious and Parasitic Diseases the health system and making care more accessible. The use of oral rehydration therapy has brought about Although it is unclear why Brazilian women began a recent sharp decline in diarrhea among children. to have fewer children, it appears to be a demand-side Diseases preventable through vaccination are largely story. Social scientists identify the high rate of abortion under control, although sustaining highly successful and contraceptive use as the most important determi- vaccination programs will be a challenge, given the nants of lower fertility. The two most popular methods inefficiencies in the purchase and distribution of pharma- of limiting fertility are female sterilization and the pill. ceuticals. Tuberculosis is on the rise as a result of the AIDS The health consequences of these practices have been epidemic, increasing worldwide immigration, and rela- complex and damaging: legal restrictions, financial tive neglect by the international public health community incentives, cultural norms, and misinformation fre- during the 1970s and 1980s. Leprosy incidence, while quently lead women to use delivery as an occasion for falling in most countries, is on the rise in Brazil. sterilization, which is one reason that the cesarean Endemic parasitic diseases continue to threaten delivery rates in Brazil are the highest in the world, and rural and remote areas, and malaria is almost exclu- maternal mortality is unusually prevalent. sively an illness of the Amazon region. In the North- east, urbanization and the government's endemic dis- Northeast Basic Health Services Projects ease program are lowering the threats of leishmaniasis, The World Bank financed two Northeast Basic Health schistosomiasis, and Chagas' disease. Yellow fever, Services projects beginning in the mid-1980s, as part of which disappeared from Brazil in mid-century, has the Brazilian government's 15-year development plan again become a threat to parts of the country. Dengue 34 Annexes and cholera, thought to be under control, have resur- launched a National AIDS Prevention and Treatment faced in recent years. The AIDS epidemic, originally program that is increasing capacities for surveillance, the most intense among bisexual and homosexual men, treatment, institution building, and prevention by is increasingly affecting women, heterosexuals, and working with NGOs. The program has been well- intravenous drug users. designed and effectively implemented. It is too early to tell, however, whether the project has slowed the rate of Disease Control Projects increase in disease incidence. Bank-financed disease control projects have focused on diseases-malaria, leishmaniasis, schistosomiasis, Chronic and Degenerative Diseases Chagas' disease, and AIDS-that are significant problems New demographic patterns emerge as a country modernizes in Brazil, strike young people, and disproportionately and develops-leading to what is known as the epidemiologi- afflict poor and marginalized groups. At least two of the cal transition. Infant mortality and fertility decline, life four World Bank disease-control projects have contributed expectancy rises, and infectious and parasitic diseases are no to declines in the incidence of those diseases and mitigated longer the leading causes of death. These new demographic their effects on afflicted individuals. The Endemic patterns emerged in Brazil's South and Southeast regions Disease project and the Second Malaria Control project decades ago, and by 1980 all regions were undergoing the helped to slow the spread of disease and promoted transition. By 1980, cardiovascular disease had become the treatment programs. Although not all the reductions in leading cause of death in all major regions, and almost all incidence can be attributed to the projects, they certainly states (table 1). The prevention and treatment of conditions contributed. The First Malaria Control project did not more common among the aging-including screening for reduce the incidence of malaria in Rondonia, however, cancer treatment of strokes, and care for long-term and and was rated unsatisfactory. chronic conditions that require expensive treatments-will Although the First Malaria Control project under- require reform of the health care system in the coming years. estimated the importance of institutional strengthening To make the necessary investments in medical infrastructure, and behavior change in public health, subsequent equipment, and training, it will be necessary to ration free disease control projects have helped build Brazil's and universal health care, rely more heavily on private human, physical, and information systems for disease financing, or do both. surveillance. The Brazilian government, with signifi- In 1989, the World Bank sponsored high-quality cant Bank support, has expanded its ability to combat analytic work that identified priorities for improving infectious and parasitic diseases in a modern, more adult health, including maternal health and promotion comprehensive manner. Instead of relying on vector of healthy behaviors, such as exercise, diet, smoking eradication, which may not be possible, the govern- cessation, and injury prevention. The Bank, however, ment has shifted its emphasis to encompass overall was unable to develop public health projects with the disease control, stressing the importance of behavioral Government of Brazil to address these concerns. changes by individuals through information cam- paigns, community mobilization, leadership by au- The Health Care System in Brazil thorities, and the treatment of those infected. The constitution of 1988, following a decades-long With support from the Bank, the government has social movement to combat the inequitable health care TABLE 1: THE LEADING CAUSES OF DEATH: PROPORTIONATE MORTALITY IN BRAZIL (PERCENT OF ALL DEATHS), 1994 South Southeast Center-West North Northeast O%erall Circularor\ Ji.i, 32.6 "11.j 2-.1 F q 19.5 . External cIAuses 11.8 1- 1.6 1 3. 9. 12 1 Neoplani 15.2 11.4 9.9 C.S 6.3 Respitarorn J;.ea,. 11.. li -..h t .- 5.9 9 2 Glandular. nietabolic, arid irnune rrerruprions 4.v . 4.1 1.4 3.9 5 2 Infections and parasiare 3.2 4.1 6.4 - II 6.4 4.1 Unknown causc, 9.5 12 i 12.8 28 3 37.0 1>. 35 Investing in Health Development Effectiveness in the Health, Nutrition. and Population Sector policies of the departed military regime, mandated a tion and high-technology care, and Brazil has one of free, universal health care system, Unica da Saide the lowest ratios of nurses to doctors in the developing (SUS). SUS contracts out a large majority of inpatient world. care and a substantial portion of outpatient services to a network of private and philanthropic hospitals, Promoting Health Sector Reform clinics, and other facilities. The government manages In the mid-1980s, the Bank sought to support health and owns just 31 percent of the hospital beds it supports decentralization through the Sio Paulo Basic Health and has slowly been decentralizing control of publicly project. The project design did not adequately account owned facilities to states and municipalities. Privately for complexities in state politics and federal-state financed health care has grown rapidly: 25-26 percent relationships, however, which limited its impact. of Brazilians are covered by private plans. These plans The current health sector reform project, Reforsus- vary widely in quality and price, but generally exclude cofinanced by the World Bank and the Inter-American coverage of expensive, catastrophic conditions, leav- Development Bank-is predicated on a much more ing that job to the public system, and are subject to sophisticated understanding of economic incentives than almost no regulatory oversight. earlier projects. It has established an innovative instru- Although Brazil's health system might appear to be ment to disburse grants to health facilities on the basis of efficient-it substantially "separates financing from the competitive bids. Reforsus also aims to improve the provision of services"-it is instead inching toward efficiency of the health care system by changing govern- crisis. The public system is severely underfinanced, ment payment systems, so that doctors and hospitals will resulting in regional inequalities, rationing of services, be paid more when they provide particularly cost- and a perceived decline in quality. The hyperinflation effective services. Unfortunately, this strategy may not of the late 1980s and early 1990s and the irregular flow work. Partly as a result of interest group pressure, rates of resources to health have contributed to the evolution continue to be set in a nontransparent process, and a of a fee structure for medical treatment that has not variety of other rules also influence the payments provid- kept pace with costs, and payment can be sporadic. ers receive. As a result, the behavior of doctors may not Doctors frequently must work at several sites to make change significantly. ends meet. Stories of long lines for hospital services, Given the complexity of the Brazilian health mistakes in emergency care, strikes and walkouts by system and the interests at stake, reforms will require medical professionals, arbitrary triage, and other crises commitment, persistence, and the nurturing of relation- are reported daily in the press. ships with strategically important partners over at least The structure of the health system in Brazil a 10-to-15-year period. Changing the education of provides weak incentives for quality and cost-effec- health care providers alone will take more than a tiveness. The government is only now beginning to generation. To date, the Bank has no long-term develop information, monitoring, and evaluation strategy for-and has little experience in-coalition- systems for the health care system; it will be some building. And the Bank's sector studies, although of time before that data can be tied to incentives and high quality, are not widely disseminated. A visible, other quality assurance mechanisms. permanent, and informed presence in Brazilian policy Local governments have been given wide responsi- debates should be the first step in becoming involved in bilities under decentralization to manage all aspects of the reform of the health system. health care, but they do not necessarily have the capacities and incentives to deliver coordinated and Recommendations cost-effective services. In addition, the old military * Coalition building. The Bank must grapple with regime left a legacy of an exclusionary and highly the difficult, institutionally embedded problems centralized health system that has little capacity and is of the Brazilian health sector. Problems require unresponsive to local needs and Brazil's enormous long-term solutions in areas such as medical regional diversity. The system is distorted and expen- education, the labor market for health care sive, expenditures do not target the poor, the health providers, and the political economy of budget- lobby is strong and well-organized, institutions are ing. The Bank must adopt at least a 10-year fragmented, medical training encourages specializa- timeframe for reform, first achieving a highly 36 An nex es visible, permanent, and informed presence in * Measures of health system performance. If health Brazilian policy debates. providers and systems are to be held accountable * Regulation of private health care. The private for the quality of services they offer, monitorable sector might provide health care more efficiently indicators of health system performance must be and effectively to poorer and middle-income seg- implemented. ments of the population with the implementation of appropriate regulation and targeted subsidies. FIGURE 1: THE EPIDEMIOLOGY OF MALARIA IN BRAZIL, * Providing basic health care services. New 1960-96 approaches are needed in the financing of pro- grams that address the needs and health condi- Number of Cases tions of poor and marginalized citizens. The 600,000 - Bank should encourage and pilot such innovative projects. * Chronic and degenerative diseases. The preven- 400,000 - tion and treatment of conditions associated with the epidemiological transition will require the 300,000 - health system to expand, improve, and develop 200,000* new delivery systems. The Bank could be useful * in experimenting with ways to reduce the inci- 100,ooo - dence of lifestyle-related risk factors among poor 0 / 0 _ and marginal groups. 1960 64 68 72 76 80 84 88 92 96 MALARIA: A CASE OF BANK INVOLVEMENT he Brazil Brazil's leading federal malaria deaths. Malaria with malaria is em- Amazon Basin public health agency at rates had begun to fall blematic of the Bank's Malaria Con- that time. before the shift in strat- work on specific dis- trol project was initi- During the program egy, however, which eases in Brazil. The ated in response to a period, the incidence of clearly signals that addi- early malaria work fo- dramatic upsurge of both strains of malaria- tional factors-perhaps cused exclusively on malaria in the Amazon P. vivax and P. including the earlier in- eradicating malaria- Region. The program falciparum-among troduction of carrying mosquitoes, had two goals: first, to indigenous peoples in the mefloquine-were at without much effect. reduce the prevalence region fell significantly. work. Other events also The second phase ad- of malaria to a level The decline coincided are probable contributors dressed the motives and that no longer repre- with a shift in project to the falling rates, incentives of patients, sented a public health strategy from malaria including migration; health care providers, problem and to reduce eradication to malaria land-settlement patterns; and other key actors; the risk of reintroduc- control and management, and a slowdown in the promoted behavioral tion in areas of low with a particular focus arrival of oldminers, change; and targeted prevalence, and, on reducing the incidence loggers, and other rapid diagnosis and second, to enhance the of falciparum cases and fortune-seekers, fertile treatment, with good organizational treating those infected targets for the parasite, results. efficiency and respon- with the strain, which in the Amazon area. siveness of SUCAM, was responsible for The effort to deal 37 Annexes HEALTH CARE IN INDIA: LEARNING FROM EXPERIENCE The World Bank has emerged as the world's largest lender in the health, nutrition, and popula- tion (HNP) sector of developing countries. The Bank also plays a major role advising on national health policies. But in India, where the Bank has invested more in HNP than in any other country-US$2.6 billion-over the past three decades, progress, particularly for the poor, has been slow and uneven. While India's health status has improved substantially, it still is not on a par with other countries at a comparable level of development. The root causes of this halting progress are poverty and low levels of education, particularly among women, but public health programs bear a share of the responsibility. The Operations Evaluation Department (OED) found health assistance is reflected in the observation that 80 that, in the 1970s and 1980s, the Bank supported percent of health spending is for private health services, government programs that were seriously flawed. But in and that the poor frequently bypass public facilities to 1988 the Bank began to work more collaboratively with seek private care. Indian experts to identify determinants of program con- In addition, inadequate management and person- straints, as well as possible solutions. This enabled the nel policies limit the effectiveness of many initiatives. Bank to push for better programming and policies and to At the national level, management is highly central- propose new ways to address fundamental problems in the ized, leading to a uniform, inflexible approach Indian health system. The results of this more recent throughout the country, despite major interdistrict approach have been encouraging. This experience with disparities in fertility, health, and cultural and institu- innovative projects, sector work, and policy dialogue in tional characteristics. This overcentralization contrib- India's HNP sector offers important lessons for improving utes to weaknesses in local service delivery, with local health in countries around the world. managers often unable to provide adequate support, supervision, and training to front-line workers. These Getting It Right problems are compounded by personnel policies that In analyzing why health progress in India has not been as fail to provide incentives for better performance or for rapid as it has in other settings, and why age-specific learning new skills. mortality and disability rates remain higher than Another reason for poor performance is that the in other countries and regions (table 1), OED identified a limited resources devoted to health have not been used number of factors, including: (1) a population growth rate strategically. Many programs have failed to effectively that puts a strain on government resources; (2) per capita target the most vulnerable groups. Nutritional supple- public health expenditures that are half those of compa- ments in some programs, for example, have been rable countries and one-third the estimated cost of an available to everyone, reducing the quantities avail- essential package of health services; (3) inadequate able for poor women and children. funding of programs used mainly by the poor, and limited And finally, there has been a tendency to allocate access for the poor to the programs that are available; (4) expenditures to India's 25 states on a per capita basis, insufficient provision of safe drinking water and sanita- ignoring the enormous differences in their need and tion; (5) poor quality of service, as a result of supply capacity to utilize such resources. Indeed, these states shortages, absenteeism, improper staff behavior, unrealis- are as diverse in language, religion, level of develop- tically large workloads, and low staff morale, and ment, administrative efficiency, and quality of gover- consequent underutilization of facilities; (6) inadequate nance as the nations of Latin America or Africa. The mobilization of private and NGO resources; (7) excessive size and diversity of the country thus present unique focus (until recently) on sterilization and use of financial challenges for the design and management of health incentives to achieve targets; and (8) inadequate focus on programs. Clearly, one-size-fits-all programs do not maternal and child health. The tenuous quality of public belong here. 39 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Evolution of Programs and Projects policy changes, including a new emphasis on outreach, Since the early 1970s, the Bank has funded 23 HNP maternal and child health, temporary contraceptive projects in India, while also sponsoring important measures, and education campaigns, which became the sector work and engaging the government in an basis of the Bank's newer population projects. The ongoing policy dialogue. From rather simple begin- Bank also began to focus on high-fertility states and nings, support has evolved slowly, in phases, as the urban slums-areas with the greatest need. Bank and the government have learned to tackle the Despite these improvements, it was more difficult weaknesses and limitations of India's health system in to reorient practices and programming than either the increasingly sophisticated ways. Bank or the government had anticipated. Staff who designed these initiatives were perhaps too optimistic Population about what could be accomplished in states with weak In the area of population policy, Bank support has been administrations. Project development also suffered separated into three distinct phases. Early projects, from the failure to involve local stakeholders in project carried out from 1972 through 1988, had the narrow design. Had there been more effective stakeholder aim of helping the government carry out its Family participation, project feasibility and risk might have Welfare Program. While designed to integrate family been more accurately gauged. planning and maternal and child health services, in Another shift came in 1996, when the government actuality the program emphasized sterilization and the dropped sterilization and numerical targets as the focus expansion of facilities. The program gave little of its population program, adopting a "target-free" emphasis to increasing demand or improving the approach that gave greater emphasis to meeting quality of family planning services, which reduced its women's reproductive health needs. The immediate impact on both contraceptive prevalence and total result of the new policy was a reduction in contracep- fertility rates. The Bank had little influence on the tive acceptance rates, in part because previously direction of this program. The government's approach exaggerated rates were now more realistically re- was firmly established long before Bank involvement, ported. Recent data suggest that acceptance rates are the Bank's lending represented only 3.6 percent of total recovering. To help this new policy succeed, the Bank program funding, and the Bank was generally poorly is designing support programs that are based on need, positioned to suggest improvements or alternatives, that closely monitor results, and that provide timely In 1987, a new and somewhat larger Bank team- feedback. one with a wider view of human resource develop- The design of the recent Reproductive and Child ment-undertook a series of sector studies that offered Health Project, for example, is based on related sector excellent diagnoses of the problems in India's popula- work and consultations with stakeholders and NGOs. tion program. Yet the impact of the initial studies was This project offers practical ways to promote family limited. Over time, however, the Bank's sector work planning without emphasizing sterilization targets and increased in influence, in large part because the Bank allows for different implementation models in different involved the government in selecting and designing the situations. It also introduces some elements of perfor- studies, and local experts were hired to carry them out. mance-based budgeting to increase accountability and The sector studies helped to generate important puts monitoring and client feedback at the center of the TABLE 1: BURDEN OF DISEASE: DISABILITY-ADJUSTED LIFE YEARS (DALYS) PER THOUSAND POPULATION LOST TO MORTALITY AND DISABILITY IN INDIA, CHINA, AND TWO REGIONS, 1990 Percent ol DAL)s Countr or Rcgion NtorialirN Disahilir Total lo;i. ags 0-4 InJia 235 l11; 339 -lS Chin.i 104 %il 184 214 Oihcr l"a ind Idand, 16S ' 260 3 NfiJJki E:',ttrin ( recnr 209 1 300 5'' 40 Ann exes project. These features should bring about improve- program has developed widespread political support, in ments in program effectiveness. part because of its widely distributed benefits. Bank staff involved in designing the next iteration of the project are Nutrition trying to find practical ways to implement such changes. The Bank has supported two quite different nutrition programs. The first, the Tamil Nadu Integrated Nutri- Health Projects tion project (TINP), was an innovative program that The Bank began to support freestanding health projects operated from 1980 to 1997. Designed by Bank staff in the early 1990s. Until that time, the government and Indian consultants, it focused on changing the way funded primary care on its own and did not seek policy mothers feed themselves and their infants and pre- advice from the Bank in the health sector. Financial school children. Mothers kept records of their difficulties in the early 1990s and new leadership in the children's weight, and received nutrition education, Ministry of Health and Family Welfare, however, primary health care, supplemental feeding, and other provided an opening for the Bank to fund two types of medical interventions when necessary. Considerable projects: disease-specific interventions and broader, care was taken in designing work routines, training state-level health system reforms. and supervising staff, and ensuring that supplemental feeding was targeted only at underweight children. The Disease Control Projects program was quite successful in reducing severe malnu- The Bank's 1993 study, Disease Control Priorities in trition, but less so in reducing moderate malnutrition. Developing Countries, stimulated interest among In- This difference may suggest that improvements in dian health officials to request Bank support to develop feeding practices can only go so far, and that further a series of disease control programs. The projects have gains require poverty reduction as well. introduced important innovations, such as greater Despite the relative success of the Bank's Tamil Nadu integration of the private sector and nonprofit regis- project, the Indian government showed little interest in tered societies into the government's health efforts and continuing or expanding it. Rather than pressing for its new ways of fighting cataract blindness, tuberculosis, expansion, in 1990 the Bank also began to support the leprosy, and malaria. Implementation experience var- government's predominant initiative for preschool chil- ies widely, but there have been notable successes. dren, the Integrated Child Development Services (ICDS) The Bank was instrumental in bringing the Indian program. The Bank advocated incorporation of elements government to move against AIDS seven years ago, of the Tamil Nadu project into ICDS, which was meant to and the Bank-financed National AIDS Control project be a holistic child development program, offering nonfor- has established state and national HIV control pro- mal preschool education for children 3 to 6 years of age; grams, instituted better diagnosis and treatment, and supplemental nutrition, immunization, and regular health moved to change risky behaviors. Over 90 percent of checkups for children ages 0 to 6; and nutrition and health the blood supply is now tested for AIDS, a threefold education for pregnant and nursing women. Outcomes increase. While it is not easy to determine the exact thus far have been disappointing. The TINP experience number of HIV cases averted, an estimated one-third of seems to have been lost on India, although the design has a million cases may have been prevented. been used in other parts of the world. To fight the resurgence of tuberculosis, the Bank While ICDS was eventually able to reach 80 percent has supported the introduction of Directly Observed of the development blocks in the country, it had no Treatment, which now covers 115 million TB sufferers. mechanisms to ensure that its services and supplemental The Cataract Blindness Control project has surgically food actually reached those most in need. In addition, restored sight to 8 million people, and 30 percent of workers were inadequately trained and were overex- these surgeries were performed with the advanced IOL tended, and the program's outreach, health, and educa- method. And leprosy victims have benefited from the tional components were often neglected. As a result, the National Leprosy Elimination project-almost 12 mil- Bank rates its ICDS projects as unsatisfactory. While the lion have been cured. It is estimated that 18 of 32 Bank originally attributed program flaws to rapid expan- Indian states/UT will eliminate leprosy by the end of sion and implementation problems, it now appears that 2000, an additional 8 states by end-2002, and the final significant changes in direction are required. Yet the 5 states by end-2005. 41 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector These programs have focused on diseases that, and secondary care, but no evidence yet of significant while serious, together represent only about 6 percent progress on cost recovery or referral. The Second State of the mortality and morbidity burden. Cardiovascular Systems project is progressing better in some states disease, cancer, trauma, mental illness, and tobacco- than in others. related diseases have yet to be addressed. These The use of different approaches tailored to the illnesses will almost surely be more difficult to handle, needs of individual states provides a unique opportu- they are less concentrated among the poor, and the nity to learn what does and does not work in different government has yet to come up with proposals for their settings. But OED's study concludes that plans for management. monitoring and evaluation must be further strength- ened if this is to take place. State Health Reform Projects State personnel policies and management systems play a Looking Toward the Future fundamental role in determining system performance and The difficulties experienced in gaining the desired health outcomes. To gain leverage over these critical results from HNP projects in India before 1988 can be determinants of success and to tailor programs to each attributed to a number of factors: there were no state's needs and capabilities, the Bank has initiated four freestanding health projects; population projects were state-level health reform projects since 1995. These usually supply-oriented; the Bank was not forceful projects also offer the Bank a long-sought opportunity to enough in addressing weak performance or in pressing influence the more fundamental determinants of how the for policy changes; and the Bank did little sector work public health system works at the state level, where the to identify the most pressing issues and needs. Bank can provide assistance that is tailored to the locality. Specific factors that inhibited the success of The first state-level effort was the Andhra Pradesh projects included: First Health Referral System project, a $159 million project approved in 1995. Its aim is to establish * Lack of adequate information meaningful referral systems, provide training and * The Bank's image of itself as a provider of equipment to strengthen management of the state hardware and infrastructure rather than a devel- public health system, introduce a cost-recovery mecha- oper of human resources nism, and improve resource allocation. The Second * Resistance from Indian counterparts to address- State Systems project extends the principles of the first ing systemic issues project to three other states. It is the largest health * Shortages of resources and effective managers project the Bank has ever funded ($350 million), and is * Focus on the public sector and on expanding the showing signs of being too large and complex to be public health system managed satisfactorily. Subsequent projects in Orissa * Application of a single model to areas with very and Maharashtra each focus on one state. Activities different characteristics range from increasing access to primary care in remote * Inadequate attention to changing health-related areas, to establishing a new institution to manage the behavior hospital system, to improving service quality at com- * Neglect of important determinants of health and munity health centers, focusing on maternity cases. demographic status, such as the education of The Maharasthra project also includes an innovative women. component to establish a new, specialized hospital that operates according to modern hospital management After 1988, however, sector work helped to initiate practices. a policy dialogue. This led to important changes in Supervision reports indicate that these projects are approach, including a focus on health system reform, a progressing satisfactorily. The Andhra Pradesh project, shift from family planning to maternal and child in particular, is progressing well, with some elements- health, and a more collegial and collaborative rela- notably management and monitoring and evaluation- tionship between the Bank and the government. rated highly satisfactory. There has been a modest A number of factors contributed to this change in increase in the share of the state budget spent on health approach, including evidence that old approaches were and a slight increase in the proportion spent on primary not working; pressures to pay more attention to the needs 42 Annexes of women; and, perhaps most important, a deterioration Bank has done little in this area except to provide in economic conditions in 1990-91, which increased the managerial and technical training, which fails to get at government's interest in acquiring foreign assistance. Also the heart of the problem. Complementary changes in significant was a renewed acceptance of decentralization, management practices, work routines, and career which allowed the development of promising state system development policies-including incentives for staff to reform projects. While there is some controversy over get more training-are needed to permanently modify how instrumental the Bank has been in effecting policy the behavior of health workers. This cluster of prob- changes, it was prepared with new kinds of projects when lems needs careful, detailed study. the opportunity arose. Accountability for performance in the Indian Most criticisms of the Bank's program pertain to its health sector is also weak. Performance-based budget- first 20 years; the program as it is now constituted is ing, by linking disbursements to performance, would essentially on the right track. There are, however, better engage implementing agencies in designing and several cross-cutting areas where there is room for managing programs and increasing accountability. additional analysis and improvement. The Bank is promoting performance-based budgeting Referral is arguably the crucial feature of a well- in some new projects, and must carefully study and test functioning health system. Programs promoting a initiatives that use it. referral system must improve the functioning and skills Another cross-cutting issue is the need to better of health workers at the secondary and primary levels integrate NGOs and the private sector, which provide and develop linkages among them. They must also the vast majority of health services, into health sector address the transport and communications problems programming. Efforts to incorporate NGOs and pri- that constrain development of an effective referral vate organizations have had mixed results. These system. Flexibility in programming as well as addi- efforts have been most successful when such organiza- tional study of relations among institutions at different tions work alongside government agencies in comple- levels, both public and private, will be needed to build mentary rather than competitive roles, and where good referral systems in India. government staff are sympathetic and effective manag- Information, education, and communication (IEC) ers. A strategy needs to be developed to involve the are also necessary, since many health problems can private sector that considers the division of labor, only be remedied by changing behavior. While the pricing and subsidy policies, licensing and regulation Bank has encouraged the government to allocate more of private providers and health insurers, and appropri- technical and financial resources to IEC, this area of ate training programs. The Bank has little experience health programming continues to be relatively ne- with these challenges in India, but can help by glected, and government IEC programs are often not examining experience in other countries and in other well implemented. The Bank must continue trying to sectors in India; encouraging the private provision of build IEC into new projects, using a client-oriented services, where appropriate; and encouraging and approach to formulate messages, train outreach work- evaluating experimental programs. ers in interpersonal communication and counseling, Finally, it is clear that India's diverse health carefully research campaigns, and monitor impact. problems, needs, and health sector capabilities require In addition, personnel problems, performance in- multiple approaches. The Bank should therefore con- centives, and accountability continue to be difficult tinue to support decentralization and experimentation challenges in improving service delivery quality. The in order to better meet the needs of individual states. 43 Annexes HEALTH CARE IN MALI: BUILDING ON COMMUNITY INVOLVEMENT The assistance provided by the World Bank to the health sector in Mali has contributed to improved access to rural health services and increased availability of affordable essential drugs, according to a recent Operations Evaluation Department (OED) study. Lessons learned through an early, unsuccess- ful health project in the country helped the Bank and its partners identify key strategies-including establishing a community-managed health sector with services financed through cost recovery, reform- ing the state pharmaceutical agency, and creating a regulatory framework to promote essential generic drugs. Remaining challenges include increasing the utilization of health services, addressing malnutri- tion, alleviating staff shortages in the community sector, and improving the equity of government health expenditures. Background When the Bank first embarked on health sector Mali is one of the poorest countries in the world. Most of operations in Mali 20 years ago, the country was faced the population engages in rain-fed cultivation of subsis- with a centralized health system that was unresponsive tence crops, but the country's climate is harsh and to the population. Government policy was biased unpredictable, with an ever-present threat of drought. toward urban, curative health care, leaving a splin- The adult literacy rate is less than 20 percent, among the tered and inaccessible system for the rural majority of lowest in the world. Education services are poorly the population. A state-owned monopoly controlled the developed, particularly at the primary level. School distribution and cost of drugs, rendering them inacces- enrollment among girls is less than one-third the Sub- sible to all but a few. Although the government in the Saharan average, and up to 80 percent of school-age mid-1980s ended guaranteed employment for medical children in rural areas do not attend primary school. school graduates and opened up the sector to private These indicators are important because poor eco- practice, it fixed fees at rates that most could not nomic conditions and low incomes depress demand for afford, and created a pool of unemployed practitio- health services, and foster conditions that make the ners. An added influence was the prevailing attitude population susceptible to disease and ill health. The toward health care: the population was accustomed to low level of education, particularly among girls, tending to its own needs with the guidance of indig- exacerbates health and nutrition indicators for children enous practices and beliefs. and contributes to low contraceptive use and high The Bank's assistance in addressing these problems fertility rates. has yielded substantial dividends, but stubborn obstacles As in most other Sub-Saharan countries, the main remain. The Bank's policy dialogue and sector analysis health problems are infectious and parasitic diseases, helped the government develop a national health policy, and the leading causes of death are such preventable which established a framework for expanded access to diseases as malaria, measles, tetanus, acute respiratory rural health services and increased availability and infections, and diarrhea. The burden of disease falls affordability of essential generic drugs. At the same time, disproportionately on children and women of reproduc- the government still has not let go of its urban bias, and tive age, and health indicators are worse in rural areas does not devote an appropriate share of expenditures to than in urban centers. Malnutrition is a severe prob- primary health care or the rural sector. The Bank's lem; one-third of children under age 5 and one-fourth of lending and project operations also helped the govern- infants under 6 months of age are stunted. And the ment create a workable community-based health care emerging problem that must now be targeted is the system that has extended coverage to rural areas through spread of the HIV virus, which currently infects 5 a more coordinated network of donors and NGOs, but percent of the population, a figure that is on the rise. overall utilization remains low, and it is difficult for 45 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector clinics to recruit practitioners and provide a clear career Cost Recovery path for health professionals. The Bank, government, and The failure of the Bank's first project and an interna- other partners are well aware of these unresolved issues, tional shift in the development model for health and are now engaged in a comprehensive Sector Invest- prompted a reconsideration of the country's health ment Program (SIP) to address them. sector strategy. The principles of the new strategy were based on models of community participation and cost The Bank's Work in the Mali Health Sector recovery articulated in the Bamako initiative, endorsed The policy dialogue and preparatory work leading to by the African ministers of health in 1987. Under the the approval in 1983 of the Bank's first health project initiative, the Ministry of Health began work with in Mali-the Health Development Project (PDS)- UNICEF in 1989 to develop an action plan for Mali represented the first time that government had partici- that would include community-managed cost recovery pated directly in the preparation of a donor project. to enhance the availability of health care services. The The project and its strategy drew on a 1981 epidemio- first step was to develop a national plan for the logical study that showed that villagers visited govern- decentralization of health planning and management, ment dispensaries only an average of once every two the provision of essential drugs, and community in- years. Three-quarters of the sample population had not volvement in the management and financing of local visited at all during the year preceding the survey. It health centers. also found that the average number of visits was a Based on the PDS experience, Bank staff were function of the distance from the dispensary-critical in convinced that the community-financed health center a country that is predominantly rural. approach could work, assuming that a reliable supply The survey suggested that the population at greatest of low-cost essential drugs could be guaranteed. The risk--children and pregnant women-was not receiving Bank proceeded by developing strong alliances with priority in treatment. People relied primarily on tradi- government policymakers and with UNICEF and other tional healers and herbal medicines for the treatment of donors. The Bank provided advice and technical illnesses. Although modern drugs were sold by the assistance during policy development, but, perhaps government pharmaceutical parastatal, they were over- more significantly, it made the approval of a well- priced and in chronically short supply. Moreover, donor defined national health policy a condition for further agencies and NGOs were attempting to expand services support. This condition gave additional momentum to at the rural level, but lack of coordination, staff shortages, the policy development process, and tied the policy to and inadequate financing for recurrent costs reduced their the new project. impact and sustainability. Although the national policy was developed by a The PDS was designed to enhance service delivery small team of technocrats from the Ministry of Health, by working with the existing government health system the government's strong ties with the Bank led some to construct additional health centers and train new donors to call it a "Bank policy." Yet the policy staff. But significant implementation delays hindered survived the civil unrest and political transitions of the nearly every component. Among other problems, the early 1990s-including four ministers of health and the NGO contracted to construct facilities performed transition to a democratically elected government in poorly. The project piloted community-managed re- 1994-because of the strong support of senior civil volving drug funds to help address chronic drug servants at the Ministry of Health. shortages in rural areas. But because efforts to reform The new policy shifted the organizational model for the pharmaceutical parastatal and reduce drug prices rural service delivery away from the administrative were unsuccessful, drugs provided by village pharma- structures of the state. It introduced the partnering of the cies remained unaffordable for most, and the pharma- public sector with the communities to broaden access to cies could not achieve financial viability. Although a health services, including community management of disappointment for both the government and the Bank, health centers and revenues from cost recovery. The the PDS-together with pilot programs sponsored by policy also called for reform of the pharmaceutical sector, other partners-provided valuable lessons that pre- including the restructuring of the pharmaceutical parast- pared the way for a significant shift in Bank and atal (PPM) to supply only essential and generic drugs, and government strategy in the early 1990s. allowing the private sector to import and sell drugs. 46 Annexes Confronting Essential Issues tion, increased community participation, and weak- The Health, Population, and Rural Water Project ened entrenched opposition to pharmaceutical reform. (PSPHR), initiated in 1991, set out to resolve the The project significantly increased access to health conundrums of rural access, drug cost, and health care facilities and, with pharmaceutical reform, increased the coordination. It helped establish and expand the new supply of affordable drugs. By 1998, nearly 300 new community health system in four districts and in the community health centers had been established, one-third capital, and was cofinanced by several other donors, of which were financed by PSPHR, and the percentage of with parallel financing and technical support from the population living within 15 kilometers of a health UNICEE The involvement of UNICEF-particularly facility increased from 17 percent in 1995 to 39 percent. the presence of a senior UNICEF technical specialist in The community clinics have been largely successful in each region-was crucial in the development of district improving service coverage and client satisfaction, and health plans and the supervision of the new community have been able to use cost recovery to finance most wage clinics. Although civil unrest initially delayed project and non-wage recurrent costs. In addition, community implementation, the 1994 democratic transition gave management committees were established for some estab- additional momentum to health service decentraliza- lished government clinics, although staff continue to be REFORM OF THE PHARMACEUTICAL SECTOR hen the market in drugs, often of it would be politically emerged, creating Bank dubious quality. infeasible to do so, and competitive pressures became The government was that the large drug in the sector. As the involved in Mali's initially reluctant to lose procurements planned for PSPHR began to health sector in the early the profits from PPM, and, the PSPHR could not be expand the community 1980s, it was recognized not surprisingly, PPM was handled immediately by health sector, the that pharmaceutical strongly resistant to any an NGO. demand for generic reform was essential. kind of reform. But by the As part of the drugs increased, The PPM earned profits late 1980s, the failure of National Health Policy, putting additional in spite of inefficient the Bank's PDS and other government, the Bank, pressure on the purchasing policies similar initiatives helped and other partners devel- government to fully because the demand for convince senior officials oped a "contract plan" implement the drugs was strong. But within the Ministry of that would guide the reforms. the drugs that PPM Health that pharmaceuti- pharmaceutical sector By 1993 the PPM imported were expen- cal reform was necessary. toward four reforms- had begun purchasing sive, brand-name The Bank realized private competition in essential generics specialty drugs, often in that the obstacles to retail drug sales, the through international short supply. Over- reform were not simply a introduction of a larger competitive bidding staffed, PPM was rife lack of capacity, and in share of generics into the (ICB). By the mid- with opportunities for the policy dialogue for government and commu- 1990s, drug prices had illicit drug sales and the PSPHR it emphasized nity health sectors, limits fallen to as little as 20 other corruption. Drugs the need to change the on sales mark-ups, and percent of their previ- were sold at a subsidized regulatory framework. the restructuring of PPM, ous levels, and they discount to government Other donors and NGOs including the elimination continued to decline, facilities, but at full had grown sufficiently of 200 positions. even after the 1994 price elsewhere. frustrated with PPM's In the early 1990s, CFA devaluation. Together with illegal inefficiency to call for its liberalization gained imports, this contributed abolition. But Bank staff support as alternative to a thriving black resisted, suggesting that channels for drug imports 47 Investing in Health Development Effectiveness in the Health, Nutrition, and Population Sector employed by the government. About half of all clinics in countries just emerging from famine or civil conflict nationwide continue to be owned and managed by the (see table below). Both stunting and wasting have been Ministry of Health. common since the late 1980s, and may have increased The Bank pressed for the restructuring of PPM, in the past decade (although the data from the 1987 removal of constraints to private competition, and DHS may not be reliable). An analysis of the 1995/96 a shift by the PPM toward the purchase of essential DHS data suggests that more than half of mortality generic drugs as conditions for project approval, among children under age 5 could be attributable to Although progress was initially slow, the eventual malnutrition (15 percent to severe malnutrition, and 42 introduction of international competitive bidding (ICB) percent to mild to moderate malnutrition). It is abun- sharply reduced official prices for drugs, and prices dantly clear that increasing access to health services have continued to decline, even with the CFA devalua- without improving nutrition will not significantly tion in 1994. Generics are now widely available, and improve child health. prices are low enough to allow community health centers to cover recurrent costs from drug sales. Most specialty drugs are now provided by the private sector. WHY PROGRESS IS SO CRITICAL: Finally, the decentralized planning process initi- THE INCREASE IN MALNUTRITION IN MALI ated under the project allowed services to be planned 1987 1998-89 1995-96 and delivered closer to the beneficiaries, and reduced 1987 1998-89 -495-96 the burdens and bottlenecks at the central level. The Stunting (hr/age) 24% 27% 30% .Wasting (wt/ht) 11% 18% 23% PSPHR provided an "umbrella" for the support of .. Undernourished (wt/ht) 31% 43% 40% several donors along with the Bank, and, together with the national policy, helped improve the coordination of donor and government activities. Utilization Must Follow Accessibility The Challenges Ahead Although the Bank and the government have made Since the first community-managed clinics were estab- substantial progress in improving physical access to lished in the mid-1990s-just before the most recent health services in rural areas and increasing the use of Demographic and Health Survey (DHS)-it is too soon to curative and preventive services in the catchment areas correlate the PSPHR with changes in health indicators in of the community health centers, utilization rates Mali. Evidence suggests some modest improvements in remain low. In 1996, Malians visited a government or the past decade, however. Both child and infant mortality community heath center for curative services only 0.16 declined slightly, but still remain high, even in the context times on average. Utilization rates are somewhat of the substantial regional rates. One-fourth of children higher at community-managed health centers, but they still die before age five. The percentage of children fully remain well below the expected average rate of one vaccinated in rural areas increased from 0 to 24 percent, visit per year. largely because rural DTC3 coverage increased. The While the PSPHR also included efforts to percentage of women with at least one prenatal visit strengthen the first-level district referral system, several during pregnancy increased from about one-third in 1987 constraints remain in the referral system between the to half in 1995, and tetanus vaccination coverage community centers and districts, including unavailable increased from 18 to 50 percent. Data also suggest that or very expensive transport; inadequate training for some health practices-including birth attendance by clinic staff on referral protocols; and, sometimes, trained medical personnel and the percentage of children inadequate skills at the district level to deal with treated for diarrhea-have improved. referrals. The prevalence and severity of malnutrition The stubbornly low utilization rates, which have among children provides a notable contrast to the resisted efforts to improve geographic access, are the successes cited. The number of children suffering from product of several factors. Costs remain a deterrent for malnutrition is daunting-23 percent of Mali's children many, despite the reduced prices for essential generic are wasting according to the 1995/96 DHS-and drugs. Physical access is still a problem, particularly prevalence of malnutrition is comparable to that found during the rainy season, because much of the popula- 48 Annexes tion must still walk more than a few kilometers to get social insurance for serious illness, those benefiting to the clinics. Community outreach remains weak- from these services are largely from the upper-income health center staff often wait for users to come to them. brackets. And despite an increase in donor support The family planning component of the PSPHR (spon- torural areas in the past decade, the growing depen- sored by USAID and relying on service delivery by dence on donors is creating its own problems. Most NGOs) was not well integrated with the community strikingly, primary health care and health education health center component. Many potential consumers programs were funded entirely by donors in 1997. still prefer traditional medicine, and attitudes change slowly in a traditional rural society. And, finally, the Next Steps decision to seek care depends on who controls house- With the right framework in place-an accessible, hold resources-often, the man. coordinated service delivery system in rural areas, and the broader, increased availability of essential generic Staff Recruitment Is Essential drugs-the government can now pursue the broader Because government clinics continue to function in goal of providing a responsive, practical health care parallel with community clinics, a major constraint in the system for all. The prospects are sound. The govern- community health sector is the difficulty of attracting and ment has taken the lead role throughout project and retaining qualified health staff. There is currently no job policy preparation, which has enhanced capacity- security, pension benefit, or opportunity for career ad- building at the Ministry of Health and "learning by vancement, and the centers are often located in remote doing" throughout the sector. areas. While staff employed by community health centers In 1998, the government completed a 10-year strate- are often more responsive to community needs, most gic plan for the sector, together with a 5-year investment health professionals express a strong preference for program, which are to guide both government and donor government service. For all these reasons, the vacancy programs. These plans were prepared in conjunction with rate for professional staff is high; some centers have the next phase of Bank support-the Sector Investment remained without a nurse for more than a year. Since the Program (SIP)-which will support the government's salaries are determined by the communities, potential strategy and address the remaining sectorwide issues. staff members are drawn to the urban areas or the better- The SIP will form the framework for several donor off regions. In addition, professional training emphasizes programs, although most donors will continue to manage curative services rather than public health promotion. funds separately. The SIP includes four broad components. The first Government Health Spending Is Still Biased is the construction, renovation, and equipping of Toward Urban Areas community health clinics at the village and district The financing of basic and preventive health services in level, and some renovation and equipping of regional Mali faces several fundamental challenges and con- andnational hospitals. The second comprises manage- straints. The foremost is the ability of the government to ment and technical training of the health sector mobilize resources for health, given a per capita annual workforce. The third calls for expanding health insur- income of US$250. The government has historically ance and cost-sharing mechanisms, and the fourth will generated only about 10 percent of GDP in revenue. entail restructuring government hospitals to improve Government provides less than one-fifth of total cost recovery, cost-effectiveness, and technical effi- resources for health-far less than households (at 50 ciency. The program also seeks to improve district percent) or external aid (25 percent). But even the health management, including referral services; money that the government does allocate to health strengthen management information systems; increase spending is not being used efficiently or with equity. community involvement in health centers, and develop The majority of the government's limited resources are management modules for hospitals. spent for urban-based tertiary care and central admin- istration. Public subsidies remain highest, and cost Lessons to Guide Future Initiatives recovery lowest, at urban tertiary facilities; the oppo- The Bank's support has progressed over the past 20 years site obtains at rural facilities. Although public subsidies from a (unsuccessful) health pilot project in a single for higher-level care could be considered a form of region, to a nationwide "umbrella project" that embodied 49 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector a new national health policy and was cofinanced by * The cost-recovery mechanism of community- several donors, to an SIP under a national strategy that is based facilities for curative care does not create to encompass all investment in the sector. This progres- incentives for locally based health promotion sion has corresponded with increased government will- activities. Strengthening these activities will be ingness to establish sectoral strategies and priorities. necessary to achieve further improvements in Although the Bank strongly encouraged govern- health and nutritional status. ment to take a sectorwide approach, government has * Incorporating health sector concerns effectively full ownership of the program. Among donors, the into macroeconomic and budgetary dialogue will Bank has taken a leadership role, but has worked to continue to require regular communication be- bring in other partners. Government officials have tween Bank macroeconomic and sector specialists, reported that the Bank has been effective in linking its and sufficient sector work to match priorities with support to key sector reforms and issues. Some donor- sector budgets and staffing patterns. partners have expressed concerns, however, that the * The community-based agenda must move be- policy development and SIP appraisal processes were yond access-by targeting continued cost barri- unduly influenced by the Bank's timetable and agenda. ers, inadequate outreach, and preferences for Given the Bank's influence in Mali's health sector, it traditional medicine or self-treatment. must be cautious and collaborative to ensure that it * Establishing a community sector outside govern- does not dominate the policy process. The Bank must ment may have made providers more responsive also engage in more rigorous monitoring and evalua- to community centers, but uncertain job security tion, particularly when new service delivery mecha- and career paths must be addressed if commu- nisms are piloted for nationwide replication. nity-based facilities are to attract and retain a Other lessons from experience in Mali may also be full cadre of health professionals. helpful to the Bank's new SIP project work: * Curative services alone will not improve health outcomes. They must be combined with appropri- ate health education and outreach programs, fam- ily planning promotion, and nutritional surveil- lance and intervention, and they must be integrated effectively with those offered by NGOs. 50 Annexes MEETING THE HEALTH CARE CHALLENGE IN ZIMBABWE The World Bank has usually "done the right thing" in the Zimbabwe health sector, but has not always "done things right," according to a recent Operations Evaluation Department (OED) study. Bank policy advice and project support have been well-crafted to address Zimbabwe's epidemiological profile and health sector needs, and have helped integrate family planning into health services, improve service quality, and increase facility deliveries, inpatient attendance, and contraceptive prevalence, among other benefits. But programs have often encountered difficulties in implementation. As Zimbabwe confronts the combined challenges of severe financial constraints and the growing AIDS epidemic, a reexamination of Bank work in the sector is appropriate in preparation for the next phase of Bank support. Background increasing. The implications for the nation's health Over the past 15 years, the World Bank has provided system, economy, and society are staggering. This is policy advice and project support to health, nutrition, and particularly tragic, because although AIDS cannot be population (HNP) programs in Zimbabwe, and in 1991 cured, it can be prevented through modifications in sexual became involved in the design and support of Zimbabwe's practices. Yet government has not offered leadership in Economic Structural Adjustment Program (ESAP). The behavior change, the strategy is not focusing on the most Bank's initial effort in the sector-a 1986 loan to support cost-effective approach (working with those most likely to improvement in the quality and availability of health transmit the virus), and several high-transmission areas in services in 8 target districts, including expansion of the country have not yet benefited from intervention. infrastructure and in-service training for nurses-was expanded in 1991 to include an additional 16 districts. A Health and the Health System 1993 loan funded the acquisition of drugs to treat sexually In the decade following independence in 1980, Zimba- transmitted infections (STIs), as well as medical supplies bwe experienced some of the most rapid improvements and laboratory equipment. in HNP indicators in all of Sub-Saharan Africa. Infant Bank project support and policy advice have mortality declined from 90 per thousand in 1980 to 53 proven valuable to the Zimbabwe health sector, but the per thousand in 1988. Household incomes increased impact on health system performance and health only modestly during this period, however, suggesting outcomes has been undermined by economic stagnation that the government's strong emphasis on basic health and a devastating AIDS epidemic. Flaws in the design and family planning services, health education, and of ESAP contributed to strains on the health sector, community outreach, bolstered by a strong focus on particularly with regard to civil service reform and prevention, were responsible for the improvements. health sector staffing. Yet the government's failure to In the 1990s, health and health service indicators control the budget deficit-and its financing of that stagnated or declined under the combined burdens of debt by borrowing domestically at high interest rates- AIDS, economic crisis, and drought, although fertility has led to escalating interest payments that now equal continued to decline. After a decade of decline, both one-fourth of all government revenue. This mounting infant and adult mortality are increasing, as are debt threatens to lead Zimbabwe into deeper economic opportunistic infections such as tuberculosis. Although crisis, and without a concerted effort at deficit reduc- economic crisis may have a role in weakening the tion, it will further undermine the health sector. health system, these increases in mortality are prima- Zimbabwe is faced with the world's most severe AIDS rily attributable to AIDS (see figure 1). Although HIV/ epidemic. According to UNAIDS, 26 percent of the adult AIDS is best addressed through prevention and behav- population in Zimbabwe is infected with the HIV virus ior change, declining per capita health spending and that causes AIDS, and the percentage may still be growing demands for curative care have weakened the 51 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector preventive focus that characterized the successful pro- FIGURE 1. DEATHS OF CHILDREN UNDER 5 grams of the 1980s. 300,000 - Impacts 250,000 - ESAP and Health 200,000 - Zimbabwe's 1991 ESAP liberalized the economy but 150,000 - failed to control the government's budget deficit, which 100,000 - has averaged nearly 10 percent of GDP annually since independence. Economic liberalization without deficit 50,000 reduction contributed to economic stagnation and 0 limited job creation. Higher costs for food and social 1986-90 1991-95 1996-00 2001-05 services, combined with declining formal sector wages O AIDS Deaths 0 Total Deaths and the lingering effects of severe drought in 1991-92, Source: NACP 1997. have left many of the poor worse-off than before adjustment began. Although both the government and mobilizing additional resources for health, improving the Bank tried to protect spending for health and quality and efficiency, and protecting the poor. The education, large budget deficits fueled inflation and led Bank persuaded the MOH to increase user fees in the to growing interest payments, which contributed to early 1990s, but the Ministry of Finance (MOF) did not declines in real health spending and real wages for permit fee retention at health facilities until late 1997. health workers. These interest payments-three-quar- Because fees were not retained, the quality of care did ters of which are for domestic debt-now represent over not improve. The Bank encouraged adoption of an three times the government spending on health (see exemption system to protect the poor from increased figure 2). Program design failed to give priority to cost recovery in the social sectors, but shortcomings in deficit reduction (for example, by cutting taxes), but the government regularly missed deficit targets by imple- FIGURE 2. GOVERNMENT EXPENDITURE menting unbudgeted programs (such as the recent ALLOCATIONS, 1989/90 AND 1996/97 payments to war veterans). Although health workers were protected from Expenditure 1989/90: Z$6,446 m retrenchments, downsizing of Ministry of Health Health (MOH) administrative and maintenance staff reduced Other 7% Education efficiency and added to morale problems without 49% generating significant savings. Although macroeconomic policies and perfor- mance have had a greater influence on the health sector than Bank project lending, the Bank has not effectively Interest on Debt linked health sector investments and strategies to Defense 15% macroeconomic dialogue, particularly in health staff- 13% ing and civil service reform. To prevent further deterio- ration in the public health sector, government must Expenditure 1996/97: Z$6,247 m (1990 Z$) give priority to reducing the budget deficit and restruc- turing debt service. In the medium-term, the budget for Health health will remain constrained, and it will be necessary Other 7% to focus on increasing efficiency and making the 41% Education difficult choices necessary to fund priorities such as AIDS prevention and basic services for the poor. Health Financing and Cost Recovery Bank work in health financing led to increased cost- Interest on Debt Defense 25% recovery efforts, but it has had limited success in 8% 52 An n e xe s design and implementation of the system meant that the underserved districts, such as Tsholotsho, maternal program reached only a small percentage of the deliveries increased markedly following facility intended beneficiaries. Following fee increases, atten- completion, while in others, deliveries stagnated and dance for some preventive services shifted from hospi- inpatient attendance fell, usually because the upgraded tals to clinics, suggesting improved efficiency, but government hospital was near a mission hospital that outpatient attendance by the poor declined in response was preferred by many patients. Domestic political to the fee increases, as prices increased and quality influences and Bank insistence on upgrading existing declined. In 1995, the government-with Bank agree- facilities contributed to inappropriate site selection. ment-abolished fees at rural health facilities, stating The Second Family Health Project (FHP2) im- that the revenue collected did not justify administrative proved facility design and site selection and built 16 costs. Service quality rather than cost is a major district hospitals for the cost of the original 8. For concern for the rural poor, but cost has become a major FHP2, the Bank placed an architect within the MOH to barrier for the urban poor; in a recent survey 40 percent ensure maximum efficiency in facility design. Interna- of the urban poor gave "too expensive" as the reason tional competitive bidding (ICB) yielded construction for not seeking treatment when ill. costs that were 40 percent below government estimates, Ironically, total cost recovery declined-from 3 to and facilities were completed on time, and below 2 percent of the MOH budget-primarily because budget, in 1998. But the severe shortage of health government made little progress in improving hospital personnel is making it difficult to staff the new facilities billing. The government continues to lose millions of and threatens to undermine their impact. Once con- dollars through inadequate billing of medical aid struction began, it was impossible-contractually or societies (health insurance companies) for expenses of politically-to delay the projects or to reduce the insured patients in government central hospitals. This number of hospitals pending resolution of staff short- experience suggests that the Bank must complement its ages. This emphasizes the importance of flexibility in broad policy recommendations with detailed dialogue project design and of focusing on service quality and on implementation, give greater attention to the institu- staffing issues rather than facility construction in the tional context, and coordinate sector and macroeco- next phase of Bank support. nomic dialogue. Now that facilities are allowed to The Bank has been well-positioned and effective in retain fees, hospital cost recovery has increased sub- promoting the integration of key HNP interventions. stantially, but billing remains inadequate. In the com- The in-service nurse training supported by the Family ing years, local districts and communities may wish to Health Projects improved staff skills and contributed to experiment with community-managed health centers, the integration of nutrition and family planning into in which fee revenues are used to purchase drugs and health services. The percentage of women obtaining improve service quality. contraceptives in health facilities has increased since the late 1980s, which is partly attributable to Bank- Strengthening Health Service Delivery sponsored training in family planning. Project efforts to Bank support for expanding district infrastructure and improve the quality of maternal delivery services in staff training through the First Family Health Project rural areas was undermined, however, by high turnover (1986-91) improved service quality and contributed to of trained nurse-midwives, who were often promoted or increased facility deliveries, inpatient attendance, and hired by the private sector soon after training. contraceptive prevalence, but has had no measurable impact on outpatient attendance or disease patterns. Health Work Force Outpatient attendance in project districts actually de- The current staff shortages were created by recent clined following facility completion in 1991, coinciding political decisions by government (abolishing training with drought, increased fee enforcement, and drug short- for state-certified nurses and firing striking health ages, which suggests that improved infrastructure and workers), high turnover of health staff, and the absence training alone will not improve service quality or access. of effective manpower planning. Erosion of real wages The impact of upgraded facilities on maternal in the public sector and increasing workloads have attendance varied considerably, depending on the contributed to turnover and low morale, as has rapid appropriateness of site selection. In genuinely growth of private health care-primarily serving urban 53 Investing in Health; Development Effectiveness in the Health, Nutrition, and Population Sector populations-in the 1990s. Although Bank staff peri- sion and management attention by both government and odically raised concerns regarding health staffing, they Bank staff contributed to a recovery in STI drug availabil- were not effective in addressing the institutional con- ity to 87 percent in 1998. straints to action. The Bank has supported technical Projects with a major pharmaceutical component assistance for work force planning and discussed health require up-front training for both government and Bank staffing during supervision missions, but did not spon- staff-with periodic follow-up training-to avoid sor sector work (research) on health staffing issues until bottlenecks that could interrupt drug availability. 1998. Responsibility for health personnel is divided Bank procurement procedures could be streamlined to among several ministries (MOH, MOF, and the Public reduce the burden on borrowers, but the cost savings Service Commission), and the Bank did not use its achieved through ICB are essential to ensure drug leverage at the macroeconomic level to elevate and add availability in the face of tight budgets and growing urgency to the dialogue. The MOH, Bank, and donors demand for drugs. have made health staffing a priority for future support, The AIDS epidemic is the most serious problem but all parties should ensure that remediation is facing the health system and, along with the deficit, the coordinated. economy as a whole. But treating STIs is resource To address staff shortages, government will need to intensive, and unless done in conjunction with a establish economic stability (to reduce inflation and concerted campaign to change sexual practices, it is prevent further budgetary declines) and develop a unlikely to have a significant impact on AIDS. Bank- comprehensive health staffing strategy. The challenge funded research has helped raise awareness in Zimba- is that budget constraints will not permit significant bwe regarding the seriousness of the AIDS epidemic, increases in personnel expenditures. Designing and and the Bank has cosponsored innovative community implementing the strategy will require negotiation AIDS prevention initiatives. The government's re- among the stakeholders, including the MOH, MOF, sponse, however, has not been commensurate with the Public Service Commission, and health professionals. scale of the epidemic, which may claim 1 million lives The Bank could assist by providing analysis and in the next decade. Experience elsewhere has shown facilitating consensus among stakeholders. that strong leadership and political commitment can halt the growth of the epidemic and save hundreds of AIDS and STIs thousands of lives. Recent pilot experiments in Zimba- The presence of an STI considerably increases the bwe show that HIV transmission rates among high-risk likelihood that an individual will contract the HIV virus. groups can be reduced by 30 percent or more in just a Treating STIs can thus be one component of an AIDS few years. The government has developed a prevention strategy. In the early 1990s, declines in the multisectoral strategic plan to combat AIDS; the government's drug budget and growing demand for challenge now is to implement it. antibiotics led to critical drug shortages. Through the STI Project, the Bank has financed half of the government's Conclusions drug budget for the past five years. Bank support for the The Bank can increase its effectiveness in the sector by purchase of STI drugs closed a major financing gap, fitting program design to accommodate institutional contributed to significant cost savings in drug procure- and political constraints and to take advantage of ment, and initially increased drug availability. Bottle- existing capacities. It can also build on its record of necks later emerged that reduced drug availability, effectiveness in promoting integration of programs and undermining program effectiveness. STI drug availability cooperation among government ministries to address increased to 89 percent in the first two years of the project, the increasingly complex challenges confronting Zim- but fell to 73 percent in 1996, primarily because of babwe. The approach would be particularly useful in reversals of government contract awards by Bank pro- establishing a comprehensive health staffing strategy. curement specialists and delays in registering drugs Because budget constraints will not permit significant purchased through ICB. Government staff did not initially increases in personnel expenditures, the design and receive adequate training in Bank procurement proce- implementation of the strategy will require negotiation dures, and Bank supervision of procurement was initially among the many stakeholders. inadequate to resolve the bottlenecks. Increased supervi- To prevent further deterioration in the public 54 Annexes health sector, government must give priority to reduc- The challenges of the next decade are considerable, ing the budget deficit and restructuring debt service. but Zimbabwe has the tools, the experience, the Until this has been accomplished, maximizing effi- innovative spirit, and the support of partners-includ- ciency and redistributing available funding can do ing the Bank-needed to meet them. Past successes much to achieve greater balance and effectiveness in clearly demonstrate that once the decision is made to service provision. Government must also take immedi- take on a problem, remarkable progress can be ate steps to give priority to AIDS prevention-particu- realized. larly to substantially increase the public and private resources devoted to behavior change-and mount an effective intersectoral response to the epidemic. 55 Annexes ANNEX B: PROJECT OUTCOME-SOURCES OF INFORMATION The inputs to this study included a statistical model influence in the health sector). that was used to test the impact of factors related to Quality of Bank project appraisal, as rated by country context, project design, and implementation on OED at project completion, is most strongly associated project outcome.' Another tool was an analysis of with satisfactory outcomes, closely followed by Bank lessons cited in Implementation Completion Reports supervision quality. Although quality seems to matter (ICRs). This annex describes the results of those more, the number of staffweeks spent on preparation analyses. and supervision may contribute to improved outcome, even controlling for quality. Quality of institutional Modeling HNP Project Performance analysis during project preparation, including assess- Consistent with the OED Annual Review of Develop- ment of ownership and demand for health services, was ment Effectiveness (OED 1998), the model included significantly correlated with project outcomes, while three categories of variables: borrower performance, economic analysis was not. A review of individual country context, and Bank performance. projects found that those with strong institutional Borrower performance, as rated by OED at project analysis also tended to have good economic analysis, completion, is the most important determinant of project and both contributed to success. Those that focused on outcome. Borrower implementation performance appears economic justifications without consideration of the to be most important element, followed by appraisal institutional context were more prone to failure. performance and compliance with conditions. Country context, particularly the quality of bor- Lessons from OED Review of ICRs rower institutions-including the level of corruption- As an exercise independent of the econometric model, is strongly correlated with project outcome. Macroeco- OED reviewed ICRs for 80 projects completed through nomic policy, represented by an indicator for the FY97, and tabulated the most frequently cited lessons. openness of the economy, was not significantly corre- Table A-1 lists the percentage of all projects that cited a lated with project outcomes (although it is significant given lesson, for both satisfactory and unsatisfactory for the entire Bank portfolio, and experience suggests projects. The data should be treated with some caution, that economic performance can also be a significant however. Terms such as capacity and complexity have TABLE 1: LESSONS CITED BY TASK MANAGERS IN ICRS Percentage of ICRs citing lesson Le-on All Projects Satisfaclor) . Unsatisfactor Prolect Dmeiihitot Ne.d herner ank .upen r ot api.t 44 26 69 36 24 52 Nccd imci ~c 'l.ar ir1 1 0 ii111111-0 - 34 19 5 Nv \d rni .i ..r 11 1[K 302 1 41 Mcdn runc hi Iik nt ..crunrirpu tund 39 - 41 IiI1i,u1W 01fill n FI. .iL32 Ir bcl . dil . ri c J b . , ur enient 23 26 AlannUIhIg .11nd r..eltlrnon Ineedquar anientIO n o Mpea 54 48 62 VrhIridcI li V rri01 iiuroNc in riin ~ i4 1 29l 3.9 Source: OED data. 57 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector different meanings to different authors, and there may be a bias in unsuccessful projects to attribute failure to factors such as low capacity or excessive complexity. Still, the findings are illustrative, particularly of the extent to which similar problems have recurred in the portfolio. 58 Annexes ANNEX C: LEDGER OF DED RECOMMENDATIONS AND MANAGEMENT RESPONSE OED Recommendation Management Response Increase Strategic Selectivity Increasing strategic selectivity is an important part of By the end of calendar year 1998, Bank management the sector/region management's continuing responsi- and the HNP Sector Board, in consultation with staff bilities, and is already part of their regular discussions and partners, should establish priorities and guidelines on work programming. Management has been work- for staffing, lending, and administrative resources ing for some time to allocate resources in line with the (including project supervision and ESW) in light of priorities identified in the HNP Sector Strategy Paper. overall objectives in the 1997 sector strategy. Particu- Additional focus was provided by the Sector Board in lar attention should be given to: (a) how the issues the Fall of 1998 with the agreement to concentrate on raised in this OED assessment will be addressed, five priority public health areas. OED's proposal for including budgetary implications; (b) how the sector even greater attention to selectivity is timely and plans to focus activities and budgets to sustain quality welcome. The Sector Board has prepared an Action in light of staff overprogramming and pending de- Plan with appropriate process and outcome indicators clines in administrative budgets under the Strategic to address the priority issues raised by OED. The Compact; and (c) how country directors will be Sector Board will address priorities for staffing, brought on board with the recommendations and lending, and administrative resources in a paper to be guidelines. prepared in FY00. Enhance Quality Assurance and Results-Orientation Management recognizes quality assurance as a high The HNP Sector Board, in conjunction with regional priority. The HNP Sector Board recently approved a sector leaders, should strengthen its role in monitoring program to strengthen its activities in portfolio quality and strengthening portfolio quality and results-orien- monitoring and enhancement. The program includes tation. Measures would include: (a) establishing a regular Sector Board discussions of portfolio quality, regular system of reviewing portfolio quality indica- enhancing direct support for task teams, experiment- tors, including identifying priorities for remedial ing with different types of support panels, and staff actions; (b) establishing supportive mechanisms to training. The sector is taking this subject very help task teams improve performance; (c) in conjunc- seriously and has already started implementation of tion with Bank management, identifying steps to several of these measures. The Sector Board will strengthen routine quality assurance mechanisms; and monitor progress in accordance with Strategic Com- (d) in annual reports on the HNP sector strategy, pact and OED indicators set out in the above- increasingly present evidence regarding progress to- mentioned Action Plan, and will review evidence of ward sector goals. progress toward sector priorities in its Annual Strat- egy Progress reports. The Bank should seek ways to strengthen the incen- Management endorses the need to develop and tives for monitoring, evaluation, and results-orienta- Implement more effective systems of, and capacity tion within client countries through: (a) promoting for, HNP project monitoring and evaluation in client wider experimentation with and use of performance- countries. However, the recommendations on moni- based budgeting systems in its lending and policy toring and evaluation, taken together, would involve dialogue, particularly in the Comprehensive Develop- a large-scale effort, with significant resource implica- ment Framework (CDF) pilot countries; (b) by the end tions. This recommendation on strengthening incen- of FY00, producing a preliminary "lessons learned" tives for monitoring and evaluation and greater paper on experience in performance-based budgeting results orientation is closely linked to the recommen- in HNP, including implications for the CDF, in dations to enhance country monitoring and evalua- conjunction with partner organizations; and (c) the tion capacity (discussed two items below). Beyond 59 Investing in Health: Developmuent Effectiveness in the Health, Nutrition, and Population Sector Bank should increasingly engage independent evalua- what can be done to strengthen incentives for monitor- tive organizations, preferably based in borrower ing and evaluation through individual operations and countries or regions, to provide periodic assessments of in CDF countries in our continuing HNP work Bank-financed activities. program, work in this area, including performance- based budgeting, will be phased in systematically, later in the sector's Action Plan. To strengthen Bank performance in monitoring and Agreed. Monitoring and evaluation is weak in the evaluation, management should: (a) identify a core HNP sector and should improve. Management sees as group of HNP staff and consultants with experience an immediate priority the development of effective- implementing HNP monitoring and evaluation who and above all practical-monitoring and evaluation could be available to assist other staff during project tools and good practice information for the HNP design and supervision; (b) develop a "good practices" sector, and their dissemination through training, both manual of M&E design and use for decisionmaking, of Bank staff and counterparts. Implementation is both at the project and systemic levels, including scheduled to start in the coming financial year. lessons from partner organizations; (c) in collabora- Nonetheless, because of the intrinsically complex and tion with the World Bank Institute, develop M&E case long-term nature of the HNP sector, this is a difficult studies and training modules; (d) periodically give subject, and one in which making progress will take recognition to task teams that demonstrate measurable significant continuing efforts. results from Bank-supported activities; and (e) in parallel with the CDF pilots, report by the end of FY00 on how Bank business practices and procedures could be modified to allow greater results-orientation in Bank lending, and to increase internal incentives for monitoring and reporting on results. To strengthen borrower capacity and incentives for This is closely linked to the recommendations to monitoring and evaluation in the HNP sector, sector enhance borrower monitoring and evaluation ca- strategies and project designs should include: (a) pacity (discussed two items above), and similar assessments of borrower incentives and capacity for considerations apply. The sector will do what it can monitoring and evaluation and (b) where appropri- through individual operations to give attention to ate, recommendations and measures to better enable the issue in project designs and sector strategies, borrowers to monitor and report on results, includ- and to support borrower capacity development; ing strengthening health information and vital priority will be given to CDF countries and other registration systems, and a description of the role of countries with special needs. But systematically the Bank relative to other partners in this process. addressing this issue will need to await the comple- tion of higher-priority work as identified in the Action Plan. For implementation, this topic will be combined with its companion item above. Intensify Learning from Lending and Nonlending Management agrees that strengthening institutional Services effectiveness is a high priority for better overall To strengthen the institutional development effec- sector and project performance. However, it is also tiveness of the Bank's work in HNP, management an area where the HNP sector literature is weak, should: (a) in coordination with PSM and other and where there are few well-established, prag- internal and external partners, develop appropriate matic guidelines, and little good practice to draw 60 Anne xes tools, guidelines, and training programs for institu- upon. OED's recommendations will be taken up tional and stakeholder analysis in HNP, both for progressively as we strengthen capacity in this area, targeted interventions and systemic reforms; (b) and the Sector Board will ensure that knowledge clarify the requirements for institutional analysis in and practice gains are incorporated in project project appraisal documents; (c) establish a core of design and appraisal documents. HNP staff and consultants with experience in institutional design and stakeholder analysis, who could be available to assist other staff. To strengthen the analytic base for Bank advice Management accepts the need to examine whether and lending: (a) management should increase fund- ESW is adequately underpinning lending opera- ing for HNP sector work; (b) the Sector Board tions. Good ESW is the foundation of sound project should sponsor operational research and provide design. A clearer picture is needed of the amount good practice guidelines on improving effectiveness and effectiveness of ESW, and whether analytical and efficiency of ESW and other Bank advisory and work is being undertaken through non-ESW chan- analytic services; and (c) management should shift nels. Management believes, therefore, that without some of the ESW budgets from country departments further study it would not be appropriate to make to regional technical managers to encourage re- specific budget allocations to HNP ESW This topic gional research on priority issues. will be taken up fully in FY01. To ncrease che Bank%s abilirN to su[rain 3 continued A1a?1,gL'mC1t aIeCS rat p.-ma% cq HNP stati to presence in borrower country health policy debates, the field normally strengthens policy dialogue and and to develop long-term partnerships with various links with stakeholders. The numbers have in- stakeholders in client countries: (a) the Bank should creased substantially over the last two years. continue its current efforts to base sector specialists in However, much depends on the individual country countries or regions, with a clear mandate for circumstances, the degree of HNP involvement in collaborative policy dialogue with stakeholders inside that country, and on regional policy. It is and outside government; and (b) for projects and management's view that the HNP sector has a reform programs requiring intensive stakeholder con- creditable record in reaching out to stakeholders sultation, country directors and sector managers during project preparation and supervision. The should ensure that these time requirements are re- Sector Board will look again at this topic in FY02. flected in project preparation and supervision budgets. To strengthen the Bank's effectiveness in health Health promotion is a somewhat neglected area promotion and in addressing the intersectoral di- throughout the sector, not just in Bank work. When mensions of health: (a) the Bank's HD Network and done well, it can be highly cost-effective. As suggested regional vice presidents should identify several key by OED, staff need increased technical support in this areas for improving intersectoral collaboration area, and this will be taken up by the Sector Board as within the Bank, including coordination of macro- a medium-term priority. On the second issue, man- economic and sectoral dialogue on social sector agement accepts that important determinants of work force issues, HIV/AIDS prevention and miti- health lie outside the health sector. Intersectoral gation, and key health promotion activities (defined collaboration is important, but notoriously difficult on a regional basis); and (b) the HNP network for any sector to achieve. Limiting the scope of work should strengthen staff skills in health promotion to a few key areas will be essential to achieving 61 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector and establish "good practice" guidelines and ex- concrete results. Work in both these areas will be amples for task managers. taken up as a medium-term priority. The Bank should strengthen work with HNP devel- Management accepts the spirit of this recommenda- opment partners (such as WHO, UNICEF, bilateral tion, but not as expressed. A great deal of work is donors) in several key areas, including strengthen- currently going into partnerships with many orga- ing HNP M&E systems and incentives and assessing nizations on a broad range of issues. Management progress and strategies in the current generation of will continue these partnerships and seek to health sector reforms. strengthen them, but not as a separate "partner- ship" activity. 62 Annexes ANNEX 0: REPORT FROM THE COMMITTEE ON DEVELOPMENT EFFECTIVENESS (CODE) Development Effectiveness in Health, Nutrition, and strengthening borrower capacity for monitoring and Population: Lessons from World Bank Experience evaluation is a long-term challenge, but it is a priority The Committee met on June 9, 1999, to discuss the that needs to be worked on now. Management reported report Development Effectiveness in Health, Nutrition, that several actions were already under way to improve and Population: Lessons from World Bank Experience quality. The HNP Board had already appointed a lead (SecM99-322), together with the draft management person on quality assurance, hired implementation response (CODE99-40). The Committee thanked OED specialists, established peer review panels, and training for an excellent report and management for a candid was underway in the Regions. An action plan was and constructive response, and appreciated the collabo- being developed with outcome indicators to track ration between OED, the Sector Board, and the progress in quality enhancement. Regions. The Committee recognized the need for a Analytical Work on Institution Building and other phased implementation of OED's recommendations but Cross-Sectoral Linkages. The Committee stressed the stressed the importance of beginning the implementa- fundamental importance of strengthening the analyti- tion process immediately. In that regard, the Commit- cal base in this sector, particularly with respect to tee welcomed management's action plan to address institution building and public sector reform, and the these recommendations. linkages between health and other sectors. More The OED review assesses the Bank's involvement analytical work is also needed on the impact of cost in the health, nutrition, and population (HNP) sector recovery on the poor, and the role of the private sector over the last 30 years, from early population projects in in the financing of health services. the 1970s, to direct lending for primary health care in Bank Organizational Issues. The Committee noted the 1980s, to a focus on health financing and system that there are a number of positive lessons emerging from reform in the 1990s. Today the World Bank is the single the close collaboration between OED and the HD largest external financier in the health sector in network in the HNP sector, and from how the Sector developing countries. The study found that although 79 Strategy has been developed and is being implemented, percent of HNP projects completed in FY97/98 had and that other networks could learn from this experience. satisfactory outcomes, some major weaknesses persist. Next Steps. The Committee looks forward to The report confirmed that the Bank overall has been effective dissemination of this report and successful in expanding health systems, providing inputs, management's action plan. improving treatment within the medical care system, and promoting dialogue on major policy issues such as the financing of health systems and HIV/AIDS. The Bank has not been as effective in improving service quality, efficiency, institutional development, and monitoring and evaluation systems, and addressing determinants of poor health that lie outside of the health system, such as how to Jan Piercy, Chairperson promote behavioral change. July 23, 1999 The main issues discussed by the Committee included: Strategic Selectivity. The Committee recognized the magnitude of the problems in the sector, but also agreed with the emphasis on "doing better, not more" and the importance of partnerships with other actors in the sector to maximize impact. Quality Assurance. The Committee stressed the critical importance of enhancing monitoring and evalu- ation in the sector, both within the Bank and in borrowing countries. Members acknowledged that 63 ENDNOTES Chapter 1 wise) interventions, and both developed and developing countries 1. The first report in the series- Evaluating Health Projects: are still in the early stages of developing systems and indicators to Lessons from the Literature (World Bank Discussion Paper No. monitor the efficiency, efficacy, and client responsiveness of 356)-surveyed the literature on the evaluation of health projects health facilities or systems. Yet plausible conclusions regarding and presented a design for this study. The second stage of the project effectiveness, system performance, and HNP outcomes are study, "Lessons from Experience in HNP" (Report No. 18642), possible using existing systems and methodologies (McPake and describes the evolution and performance of the HNP lending Kutzin 1997). portfolio, based on a review of the 224 HNP projects approved 7. For further discussion of health information systems in between FY70 and FY97. In addition, OED conducted a series of developing countries, see Sandford and others 1992. four country sector evaluations, which examined the history and 8. OED used a crude measure of complexity, which com- performance of HNP nonlending and lending activities in Brazil bined the total number of project components with the number of (Report No. 18142), India (Report No. 19537), Mali (Report No. government ministries directly involved, to assess a complexity 18112), and Zimbabwe (Report No. 18141). "score" for each project. The measure of institutional quality is taken from the World Development Report 1997. Chapter 2 9. The HNP sector has recognized this problem, and has 1. The Bank did not want to "dilute" its family planning initiated a number of research and operational activities to efforts, and was initially uncertain of its role in international improve the poverty focus of the Bank's work. health, particularly relative to that of the World Health Organi- 10. The path and opportunities for policy change depend on zation (WHO). several factors, including whether it occurs in a time of political 2. Project designs included a broadly standard package of crisis or transition, the political and technical skills of key inputs: financing for new or upgraded facilities (usually about technocrats and political leaders, and the characteristics of the two-thirds of the Bank financing); training for health or family reform itself, which indicate the type of conflict and opposition planning staff; technical assistance and training for the central generated (Grindle and Thomas 1991). ministry; vehicles and equipment; and, often, an information, 11. For an example of a computer-based instrument for education, and communication (IEC) component. Projects some- stakeholder analysis of health sector reforms, see Reich and times also financed drugs or a small nutrition component. Cooper 1996. 12. Parallel financing from UNICEF in Mali funded techni- Chapter 3 cal specialists to train district health officers in community . e .o mobilization. This proved successful, although NGOs might have 1. The discussion in this chapter draws on reviews of project moeficntyadfetvlyobizdomutes completion documents, OED assessments of project design qual- ity, an econometric model of project outcome (see Annex B), the 13. In Zimbabwe, for example, the Bank helped strengthen OED country case studies, and data from the Bank's Quality the hand of the Ministry of Health in its dealings with the Assurance Group (QAG). Ministry of Construction, both by placing an architect in the Assurance G roup (G Ministry of Health and by convening regular meetings during 2. ED ate a rojctas satsfator" i i ahedmot supervision visits to jointly solve implementation problems. of its stated objectives. The poor project performance in the late 1980s probably reflects weak institutional design in the first 14. This section draws primarily on the results of OED- generation of Bank health projects and shortfalls in funds for sponsored GroupWare sessions with Bank HNP staff, together recurrent costs as a result of economic crisis in many borrower with analysis of trends in the HNP portfolio, QAG findings, and countries. the OED country case studies. 3. See Annex B for a description of the model and detailed 15. In response to the findings and recommendations of this results. study, the Bank's HNP Sector Board has given priority to developing 4. or adapting institutional assessment tools for HNP staff (see Annex 4. .inBril for exap, a number of H proe C). A number of tools and methodologies already exist, however, or languished in the 1980Os, but the overall portfolio improved. are in the process of being developed. These include a toolkit for considerably in the 1990s when the government, with strong assessing the capacity of an education ministry to implement a encouragement from the Bank, agreed to make a concerted effort project (Orbach 1999), which could be adapted to the health sector. to improve implementation. In India, Bank efforts to encourage QDdvlpdacmue-ae raiainlcpct ef chne.ntegvrmn' NPsrtg aeltl eda OED developed a computer-based organizational capacity self- changes in the government's HNP strategy made little headway assessment toolkit, focused on the planning, management, and unmmiteditil-fllon va ainl criss inproah 19 the goern t delivery of basic health services. The questionnaire for this toolkit, committed itself to reevaluating its approach to the sector. by Michael Bernhardt, is annexed in Stout and others 1997. The 5. Institutional quality encompasses the formal and infor- Bank's HNP Network is in the process of developing a toolkit for mal rules governing the behavior of individuals and organizations assessing private sector health service delivery. The Bank's Poverty in a society, not merely the capacity of particular individuals or Reduction and Economic Management Network (PREM) is devel- organizational units (World Bank 1997d). oping general tools for assessing governance, budgetary, and service 6. Methodological problems can make it difficult to at- delivery systems, which could be further adapted to the social tribute changes in outcomes to specific (Bank-financed or other- sectors. Outside the Bank, the International Development Research 65 Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Center has developed a comprehensive toolkit for organizational Annex B self-assessment (Lusthaus and others 1999), and the U.S. Agency for 1. The study used a probit model, with OED outcome ratings International Development's MEASURE Project is developing an from 80 projects completed by FY97 as the dependent variable. institutional and organizational assessment methodology for the Independent variables were drawn from several sources, including health sector. ex post OED ratings for Bank and borrower performance (ap- praisal and implementation); institutional quality ratings from Chapter 4 the World Development Report 1997; and the corruption index 1. This emphasis on expanding access to government services from the International Country Risk Guide. In addition, the study was consistent with the objectives of the 1978 Health for All constructed summary indicators for the quality of economic and conference. institutional analysis, based on OED ratings of project appraisal 2. In Kenya, Bank support helped improve efficiency and reports for all projects approved through FY97 (see Stout and cost recovery at Kenyatta National Hospital, but the percentage Johnston 1998). In the text, "significant" implies a confidence of the health budget going to the hospital increased rather than level of 95 percent or more. decreased over the life of the project. Annex A The Precis that appear in this Annex are reprints of OED Pr6cis 176 (Zimbabwe), 187 (India), 188 (Mali), and 189 (Brazil). Copies of individual Precis are available from OED. 66 BIBLIOGRAPHY Abbasi, Kamran. 1999. "The World Bank and World Grindle, Merilee S., ed. 1997. Getting Good Govern- Health: Health Care Strategy." British Medical ment: Capacity Building in the Public Sectors of journal 318. Developing Countries. Cambridge, MA: Harvard Adams, Isaac. 1998. "Country Experience: Sectorwide University Press. Approach to M&E in the Health Sector in Ghana." Grindle, Merilee S, and John W. Thomas. 1991. Public In Evaluation Capacity Development in Africa: Choices and Policy Change: The Political Economy Selected Proceedings from a Seminar in Abidjan. of Reform in Developing Countries. Baltimore, MD: Washington, D.C.: African Development Bank and Johns Hopkins University Press. the World Bank Operations Evaluation Department. Hammer, Jeffrey S. 1996. "Economic Analysis for Aday, Lu Ann, Charles E. Begley, David R. Lairson, Health Projects." World Bank Policy Research Work- and Carl H. Slater. 1993. Evaluating the Healthcare ing Paper No. 1611, Washington, D.C. System: Effectiveness, Efficiency, and Equity. Chi- Heaver, Richard. 1988. "Reaching People at the Periph- cago: Health Administration. ery: Can the World Bank's Population, Health, and Belli, Pedro. 1996. Handbook on Economic Analysis of Nutrition Operations Do Better?" World Bank Working Investment Operations. Washington, D.C.: World Paper Series 81, Population and Human Resources Bank. Department, Washington, D.C. Castro, Claudio de Moura, and Phil Musgrove. n.d. Israel, Arturo. 1987. Institutional Development: Incen- "On the Non-existence of 'The Social Sector': Or tives to Performance. Washington, D.C.: World Bank. Why Education and Health Are More Different than Jamison, Dean, W. Henry Mosely, Anthony Measham, Alike." Sustainable Development Department, Inter- and Jose Luis Bobadilla. 1993. Disease Control American Development Bank, Washington, D.C. Priorities in Developing Countries. Washington, Dayton, Julia. 1998. World Bank HIVIAIDS Interven- D.C.: Oxford University Press for the World Bank. tions: Ex-ante and Ex-post Evaluation. World Bank Johnson, John, and Sulaiman Wasty. 1993. Borrower Discussion Paper No. 389. Washington, D.C. Ownership of Adjustment Programs and the Political Durch, Jane, Linda Bailey, and Michael Stoto, eds. Economy of Reform. World Bank Discussion Paper 1997. Improving Health in the Community: A Role No. 199, Washington, D.C. for Performance Monitoring. Committee on Using Johnston, Timothy. 1998. The Impact of World Bank Performance Monitoring to Improve Community Support to the HNP Sector in Zimbabwe. Report No. Health. Washington D.C.: National Academy Press. 18141, Operations Evaluation Department, World Filmer, Deon, Jeffrey Hammer, and Lant Pritchett. Bank, Washington, D.C. 1998. "Health Policy in Poor Countries: Weak Links Johnston, Timothy, Sheila Dohoo Faure, and Laura in the Chain." World Bank Policy Research Working Raney. 1998. The World Bank and the Health Sector Paper 1874, Poverty and Human Resources, Devel- in Mali: An OED Country Sector Report. Report No. opment Working Group, Washington D.C. 18112, Operations Evaluation Department, World Gauri, Varun. 1998. The Brazil Health System: Impact Bank, Washington, D.C. Evaluation Report. Report No. 18142, Operations Knowles, James, Charlotte Leighton, and Wayne Stinson. Evaluation Department, World Bank, Washington, 1997. "Measuring Results of Health Sector Reform for D.C. System Performance: A Handbook of Indicators." Gilson, Lucy. 1997 "The Lessons of User Fee Experience Special Initiatives Report No. 1, Partnerships for in Africa." Health Policy and Planning 12 (4). Health Reform, Abt Associates, Bethesda, MD. Girishankar, Navin. 1999. "Reforming Institutions for Lerer, Leonard, Alan Lopez, Tord Kjellstrom, and Service Delivery: A Framework for Development Derek Yach. 1998. "Health for All: Analyzing Assistance with an Application to the Health, Nutri- Health Status and Determinants." World Health tion, and Population Portfolio." World Bank Policy Statistics Quarterly 51 (1). Research Working Paper No. 2039, Operations Lusthaus, Charles, Marie-Helene Adrien, Gerry Ander- Evaluation Department, Poverty Reduction and Eco- son, and Fred Carden. 1999. Enhancing Organiza- nomic Management Network, and Human Develop- tional Performance: A Toolbox for Self-Assessment. ment Network, Washington, D.C. Ottawa: International Development Research Center. 67 [nvesting in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Martinez, Javier, and Tim Martineau. 1998 "Rethink- Orbach, Elie. 1999. "Organizational Capacity and ing Human Resources: An Agenda for the Millen- Capacity Building: An Operational Definition and nium." Health Policy and Planning 13 (4). an Assessment Methodology." Capacity Building McPake, Barbara, and Joseph Kutzin. 1997. Methods Unit, Africa Region, World Bank, Washington, D.C. for Evaluating Effects of Health Reforms. Geneva: Photocopy. World Health Organization. Over, Mead, and Naoko Watanabe. 1999. "Evaluating Morgan, Peter, and Ann Qualman. 1996. "Institutional the Impact of Organizational Reforms and Hospi- and Capacity Development, Results-Based Manage- tals." Development Economics Department, World ment and Organisational Performance." Political Bank, Washington, D.C. Photocopy. and Social Policies Division, Policy Branch, Cana- Overseas Development Council. 1998. "Rx for Global dian International Development Agency, Toronto. Health Cooperation Beyond 2000: Report on the Musgrove, Phillip, and Ronald Ridker. 1999. Case Conference on Global Health Cooperation Beyond Study of World Bank Activities in the Health Sector 2000." Washington, D.C. in India. Operations Evaluation Department, World Pavignani, Enrico, and Joaquim Ramalho Durao. Bank, Washington, D.C. 1997. "Aid, Change and Second Thoughts: Coordi- Nelson, Joan M. In press. "Reforming Social Sector nating External Resources to the Health Sector in Governance: A Political Perspective." In J. Tulchan, Mozambique." Department for International Devel- ed., The Politics and Administration of Social Develop- opment, London. ment in Latin America. Boulder, CO: Lynn Rienner. Pelletier, David L., and Roger Shrimpton. 1994. "The Nelson, Joan M., and Stephanie J. Eglington. 1993. Role of Information in the Planning, Management, Global Goals, Contentious Means: Issues of Multiple and Evaluation of Community Nutrition Programs." Aid Conditionality. Washington, D.C.: Overseas Health Policy and Planning 9: 171-84. Development Council. Peters, David, and Shiyan Chao. 1998. "The Sector- Nolan, Brian, and Vincent Turbat. 1995. Cost Recov- Wide Approach in Health: What is It? Where Is It ery in Public Health Services in Sub-Saharan Africa. Leading?" International Journal of Health Planning Washington D.C.: Economic Development Institute, and Management 13: 177-90. World Bank. Piccioto, Robert, and Eduardo Wiesner. 1998. Evalua- North, Douglas. 1990. Institutions, Institutional tion and Development: The Institutional Dimension. Change, and Economic Performance. Cambridge, London: Transaction Publishers for the World Bank. U.K.: Cambridge University Press. PAI (Population Action International). 1997. Falling Nutbeam, Don, and Elizabeth Harris. 1998. Theory in Short: The World Bank's Role in Population and a Nutshell: A Practitioner's Guide to Commonly Reproductive Health. Washington, D.C. Used Theories and Models in Health Promotion. Preker, Alexander, Logan Brenzel, and Annu Ratta. Sydney, Australia: Department of Public Health and 1997. "Project and Economic Analysis in the Health, Community Medicine, University of Sydney. Nutrition and Population Sector: A Conceptual OED (Operations Evaluation Department). 1998. An- Framework and Portfolio Review." World Bank nual Review of Development Effectiveness. Wash- Human Development Network, Washington, D.C. ington, D.C.: World Bank. Preker, Alex, April Harding, and Navin Girishankar. _. 1996. Social Dimensions of Adjustment. Wash- 1999. "The Economics of Private Participation in ington, D.C.: World Bank. Health Care: New Insights from Institutional Eco- 1994. An Overview of Monitoring and Evalua- nomics." Human Development Network, World tion in the World Bank. Report No. 13247. Washing- Bank, Washington, D.C. Photocopy. ton, D.C.: World Bank. Preker, Alex, and April Harding, eds. 1999. Innovations - . 1992. Population and the World Bank: Impli- in Health Care Delivery: Organizational Reforms cations from Eight Case Studies. Washington, D.C.: Within the Public Sector. Washington, D.C., and World Bank. London: World Bank and Oxford University Press. 68 B3ibliography Prescott, Nicholas, ed. 1997. Choices in Financing Williamson, John, ed. 1994. The Political Economy of Health Care and Old Age Security. World Bank Policy Reform. Washington, D.C: Institute for Inter- Discussion Paper No. 392, Washington D.C. national Economics. Reich, Michael. 1997. "Book Review of World Devel- World Bank. 1999. Population and the World Bank: opment Report 1993: Investing in Health." Eco- Adapting to Change. Human Development Network, nomic Development and Cultural Change 45 (4). Washington, D.C. Reich, Michael R., and David M. Cooper. 1996. _. 1997a. Confronting AIDS: Public Priorities in "PolicyMaker: Computer-Assisted Political Analysis, a Global Epidemic. Washington, D.C. Software and Manual," Version 2.2. Brookline, MA: _. 1997b. Health, Nutrition and Population PoliMap. Sector Strategy. Human Development Network, Sandford, Peter, Hugh Annett, and Richard Cibuslkis. Washington, D.C. 1992. "What Can Information Systems Do for Pri- . 1997c. Innovations in Health Care Financing: mary Health Care? An International Perspective." Proceedings of a World Bank Conference, March 10- Social Science and Medicine 34 (10). 11. World Bank Discussion Paper No. 365, Washing- Schieber, George J. 1995. "Preconditions for Health ton, D.C. Reform: Experiences from the OECD Countries." _. 1997d. World Development Report 1997: The ELSEVIER Heath Policy 32: 279-93. State in a Changing World. Washington, D.C. Stout, Susan, Alison Evans, Janet Nassim,,and Laura . 1996. "Performance Monitoring Indicators: A Raney. 1997. Evaluating Health Projects: Lessons Handbook for Task Managers." Operations Policy from the Literature. World Bank Discussion Paper Department, Washington, D.C. No. 356, Washington, D.C. _. 1994. Better Health in Africa: Experience and UNICEE 1998. Implementing Health Sector Reforms in Lessons Learned. Washington, D.C. Africa: A Review of Eight Country Experiences. Divi- . 1993a. World Development Report 1993: sion of Evaluation, Policy, and Planning, New York. Investing in Health. Washington, D.C. Valdez, Joseph, and Michael Bamberger. 1994. Moni- _. 1993b. Disease Control Priorities in Developing toring and Evaluating Social Programs in Develop- Countries. Washington, D.C. ing Countries. Washington D.C: Economic Develop- -. 1987. Financing Health Services in Develop- ment Institute, World Bank. ing Countries. Washington, D.C. van de Walle, Nicolas, and Timothy Johnston. 1996. WHO (World Health Organization). 1997. "Monitor- Improving Aid to Africa. Washington, D.C: Overseas ing Reproductive Health: Selecting a Short List of Development Council. National and Global Indicators." Division of Repro- Walt, Gill, and Lucy Gilson. 1994. "Reforming the ductive Health. Geneva. Health Sector in Developing Countries: The Central _. 1994. Global Comparative Assessments in the Role of Policy and Analysis." Health Policy and Health Sector. Geneva. Planning. Oxford, U.K.: Oxford University Press. _. 1993. "Rapid Evaluation Methods (REM) of Walt, Gill, Enrico Pavignani, Lucy Gilson, and Kent Health Services Performance: Methodological Ob- Buse. 1999. "Managing External Resources in the servations." Bulletin of the World Health Organiza- Health Sector: Are There Lessons for SWAps?" tion 71 (1). Health Policy and Planning 14 (3). Watkins, Kevin. 1997. "Cost-Recovery and Equity in the Health Sector: Issues for Developing Countries." London: Oxfam UK & Ireland Policy Department. 69 OPERATIONS EVALUATION DEPARTMENT PUBLICATIONS The Operations Evaluation Department (OED), an in- Documents listed with a stock number and price dependent evaluation unit reporting to the World code may be obtained through the World Bank's mail Bank's Executive Directors, rates the development order service or from its InfoShop in downtown impact and performance of all the Bank's completed Washington, D.C. For information on all other docu- lending operations. Results and recommendations are ments, contact the World Bank InfoShop. reported to the Executive Directors and fed back into For more information about this study or OED's the design and implementation of new policies and other evaluation work, please contact Elizabeth projects. In addition to the individual operations and Campbell-Page or the OED Help Desk. country assistance programs, OED evaluates the Bank's policies and processes. Operations Evaluation Department Summaries of studies and the full text of the Pr9cis Partnerships & Knowledge Programs (OEDPK) and Lessons & Practices can be read on the Internet at E-mail: ecampbellpage@worldbank.org http://www.worldbank.org/html/oed/index.htm E-mail: OED Help Desk@worldbank.org Telephone: (202) 473-4497 How To Order OED Publications Facsimile: (202) 522-3200 Operations evaluation studies, World Bank discussion papers, and all other documents are available from the World Bank Infoshop. Ordering World Bank Publications The World Bank InfoShop serves walk-in customers Customers in the United States and in territories not only. The InfoShop is located at: served by any of the Bank's publication distributors may send publication orders to: 701 18th Street, NW Washington, DC 20433, USA The World Bank P.O. Box 960 All other customers must place their orders through Herndon, VA 20172-0960 their local distributors. Fax: (703) 661-1501 Telephone: (703) 661-1580 Ordering by e-mail The address for the World Bank publication database If you have an established account with the World on the Internet is: http://www.worldbank.org (select Bank, you may transmit your order by electronic mail publications/project info). on the Internet to: books@worldbank.org. Please in- E-mail: pic@worldbank.org clude your account number, billing and shipping Fax number: (202) 522-1500 addresses, the title and order number, quantity, and Telephone number: (202) 458-5454 unit price for each item. 71 OED SERIES 1997 Annual Review of Development Effectiveness 1998 Annual Review of Development Effectiveness Agricultural Extension and Research: Achievements and Problems in National Systems Bangladesh: Progress Through Partnership Developing Towns and Cities: Lessons from Brazil and the Philippines Financial Sector Reform: A Review of World Bank Assistance Fiscal Management in Adjustment Lending India: The Dairy Revolution Mainstreaming Gender in World Bank Lending: An Update Nongovernmental Organizations in World Bank-Supported Projects: A Review Paddy Irrigation and Water Management in Southeast Asia Poland Country Assistance Review: Partnership in a Transition Economy Reforming Agriculture: The World Bank Goes to Market The World Bank's Experience with Post-Conflict Reconstruction Zambia Country Assistance Review: Turning an Economy Around Proceedings Lessons of Fiscal Adjustment Lesson from Urban Transport Evaluation and Development: The Institutional Dimension (Transaction Publishers) Monitoring & Evaluation Capacity Development in Africa Public Sector Performance - Lessons from Urban Transport Public Sector Performance - The Critical Role of Evaluation Multilingual Editions Assessing Development Effectiveness: Evaluation in the World Bank and the International Finance Corporation Apprciation de l'efficacite du d6veloppement: L'dvaluation a la Banque mondiale et a la Societe financiere internationale Determinar la eficacia de las actividades de desarrollo: La evaluaci6n en el Banco Mundial y la Corporaci6n Financiera Internacional C6te d'Ivoire : Revue de l'aide de la Banque mondiale au pays Philippines: From Crisis to Opportunity Filipinas: Crisis y oportunidades Rebuilding the Mozambique Economy: Assessment of a Development Partnership Reconstruir a Economia de Moqambique 0 THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. Telephone: (202) 477-1234 Facsimile: (202) 477-6391 Telex: MCI 64145 WORLDBANK MCI 248423WORLDBANK Web: http://www.worldbank.org Email books@worldbank.org lmes M. HttUn"r H B177 143 10 343104 ISBN 0-8213-4310-6