19 9 4 -15425 A61 l9 Evaluation Results World Bank Operations Evaluation Department 1994 Evaluation Results Other Titles in the Series Prepared by the World Bank Operations Evaluation Department Ghana Country Assistance Review: A Study in Development Effectiveness (1996; contains summaries in French and Spanish) Evaluation and Development: Proceedings of the 1994 World Bank Conference (1995) Developing Industrial Technology: Lessons for Policy and Practice (1995; contains summaries in French and Spanish) The World Bank and Irrigation (1995; contains summaries in French and Spanish) Structural and Sectoral Adjustment: World Bank Experience, 1980-92 (1995; contains summaries in French and Spanish) Gender Issues in World Bank Lending (1995; contains summaries in French and Spanish) The World Bank's Role in Human Resource Development in Sub-Saharan Africa: Education, Training, and Technical Assistance (1994; contains summaries in French and Spanish) 1992 Evaluation Results (1994; contains summaries in French and Spanish) New Lessons from Old Projects: The Workings of Rural Development in Northeast Brazil (1993; contains summaries in French, Portuguese, and Spanish) World Bank Approaches to the Environment in Brazil (1993; contains summaries in French, Portuguese, and Spanish) Evaluation Results for 1991 (1993; contains summaries in French and Spanish) Trade Policy Reforms under Adjustment Programs (1992; contains summaries in French and Spanish) World Bank Support for Industrialization in Korea, India, and Indonesia (1992; contains summaries in French and Spanish) Population and the World Bank: Implications from Eight Case Studies (1992; contains summaries in French and Spanish) Forestry: The World Bank Experience (1992; contains summaries in French and Spanish) The Aga Khan Rural Support Program in Pakistan: Second Interim Evaluation (1990; contains summaries in French and Spanish) 1994 Evaluation Results THE WORLD BANK WASHINGTON, D.C. Copyright @ 1996 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First printing April 1996 The opinions expressed in this report do not necessarily represent the views of the World Bank or its member governments. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. Some sources cited in this paper may be informal documents that are not readily available. The material in this publication is copyrighted. Requests for permission to reproduce portions of it should be sent to the Office of the Publisher at the address shown in the copyright notice above. The World Bank encourages dissemination of its work and will normally give permission promptly and, when the reproduction is for noncommercial purposes, without asking a fee. Permission to copy portions for classroom use is granted through the Copyright Clearance Center, Inc., Suite 910, 222 Rosewood Drive, Danvers, Massachusetts 01923, U.S.A. The complete backlist of publications from the World Bank is shown in the annual Index of Publications, which contains an alphabetical title list (with full ordering information) and indexes of sub- jects, authors, and countries and regions. The latest edition is available free of charge from Distribution Unit, Office of the Publisher, The World Bank, 1818 H Street, N.W., Washington, D.C. 20433, U.S.A., or from Publications, The World Bank, 66, avenue d'I6na, 75116 Paris, France. Cover: A woman in Maijpara drying rice to be stored for the year's supply (Bangladesh) Photo credit: World Bank/Kay Chernush ISSN: 1019-4363 ISBN: 0-8213-3607-X Contents Foreword xi Prefacio xii Pr6face xiii Acknowledgments xiv Abbreviations and acronyms xv Executive summary 1 Resumen 8 R6sum6 analytique 16 1. The performance record 25 Introduction 25 1994 results 29 Lessons from outstanding operations 40 2. The determinants of portfolio performance 44 Introduction 44 Key findings 44 Improving project performance 46 Improving country performance 49 Broader issues of Bank-country assistance 62 Implications for Bank operational priorities 63 3. Poverty environment, and private sector development 66 The thematic content of recently evaluated operations 66 Poverty reduction 66 Environmental sustainability 71 Private sector development 74 vii 4. Improving development effectiveness 79 Introduction 79 Learning from Bank clients and staff 79 A sound policy environment matters 80 Quality at entry matters 82 Portfolio management matters 82 Development partnerships matter 84 Boxes 1.1 Different perspectives on performance 26 1.2 Beneficiary groups 33 1.3 Social safety nets 35 2.1 Quality Bank processing overcomes unfavorable macroeconomic environment: examples from Brazil 50 2.2 Sharply contrasting portfolio performance: China and Peru 54 2.3 Argentina's experience with Bank-financed projects 56 3.1 Structural adjustment, growth, and poverty reduction 70 3.2 Support for private sector development: lessons from OED studies 77 4.1 Ghana country assistance review: generic lessons 81 4.2 Quality of economic analysis: criteria 83 Figures 1.1 Outcome, sustainability, and institutional development 28 1.2 Distribution of evaluated operations by region, 1994 and 1974 cohorts 29 1.3 Distribution of the FY95 ongoing portfolio, by region 29 1.4 Project demandingness, complexity, and riskiness, by approval year, 1980-89 30 1.5 Performance of operations evaluated, by sector and region, 1994 and 1990-94 cohorts (n>=10) 31 1.6 Sustainability of operations evaluated, by sector and region, 1994 and 1989-94 cohorts (n>=10) 36 1.7 Positive sustainability factors for satisfactory projects, 1994 cohort 37 1.8 Achievement of institutional objectives, by sector and region, 1994 and 1989-94 cohorts (n>=10) 38 1.9 Interrelationship of project sustainability, institutional development, and performance ratings, 1989-94 evaluation cohort 39 1.10 Appraisal and re-estimated rates of return, by approval year, 1969-85 39 1.11 Variance of re-estimated rates of return, by approval year, 1969-85 40 1.12 Re-estimated rates of return, by sector and approval year groups 40 1.13 Time overruns and outcome, 1994 cohort 41 1.14 Cost over- or underruns and outcome, 1994 cohort 41 2.1 Quality of appraisal by major areas of analysis: projects with satisfactory outcome, likely sustainability, and substantial ID impact 48 viii 2.2 Factors affecting appraisal quality, projects with satisfactory outcome, likely sustainability, and substantial ID impact 48 2.3 Performance of supervision by major areas of attention: projects with satisfactory outcomes, likely sustainability, and substantial ID impact 49 2.4 Factors affecting supervision quality, projects with satisfactory outcomes, likely sustainability, and substantial ID impact 49 2.5 Portfolio and country economic performance 52 3.1 Thematic components by country income group, operations evaluated in 1993 and 1994 67 Table 3.1 Thematic components by country income group, operations evaluated in 1993 and 1994 68 Annexes 87 ix Foreword OED's twenty-first annual review of evaluation strongest determinants of project outcomes. results reports on 246 operations evaluated for The quality of the Bank's contributions-in the first time in 1994. The review essentially appraising proposed projects and supervising confirms the findings of the last two years and projects in progress-is a close second. explores new ground in assessing the determi- nants of project performance. Three country factors have statistically signifi- cant effects on portfolio performance: macro- The share of projects with satisfactory outcomes economic stability, per capita income growth, in the 1994 cohort is 66 percent, a modest and good governance. Bank activities that help increase over recent years. While encouraging, to create an adequate enabling environment this finding underscores the need for managers also contribute handsomely to effective portfo- to focus on key performance indicators and lio performance, as does institutional develop- for the Bank to enhance accountability for ment aimed at encouraging good governance. portfolio results. In countries with adverse policy environments, Appraisal forecasts of elapsed times, costs, and the Bank should focus on nonlending services rates of return have become somewhat more that encourage the implementation of adequate realistic. The performance of adjustment opera- policies. Capacity-building assistance is also tions continued to improve, a trend that first warranted, especially where borrowers' com- became noticeable two years ago. mitment can be secured and effective benefi- ciary participation is feasible. The review breaks new ground with a rigorous quantitative analysis of the determinants of performance in more than a thousand com- pleted operations. The analysis shows that Robert Picciotto borrowers' management of operations, and Director General compliance with loan covenants, are the Operations Evaluation xi Prefacio En el vigésimo primer examen anual de los ciones de los préstamos son los factores más resultados de la evaluación ex post, se pasa importantes del éxito de los proyectos, segui- revista a 246 operaciones que se evaluaron por dos muy de cerca por la calidad de las contribu- primera vez en 1994. Básicamente, el examen ciones del Banco -en la evaluación de las confirma los resultados obtenidos en los últi- propuestas de proyectos y en la supervisión de mos dos años y en él se analizan nuevos meca- los proyectos en curso. nismos para evaluar los factores que determinan los resultados de los proyectos. Tres factores vinculados a la situación particu- lar de cada país -estabilidad macroeconómica, El 66 por ciento de los proyectos evaluados en crecimiento del ingreso per cápita y buen 1994 dio resultados satisfactorios, lo que repre- gobierno- son importantes, desde el punto de senta un cierto aumento con respecto a los últi- vista estadístico, para el desempeño de la car- mos años. Si bien son alentadores, estos tera. Las actividades del Banco que contribuyen resultados ponen de manifiesto la necesidad de a crear un entorno favorable tienen también un que los gerentes se concentren en los indicado-, efecto muy positivo en el desempeño, al igual res principales del desempeño y de que el que los programas de desarrollo institucional Banco mejore la rendición de cuentas relativa a destinados a fomentar el buen gobierno. los resultados de la cartera. En los países con un marco de política desfavo- Actualmente, las previsiones de las evaluacio- rable, el Banco debería concentrarse en los servi- nes iniciales sobre plazos, costos y tasas de ren- cios no financieros que facilitan la aplicación de tabilidad son algo más realistas. Los resultados políticas apropiadas. También es necesario pres- de las operaciones de ajuste siguieron mejo- tar asistencia para el fortalecimiento de la capa- rando, tendencia observada por primera vez cidad, sobre todo en los casos en que es posible hace dos años. garantizar la adhesión de los prestatarios y la participación eficaz de los beneficiarios. El examen incluye un análisis cuantitativo rigu- roso e innovador de los factores determinantes de los resultados de más de mil operaciones ter- minadas. Del análisis se desprende que la ges- Robert Picciotto tión de las operaciones por parte de los Director General prestatarios y el cumplimiento de las estipula- Evaluación de Operaciones xii Acknowledgments This study was prepared by Pablo Guerrero Christopher Kilby, Robert van der Lugt, and Ulrich Thumm, and Rachel Weaving. Contribu- Anwar Shah. Support was provided by William tors included Omar Arias, John English, Battaile and Charlie Sheerin. The study was copy- Christopher Gibbs, Juan Giral, John Johnson, edited and produced by Angie Gentile-Blackwell. xiv Préface Le vingt et unième rapport annuel de l'OED des clauses de l'accord de prêt. Vient immédia- sur les résultats de l'évaluation rétrospective tement derrière ces conditions la qualité de la des projets de la Banque rend compte de contribution de la Banque - lors de l'évalua- 246 opérations ayant fait l'objet d'une première tion préalable des projets proposés et de la évaluation rétrospective en 1994. Ce rapport supervision de leur exécution. confirme pour l'essentiel les conclusions des rapports des deux années précédentes et expé- Trois facteurs liés au pays ont en outre des effets rimente de nouveaux moyens de juger des élé- statistiquement sensibles sur les résultats du ments déterminants des résultats des projets. portefeuille: la stabilité macroéconomique, la croissance du revenu par habitant et la bonne Les résultats de 66 pour cent des projets de la gestion des affaires publiques. Les activités de cohorte de 1994 ont été déclarés satisfaisants, ce la Banque qui facilitent la création de condi- qui représente un léger progrès par rapport à tions propices à la bonne exécution d'un projet ces dernières années. Pour encourageante contribuent pour beaucoup, elles aussi, à la qu'elle soit, cette constatation montre à quel bonne gestion du portefeuille, au même titre point il importe que les chefs de projet concen- que les mesures de renforcement institutionnel trent leur attention sur les principaux indica- visant à favoriser une saine conduite de la teurs de performance et que la Banque insiste chose publique. davantage sur la responsabilité à l'égard des résultats du portefeuille. Dans les pays où les politiques suivies ne sont guère favorables au succès de ses opérations, la Les prévisions de délai, de coût et de taux de Banque devrait se concentrer sur les services rentabilité établies lors de l'évaluation préalable hors prêts susceptibles de promouvoir la mise des projets sont aujourd'hui plus conformes à la en oeuvre de politiques plus adéquates. Une réalité. Les résultats des opérations d'ajuste- aide au renforcement des capacités serait égale- ment ont continué de s'améliorer, poursuivant ment judicieuse, surtout s'il est possible d'inci- ainsi la tendance apparue deux ans plus tôt. ter l'emprunteur à s'engager et d'obtenir une participation effective des bénéficiaires. Cette année, l'examen innove avec une analyse quantitative rigoureuse des éléments détermi- nants des résultats de plus de mille opérations achevées. Cette analyse montre que l'issue d'un Robert Picciotto projet dépend principalement de sa bonne ges- Directeur général tion par l'emprunteur et du respect par celui-ci Evaluation des opérations xiii Abbreviations and acronyms AFR Africa region ARPP Annual Report on Portfolio Performance CAS Country assistance strategy CFA Central African Franc CY Calendar year EAP East Asia and Pacific region ECA Europe and Central Asia region ERR Economic rate of return ESD Environmentally sustainable development ESW Economic and sector work FY Fiscal year G7 Group of Seven (industrialized nations) GDP Gross domestic product GNP Gross national product IBRD International Bank for Reconstruction and Development ICR Implementation completion report ID Institutional development IDA International Development Association IFC International Finance Corporation IMF International Monetary Fund KIP Kampung Improvement Program LAC Latin America and Caribbean region LIBOR London interbank offered rate M&E Monitoring and evaluation MNA Middle East and Northern Africa region NGO Nongovernmental organization O&M Operation and maintenance OED Operations Evaluation Department PAR Performance audit report PCR Project completion report PSA Private sector assessment PSD Private sector development PTI Program of targeted interventions xv SAL Structural adjustment loan SAS South Asia region SECAL Sectoral adjustment loan xvi Executive summary This twenty first Annual Review of Evaluation cohort of operations evaluated last year. Results analyzes evaluation findings for 1994 While it is better than the average of 63 per- and traces their implications for the manage- cent for 1990-94, it is still unacceptably low. ment of Bank operations. It is based on comple- tion reports by operational staff and on * Forty four percent of the evaluated opera- performance audit reports, impact evaluations, tions are expected to sustain their benefits and evaluation studies by the independent throughout the operational phase that fol- Operations Evaluation Department (OED). lows the completion of Bank loan disburse- ments. This proportion is little different In 1994,246 completed operations were evalu- from last year's evaluation cohort and the ated for the first time (the "1994 cohort"). Most average for 1989-94. But the proportion of of these loans and credits, representing disburse- operations judged unlikely to sustain their ments of $17 billion, are recent; about three benefits has dropped to 20 percent from fourths were approved in 1985 or later, and 24 percent in 1993. nearly half were closed in 1993 or later. In the same year, OED produced performance audit = Institutional development goals were sub- reports for 108 operations whose completion stantially achieved in 39 percent of the oper- reports had been prepared in earlier years. ations-better than the 30 percent in the Where relevant the review draws on information 1993 cohort and the average of 31 percent for from all the 354 operations evaluated in 1994. the last five years, but again a low figure. Adjustment operations kept up the The performance record (Chapter 1) improvement in performance that first became noticeable two years ago, and they This year's results essentially confirm and performed better, on average, than invest- consolidate those of the last two years. They ment projects. Adjustment operations had a reflect a number of encouraging signs, but con- 70 percent satisfactory rating, versus firm the need to further improve accountability 65 percent for investment operations. for results. Outcomes * Sixty six percent of the operations evaluated in 1994 had satisfactory outcomes. This year's More than two thirds of the operations analyzed percentage compares with 64 percent in the had goals that were highly or substantially relevant to development priorities. Forty five Sustainability ratings remain unchanged, on aver- percent of the operations were considered age, from last year. In East Asia, the proportion of highly or substantially effective in meeting their operations rated likely to be sustainable goals, and another 34 percent as moderately improved to 73 percent. In Africa, 23 percent of effective. As in last year's cohort, physical goals operations were rated likely to be sustainable. were met more readily than goals such as mac- On the other hand, only 26 percent-down roeconomic and sectoral reform, financial from 31 percent in 1989-94-were rated as enhancements, and institutional development. unlikely to be sustainable. Forty one percent of the operations were rated as highly or substantially efficient and another Looking at operations by their year of 32 percent as moderately efficient. approval, the proportion judged likely to sustain their benefits increased from 42 percent East and South Asia had the best record on all for those approved in 1984 to 51 percent for measures of overall performance. those approved the following year. But for later approval years, not enough operations have By sector, performance was best in human yet been evaluated to be certain of an resources, with 83 percent of its operations improving trend. rated satisfactory, and poorest in agriculture, where operations performed better than their Reinforcing previous evaluation findings, gov- 1990-94 average but still achieved only a 60 per- ernment commitment to project goals, institu- cent satisfactory rating. tional and management effectiveness, an enabling policy environment, technical viabil- Comparing the performance of operations ity, and cost recovery had positive influences on approved in 1974-79 with that of operations sustainability in the 1994 cohort. approved in 1980-85, outcomes improved nota- bly in transport operations. But overall, there Institutional development was a decline in performance, led by water sup- ply, power, and finance. Progress was made on institutional development, but performance was still poor. The share of oper- Sustainability ations in the 1994 cohort that substantially achieved their institutional development goals Sustainability measures the extent to which an improved to 39 percent, better than last year's operation is likely to maintain its achievements, 30 percent and the 1989-94 average of 31 per- or expected achievements, during its opera- cent. Almost all sectors and regions improved tional phase. To arrive at a sustainability their record to some extent. rating evaluators assess many of the same factors that underlie the appraisal of an opera- Borrowers' commitment was the factor with the tion, from economic, social, financial, institu- most prominent influence on the achievement tional, and environmental perspectives, as of institutional development goals. appropriate. Sustainability is an even more demanding test than the outcome criterion. Returns, implementation times, and costs Experience shows that the achievement of sus- tainability may call for a series of operations, Projects realized high returns. Rates of return since it often depends on more favorable poli- were calculated at appraisal and re-estimated cies or stronger institutions than may prevail at evaluation for 82 projects-one third of the when the Bank is first involved. 1994 cohort. Most of these projects achieved 2 re-estimated returns much higher than 10 per- * The Bank was closely involved and carefully cent. By sector, re-estimated rates of return monitored progress. were: agriculture 17 percent, power 17 percent, transport 28 percent, and urban development * They were implemented pragmatically and 25 percent. The gap seems to be narrowing flexibly. between returns estimated at appraisal and re-estimated at completion. m Many of them enjoyed the benefits of long- standing relationships with the Bank, prior Forecasts of implementation time and cost are lending operations, and/or extensive eco- becoming more realistic. nomic and sector work. * Time overruns have gone down, but fore- casts are still overoptimistic. On average, Determinants of portfolio performance operations evaluated in 1994 took (Chapter 2) 37 percent longer to implement than originally scheduled, down from What should be the focus of the Bank's atten- 48 percent in 1993, and 54 percent in tion in the project cycle to bring about satisfac- the 1974-94 cohorts. tory portfolio performance- How effectively can the Bank mitigate the risks inherent in a " Cost estimates are also improving. Africa country's economic and sociopolitical environ- was the only region where operations cost ment and the borrower's behavior? more than expected (overruns were at 16 percent). The other regions had cost This year's analysis of the determinants of underruns of between 10 and 15 percent. portfolio performance looks at all operations evaluated by OED in 1991-94. It uses more Outstanding operations rigorous quantitative techniques than its predecessors, and leads to partial revisions Plenty of "best practice" examples show that signif- of OED's earlier findings. icant improvements in portfolio performance are possible. OED evaluators designate operations Of all the independent variables featured in the analy- as outstanding if they exceed all their major sis, borrower performance during preparation and goals, have highly innovative designs, and/or implementation had the widest impact on individual use approaches likely to be successful in differ- project results. Subsumed under "borrower per- ent countries or sectors. Of the 354 operations formance" are: the policy environment created that were evaluated in 1994, OED assessed for Bank-supported projects, the level of govern- 18 as outstanding, on the basis of performance ment commitment to project goals, the provision audits or reviews of completion reports. and reliability of domestic funds for the project, This diverse group of operations shared the the administrative procedures in use, and the following distinctive features: quality of decision making. Poor management by the borrower, and poor compliance with loan " They originated with borrowers, with the covenants, generally led to operations with poor Bank seen either as an objective catalyst results, uncertain or unlikely sustainability, and for change, or as providing the necessary weak contributions to institutional development. technical knowledge. Closely following borrower performance, however, * Their designs were relatively simple. was how well the Bank did its job. The quality of 3 appraisal and strong supervision were impact during implementation and the sustain- especially important. In general, quality ability of benefits. This underscores the impor- Bank processing had a dominant effect on out- tance of getting the economic fundamentals comes and a significant effect on the sustain- right-which is also the prime rationale for ability of benefits. adjustment lending. At the level of country portfolios, all the project- The current Bank practice of maintaining minimum level determinants were reaffirmed, but perfor- lending levels in countries with adverse policy envi- mance was also influenced strongly by macro- ronments should be revisited. Without a cohesive economic stability, growth in per capita income, strategy to address economic fundamentals, the and good governance. odds of success are poor, and nonlending ser- vices directed at restoring adequate policies may Determinants of portfolio performance differed in be a better option. In countries with adverse pol- importance across the spectrum of countries. The icy environments and a history of disappointing results imply that improvements in Bank pro- portfolio performance, Bank-country dialogue cess performance, particularly higher quality at and aid coordination, shaped by economic and entry, have a higher payoff in terms of better sector work, is likely to be the most cost-effective portfolio performance in countries whose port- approach. Moderate lending for capacity build- folio performance is already above average. ing may also be warranted where a government Good Bank appraisal and supervision can make is committed to institutional reform. a significant difference to outcome and sustain- ability, but the effort has to be extraordinarily great to offset the effects of a poor policy and Poverty, environmental sustainability, implementation environment. and private sector development (Chapter 3) Implications for the Bank's operational priorities How often and to what degree are the Bank's three thematic objectives-poverty alleviation, The findings suggest that the most direct way to environmentally sustainable development, and improve portfolio performance is to lend more private sector development-reflected in spe- selectively. Of course, given the Bank's cooper- cific components of project design? How suc- ative character, selectivity has to be weighed cessfully have operations met their stated against member countries' legitimate entitle- thematic objectives, and what accounts for their ment to services. In countries where the eco- varying performance? nomic environment is inimical to lending, use of nonlending services may be a cost-effective The 737 operations that OED evaluated way to maintain contact with countries and in 1993 and 1994, through performance contribute to needed policy improvements. audits and reviews of completion reports, were analyzed to identify best practices and In countries with a history of poor portfolio lessons of experience. As in last year's Annual performance, the Bank should focus its contributions Review, only those operations with compo- on macroeconomic stabilization and growth- nents that specifically targeted the thematic promoting structural reforms. Sustained objectives were featured; many others may improvement in the policy environment is have had positive or negative effects likely to result in much better operational on poverty, the environment, or private performance-in terms of both development sector development. 4 Poverty reduction resources and rural environmental projects; and projects to strengthen national and local envi- Among the 737 operations that OED evaluated in ronmental management capacity and public 1993 and 1994, most of those with poverty reduction involvement in environmental decision mak- components supported efforts to increase the income ing. Of the 737 operations reviewed, 196, levels of relatively poor beneficiaries. They focused accounting for $13.4 billion in lending commit- on improving farming practices, creating job ments, had environmental components. opportunities, and providing access to credit, better shelter, and enhanced social services and Experience in environmental components social "safety net" programs. Nearly half the points to the following lessons: operations with poverty components were in the lowest-income group of countries. * Environmental impact assessments should continue to be done after projects are Experience with poverty components high- commissioned. lights the importance of: * Building institutional capacity in environ- " Bottom-up participatory processes that mental management can be the Bank's most enhance beneficiaries' role in designing and important long-term contribution. Where implementing operations. this succeeds, the institutional experience acquired should be disseminated to other * Cooperative approaches that boost benefi- countries and regions. ciaries' confidence and encourage entrepre- neurship and innovation. * Sensitizing target populations to environ- mental hazards can have long-lasting bene- * Regularization of land-use titles to encour- ficial results. age beneficiaries' involvement in and com- mitment to development efforts. * Efforts to enhance preparedness for environ- mental emergencies need early preparation * Innovative financial measures that pro- and, often, a rapid response with financial mote self-financing by beneficiaries and and technical support. strengthen the links between public and pri- vate sector institutions. Private sector development * Social action programs, such as safety nets The Bank supports private sector development and the protection of basic services, to pro- (PSD) by promoting efforts to improve the busi- tect the poor from the adverse consequences ness environment; promoting public enterprise of adjustment. reform and privatization in cases where the pri- vate sector would clearly use the resources more Environmental sustainability efficiently; promoting private participation in infrastructure; and assisting with financial sec- The Bank pursues targeted programs, including tor reform. More broadly, satisfactory adjust- specific projects, regional and global efforts, ment operations contribute to private sector and strategies and policies in support of envi- development by helping to create an enabling ronmentally sustainable development. Three and competitive business environment for the types of projects are typical: pollution manage- private sector. Of the 737 operations reviewed, ment and urban environment projects; natural 254 had PSD components, which absorbed 5 $22 billion of lending commitments. Policy those evaluated in 1994 are judged to have sub- reform operations absorbed nearly one fourth stantially stretched the implementation capac- of the resources committed to PSD components. ity of the countries that borrowed for them. Nearly half involve a multiplicity of goals and Experience in private sector development com- agencies, and are substantially vulnerable to ponents highlights the following lessons: risks. Like their recent predecessors, this year's evaluations show that it is in the more demand- * The health of the macroeconomic environ- ing areas of capacity building and policy reform ment matters most. that outcomes have often failed to meet targets; physical goals were achieved in 77 percent of " In countries undergoing a dramatic trans- the operations evaluated in 1994. formation from a controlled to a market economy, pursuing a gradual but steady OED evaluations repeatedly show the importance program of stabilization and liberalization of a sound economic policy environment. Experi- may be the key to improving the economic ence confirms that a stable and open economic climate and inducing private sector interest. environment positively affects project outcome. Government decisions at the micro level, or at * Government commitment must be the level of individual projects, must take full expressed through unmistakable signals that account of the economic environment in which the reforms introduced will not be reversed. projects are set if their expected goals are to be attained. * Where governments are strongly committed to PSD reforms, policy initiatives, measures, Quality at entry: The evidence points unequivo- and actions precede Bank involvement. cally to the importance of quality at entry in explaining project and portfolio performance. * In successful PSD reforms, the Bank pro- The quality of project preparation and of vided quality sector work and kept a low appraisal have significantly more influence profile, while providing effective technical on satisfactory project performance than key and policy support. country macroeconomic variables, external factors, or governance considerations. * Where capital markets are still developing, financial intermediation loans-designed Management of the ongoing portfolio: Sound port- with minimum regulation and allowing folio management, including the management market forces to dictate the terms-can be of risks, makes a substantial difference to devel- effective tools for channeling credit to opment results. So do the quality of borrower private entrepreneurs. implementation and Bank supervision. Yet much current supervision work still gives short shrift to those factors that would allow the Bank Improving development effectiveness to advise mid-course corrections where needed (Chapter 4) to achieve projects' development objectives. The gap is narrowing between supervision per- The scope, ambition, and complexity of Bank formance ratings, which often turn out to be too operations have risen considerably in recent optimistic, and evaluation ratings given to com- years. The operations evaluated in 1994 pursue pleted projects. But it is still too wide. More more goals and have more components than forceful portfolio management, focused on those evaluated in earlier years. Two thirds of development objectives, and better implemen- 6 tation of existing guidelines on monitoring and build domestic capacity, and strengthen evaluation, will be needed to close this gap. partnerships. The following measures would help to improve Development partnerships: As emphasized outcomes and the sustainability of benefits, by many examples of best practice, stakeholder without sacrificing relevance: participation in design and implementation can have a significantly positive effect on " Improve business processes through sys- project outcome. Similarly, participation is at tematic learning by staff, borrowers, and the core of enhancing government ownership other stakeholders. and commitment to sustainable policy reforms. As development operations become increas- * Enhance the role of risk assessments in set- ingly complex and taxing, both on borrowers ting country strategies and appraising qual- and the Bank, partnership can substantially ity at entry. improve the management of necessary risks, or help to minimize them. Finally, effective * Recognize and reflect political economy con- development partnerships require mutually siderations throughout the project cycle. reinforcing and coordinated Bank action with other development agencies and " Use nonlending services to improve project with nongovernmental organizations, selection, nurture borrower ownership, whenever appropriate. 7 Resumen En este vigésimo primer examen anual de los varios indicios alentadores, si bien corroboran resultados de la evaluación ex post se presentan la necesidad de mejorar el sistema de rendición las conclusiones de las evaluaciones realizadas de cuentas en relación con los resultados. en 1994 y se analiza la importancia de dichas conclusiones para la gestión de las operaciones a El 66 por ciento de las operaciones evalua- del Banco. En el examen se utilizan informes de das en 1994 dieron resultados satisfactorios, terminación de proyecto preparados por el per- frente a un 64 por ciento de las evaluadas el sonal de operaciones, informes de evaluación año anterior. Si bien esta proporción repre- ex post, informes de evaluación de los efectos y senta una mejoría con respecto al promedio estudios de evaluación realizados por el Depar- del 63 por ciento de 1990-94, sigue siendo tamento de Evaluación de Operaciones (DEO), inaceptablemente baja. organismo independiente. a Se prevé que el 44 por ciento de las operacio- En 1994 se hizo la primera evaluación de nes evaluadas producirán beneficios 246 operaciones terminadas (el "grupo de pro- durante toda la etapa operacional que se ini- yectos de 1994"). La mayoría de estos préstamos cia al terminar el proceso de desembolso de y créditos -que representan desembolsos por préstamos del Banco. Esta proporción no un monto de US$17.000 millones- son recien- difiere mucho de la correspondiente al tes; las tres cuartas partes, aproximadamente, grupo de proyectos evaluados el año ante- se aprobaron en 1985 o en una fecha posterior, rior ni del promedio registrado en 1989-94. y casi la mitad se cerraron en 1993 o después. Sin embargo, la proporción de operaciones En 1994, el DEO preparó los informes de evalua- que, probablemente, no mantendrán sus ción ex post de 108 operaciones cuyos informes beneficios es del 20 por ciento, frente al de terminación de proyecto se habían preparado 24 por ciento de 1993. en años anteriores. En el examen se utiliza, cuando es pertinente, información proveniente a Las operaciones que lograron en buena de las 354 operaciones evaluadas en 1994. parte sus objetivos de desarrollo institucio- nal representaron el 39 por ciento, frente a un 30 por ciento en el caso del grupo Reseña de los resultados (Capítulo 1) de proyectos evaluados en 1993 y al 31 por ciento registrado en los últimos En general, los resultados de este año confirman cinco años. No obstante, este porcentaje los de los dos anteriores. Además, presentan también es bajo. = En las operaciones de ajuste continuó la los resultados empeoraron, sobre todo en las mejora de los resultados que comenzó a áreas de abastecimiento de agua, energía eléc- hacerse visible hace dos años, y su rendi- trica y finanzas. miento medio fue superior al de los proyec- tos de inversión. Las operaciones de ajuste Sostenibilidad con calificaciones satisfactorias fueron el 70 por ciento del total, frente a un 65 La sostenibilidad indica en qué medida es por ciento en el caso de las operaciones probable que una operación mantenga sus de inversión. beneficios --o los beneficios previstos- durante todo su proceso de aplicación. Para Resultados medir la sostenibilidad, los funcionarios a cargo de la evaluación examinan, desde el punto de Más de las dos terceras partes de las operacio- vista económico, social, financiero, institucional nes evaluadas tenían objetivos bastante o muy o ambiental, muchos de los factores que se con- relacionados con las prioridades del desarrollo. sideran en la evaluación inicial de una opera- Se estimó que un 45 por ciento de las operacio- ción. Los criterios que se utilizan para medir la nes cumplieron sus objetivos con mucha o bas- sostenibilidad son aún más exigentes que los tante eficacia, en tanto que un 34 por ciento utilizados para evaluar los resultados de los logró un éxito moderado. Como en el caso del proyectos. La experiencia indica que para grupo de proyectos evaluados el año pasado, lograr la sostenibilidad puede ser necesario ini- las metas físicas se cumplieron más fácilmente ciar numerosas operaciones, ya que a menudo que otras, como la reforma macroeconómica y se requieren políticas más favorables e institu- sectorial, el mejoramiento de los resultados ciones más eficientes que las existentes cuando financieros y el desarrollo institucional. el Banco interviene por primera vez. El 41 por ciento de las operaciones se clasificaron como sumamente eficientes o Los niveles medios de sostenibilidad no han variado bastante eficientes y un 32 por ciento como con respecto al año pasado. En el caso del Asia moderadamente eficientes. oriental, la proporción de operaciones cuya sos- tenibilidad se clasificó como probable aumentó Asia oriental y Asia meridional fueron las regio- al 73 por ciento. En Africa, la proporción fue del nes que mejores resultados obtuvieron en todos 23 por ciento. Por otro lado, sólo en el 26 por los frentes. ciento de las operaciones -frente al 31 por ciento del período 1989-94- se consideró En lo que respecta a los distintos sectores, los improbable el logro de la sostenibilidad. mejores resultados correspondieron a recursos humanos, en el que un 83 por ciento de las ope- Si se consideran las operaciones atendiendo al raciones se clasificaron como satisfactorias, y año de aprobación, la proporción con buenas los más bajos a la agricultura, cuyo rendimiento perspectivas de mantener sus beneficios mejoró con respecto al promedio de 1990-94 aumentó del 42 por ciento en 1984 al 51 por pero sólo fueron satisfactorios en el 60 por ciento en 1985. Sin embargo, debido a que aún ciento de los casos. no se han evaluado suficientes operaciones es imposible determinar con certeza si en años pos- Si se comparan los resultados de las operacio- teriores la tendencia ha seguido siendo positiva. nes aprobadas en 1974-79 con los de las aproba- das en 1980-85, se observan notables mejoras en Una comprobación que confirma los resultados el sector del transporte. No obstante, en general de evaluaciones anteriores es que el firme 9 compromiso de los gobiernos con respecto a los siendo demasiado optimistas. Por término objetivos de los proyectos, la eficacia institucio- medio, el período de ejecución de las opera- nal y administrativa, un clima favorable en ciones evaluadas en 1994 fue un 37 por materia de políticas, la viabilidad técnica y la ciento más largo de lo que se había previsto, recuperación de costos son factores que influye- comparado con un 48 por ciento en 1993 y ron positivamente en la sostenibilidad de los un 54 por ciento en el caso de los grupos proyectos del grupo de 1994. correspondientes al período 1974-94. Desarrollo institucional * Las estimaciones de costos también están mejorando. Africa fue la única región en la Si bien se lograron avances en el terreno del que el costo de las operaciones fue superior desarrollo institucional, los resultados siguieron a lo previsto (los sobrecostos alcanzaron un siendo deficientes. Un 39 por ciento de los proyec- promedio del 16 por ciento). En las demás tos evaluados en 1994 lograron en buena parte regiones los infracostos fueron de entre un sus objetivos de desarrollo institucional, en 10 por ciento y un 15 por ciento. comparación con el 30 por ciento del año ante- rior y un promedio del 31 por ciento durante el Operaciones sobresalientes período 1989-94. Prácticamente en todos los sectores y regiones mejoraron en alguna Muchos ejemplos de "practicas óptimas" demues- medida los resultados. tran que es posible mejorar considerablemente el desempeño de la cartera. Los funcionarios a cargo El firme compromiso de los prestatarios fue el de la evaluación del DEO clasifican las opera- factor más importante en el logro de los objeti- ciones como sobresalientes si éstas logran todas vos de desarrollo institucional. sus metas más importantes, tienen un diseño sumamente innovador y/o utilizan un enfoque Rentabilidad, plazos de ejecución y costos que tiene buenas perspectivas de lograr el éxito en diferentes países o sectores. De las 354 opera- Los proyectos alcanzaron un alto nivel de rentabili- ciones evaluadas en 1994, 18 fueron clasificadas dad. La tasa de rentabilidad se calculó en la eva- como sobresalientes sobre la base de informes luación inicial y se revisó en la evaluación ex de evaluación ex post o informes de termina- post de 82 proyectos, es decir, de la tercera parte ción de proyecto. A pesar de su diversidad, de los proyectos del grupo de 1994. En la mayo- estas operaciones tuvieron en común los ría de los casos, la tasa revisada de rentabilidad siguientes elementos: fue muy superior al 10 por ciento: 17 por ciento en el sector agrícola y en el de la electricidad, 28 * La iniciativa partió de los prestatarios, consi- por ciento en el del transporte y 25 por ciento en derándose al Banco bien como agente catali- el del desarrollo urbano. La diferencia entre la zador de la reforma o como dispensador de tasa de rentabilidad calculada durante la eva- los conocimientos técnicos necesarios. luación inicial y al terminar el proyecto parece estar disminuyendo. * Su diseño fue relativamente sencillo. Los pronósticos sobre los períodos de ejecución y los * El Banco participó activamente y siguió costos son cada vez más realistas. minuciosamente los progresos logrados. * Las demoras respecto del plazo previsto han * Su ejecución tuvo un carácter pragmático disminuido, si bien los pronósticos siguen y flexible. 10 * Muchas de ellas habían contado con estas calidad de las intervenciones del Banco. Fueron ventajas: una relación de larga data con el especialmente importantes la calidad de la eva- Banco, operaciones de financiamiento ante- luación y la decidida labor de supervisión. En riores y/o una amplia labor de estudios eco- general, la influencia de la buena labor adminis- nómicos y sectoriales. trativa realizada por la institución fue determi- nante en los resultados obtenidos, significativa en la sostenibilidad de los beneficios y mode- Factores que influyen en el desempeño rada en el nivel de desarrollo institucional de la cartera (Capítulo 2) logrado a través de las operaciones. ¿En qué aspectos del ciclo de los proyectos debe En lo que respecta a la cartera de operaciones de los concentrarse el Banco para lograr el buen des- países, se reafirmaron todos los factores determinan- empeño de la cartera? ¿En qué medida puede el tes propios de los proyectos, aunque la estabilidad Banco disminuir los riesgos propios del macroeconómica, el crecimiento del ingreso per ambiente económico y sociopolítico de un país cdpita y el buen gobierno también influyeron y del comportamiento del prestatario? considerablemente en los resultados. El análisis de este año de los factores que deter- La importancia de los factores que determinan el des- minan el desempeño de la cartera incluye todas empeño de la cartera variaron según el país. Los las operaciones evaluadas por el DEO en 1991- resultados indican que, en los países en que al 94. Se utilizan técnicas de análisis cuantitativo iniciarse una operación el desempeño de la car- más rigurosas que en evaluaciones anteriores y tera es mejor que el promedio, los progresos en se hace una revisión parcial de algunas de las la tramitación de los proyectos por parte del conclusiones anteriores del DEO. Banco -sobre todo en la calidad inicial- resul- tan más rentables, en lo que al desempeño de la De todas las variables independientes consideradas cartera se refiere, en los países donde éste es ya en este andlisis, el factor que mds influyó en los superior a la media. Una buena labor de evalua- resultados de los distintos proyectos fue el desem- ción inicial y supervisión por parte del Banco peño del prestatario durante la etapa de preparación puede representar una gran diferencia en lo que y ejecución. En el "desempeño del prestatario" respecta al desempeño y la sostenibilidad, pero se incluyen los siguientes elementos: marco se requiere un esfuerzo realmente extraordina- normativo creado para los proyectos respalda- rio para contrarrestar los efectos de unas políti- dos por el Banco, nivel de compromiso del cas y condiciones de ejecución inadecuadas. gobierno con las metas del proyecto, suministro y la fiabilidad del financiamiento interno, nor- Consecuencias para las prioridades mas administrativas utilizadas y calidad del operacionales del Banco proceso de toma de decisiones. En general, una gestión deficiente por parte del prestatario Los resultados indican que el otorgamiento más y la falta de cumplimiento de las cláusulas de selectivo de los préstamos es la forma más los contratos se tradujo en operaciones con directa de mejorar los resultados de la cartera. resultados deficientes, sostenibilidad incierta No obstante, dado el carácter cooperativo de la o improbable y escasa contribución al desarro- labor del Banco, hay que compaginar la selecti- llo institucional. vidad con el derecho legítimo que tienen sus paí- ses miembros a los servicios de la institución. En No obstante, inmediatamente después del comporta- los países en que el clima económico es hostil al miento del prestatario el factor más importante es la otorgamiento de préstamos, el uso de servicios 11 no financieros podría ser una forma eficaz en bilidad ambiental y el desarrollo del sector función de los costos de mantener contacto con privado- en los componentes específicos de la esos países y contribuir a mejorar sus políticas. formulación de proyectos? ¿En qué medida se han logrado esos objetivos en las operaciones y En el caso de los países con una trayectoria decepcio- cuáles son los factores que determinan las dife- nante en lo que respecta al desempeño de la cartera, rencias en los resultados? el Banco debe orientar su labor hacia la estabilización macroeconómica y la adopción de reformas que Se han analizado las 737 operaciones evaluadas favorezcan el crecimiento. Es probable que un por el DEO en 1993 y 1994 a través de informes mejoramiento ininterrumpido del marco nor- de evaluación ex post e informes de termina- mativo se traduzca en un desempeño operacio- ción de proyecto a fin de identificar las prácticas nal mucho mejor, tanto en lo que respecta a su óptimas y las enseñanzas adquiridas. Como en repercusión sobre el desarrollo durante la etapa el examen anual del año anterior, sólo se han de ejecución como a la sostenibilidad de los incluido las operaciones con componentes beneficios. Esto pone de manifiesto la impor- específicamente orientados a lograr los citados tancia de sentar una sólida base económica ade- objetivos; es posible que muchas otras operacio- cuada, lo cual es también la principal razón nes hayan tenido efectos positivos o negativos para otorgar préstamos para fines de ajuste. sobre la pobreza, el medio ambiente o el desa- rrollo del sector privado. El Banco debe revisar la práctica actual de otorgar niveles mínimos de préstamo a los países cuyo marco Reducción de la pobreza de políticas es desfavorable. Sin una estrategia coherente para abordar los problemas económi- La mayoría de las operaciones con componentes de cos fundamentales las posibilidades de éxito reducción de la pobreza evaluadas por el DEO en son escasas, y en ese caso podría ser más ade- 1993 y 1994 tuvieron como fin respaldar los esfuer- cuado utilizar los servicios no financieros para zos por aumentar el ingreso de algunos beneficiarios restablecer políticas adecuadas. Es posible que relativamente pobres. Dichas operaciones se cen- en el caso de los países con un marco inade- traron en el mejoramiento de las prácticas agrí- cuado de políticas y un desempeño de la cartera colas, la creación de oportunidades de empleo, decepcionante el diálogo del Banco con los paí- el acceso al crédito, el mejoramiento de la ses y la coordinación de la asistencia, junto con vivienda y la ampliación de los servicios socia- una labor económica y sectorial, sean la estrate- les y los sistemas de protección social. Casi la gia más eficaz en función de los costos. Si un mitad de las operaciones con componentes de gobierno manifiesta su firme propósito de apli- reducción de la pobreza se llevaron a cabo car reformas institucionales podría estar justifi- en el grupo de países de ingreso más bajo. cada la concesión de pequeños préstamos para el desarrollo de la capacidad. La experiencia adquirida en el uso de componentes relacionados con la pobreza indica la importancia de: La pobreza, la sostenibilidad ambiental * Los procesos de participación "de abajo y el desarrollo del sector privado arriba" que favorezcan la intervención de (Capítulo 3) los beneficiarios en la formulación y ejecu- ción de las operaciones. ¿Con qué frecuencia y en qué grado se reflejan los tres grandes objetivos del Banco -la reduc- * Los enfoques basados en la cooperación ción de la pobreza, el incremento de la sosteni- que fomenten la confianza de los beneficia- 12 rios y alienten el espíritu de empresa y capacidad institucional de ordenación la innovación. ambiental. La experiencia adquirida en este terreno debe transmitirse a otros países " La regularización de los títulos de propie- y regiones. dad de la tierra a fin de alentar la participa- ción de los beneficiarios en los programas a La sensibilización de los beneficiarios sobre de desarrollo. los peligros para el medio ambiente puede conseguir resultados positivos y duraderos. " La adopción de medidas innovadoras de financiamiento que fomenten el autofinan- a Las medidas destinadas a incrementar el ciamiento por parte de los beneficiarios y nivel de preparación ante las emergencias refuercen los vínculos entre las instituciones ambientales deben formularse con anticipa- públicas y privadas. ción, y a menudo exigen un rápido respaldo financiero y técnico. * Los programas sociales, como los sistemas de protección y la prestación de servicios Desarrollo del sector privado básicos, a fin de proteger a los pobres de los efectos negativos del ajuste. A fin de respaldar el desarrollo del sector pri- vado, el Banco promueve la adopción de medi- Sostenibilidad ambiental das destinadas a mejorar el entorno económico, la reforma de las empresas públicas y la privati- El Banco lleva adelante programas focalizados, zación en los casos en que es indudable que el incluidos proyectos con fines específicos, progra- sector privado haría un uso más eficiente de los mas a nivel regional y mundial y estrategias y polí- recursos, y la participación del sector privado en ticas destinadas a reforzar la sostenibilidad el suministro de infraestructura, además de con- ambiental. Los tres tipos de proyecto más comunes tribuir a la reforma del sector financiero. Cuando son los de gestión de la contaminación y del medio son eficaces, las operaciones de ajuste facilitan el ambiente urbano, los de gestión de los recursos desarrollo del sector privado al contribuir a crear naturales y el medio ambiente rural, y los destina- un entorno económico favorable y competitivo. dos a reforzar la capacidad de gestión ambiental a De las 737 operaciones evaluadas, las que tenían nivel nacional y local y a incrementar la participa- componentes de desarrollo del sector privado ción de la comunidad en la formulación de decisio- eran 254 y sumaban en total US$22.000 millones nes sobre el medio ambiente. De las 737 operacio- en compromisos de préstamo. Las operaciones nes evaluadas, las que tenían componentes orientadas a la reforma del marco normativo ambientales eran 196, que representaban compro- absorbieron casi la cuarta parte de los recursos misos de préstamo por US$13.400 millones. destinados al desarrollo del sector privado. La experiencia adquirida en relación con los compo- De la experiencia adquirida en el terreno del nentes ambientales indica que: desarrollo del sector privado se desprenden las siguientes conclusiones: " Deben seguir haciéndose evaluaciones del impacto ambiental una vez que los proyec- a Lo que más cuenta es la solidez del entorno tos han sido puestos en servicio. macroeconómico. * La contribución más importante del Banco a a En los países en que tiene lugar una transfor- largo plazo podría ser el desarrollo de la mación radical desde una economía dirigida 13 a una economía de mercado la aplicación de rabilidad. Las evaluaciones de este año, como un programa gradual pero ininterrumpido las realizadas en años anteriores, indican que de estabilización y liberalización podría ser en las áreas más complejas del desarrollo de la clave para mejorar la economía y alentar la capacidad y la reforma de las políticas a la participación del sector privado. menudo no se logran los objetivos fijados; el 77 por ciento de las operaciones evaluadas en " Los gobiernos deben manifestar inequívoca- 1994 lograron sus objetivos físicos. mente su decisión de no dar marcha atrás en la adopción de las reformas. En las evaluaciones del DEO se pone de manifiesto, una y otra vez, la importancia de una política " Cuando los gobiernos están decididos a económica adecuada. La experiencia indica adoptar reformas orientadas al desarrollo que un entorno económico estable y abierto del sector privado, las iniciativas, medidas y tiene un efecto positivo en los resultados de acciones en materia de políticas preceden la los proyectos. Al adoptar decisiones a nivel participación del Banco. microeconómico, o a nivel de cada proyecto, los gobiernos deben tomar plenamente en " En los casos en que las reformas han sido efi- cuenta el entorno económico en que se ejecutan caces, el Banco contribuyó mediante una los proyectos. adecuada labor sectorial y un respaldo efi- caz en las áreas técnica y normativa, mante- Calidad inicial: Las observaciones realizadas niéndose siempre en segundo plano. demuestran inequívocamente la importancia de la calidad inicial para el resultado de los " En los casos en que los mercados de capital proyectos y el desempeño de la cartera. La cali- se encuentran en etapa de formación, los dad de la preparación de los proyectos y de su préstamos a un intermediario financiero evaluación inicial son mucho más importantes -otorgados con un mínimo de regulación para los resultados de los proyectos que las y en las condiciones que dicta el mercado- variables macroeconómicas fundamentales pueden ser instrumentos eficaces para del país, que los factores externos o que los encauzar el crédito hacia los empresarios aspectos relativos al sistema de gobierno. privados. Gestión de la cartera existente: Una gestión apropiada de la cartera, incluida la gestión de Aumento de la eficacia del desarrollo riesgos, influye considerablemente en los resul- (Capítulo 4) tados de las políticas de desarrollo. La eficacia de la labor de ejecución de los prestatarios y la En los últimos años, el alcance, envergadura y supervisión del Banco son igualmente impor- complejidad de las operaciones del Banco han tantes. No obstante, en la labor de supervisión aumentado considerablemente. Las operacio- se sigue dando, en general, poca importancia a nes evaluadas en 1994 tenían más objetivos y ciertos factores que permitirían al Banco acon- componentes que las evaluadas en años ante- sejar la aplicación, cuando sea necesario, de riores. Se estima que, de las operaciones evalua- medidas correctivas a mitad de camino a fin de das en 1994, dos terceras partes pusieron a dura lograr los objetivos de desarrollo del proyecto. prueba la capacidad de ejecución de los países La diferencia entre la calificación de los resulta- prestatarios. Casi la mitad tienen múltiples dos de la labor de supervisión --que a menudo objetivos y requieren la participación de nume- resulta a ser demasiado optimista- y la evalua- rosos organismos, y presentan una gran vulne- ción de los proyectos terminados se está 14 reduciendo, si bien sigue siendo considerable. u uso de los servicios no financieros para Para salvar esta diferencia será necesario reali- seleccionar más eficazmente los proyectos, zar una gestión más decidida de la cartera, cen- fomentar la identificación de los prestata- trada en los objetivos de desarrollo, y aplicar rios, desarrollar la capacidad interna y refor- mejor las actuales directrices sobre seguimiento zar las asociaciones. y evaluación. Desarrollo de la participación: Como han demos- Las siguientes medidas contribuirían a mejorar trado numerosos ejemplos de prácticas ópti- los resultados y la sostenibilidad de los bene- mas, la participación de las partes interesadas ficios, sin disminuir la pertinencia de la en la formulación y ejecución de los proyectos labor realizada: puede tener un efecto muy positivo en los resul- tados. Del mismo modo, la participación es fun- " mejoramiento de los procesos económicos damental para alentar la identificación y el mediante una labor sistemática de aprendi- compromiso del gobierno con respecto a la zaje por parte del personal, los prestatarios y adopción de reformas viables del marco norma- las demás partes interesadas; tivo. Al aumentar la complejidad y dificultad de las operaciones en el área del desarrollo, tanto " fortalecimiento de la evaluación de riesgos para los prestatarios como para el Banco, la par- en la formulación de las estrategias para paí- ticipación puede mejorar considerablemente la ses y en la evaluación de la eficacia en la gestión de riesgos y contribuir a minimizarlos. etapa inicial; Finalmente, para que la cooperación para el desarrollo sea eficaz, es necesario que el Banco " identificación y consideración de los aspec- colabore, en forma coordinada y mutuamente tos políticos y económicos durante todo el beneficiosa, con otros organismos de desarrollo ciclo de los proyectos, y y organizaciones no gubernamentales. 15 Résumé analytique Ce vingt et unième Examen annuel des résultats qu'il faut continuer d'améliorer la justification de l'évaluation rétrospective des projets pré- des résultats. sente une analyse des conclusions des évalua- tions effectuées en 1994 et décrit les effets * Soixante-six pour cent des opérations éva- qu'elles pourraient avoir sur la gestion des opé- luées en 1994 ont eu des résultats satisfai- rations de la Banque. Il se fonde sur les rapports sants, soit un pourcentage comparable à d'achèvement établis par les services opération- celui obtenu pour la cohorte des opérations nels et sur les rapports d'évaluation rétrospec- évaluées l'an dernier (64 pour cent). Bien tive, les études d'impact et les études effectués qu'il soit supérieur à la moyenne de 63 pour par le Département de l'évaluation des opéra- cent enregistrée pour 1990-94, ce pourcen- tions (OED). tage est toujours beaucoup trop faible. En 1994, 246 opérations achevées ont fait l'objet * Quarante-quatre pour cent des opérations d'une première évaluation (la « cohorte de évaluées devraient continuer à procurer 1994 »). La plupart de ces prêts et crédits, qui des avantages durant toute la phase opéra- représentent des engagements de 17 milliards tionnelle qui débute lorsque le prêt de la de dollars, sont récents; environ les trois quarts Banque a été entièrement décaissé. Cette ont été approuvés en 1985 ou plus tard et près proportion n'est pas très différente de celle de la moitié ont été clos en 1993 ou ultérieure- de la cohorte de l'an passé et de la moyenne ment. L'OED a aussi établi des rapports d'éva- pour 1989-94. Le pourcentage des opéra- luation rétrospective pour 108 opérations dont tions qui ne continueront probablement pas les rapports d'achèvement avaient été préparés de procurer des avantages est, en revanche, les années précédentes. L'examen s'inspire, s'il tombé de 24 pour cent en 1993 à 20 pour cent y a lieu, des informations tirées de l'ensemble en 1994. des 354 opérations évaluées en 1994. Les objectifs de développement institution- nel ont été atteints pour l'essentiel dans Le bilan des résultats (Chapitre 1) 39 pour cent des opérations - un pourcen- tage supérieur aux 30 pour cent constatés Les résultats de cette année corroborent et ren- pour la cohorte de 1993 et à la moyenne de forcent pour l'essentiel ceux des deux dernières 31 pour cent enregistrée pour les cinq der- années. Ils attestent d'un certain nombre de nières années, mais un résultat lui aussi signes encourageants, mais confirment bien peu satisfaisant. 16 * Les opérations d'ajustement ont poursuivi une amélioration notable pour les projets de l'amélioration de leur performance notée transport. Les résultats se sont toutefois dégra- pour la première fois il y a deux ans, dés dans l'ensemble, en particulier dans les et ont, dans l'ensemble, obtenu de meilleurs domaines de l'approvisionnement en eau, de résultats que les projets d'investissement. l'électricité et des finances. Soixante-dix pour cent des opérations d'ajustement ont reçu une notation Viabilité à long terme satisfaisante, contre 65 pour cent des projets d'investissement. Par viabilité à long terme, on entend la mesure dans laquelle une opération pourra probable- Résultats ment maintenir ses accomplissements effectifs ou escomptés durant sa phase opérationnelle. Plus des deux tiers des opérations analysées Pour noter la viabilité à long terme, les services avaient des objectifs qui correspondaient de l'évaluation rétrospective évaluent un grand largement ou substantiellement aux priorités nombre des facteurs qui ont déjà servi de base à de développement. Quarante-cinq pour cent l'évaluation de l'opération, en se plaçant, selon des opérations sont considérées avoir réussi, le cas, dans une optique économique, sociale, dans une mesure substantielle ou importante, financière, institutionnelle et environnemen- à atteindre leurs objectifs et 34 pour cent d'entre tale. La viabilité est un test plus exigeant encore elles y sont parvenues de manière modérément que le critère des résultats. L'expérience montre effective. Comme l'année passée, la cohorte qu'il faut parfois réaliser une série d'opérations a plus facilement atteint les objectifs matériels pour assurer la viabilité voulue, car celle-ci que des objectifs tels que la réforme macroéco- dépend souvent de politiques plus favorables nomique ou la réforme d'un secteur, les amélio- ou d'institutions plus fortes que celles qui peu- rations financières et le développement vent être en place lorsque la Banque entre en institutionnel. Quarante et un pour cent scène pour la première fois. des opérations ont été jugées avoir été grandement ou sensiblement efficientes et Les notations de viabilité ne se sont, dans l'ensemble, 32 pour cent l'ont été modérément. pas modifiées par rapport à l'année précédente. En Asie de l'Est, le pourcentage des opérations L'Asie de l'Est et l'Asie du Sud enregistrent les jugées probablement viables à long terme est meilleurs résultats pour tous les indicateurs de passé à 73 pour cent. En Afrique, il est de la performance globale. 23 pour cent. Par contre, il a été jugé que 26 pour cent seulement des opérations - con- Au niveau des secteurs, les meilleurs résultats tre 31 pour cent de 1989 à 1994 - avaient peu ont été obtenus pour les ressources humaines de chances d'être viables à long terme. (83 pour cent des opérations ont été jugées satisfaisantes) et les pires pour l'agriculture, où Si l'on classe les opérations en fonction de les opérations ont donné des résultats supé- l'année où elles ont été approuvées, la propor- rieurs à la moyenne de la période 1990-94, mais tion de projets dont les bénéfices ont des n'ont toutefois obtenu une notation satisfai- chances de durer est passée de 42 pour cent sante que dans 60 pour cent des cas. pour les opérations approuvées en 1984 à 51 pour cent pour celles qui ont été approuvées Si l'on compare la performance des opérations l'année suivante. Il n'a cependant pas encore été approuvées de 1974 à 1979 à celle des opéra- procédé à l'évaluation rétrospective d'un nom- tions approuvées de 1980 à 1985, on constate bre suffisant d'opérations approuvées après 17 1984 pour pouvoir affirmer l'existence d'une Les prévisions de la durée d'exécution et des évolution positive. coûts deviennent plus réalistes. A l'appui des conclusions des évaluations * Les retards par rapport au calendrier dimi- rétrospectives précédentes, la détermination nuent, mais les prévisions sont toujours exa- des pouvoirs publics à atteindre les objectifs des gérément optimistes. En moyenne, la projets, l'efficacité des institutions et de la ges- période d'exécution des opérations ayant tion, un cadre politique favorable, la viabilité fait l'objet d'une évaluation rétrospective en technique et le recouvrement des coûts ont eu 1994 a été plus longue de 37 pour cent que une influence positive sur la viabilité à long prévu; ce retard par rapport au calendrier terme des projets de la cohorte de 1994. est inférieur à celui constaté pour 1993 (48 pour cent) et pour les cohortes de 1974 à Développement institutionnel 1994 (54 pour cent). Des progrès ont été réalisés dans le domaine du déve- * Les estimations de coûts se sont aussi amé- loppement institutionnel, mais les résultats sont liorées. L'Afrique est la seule région où les encore décevants. La part des opérations de la opérations ont coûté plus que prévu (les cohorte de 1994 qui ont en grande partie réalisé dépassements de coûts ont atteint 16 pour leurs objectifs en matière de développement ins- cent). Dans les autres régions, les coûts ont titutionnel est passée à 39 pour cent, ce qui repré- été surestimés à raison de 10 à 15 pour cent. sente une amélioration par rapport aux 30 pour cent de l'année précédente et à la moyenne de Opérations ayant obtenu d'excellents 31 pour cent constatée pour les années 1989-94. résultats Les résultats se sont améliorés dans une certaine mesure dans presque tous les secteurs et régions. Un grand nombre de « pratiques optimales » mon- trent qu'il est possible d'améliorer sensiblement les L'engagement des emprunteurs a été le facteur performances du portefeuille. Les services de l'OED qui a le plus influé sur la réalisation des objec- considèrent qu'une opération a obtenu d'excel- tifs de développement institutionnel. lents résultats lorsqu'elle a réussi à dépasser tous ses objectifs principaux, qu'elle a une con- Taux de rentabilité, durée d'exécution ception très novatrice et/ou suit une démarche et coûts qui peut donner de bons résultats dans des pays ou des secteurs différents. Sur les 354 opérations Les projets ont eu des taux de rentabilité élevés. Les qui ont fait l'objet d'une évaluation rétrospective taux de rentabilité ont été calculés au moment en 1994, I'OED a déterminé que 18 d'entre elles de l'évaluation et ont été réestimés au moment ont produit d'excellents résultats sur la base des de l'évaluation rétrospective pour 82 projets, évaluations rétrospectives ou des examens des soit un tiers de la cohorte de 1994. Le taux de rapports d'achèvement. Les opérations diverses rentabilité réestimé de la plupart de ces projets qui composent ce groupe ont certaines caracté- est bien supérieur à 10 pour cent. Il est, par sec- ristiques en commun: teur, de 17 pour cent pour l'agriculture, 17 pour cent pour l'électricité, 28 pour cent pour les * Elles ont été suggérées par les emprunteurs, transports et 25 pour cent pour le développe- la Banque étant considérée soit comme un ment urbain. L'écart entre le taux de rentabilité agent catalyseur objectif du changement, estimé à l'évaluation et le taux réestimé une fois soit comme la source des connaissances le projet achevé semble diminuer. techniques nécessaires. 18 " Elles avaient une structure relativement ment en monnaie nationale pour le projet, simple. les procédures administratives utilisées et la qualité de la prise de décisions. Les " La Banque a pris une part active à l'opération projets mal gérés par l'emprunteur et dont et en a soigneusement suivi le déroulement. les clauses du prêt n'étaient guère respectées ont généralement donné des résultats médio- " Les opérations ont été exécutées de manière cres, d'une viabilité incertaine ou très pragmatique et avec souplesse. douteuse, et n'ont guère contribué au déve- loppement institutionnel. " Nombre de ces opérations ont tiré parti de relations établies de longue date avec La manière dont la Banque s'acquitte de sa tâche la Banque, de prêts antérieurs et/ou est toutefois presque aussi importante. La qualité d'études économiques et sectorielles de de l'évaluation et une supervision étroite jouent grande portée. un rôle crucial. De manière générale, la qualité de l'instruction des dossiers par la Banque influe dans une mesure considérable sur les Facteurs déterminant la performance du résultats, dans une mesure sensible sur la viabi- portefeuille (Chapitre 2) lité à long terme des avantages procurés par le projet et dans une mesure limitée sur le degré Sur quel aspect du cycle du projet faudrait-il de développement institutionnel. que la Banque porte son attention pour assurer une performance du portefeuille satisfaisante? Au niveau des portefeuilles-pays, l'effet de tous Dans quelle mesure la Banque peut-elle atté- les facteurs déterminants au niveau du projet a nuer les risques inhérents à l'environnement été confirmé par les analyses qui montrent toutefois économique et socio-politique d'un pays et au que les résultats sont aussi fortement influencés comportement de l'emprunteur? par la stabilité macroéconomique, la croissance du revenu par habitant et la bonne gestion des L'analyse des facteurs déterminant la perfor- affaires publiques. mance du portefeuille qui a été effectuée cette année couvre toutes les opérations qui ont fait Les facteurs déterminants de la performance du l'objet d'une évaluation rétrospective par portefeuille ne revêtent pas tous la même importance l'OED entre 1991 et 1994. Elle utilise des d'un pays à un autre. Les résultats impliquent techniques quantitatives plus rigoureuses que les améliorations de la performance de la qu'auparavant et modifie partiellement les Banque au niveau de l'instruction du dossier, conclusions antérieures de l'OED. notamment un relèvement de la qualité des projets au moment où ils sont inscrits au porte- De toutes les variables indépendantes incluses feuille, permettent d'améliorer dans une plus dans l'analyse, c'est la performance de l'emprunteur large mesure la performance du portefeuille durant la préparation et l'exécution du projet qui dans les pays où cette dernière est déjà supé- a le plus d'effet sur les résultats de ce projet. rieure à la moyenne. Une évaluation et une Par « performance de l'emprunteur », on supervision satisfaisantes de la Banque peuvent entend : le cadre de politique générale mis en influer sensiblement sur les résultats et la viabi- place pour les projets financés par la Banque, lité à long terme, mais l'effort qui doit être la mesure dans laquelle les pouvoirs publics fait pour compenser les effets d'un cadre de sont déterminés à atteindre les objectifs du politique générale et d'exécution peu propice projet, la fourniture et la fiabilité d'un finance- est considérable. 19 Implications pour les priorités un dialogue avec la Banque et à coordonner opérationnelles de la Banque l'aide, sur la base d'études économiques et sec- torielles. Il peut aussi être justifié de consentir Les conclusions suggèrent que le moyen le plus des prêts de montant limité à l'appui d'un ren- direct d'améliorer la performance du porte- forcement institutionnel lorsque le gouverne- feuille est de consentir des prêts de manière plus ment s'est engagé à entreprendre une réforme sélective. Etant donné le caractère coopératif de en ce domaine. la Banque, il faut évidemment mettre en balance ce besoin de sélectivité et le droit légitime des pays membres d'obtenir les services de l'institu- Pauvreté, viabilité de l'environnement tion. Dans les pays où l'environnement écono- et développement du secteur privé mique est peu propice aux prêts, la fourniture (Chapitre 3) de services autres que des prêts peut être un moyen efficace au plan des coûts de maintenir le Avec quelle fréquence et dans quelle mesure les contact et de contribuer aux améliorations qui trois objectifs thématiques de la Banque - lutte doivent être apportées à la politique générale. contre la pauvreté, environnement et développe- ment durable et développement du secteur Dans les pays où la performance du portefeuille est privé - sont-ils poursuivis par des composantes généralement médiocre, la Banque devrait axer son spécifiques des projets? Avec quel succès les opé- appui sur la stabilisation macroéconomique et des rations ont-elles atteint leurs objectifs thémati- réformes structurelles favorables à la croissance. ques déclarés et comment peut-on expliquer les Une amélioration systématique du cadre de différences constatées au plan des résultats? politique interne devrait permettre d'accroître dans une large mesure la qualité des résultats Les 737 opérations que l'OED a évaluées des opérations - tant en ce qui concerne leur en 1993 et 1994, par le biais d'évaluations impact sur le développement durant la mise en rétrospectives et d'examens des rapports oeuvre que la durabilité des avantages qu'ils d'achèvement, ont été analysées en vue d'iden- procurent. On voit donc à quel point il importe tifier les pratiques les meilleures et les enseigne- que les aspects fondamentaux de l'économie ments à tirer de l'expérience. Comme pour soient bien établis - ce qui est aussi la princi- l'Examen annuel de l'an passé, seules les opéra- pale raison d'être des prêts à l'ajustement. tions dont certaines composantes visaient de manière spécifique les objectifs thématiques Il conviendrait de revoir la pratique actuelle de ont été prises en compte; beaucoup d'autres ont la Banque qui consiste à maintenir des niveaux pu avoir des effets positifs ou négatifs sur la minimums de prêt dans les pays dont le cadre de pauvreté, l'environnement ou le développe- politique générale est défavorable. En l'absence ment du secteur privé. d'une stratégie cohérente concernant les aspects fondamentaux de l'économie, les chances de Réduction de la pauvreté succès d'un projet sont médiocres et il peut être plus intéressant de fournir des services hors Parmi les 737 opérations que l'OED a évaluées en prêts visant à rétablir des politiques appro- 1993 et 1994, la plupart de celles qui étaient dotées priées. Dans les pays dont le cadre de politique de composantes axées sur la lutte contre la pauvreté générale est peu favorable et dont le porte- ont appuyé des efforts qui visaient à relever le niveau feuille ne produit généralement que des résul- des revenus de bénéficiaires relativement pauvres. tats décevants, la démarche la plus effective au Elles ciblaient l'amélioration des pratiques agri- plan des coûts consiste probablement à établir coles, la création d'emplois, l'accès au crédit, la 20 fourniture de meilleurs logements, ainsi que milieu rural; et les projets visant à renforcer les l'amélioration des services sociaux et de pro- capacités nationales et locales de gestion de grammes de « filets de sécurité sociale ». Près de l'environnement et la participation du public à la moitié des opérations dotées de composantes la prise des décisions liées à l'environnement. axées sur la lutte contre la pauvreté ont été réa- Sur les 737 opérations examinées, 196, pour les- lisées dans des pays appartenant au groupe de quelles 13,4 milliards de dollars ont été engagés revenu le plus faible. sous forme de prêts, avaient des composantes relatives à l'environnement. L'expérience acquise dans le cadre de ces compos- antes axées sur la lutte contre la pauvreté fait ressor- De l'expérience acquise dans le cadre de ces compo- tir l'importance de: santes environ nementales se dégagent les enseigne- ments suivants " méthodes participatives partant de la base qui renforcent le rôle des bénéficiaires dans a Il conviendrait de poursuivre les évalua- la conception et l'exécution des opérations; tions des effets des projets sur l'environne- ment après leur adoption. " méthodes coopératives qui accroissent la confiance des bénéficiaires et encouragent a Le renforcement de la capacité institution- l'esprit d'entreprise et l'innovation; nelle de gestion de l'environnement pour- rait être la contribution à long terme la plus " la régularisation des titres de propriété fon- importante de la Banque. Lorsque les efforts cière pour encourager la participation des déployés à ce titre sont couronnés de succès, bénéficiaires et leur engagement face aux il faudrait faire profiter d'autres pays et efforts de développement; régions de l'expérience acquise au plan des institutions. " mesures financières novatrices qui encoura- gent l'autofinancement par les bénéficiaires * La sensibilisation des populations ciblées et renforcent les liens entre les institutions aux dangers environnementaux pourrait des secteurs public et privé; avoir des effets positifs durables. " programmes d'action sociale, comme des a Les efforts de renforcement de la planifica- filets de sécurité et la prestation de services de tion préalable aux catastrophes environne- base, pour protéger les populations pauvres mentales doivent être entrepris le plus tôt des conséquences négatives de l'ajustement. possible et, souvent, doivent comporter une réponse rapide assortie d'un soutien finan- Viabilité écologique cier et technique. La Banque poursuit des programmes ciblés - Développement du secteur privé parmi lesquels des projets spécifiques, des initiatives régionales et mondiales, et des straté- La Banque appuie le développement du secteur gies et politiques - à l'appui d'un développe- privé en encourageant les efforts visant à amé- ment écologiquement viable. On peut citer ici liorer le cadre des affaires, en favorisant la trois types de projets représentatifs : les projets réforme et la privatisation des entreprises concernant la gestion de la pollution et l'envi- publiques lorsque le secteur privé ferait mani- ronnement urbain; les projets portant sur des festement meilleur usage des ressources exis- ressources naturelles et l'environnement en tantes, en encourageant la participation du 21 secteur privé dans l'infrastructure et en l'arrière-plan, tout en offrant un appui tech- appuyant la réforme du secteur financier. De nique et des directives efficace. façon plus générale, de bonnes opérations d'ajustement contribuent au développement du * Lorsque les marchés financiers sont en cours secteur privé en aidant à créer un cadre compé- de développement, l'octroi de prêts à des titif et propice pour les activités industrielles et intermédiaires financiers - avec un mini- commerciales du secteur privé. Sur les 737 opé- mum de réglementations et à des conditions rations examinées, 254 étaient dotées de com- dictées par les forces du marché - peut être posantes axées sur le développement du un moyen utile de mettre des crédits à la dis- secteur privé, au titre desquelles 22 milliards position des entrepreneurs du secteur privé. de dollars ont été engagés sous forme de prêts. Les opérations de réforme des politiques ont absorbé près du quart des ressources engagées Amélioration de l'efficacité du pour ces composantes. développement (Chapitre 4) De l'expérience acquise dans le cadre des compo- La portée, les visées et la complexité des opéra- santes de développement du secteur privé, on peut tions de la Banque ont considérablement aug- tirer les enseignements suivants: menté ces dernières années. Les opérations qui ont fait l'objet d'une évaluation rétrospective en * L'environnement macroéconomique est le 1994 poursuivent plus d'objectifs et ont plus de facteur le plus important. composantes que les opérations évaluées anté- rieurement. Les deux tiers de celles qui ont été " Dans les pays qui connaissent une transfor- évaluées en 1994 sont jugés avoir poussé à mation radicale puisqu'ils passent d'une l'extrême limite la capacité d'exécution des économie dirigée à une économie de mar- pays qui ont contracté des emprunts à leur titre. ché, la poursuite d'un programme progres- Près de la moitié poursuivent des objectifs mul- sif mais continu de stabilisation et de tiples avec l'aide de nombreux organismes et libéralisation pourrait être le moyen d'amé- s'exposent à divers risques. Comme celles qui liorer le climat économique et de stimuler les ont tout juste précédées, les évaluations l'intérêt du secteur privé. rétrospectives de cette année montrent que c'est dans les domaines plus délicats du renforce- " Les pouvoirs publics doivent témoigner de ment institutionnel et de la réforme de la poli- leur engagement en indiquant sans aucune tique générale que les résultats n'ont le plus équivoque qu'ils ne reviendront pas sur les souvent pas répondu à l'attente; les objectifs réformes introduites. physiques ont été atteints dans 77 pour cent des opérations évaluées en 1994. " Lorsque les gouvernements sont fermement déterminés à poursuivre le développement Les évaluations rétrospectives de l'OED ont montré du secteur privé, des initiatives, mesures et à maintes reprises l'importance d'un bon climat éco- actions sont prises par les pouvoirs publics nomique. L'expérience confirme qu'un environ- avant que la Banque ne participe à ces efforts. nement économique stable et ouvert influence de manière positive les résultats d'un projet. " Chaque fois que les efforts de réforme liés au Les décisions du gouvernement au niveau développement du secteur privé ont réussi, micro, ou au niveau des projets spécifiques, la Banque avait produit des études sectori- doivent tenir pleinement compte du cadre éco- elles de haute qualité et était restée à nomique dans lequel les projets sont mis en 22 place pour que les objectifs recherchés puissent * améliorer les procédures commerciales en être atteints. formant systématiquement à cet effet les ser- vices concernés, les emprunteurs et autres Une qualité élevée des projets dès l'inscription au parties intéressées; portefeuille: La mesure dans laquelle la perfor- mance des projets et du portefeuille dépend de * renforcer le rôle de l'évaluation des risques la qualité des projets inclus dans le portefeuille lorsque sont mises au point les stratégies- ressort très clairement des observations. La pays et qu'est évaluée la qualité des projets qualité de la préparation et de l'évaluation au moment de l'inscription au portefeuille; influe sensiblement plus sur la performance d'un projet que les variables macroécono- * reconnaître durant tout le cycle d'un projet miques fondamentales d'un pays, des facteurs les considérations d'économie politique et extérieurs ou des considérations liées à la en tenir compte; gestion des affaires publiques. * utiliser des services hors prêts pour amélio- La gestion du portefeuille en cours : Une bonne rer la sélection des projets, encourager les gestion du portefeuille, et notamment la gestion emprunteurs à prendre en charge les opéra- des risques, influe grandement sur les résultats tions, développer les capacités nationales et obtenus au plan du développement. Il en est de renforcer les partenariats. même de la qualité de l'exécution par l'emprun- teur et de la supervision par la Banque. Or, une Les partenariats pour le développement : Ainsi grande partie du travail actuel de supervision que le montrent de nombreux exemples de pra- n'accorde pas l'attention nécessaire aux facteurs tiques optimales, la participation des intéressés qui permettraient à la Banque de conseiller les à la conception et à l'exécution d'un projet peut modifications à mi-parcours qui peuvent être avoir un effet nettement positif sur les résultats. nécessaire pour atteindre les objectifs de déve- Ce sens de la participation est aussi crucial loppement des projets. L'écart diminue entre les pour encourager les pouvoirs publics à notations de la performance données à l'occa- prendre en charge les opérations et à s'engager sion de la supervision, qui sont souvent trop vis-à-vis de réformes durables. Les projets optimistes, et les notations de l'évaluation de développement devenant de plus en plus rétrospective données aux projets achevés, complexes et absorbants, aussi bien pour mais il est encore trop élevé. Il faudra, pour le les emprunteurs que pour la Banque, le partena- combler, assurer une gestion plus énergique du riat peut grandement améliorer la gestion des portefeuille, axée sur les objectifs de développe- risques qui doivent être pris ou aider à les ment, et mettre mieux en oeuvre les principes réduire au maximum. Enfin, pour que des directeurs applicables au suivi et à l'évaluation. partenariats constitués aux fins du développe- ment soient fructueux, il faut que la Banque Certaines mesures contribueraient à améliorer et d'autres organismes de développement les résultats et la viabilité à long terme des béné- et des organisations non gouvernementales, fices sans toutefois sacrifier la pertinence des selon le cas, mènent une action concertée opérations, qui consisteraient à: et complémentaire. 23 1. The performance record Introduction encouraging signs and confirming the need for greater accountability for results. This twenty first Annual Review of Evaluation Results analyzes evaluation findings for 1994 Sixty six percent of the operations evaluated and traces their implications for the management in 1994 had satisfactory outcomes. Thus one of Bank operations. It is based on completion in three is judged not to have met its major reports by operational staff and on performance relevant goals and/or not to have made an audit reports, impact evaluations, and evalua- acceptable contribution to development. tion and process studies by the independent This year's percentage compares with 64 Operations Evaluation Department (OED). percent in the cohort of operations evalu- ated last year, and is better than the average In 1994, 246 completed operations were evalu- of 63 percent for 1990-94, though it is still far ated for the first time (the "1994 cohort") and too low to be acceptable. (See Box 1.1 and OED produced performance audit reports for Annex Table 1.7.) 108 operations whose completion reports had been prepared in earlier years. Where relevant * Measured in terms of real disbursements, the review draws on the information from all the share of operations with satisfactory out- the 354 operations evaluated in 1994. The 1994 comes is 65 percent, down from 71 percent cohort, representing disbursements of $17.2 bil- in 1993 and 67 percent for 1990-94. (See lion, reflects Bank performance throughout the Figure 1.1, Annex Table 1.8.) late 1980s and early 1990s; about three fourths of the operations in the cohort were approved * Forty four percent of the evaluated operations in 1985 or later, and nearly half were closed in are expected to sustain their benefits through- 1993 or later (Annex Table 1.1). Even so, many out the operational phase that follows the of them were conceived and implemented completion of loan disbursements. This pro- before the Bank began its ongoing action plan in portion is little different from last year's and response to the Portfolio Management Task the average for 1989-94. But the proportion of Force recommendations. operations rated unlikely to sustain their ben- efits has dropped to 20 percent from 24 per- Overview of results cent in 1993 (Figure 1.1, Annex Table 1.10). This year's results essentially consolidate those of * Institutional development goals were sub- the last two years, while evincing a number of stantially achieved in 39 percent of the 25 Box i.: DIFFERENT PERSPECTIVES ON PERFORMANCE Box Figure 1 shows the historical per- those that have performed much better than formance trend including the 1994 evalua- average. tions. Given that these evaluations corre- spond to operations approved and closed in Box Figure 3, with operations grouped by clos- different years, the graph may not reflect a ing year, provides yet another perspective in true trend. which operations evaluated in 1994 are added to the previous evaluations. From this view the In Box Figure 2, however, the 1994 evalua- performance record benefits from the inclusion tions have been added to prior evaluations of more recent evaluations, including a large and grouped according to the year in which number of operations that were under supervi- the operations were approved, providing a sion through 1993. This graph, while it con- more meaningful basis for interpreting the firms improved performance for the period historical performance trend. The graph 1990 to 1992 relative to a low in 1988, shows an shows that for operations approved since estimated decline in performance in 1993. about 1982, performance appears to be on a modest upward trend, particularly if Finally, Box Figure 4 provides a look at the weighted by disbursements. performance of ongoing operations, as mea- sured by their expected achievement of Although many other operations approved development objectives, as displayed in the since 1985 have been evaluated, their Annual Reports on Portfolio Performance, numbers are not yet representative enough to that is, on the basis of supervision ratings. provide a reliable estimate of performance This graph cannot be directly compared with for these years. Adding them to the graph the previous graphs, and it could be affected would impart an upward bias to the histori- by recent efforts to make supervision ratings cal record since the operations evaluated more realistic. Yet, the trend displayed in the earliest for each approval year are typically graph leaves little room for complacency. operations-better than the 30 percent in on average, than investment projects the 1993 cohort and the average of 31 per- (Annex Table 1.9). cent for the last five years (Figure 1.1, Annex Table 1.12). The 1994 cohort in context The adjustment operations evaluated in The range, ambition, and complexity of the Bank's 1994 kept up the improvement in perfor- activities have grown considerably over the past 20 mance that first became noticeable two years years. The 1994 cohort is almost five times larger ago, and they performed somewhat better, than its 1974 counterpart. It involves $17 billion 2o (Box 1.1 continued) Box Figure 1: Performance of operations evaluated Box Figure 2: Performance of operations evaluated by year of evaluation, 1974-94 by year of approval, 1967-85 100 - --100-- - -- 90 Y 90 a ¶ / 80 S 70 .070 60~ 60 50 50 40 40 30 30 20 20 10 10 0 ---'-0 - '-- 74 76 78 80 82 84 86 88 90 92 94 67 69 71 73 75 77 79 81 83 85 Evaluation year Approval year - projects - real disbursements - projects - real disbursements Box Figure 3: Performance of operations evaluated Box Figure 4: Performance of ongoing operations In year of closing, 1980-93 by ARPP fiscal year, 1980-95 100 --- 100 90 8') 'w--. 70' 60 50 90 40 30 20 10 0 1 80 80 82 84 86 88 90 92 80 82 84 86 88 90 92 94 Closing year ARPP fiscal year - projects a real disbursements of disbursements for 80 countries, compared cohort, with nearly two thirds of the disburse- with $4 billion for 50 countries in 1974 (Figure ments, agriculture claimed the largest share of 1.2 and Annex Figure 1.1). the 1994 cohort, with 28 percent of the disburse- ments (Annex Figure 1.2). Adjustment lending, The largest borrower of the 1994 cohort was which began in 1980, made up 12 percent of the Brazil, whereas the largest in the 1974 cohort 1994 cohort. was Spain. The 1974 cohort also included loans to Iceland, Israel, Taiwan, and New Zealand. The 1994 cohort is also broadly similar in its sectoral Whereas transport projects dominated the 1974 composition to today's portfolio of ongoing opera- 27 tions. The main difference is that agriculture's FIGuRE 1.1: OUTCOME, SUSTAINABILITY, AND share of the number of operations has declined INSTITUTIONAL DEVELOPMENT as the share of human resources (education, outcome population, health, and nutrition) has grown; 80 today each accounts for roughly one fifth of the portfolio. Smaller changes have occurred in 60 power, industry, and finance, whose shares of the portfolio have dwindled, while program and policy and water and sanitation have expanded. 40 (See Figure 1.3 and Annex Figure 1.3.) 20 Unlike those of 20 years ago, however, most of the operations in the 1994 cohort have multiple goals. o OED classifies operations according to five 1994 1993 1990-94 1974-94 types of goals: macroeconomic reform, sectoral Evaluation i/ears.. Evalution earsreform, physical enhancements, fmnancial E project weight real disbursement weight enhancements, inancil enhancements, and institutional development. sustainability A fifth of the operations evaluated in 1994 had 60 four such goals, another fifth had three, and about a third had two (Annex Table 1.5). 50- 40 Complexity of operations 30 GED rates operations in terms of the demanding- ness, complexity, and riskiness of their objectives: 20 * Demandingness involves a judgment about the degree to which an operation stretches o 9the specific country's and/or the Bank's 1994 1993 1990-94 Evaluation years implementation capacity. * project weight [ real disbursement weight E Complexity is determined by factors such as institutional development the range of policy and institutional 40 -improvements to be attempted, the number of institutions involved, the number of 30 project components and their geographic dispersion, and the number of cofinanciers. g20- * Riskiness is concerned with vulnerability to variations in key factors affecting 10 the operation. 0 Demandingness, complexity, and riskiness rose 1994 1993 1990-94 significantly during the 1980s, substantially Ev,aluationz years increasing the difficulty of implementing Bank operations (Figure 1.4). 28 This year's findings confirm that the more types of FGR .:DSRBTO FEAUTDOEAIN goals an operation attempts to pursue simulta- FIGREI 1994 DISTRIBUTION ORVAUTEDOEAIN neousl, the more complex and difficult it is to imple- B EIN 94AD17 OOT ment, and the more demanding it is on the borrower. 7994 cohort: 246 operations Of operations in the 1994 cohort, 67 percent were highly or substantially demanding on the South Asia (13%)- resources of the countries that borrowed for 'yAfrica (31 %) Middle East and North ~ them; 40 percent were similarly demanding on Africa (I1 I%) __ the Bank. Fifty one percent were highly or sub- satalcope,and 47 percent were highly Latin Anmerica and - ~ L'East Asia and or substantially risky (Annex Table 1.6). Caribbean (24%)Paic(1% Europe and Central Asia (5%) Overall performance of the portfolio is affected 7974 cohort: 50 operations negatively by operations in countries affected South Asia (10%) by civil strife-the so-called "failed states." Middle East and North >.Africa (26%,') The Bank's FY95 Annual Report on PortfolioAfia(7, Performance provides the most current esti- A*East Asia and mates of the impact of those operations on the --Pacific (10%) Bank's overall portfolio performance, that is, LtnAeiaad "5 Caribbean (36%7) Europe and Central that 20 percent of the problem projects are Asia (1017) located in such states. During FY95 the "failed states" included Algeria, Angola, Burundi, Djibouti, Rwanda, Sierra Leone, Somalia, ___________________ Sudan, and Zaire. For operations evaluated in FIGURE 1.3: DISTRIBUTION OF THE FY95 ONGOING 1991-94, the percentage rated satisfactory in PORTOFOLIO, BY REGION these countries is 40 percent, well below aver- age for the Bank's portfolio as a whole, confirm- FY95 ARPP portfolio: 7,742 operations ing the sensitivity of portfolio performance to South Asia (13%) governance problems. Middle Fast and North - Africa (32%) Africa (8%) > 1994 Results Latin America and Caribbean (20%) Outcoes ad CetralEast Asia and OucmsEurope adCnrlPacific (17%) Asia 011%) To assess the outcome of each operation, the Bank's evaluators look beyond physical achievements to assess the relevance of its The 354 operations OFD evaluated in 1994, design and the efficacy, as well as the efficiency, through performance audits and completion of performance. For an operation to be rated reports, were analyzed for the relevance of their satisfactory, its goals must be consistent with objectives. (Where a project is formally restruc- the Bank's country and sectoral assistance strat- tured it is judged against its revised objectives.) egies (relevance) and it must have achieved, More than two thirds of those rated had objec- or be expected to achieve, most of its major tives highly or substantially relevant to devel- goals (efficacy) in a timely and cost-efficient opment priorities. One project in five had only manner (efficiency). moderately relevant objectives, while one in 16 29 had irrelevant objectives; 17 operations could FIGURE 1.4: PROJECT DEMANDINGNESS, COMPLEXITY, not be rated (Annex Table 1.14). AND RISKINESS, BY APPROVAL YEAR, 1980-89 demandingness To assess efficacy, evaluators analyze the extent 100 to which operations achieved their goals. Forty 90 five percent of the operations rated were con- G sidered highly or substantially effective in 70 meeting their goals, and another 34 percent as 60 moderately effective; 19 operations could not be rated (Annex Table 1.14). 50 440 As in last year's analysis, physical goals were met 30 more readily than other goals. Of the 110 opera- 20 tions that evaluators recorded as having physi- cal goals, 77 percent substantially met these o 81 82 83 84 85 86 87 88 89 goals; of the 58 with sector reform goals, only Approval year 60 percent met these goals (Annex Table 1.15). complexity 100 To assess efficiency, evaluators compare eco- 90 nomic rates of return-for types of operations 80 where these can be calculated-as calculated at S70 appraisal and re-estimated at the completion of disbursements. They also compare actual costs 60 and implementation times with the estimates 50 40 made at appraisal, and calculate overall ratings for efficiency. For 41 percent of the operations 30 rated, execution was rated as highly or substan- 20 tially efficient; for 32 percent of the operations 10 as moderate; and for 20 percent as negligible. 0__ 80 81 82 83 84 85 86 87 88 89 Twenty five operations could not be rated. Approval year (Annex Table 1.14.) riskiness 100 Regional outcomes 90 - 80 On all measures of overall performance, East Asia and South Asia had the best record. Measured in 70 terms of numbers of operations rated satisfac- 60 tory in the 1994 evaluation cohort, Africa had 50 the poorest record (57 percent), and Latin 40 America and the Caribbean (LAC) and Middle 30 - East and North Africa (MNA) were somewhat 20 below the Bankwide average of 66 percent. In 10 terms of the percentage of disbursements 0 8 - - - devoted to satisfactory operations, Africa fol- 80 8 82 83 84 85 86 87 88 89 lowed the Asia regions, with 72 percent of its Approval year disbursements invested in satisfactory opera- 30 tions, and MNA and LAC trailed behind.FIUE15PROMACOFPRTON Europe and Central Asia (ECA) had only nine FGUE15PROMACOFPRTIN operations in the 1994 cohort, too small a num- EVALUATED, BY SECTOR AND REGION, 1994 AND ber for meaningful performance comparisons. 1990-94 COHORTS (N>=10) (Annex Tables 1.7 and 1.8.)Rein East Asia and Pacitic --_______ South Asia ____________ Outcomes substantially improved in South Asia and Europe and Central Asia _________ Latin America and Caribbean_______________ LAC, compared with the average for 1990-94. Middle East and North Africa - ____ Outcomes in the Africa and the East Asia and Affica Sector, Pacific (EAP) regions remained about the same. Human resource Urban ______________ Those in MNA were poorer, at 63 percent satis- Transport factory. than in 1990-94 and in 1974-94 (78 per- IndustryMWE" Program and policy cent satisfactory). (Annex Tables 1.7 and 1.8.) Average Power Agriculture ____________________________ Sectoral outcomes Finance ____ 0 20 40 6 so 100 Measured in terms of the proportion of operations Percen saifatr with ~~~~ ~~ E satisactor oucoe inte9949vluto withSatsfator oucoms inthe199 evluaion Note: Technical assistance, telecommunications, and water were omitted cohort, human resource operations performed the due to small sample size. best, with 83 percent of its operations rated satisfac- Source: Annex Table 1.7. tory. (See Figure 1.5 and Annex Table 1.7.) Human resources UComplex projects are institutionally difficult to implement, and unduly rapid expansion Human resource operations performed consistently can compromise quality. Many of the les- well. Historically, these operations have per- sons learned from past rural development formed at levels substantially higher than projects are valid for today's human Bank averages, and this year's results reinforce resource development projects. this pattern. Of the 24 evaluated operations in this sector, 83 percent achieved satisfactory *In creating or expanding vocational training performance; 74 percent were sustainable; programs, the size and occupational cover- and 59 percent (more than in any other sector) age of individual centers should reflect the made substantial contributions to institutional diversity and requirements of the local econ- development. omy. And where industry is playing an increasing role in vocational training, the Evaluation experience points to lessons that centers should fill gaps, protecting the needs may help to safeguard and improve quality in of disadvantaged groups, and meet newly these operations. emerging requirements. *Recurrent expenditure and counterpart *The actual or perceived need for speed funding problems are highlighted in almost should not compromise quality; to make all performance audits. This implies that an effective contribution human resource project design should pay more attention to operations need flexible implementation the potential of institutional and community and ongoing monitoring to identify organizations in operating and maintaining impediments to the effective use of these investments, facilities offered. 31 " The architecture of basic infrastructure goals for installing new facilities but were ulti- should be low-cost and simple enough for mately judged unsatisfactory and/or unlikely local community involvement. It should to maintain their benefits. maximize use of local material, adapted to local conditions, and be easy to maintain Several agricultural projects tried to accomplish and operate. "too much, too soon," given the caliber of the organizations they were working through and * Relevance and efficiency call for a focus on the newness of the ideas they were promoting; community and preventive health rather examples include attempts to develop decen- than hospital-based curative care. tralized extension services with nationwide coverage in Gambia, Indonesia, and Malawi. * Human resource interventions should be Several were introduced without enough pilot- carefully designed and differentiated to ing, with weak arrangements for monitoring respond to local geographic, economic, and evaluation (M&E), lack of ownership, and and social differences. insufficient participation. Projects in several countries, including Mauritania, Nepal, and " Expanding and improving social services is Yemen, suffered from designs based on insuffi- often very difficult, especially in impover- cient knowledge of local farming systems and ished areas. Experience emphasizes the social characteristics. Some were well-inten- value of piloting and of flexible, rather than tioned about having beneficiaries participate in "blueprint," approaches to project design, to project design, but naive about how to get this induce learning from experience. to happen. Many projects suffered from diffi- culties in the surrounding economies. In Brazil, a Social funds, under the right circumstances, Cameroon, Indonesia, Kenya, Malawi, and are cost effective ways to assist the poor. Zaire, difficult and/or deteriorating macroeco- Carefully defined goals, quality staff, com- nomic conditions affected projects, causing mitment by higher authorities, openness shortages of operating funds. and transparency, professional oversight, and absence of political interference Recent performance audits of agricultural projects contribute to their effectiveness. suggest the following lessons: Agriculture * Participation by intended beneficiaries in project design and management plays a vital Agriculture is the sector with the poorest perfor- role in agricultural projects (Box 1.2). Much of mance in the 1994 evaluation cohort, and evaluation the experience shows that project design and data by approval year show that agriculture's overall government administration of services can performance has consistently been relatively poor. help make beneficiary groups such as irriga- Eighty one agricultural operations were evalu- tors' associations more effective. Community ated in 1994. They performed somewhat better dynamics and the relations among group than the average for 1990-94, but still achieved members also have a major influence and only a 60 percent satisfactory rating. need to be understood by project designers. Looking at the agricultural projects rated unsat- U Pilot operations are needed wherever there isfactory, many of their characteristic problems is lack of experience with the environmental occurred in widely differing countries and conditions, the technology to be adapted, or regions. Many managed to meet their physical the institutional arrangements. 32 Box 1.2: BENEFICIARY GROUPS Irrigator groups: OED's 1994 study of irriga- audited in 1994, worked to develop such tion outlined the conditions associated with local producers' associations, generally with durable irrigator groups: some success. " Geographic and jurisdictional boundaries Their experience showed that herders and are clear. farmers' associations can successfully take " Members' financial obligations are pro- over some of the functions of government portional to their benefits. services, if carefully established. Herders' " Members are represented in decisions on associations are easiest to form and sustain if water allocation and financial contributions. members have few and real collective inter- * Auditors of financial statements are ests. In the three projects, the leading collec- accountable to irrigators. tive interests were: potable and stock water; * Rule violators are punished according to better human and animal health; grazing their transgressions. management; and literacy programs--espe- * Disputes are resolved locally and cheaply. cially if targeted to women. * Governments do not challenge decisions made by irrigator groups. The northern pastoral development sub- project in Senegal showed the effectiveness of Herders' associations: One of the best ways to a long-term, cumulative approach to institu- create the basis for sustainable development tion building. Careful preparation, with among pastoralists and agro-pastoralists is to important contributions from social scientists promote the creation and institutional devel- with reasonably good socioeconomic data; opment of herders' associations. Projects in continuity of approach; commitment; and Central African Republic and Senegal, suitable personnel were all very important. * Agricultural research and extension projects * Policy-based lending for agriculture require detailed assessments of the technol- succeeds under the right circumstances: ogy extended, available resources, and mon- country-based preparation, ownership, itoring and evaluation arrangements. timeliness, politically viable reforms. Their sustainability depends on adequate budgetary provisions. * Agricultural credit projects require fine- tuning to market conditions to ensure their * Agricultural projects suffer when their onlending arrangements are realistic, that planners take inadequate notice of a the rural financial sector is adequately distorted macroeconomic or sector understood, and the participating financial environment. institutions will remain financially viable. 33 * Fisheries projects need to pay careful atten- of a reform process that has been sustained, tion to the danger of depleting fish resources despite changes in government. and to the adequacy of related monitoring, information, and management systems. Experience in adjustment operations in several poor countries, including Guinea, Nepal, and Sao Sectoral performance trends Tome and Principe, emphasizes the importance of identifying and concentrating on essentials, Comparing the performance of operations leaving enough time for implementation, espe- approved in 1974-79 with that of operations cially where institutions are to be restructured approved in 1980-85, outcomes improved nota- and processes changed. bly in transport operations. But overall, there was a decline in performance, led by water sup- Social safety nets. Safety net instruments that the ply, power, and finance. Bank has supported to accompany adjustment efforts include direct cash transfers, subsidies on Adjustment operations basic goods and services, social infrastructure development in deprived areas, employment- The adjustment operations evaluated in 1994 kept generating public works schemes, targeted up the improvement in performance that first became nutrition programs, employment services and noticeable two years ago, and they performed better, retraining programs, and direct delivery of basic on average, than investment projects. Adjustment goods and services to the poor. Experience operations had a 70 percent satisfactory rating, emphasizes the need for selective targeting of versus 65 percent for investment operations. social safety nets, so that they safeguard people (See Annex Table 1.9.) who are most vulnerable, but are affordable and avoid giving perverse incentives, for example to Among the satisfactory adjustment operations, draw unemployment benefits. (See Box 1.3.) Argentina's $300 million Public Enterprise Reform Adjustment Project (implemented in Sustainability 1991-93) supported one of the most far-reaching reforms of the public enterprise sector under- Sustainability measures the extent to which an taken by any country. The extensive sector work operation is likely to maintain its achievements that had been carried out in the targeted sectors or expected achievements during its opera- helped the Bank to respond quickly to the gov- tional phase. This is an even more demanding ernment's request for assistance. Strong govern- test than the outcome criterion. Experience ment ownership and a highly effective project shows that to achieve sustainability may call for unit contributed to this operation's success. a series of operations, since it often depends on Poland's reforms, supported by the $300 million more favorable policies, and on stronger insti- First Structural Adjustment Loan (SAL) (imple- tutions, than prevail when first operations are mented in 1990-91), were among the first and undertaken. To arrive at a sustainability rating most radical of their kind in Eastern Europe. evaluators assess many of the same factors that The program halted hyperinflation, opened underlie the appraisal of an operation, from Poland's borders to trade, liberalized most economic, social, financial, institutional, and prices, eliminated rationing, privatized small environmental perspectives.1 businesses, and made progress in reducing the debt burden. Though output and employment Sustainability ratings remain unchanged, on aver- fell by more and for longer than the program's age, from last year. Sustainability of operations architects expected, the SAL supported the start was rated as likely in 44 percent of the opera- 34 Box 1.3: SOCIAL SAFETY NETS Few of the Bank's early adjustment loans needed to develop appropriate policy pre- provided for safety net programs to compen- scriptions for the design of cost-effective sate people displaced by reforms and to meet safety nets. Experience shows that: the basic needs of groups excluded from growth, but such programs have become * Programs that were implemented quickly more common, especially since the late 1980s. and served large numbers of people had Over the long term, safety nets can high-level political support, help with strengthen human capital development, design from their targeted beneficiaries, especially among the most deprived parts of strong motivation, good information, society. timely procurement, transparency, and public accountability. Not all social safety nets are replicable or sus- * Programs that had difficulty getting tainable. They run the risk of entrenching started cited political interference, poor themselves in a government's social develop- coordination, limited participatory pro- ment strategy and burdening public finances cesses, inadequate staffing, and cumber- and institutions. More systematic piloting is some bureaucratic procedures. tions in the 1994 cohort, the same as in the 1993 Improvement in human resources. Three fourths cohort, and slightly lower than the 45 percent of the human resources operations in the 1994 over 1989-94 (Figure 1.6 and Annex Table 1.10). cohort were rated likely to be sustainable, Looking at real disbursements, the proportion up from 61 percent in 1989-94. With 23 opera- of lending likely to yield sustainable benefits tions assessed for their sustainability, human was 49 percent in 1994, down from 54 percent in resources had the highest sustainability rating 1993, and 50 percent for 1989-94. of any sector in the 1994 cohort. Sustainability ratings improved to 73 percent in Sustainability ratings also improved notably East Asia; fell to 23 percent in Africa. Substantial in finance and industry, although to levels no improvements in likely sustainability were higher than 50 percent. (See Figure 1.6 and made in East Asia and Pacific from the 66 per- Annex Table 1.10.) Chile's Financial Markets cent average for 1989-94. In Africa, the propor- Loan (approved 1989, closed 1994) helped to tion of operations rated likely to sustain their consolidate the gains of a successful adjustment benefits fell from 33 percent to 23 percent. Eval- program. Reforms supported by the project for- uators' uncertainty about governance issues led tified the policy environment of Chile's finan- them to rate 51 percent of the African opera- cial system, and several of them were bolstered tions as uncertain to be sustainable; but only 26 by the capital markets law approved in January percent of the African operations--down from 1994. In industry, Jamaica's Kingston Free Zone 31 percent in 1989-94-clearly seemed unlikely Project (approved 1982, closed 1990) promoted to be sustainable. (See Annex Table 1.10.) industrial exports, generated foreign exchange 35 goals were met relatively readily, but often FIGURE 1.6: SUSTAINABILITY OF OPERATIONS EVALU- improvements in management fell short of 4,TED, BY SECTOR AND REGION, 1994 AND 1989-94 expectations, and cost recovery and operating COIIORTS (N>=10) efficiency issues posed difficulties that made Regions project benefits hard to sustain. East Asia and Pacific Middle East and North Africa Lahn America and Caribbean In many of the power projects, borrowers had Europe and Central Asia South Asia difficulty mobilizing counterpart funds. Often, Afios financial management was poor, operating Human resource expenses were higher than expected, and sales Pow'er Transport grew less than expected. During the 1980s many Industry power utilities had difficulty servicing their Average Finance -_--_-- _debt, and few had enough resources available Program and policy Agriculture for new investment. Many countries had diffi- Water and sanitation ~culty raising power tariffs, but evaluations point P- rcent sati,tai ton out that often projects could have been more S89-94 M 1994 imaginative in seeking for alternative ways to Note: Technical assistance, telecommunications, and water were omitted increase revenues and, even more important in due to small sample size. many cases, to rein in operating costs. Evalua- Source: Annex Table 1.10. tions of projects in Peru, Sierra Leone, and Turkey emphasize the difficulties caused by lack of borrower commitment to sound financial earnings, and created new jobs near one of the management in power utilities. Another recur- highest unemployment areas of Kingston. The ring theme is the need for much more proactive project has had a lasting impact on employment supervision in power projects. If the borrower and foreign exchange earnings. is to be fully committed to agreed project goals, the Bank must show itself to be so, too. Staff Sustainability ratings declined in urban opera- continuity can play an important role here. tions...In urban development, only a third of operations were rated likely to be sustainable, In water supply and sanitation, many projects down from nearly half for the period 1989-94. sought to strengthen the management of water Many of the urban projects had substantial phys- utilities and to win greater autonomy for them ical achievements. They built houses, equipped from municipal authorities, but very few made marketplaces, and provided basic utilities and sustainable improvements in these areas. Often services in low-income neighborhoods. Yet, in projects were delayed by management weak- countries as different as Chile, The Gambia, nesses, excessive centralization of authority, Indonesia, Madagascar, and Tunisia, many and difficulties in obtaining counterpart funds. projects closed without adequate arrangements Low tariffs, inefficient operation, with high lev- for recovering their costs or for meeting their els of unaccounted-for water, and high levels of requirements for operation and maintenance. arrears-with the largest arrears sometimes owed by branches of government-threatened ...and public utilities. Sustainability ratings projects' financial sustainability. In Lagos, declined in power and in water and sanitation, the Solid Waste Management Project (approved compared with the 1990-94 averages. In both 1985, closed 1993) had a big but only temporary power, with 22 projects in the 1994 cohort, and impact on sanitation in the city; the authorities water supply/sanitation, with nine, physical lacked the funds to keep the collection equip- 36 ment running, and by the time the project closed only a third of the solid waste FIGURE 1.7: POSITIVE SUSTAINABILITY FACTORS FOR generated daily was being collected. SATISFACTORY PROJECTS, 1994 COHORTS By year of approval, the proportion of opera- Gov't. commitment tions judged likely to sustain their benefits Institutional/mgmt. effectiveness increased from 42 percent for those approved in Policy environment 1984 to 49 percent for those approved the follow- Technical viability ing year. For later approval years, not enough operations have yet been evaluated to be certain Economic viability of an improving trend. (Annex Table 1.11.) Financial viability Local participation Factors that encouraged sustainability Environ. viability Reinforcing previous evaluation findings, gov- 0 20 40 60 80 100 120 140 ernment commitment to project goals, institu- Number of positive responses tional and management effectiveness, an enabling policy environment, technical viabil- ity, and cost recovery were found to have a pos- uated in 1994 substantially achieved their itive influence on sustainability in the 1994 institutional development goals, compared with cohort (Figure 1.7). 34 percent in the 1989-94 period. Zimbabwe's successful Family Health Project is illustrative. Institutional development The project was based on a government policy paper, "Planning for Equity in Health."It Improving a country's ability to make effective increased the availability and use of family plan- use of its human, organizational, and financial ning services, improved the health of mothers resources provides a crucial link between the and children, and strengthened the govern- development impact of a project and the sus- ment's capacity to plan, manage, and evaluate tainability of its benefits. mother and child health and family planning services. The project was well calibrated to the Progress was made on institutional development, Ministry of Health's institutional capacity and but performance was still poor. The share of oper- financial constraints. The district and peripheral ations in the 1994 cohort that substantially facilities it supported are functioning well and achieved their institutional development goals are well maintained. District health administra- improved to 39 percent, better than last year's tors trained under the project are now the focal 30 percent and the 1989-94 average of 31 per- points for decentralized planning. A follow-on cent. (See Figure 1.8 and Annex Table 1.12.) project continues many of the activities. The Almost all sectors and regions improved their project's positive effects on knowledge, atti- record to some extent, except water and sanita- tudes, and practices are likely to continue to tion, energy, with its small sample size, and influence decisions about family size and health. ECA, with its small number of operations and rapidly changing institutional environment. Factors that encouraged institutional development The improvement in the institutional development performance of human resource operations is strik- Borrower commitment was the factor with the most ing. Fifty nine percent of the 22 operations eval- prominent influence on achievement of institutional 37 highly unsatisfactory projects are 100 percent FIGURE 1.8: ACHIEVEMENT OF INSTITUTIONALunieytbestaal. OBJECTIVES, BY SECTOR AND REGION, 1994 AND 1989-94 unlikely to be sustainable. COHORTS (N>=) Institutional development and sustainability are Regions strongly related. Just over three fourths of the East Asia and Pacific Latin America and Caribbean operations that achieved substantial institu- Middle East and North Afnca - -- South Asia tional development were also judged likely to be Africa dvlpetj~e Europe and Central Asia sustainable. Conversely, about 60 percent of the Human rect trs operations judged likely to be sustainable also Inclustrv___________dvlpet Power achieved substantial institutional development. Program and lIc~ Prgrm and Most sustainable operations invested in the insti- Agriculture Water and samtation tutional capacity needed to sustain benefits after Urban Transport completion. To yield sustainable benefits, opera- Fimance Energy tions may need to address problems of public 0 10 20 30 40 50 6o sector management, civil service reform, decen- Percent ubstanfial S1989-94 g 1994 tralization, deregulation, private sector develop- ment, promotion of civil society, and enhanced Note: Telecommunications, tourism, pollution, and technical assistance were partcpation. S civ e a f n ays omitted due to small sample size. participation. Such objectives call for new ways Source: Annex Table 1.12. of doing the Bank's business, including a multi- disciplinary approach to problem solving, a vastly different role for field offices, and a recog- development goals. Out of 36 cases where evalua- nition that systematic burden sharing with part- tors judged borrower commitment as strong, ners enhances development impact. 64 percent achieved substantial institutional development. Conversely, among 30 operations Returns, implementation times, and costs where borrower commitment was judged to be weak, 40 percent achieved negligible progress Undue optimism still characterizes the appraisal of toward their institutional development goals. operations, but forecasts of both time and cost are (Annex Table 1.13.) becoming more realistic. Relationships between sustainability, Projects realized high returns. Rates of return institutional development, were calculated at appraisal and re-estimated at and overall outcomes evaluation for 82 projects (one third of the 1994 cohort). Most of these projects achieved re- This year's findings confirm the strong correla- estimated returns much higher than 10 percent; tions, pointed out in last year's Annual Review, the Bankwide average was 22 percent. By sec- between satisfactory outcomes, sustainability, tor, re-estimated rates of return were: agricul- and institutional development (Figure 1.9). ture 17 percent; power 17 percent; transport They are similar to those for the 1989-94 28 percent; and urban development 25 percent cohorts on average. (Annex Table 1.16). Serious issues about the quality of the Bank's economic analysis, and Sustainability and satisfactory outcome are strongly hence of rate of return estimates, are discussed related. Ninety one percent of the projects rated in Chapter 4. likely to be sustainable were rated satisfactory. Not unexpectedly, highly satisfactory projects Appraisal rates of return are becoming more realistic. are 100 percent likely to be sustainable, and Analyzing the data by approval year shows that 38 rates of return both at appraisal and completion F have trended upward, and that the gap between FIGURE 1.9: INTERRELATIONSHIP OF PROJECT SUSTAIN- thae twonded has arwed. (See Fige 1.0. beABILITY, INSTITUTIONAL DEVELOPMENT, AND PERFOR- MANCE RATINGS, 1989-94 EVALUATION COHORT However, there is wide dispersion around these averages. In general, projects in agriculture and power have had lower rates of return than those 100 in transport and urban development. The aver- ages should be interpreted with care, given 80 the width of the dispersion. (See Figures 1.11 60 and 1.12.) 40 20 Time variations. Time overruns have gone down, 0 Substantial but scheduling is still overoptimistic. On aver- Li e Moest a ~ Modest age, operations evaluated in 1994 took 36 per- UnceriaNegligible cent longer to implement than originally centlongr t impemet thn oiginllyProject suistainability Institutional development scheduled, down from 53 percent in the 1974-94 cohorts. Time overruns were highest for projects in industry (55 percent), followed by energy (47 percent). The better regional performers were FIGURE i.1o: APPRAISAL AND RE-ESTIMATED RATES Africa (32 percent), EAP, and ECA (each 29 per- OF RETURN, BY APPROVAL YEAR, 1969-85 cent). (See Figure 1.13 and Annex Table 1.17.) Time overruns often reflect a slow start to implemen- 30 tation, delays in procurement, problems in counter- part funding, underestimation of implementation capacitiy,and additional works. For example, in the 20 Yemen Fourth Highway Project, physical com- pletion was four years late, due mainly to diffi- :10 culties in selecting qualified consultants for technical assistance. The Vocational Training Project in Ecuador, though satisfactorily imple- 0 mented, was substantially delayed because of 1969 1971 1973 1975 1977 1979 1981 1983 1985 unrealistic implementation schedules, underes- Year of approval timation of institutional capacity, and counter- - appraisal rates of return -re-estimated rates of return part funding problems. Time underruns usually reflect conservative appraisals, but may also be due to curtailment personnel, and urban institutions. It had (or outright abandonment) of project objectives. 17 effectiveness conditions, 21 disbursement For example, the Rwanda Urban Institution categories, and 14 disbursement conditions. Sectoral Development Project fell victim to its complexity; 95 percent of the project was can- Cost estimates are improving. Continuing the celed three years before the anticipated closing recent pattern for cost underruns, this year's date. This was a very demanding project for a cohort achieved an average underrun of 4 per- country short of financial resources, skilled cent compared with an underrun of 8 percent 39 Reasons for cost underruns. In many cases, cost FIGURE 1.11: VARIANCE OF RE-ESTIMATED RATES OF underruns were due to devaluations, resulting RETURN, BY APPROVAL YEAR, 1969-85 in savings. But some reflect cancellation deci- 60 - - sions. Others reflect the impact of government budgetary constraints and the increasingly fre- 50 quent mid-term reassessment of project costs. 40 Project cost underruns are often accompanied High by counterpart funding problems. For example, 30 Morocco's Oulmes-Rommani Agricultural 20 Average Development Project was implemented at a time of severe budget difficulties; lack of coun- 10 ,terpart funds forced a reduction of project activ- 0 ities that resulted in cost "savings." Low 69 71 73 75 77 79 81 83 85 Reasons for cost overruns. Cost overruns Approval year often result from delays or design under- estimation. Implementation of Colombia's Sec- Note: For dispersion analysis the top and bottom 10 percent of re-estimated ond Bogota Water Supply Project was first returns were excluded so as to minimize the effect of outliers. delayed by slow progress by the initial contrac- tor. Then rebidding for the completion of the works caused considerable delays and substan- FIGURE 1.12: RE-ESTIMATED RATES OF RETURN, tial cost overruns. BY SECTOR AND APPROVAL YEAR GROUPS Operations that kept close to their original cost estimates performed better than those with large 30 cost overruns or underruns (Figure 1.14). 20 F Lessons from outstanding operations Of the 354 operations evaluated in 1994, OED 10 assessed 18 as outstanding, either on the basis of performance audits or on the basis of reviews of completion reports (Annex Table 1.18).2Imple- mented between 1980 and 1995, half the opera- 0 L Agriculture Power Transport Urban tions were in Latin America and the Caribbean, 1968-73 M 1974-79 F 1980-85 and all but one were investment operations. Together they accounted for $2 billion of lend- ing commitments. In size, they spanned the gamut from a $2 million pilot project in Albania to a $500 million industrial adjustment loan to last year, and 7 percent over the last 21 years. Mexico. Four of the 18 operations were urban; Africa was the only region where operations cost three were in energy (all for petroleum develop- more than expected (overruns were at 16 per- ment in China); and three were in transport (all cent). The other regions had cost underruns of for roads in Chile). Chile and China each had between 10 and 15 percent. (Annex Table 1.17.) four of the outstanding operations. 40 The operations shiared sonie distinctivefeatures: FIGURE 1.13: TIME OVERRUNS AND OUTCOME, 1994 COHORT " They originated with borrowers, with the Bank seen either as an objective over 50 catalyst for change, or as providing the necessary technical knowledge. >25, <= 50 * Their designs were relatively simple. >0, <- 25 * The Bank supervised them closely. on time or less * They were implemented pragmatically and flexibly. 0 10 20 30 40 50 60 70 80 90 100 Perce'nt satisfactory/ * In several, though by no means all, the Bank was able to build on long-standing relationships and knowledge of how things worked "on the ground," resulting FIGURE 1.14: COST OVER- OR UNDERRUNS AND OUTCOME, from prior operations and/or extensive 1994 COHORT economic and sector work (ESW). Urban development and over 4o reconstruction >20, <= 40 Z >5, <= 20 Jordan's Aimman Transport and Municipal Develop- ment Project (approved 1983, closed 1993) >=-5, <= 5 improved the transport infrastructure in Amman > and reduced traffic problems. The project's solid k waste collection and disposal system could >20, < 40 provide a model for other countries in the region. more than -40 The high degree of ownership and commitment - - - by the municipal authority, MOGA, was instru- 0 10 20 30 40 50 60 70 80 90 100 mental to success. Successful collection of Percent satisfactory user charges for the solid waste system, which were included in the monthly electricity bill, prompted MOGA to explore options for privatizing the waste collection system. * Significant design changes during imple- Evaluators drew the following lessons from mentation should be subject to rigorous the project: economic justification. * Avoid complex projects with too many * Assess, as part of the appraisal process, the integrated components when the imple- need to organize a project launch workshop menting agency has limited executing to familiarize implementing agency staff capacity. Ensure that project objectives with Bank procurement and disbursement have the full support of the borrower procedures and how these procedures and its implementing agency. accord with national ones. 41 * Ensure supervision missions have an in its use, and using the new technology to adequate skill mix. improve oilfield productivity and safety were successfully met. Two earthquake reconstruction projects. The North China Earthquake Reconstruction Project Technical assistance and training. One of the and the Popayan Earthquake Reconstruction Daqing project's most effective components Project in Colombia were typical of the Bank's was technical assistance and staff training. approach to disaster reconstruction in that they The techniques used are likely to be replicable supported measures to reduce the risk of future in other countries with large, but technically disasters and safeguard people most vulnerable outdated, petroleum industries. They included: to them. Both projects significantly helped the affected areas' economic recovery and the reha- * Permanent training institutions for engi- bilitation of living conditions. neers and skilled workers within each oil- field region. A core of instructors were In China, the implementation capacity proved trained abroad in the use of curricula and extraordinarily effective. Systematic participa- methods for teaching advanced oilfield tory planning techniques and diligent project operation techniques and practices. management by the local governments, as well as the (approved) use of windfall savings from * A national program for petroleum-related foreign exchange realignments, meant that local education. The scope and content of the resources were effectively mobilized so that courses taught were expanded, and over- physical accomplishments exceeded appraisal seas training programs for their staff expectations. were established. Experience in both projects emphasizes the * Training, in China and abroad, provided by importance of developing public understand- the suppliers of modern and sophisticated ing about disaster mitigation. Within 60 days of equipment purchased under the project. the Popayan earthquake, a new building code This training was explicitly specified in the had been imposed and a public education pro- bidding documents for equipment. gram had begun. The building code has been enforceable because it is widely understood. * Interaction between implementing agency Publications and courses were developed, staff and consultants, both in China and at aimed at constituencies ranging from the man- the consultants' headquarters, on studies, in-the-street to the masons and engineers. particularly those related to reservoir analy- sis, oilfield operations, and safety practices. Energy: oil and gas development in China * Consultants' assistance, at the oilfields, with Between 1982 and 1986 the Bank made the initial implementation of the recommen- loans to help Chinese enterprises modernize dations from their studies, particularly in their exploration and production of oil and the areas of drilling and oilfield practices natural gas. A 1994 performance audit found and safety standards. the Daqing Oilfield/Gaotaizi Reservoir Development Project, Zhongyuan/Wenliu Help on procurement procedures. The operations Petroleum Project, and Karamay Petroleum also highlight the difficulties and delays encoun- Project particularly successful. Their goals of tered when the Bank starts operations in a introducing modern technology, training staff country that has limited experience with foreign 42 suppliers and is unfamiliar with the Bank's pro- Albania: Rural Poverty Alleviation curement procedures. Training programs for Pilot Project Bank procurement and loan disbursement prac- tices should be part of any project with a new bor- A pilot project in Albania, closed in 1995, used rower or new executing agency. This is partic- Bank assistance to build on activities financed ularly important when the country itself has only by a French nongovernmental organization recently joined the Bank, and is unfamiliar with (NGO) and the United Nations Development international bidding and Bank procedures. A Program to test participatory approaches to special procurement seminar, for staff from all the alleviating rural poverty. The project's agencies involved in the procurement process, bottom-up methods worked well. It established can significantly help to foster the teamwork a rural development fund and successfully needed for efficient project implementation. tested the modus operandi for rural works and village credit systems. Importantly, it found Adjustment: industry in Mexico that rural works had less effect on employment and income for the poor than expected. Reasons This $500 million operation, prepared in 1988, included a renaissance of agriculture, with was part of a package of three adjustment opera- expansion in farm employment and incomes, tions designed to support the Mexican debt during project implementation; the need for restructuring program. It provided for sweeping technical skills not easily found among the liberalization of foreign ownership, investment poorest peasants; and the finding that members procedures, and deregulation, and opened up the of the poorest families lack time either for extra trucking industry to competition, benefiting con- farm work or for wage employment outside sumers. Many measures taken were in advance farming. The successful village credit compo- of agreed schedules and exceeded their targets. nent was designed to fit the sociocultural fea- tures of typical mountain Albanian villages; Three features stand out: the repayment rate, independently verified, is excellent. A follow-on project is being * The program was run by a centralized, appraised. well-managed unit with direct access to the president's office. Notes * It initially relied on administrative solutions and presidential decrees. 1. Sustainability is rated as being either "likely," "unlikely," or "uncertain." The last category includes (1) projects for which there is insufficient information at the time of project completion * The program was helped by the widespread report/performance audit report preparation to reach a judg- recognition of the need for fundamental eco- ment as to the likelihood of sustainability, and (2) projects whose nomic reform, private sector confidence in future sustainability is truly uncertain. the government's commitment and ability 2. OED evaluators designate operations as outstanding if they to implement reform, and the orderly exceed their major goals, have highly innovative designs, and sequencing of reforms. may be replicable to other countries or sectors. 43 2. The determinants of portfolio performance Introduction ing countries by portfolio performance, macroeconomic performance, Bank perfor- What should be the focus of Bank attention in the mance, and borrower performance. This project cycle to bring about a satisfactory portfo- approach is used to identify outliers at both lio performance? How effective are the Bank- ends of the spectrum so as to provide addi- controlled processes to mitigate the risks inher- tional, anecdotal evidence to illustrate the ent in a country's economic and sociopolitical findings of the econometric analyses. environment and the borrower's behavior? This year's analysis builds on earlier attempts, Key findings particularly the 1992 and 1993 Annual Reviews. To some extent, it constitutes an update of these Outcomes exercises, but the use of more rigorous quantita- tive techniques allows a somewhat more corn- At the individual project level: plete and nuanced assessment. The analytical approach has three prongs: *The variables that determine project out- come in a statistically significant way are: (1) Econometric analysis of the universe of Bank appraisal and supervision; borrower projects evaluated during CY1991-94, with implementation and compliance with loan individual projects as the unit of analysis.] covenants; and-negatively-macroeco- Annual averages were computed for indi- nomic instability as well as governance. vidual projects starting from two years before approval and up to three years *Borrower implementation and compliance after approval, with loan covenants are the strongest determinants. They are significantly (2) Econometric analysis of entire country port- and positively related to satisfactory folios, using the same sources of information project outcomes. as project-based analysis and covering the same periods. 2 *Bank performance greatly matters in determining whether a project achieves its (3) Complementary assessment of individual major development goals. Quality at entry project experience and-more important through sound appraisal as well as strong and novel-country-specific analysis rank- supervision are key factors. 44 * Macroeconomic instability (proxied by Bank supervision (to a lesser degree domestic inflation) and bad governance (as appraisal), borrower's compliance with loan measured by indices for political instability covenants, management of implementation and corruption) were the two country fac- agency, GDP per capita growth, gover- tors with a statistically significant adverse nance, and the loan amount. effect on project performance. m The quality of management of the project " The deleterious effects of macroeconomic implementation agency is the single most and political instability on individual project important determinant of project sustain- outcomes can be significantly mitigated by ability. Borrowers' compliance with loan strong Bank appraisal and supervision. covenants is also a powerful factor. Supervi- sion remains the only statistically significant " The strong performance of the East Asia and Bank-related determinant of project sustain- Pacific and South Asia portfolios in out- ability after borrower behavior is accounted comes, relative to Africa, is explained for. In other words, the chances of project largely by superior borrower and Bank per- sustainability increase when the borrower is formance, and-to a lesser extent-by better in the driver's seat; effective Bank supervi- macroeconomic performance. sion during project implementation to sup- port the borrowers' effort is tantamount to At the levlel of country portfolios: additional "insurance coverage." * When determinants are analyzed, not project- * Governance3 is also an important "climatic" by-project, but country-by-country, all of the aspect for the chances of sustaining project project-level determinants are re-affirmed, but results over the longer term. So is domestic the influence of a country's macroeconomic economic growth, although to a lesser climate on the whole country project portfolio, degree. as compared to an individual project, becomes much more important. In addition to the neg- * The loan amount, although clearly of minor ative effects of high domestic inflation noted importance, also exerts some influence on above, the rate of per capita GDP growth was project sustainability-most likely because found to exert the most powerful positive the borrower's and the Bank's attention impact on average outcomes at the portfolio increases as the loan amount becomes larger level. The impact of growth on portfolio per- and the stakes higher. formance is particularly strong in countries whose portfolios performed poorly in the past. * East Asia is again the region that stands out with the best average results, while * The mtensity of Bank-country dialogue (as Africa trails all other regions by a proxied by the number of ESW reports) also large margin. had some influence on portfolio performance. At the level of country portfolios: Sustainability * GDP growth was found to be very impor- At the individual project level: tant for longer-term portfolio sustainability. It was actually found to be more important * The variables that determine project sustain- for sustainability than for the outcome dur- ability in a statistically significant way are: ing implementation. 45 * Among countries where projects chronically Thus, taking borrower and Bank together, there fall short in achieving sustainability, quality are very few "outside excuses" for unsatisfac- Bank supervision appears to have a greater tory project and portfolio results. impact than quality at entry in raising sustainability indices. Global economic factors such as the interest level as measured by LIBOR (London interbank " The intensity of the Bank-country dialogue offered rate), global growth as measured by (as proxied by the number of Bank ESW economic growth in the G7 countries, or terms- reports for a given country), although not a of-trade developments were not found to be sta- major factor, was also found to be significant tistically significant factors in determining port- for longer-term sustainability. folio performance. However, to the extent that domestic economic performance as measured Institutional development4 by inflation and economic growth is influenced by these global factors, in conjunction with the Quality at entry and supervision, as deter- varying quality of domestic economic manage- mined by both Bank and borrower behavior, ment across countries, these global factors exerted powerful positive influences on tend to exert an indirect influence.6 institutional development, with borrower behavior being a stronger determinant than The following section illustrates the key find- Bank behavior. ings from the project-based analysis with spe- cific examples drawn from recently evaluated Among the country factors, only governance,5 projects. The section on country portfolio-based as proxied by an index of freedom from analysis follows a similar approach, illustrating corruption, had any significant impact on the findings with country portfolio cases. institutional development. Regional variations in performance, with East Improving project performance Asia and MNA in the lead, are almost entirely explained by differences in the average contri- Borrower performance makes the bution borrowers made to project processing. biggest difference Conclusions As noted above, borrower performance,7 compli- ance with loan covenants, and quality of manage- Of all the independent variables, one-borrower ment as well as government commitment8 have performance-had the widest impact on individ- a large impact on project outcomes, sustainabil- ual project results. In other words, poor ity, and institutional development. borrower performance will generally lead to projects with correspondingly disap- The Chile Highway Reconstruction I, II, and pointing development results, uncertain Road Sector Projects exemplify the importance or unlikely sustainability, and weak institu- of government commitment for achieving suc- tional development. cessful outcomes. Indeed, the government sup- ported the increased use of private-sector Closely following borrower performance was contractors in projects designed to upgrade how well the Bank did its job. In general, quality national, regional, secondary, and communal Bank processing had a dominant effect on out- roads that were deteriorating at alarming comes, and a significant effect on sustainability. levels by the late 1970s. This deterioration had 46 dangerously reduced the road sector's capacity cent. Overall, an OED audit found that the ben- to facilitate communication and economic efits of the investment fund projects are likely to development. But the agency responsible for be sustained, attributing this success in major managir.g roads, Vialidad, had grown in staff part to: (1) the setting of defined and monitor- because of social welfare programs. These able goals with latitude in how to achieve them, projects were instrumental in reorienting Vial- and (2) the quality of FHIS staff. idad toward increased reliance on the private sector by mandating extensive international or Bank performance matters, too local competitive bidding for most contracts, thereby keeping government employment in As highlighted above, quality at entry perfor- check while infusing government business with inance (as determined by the quality of Bank private sector expertise and efficiency. The identification and appraisal work) and Bank projects were completed within original sched- supervision have a major impact on the likeli- ules and within or below financial budgets, hood that any given project would have a satis- except for additions necessitated by floods, factory and sustainable development impact. A earthquakes, and changes in public sector good example of this can be found in the Argen- finance. Moreover, the quality of the work was tine Power Engineering Project. The project's high, and routine and periodic maintenance objective was to assist the government with exceeded quantitative targets. The projects are modernizing electricity distribution in the considered outstanding examples for other country. At appraisal, Bank staff correctly rec- countries to follow (see Chapter 1). ognized that coordination difficulties between the Federal Energy Secretariat and the provin- The Honduras First and Second Social Invest- cial utilities could jeopardize the success of the ment Fund Projects offer instructive lessons for project. To deal with this situation, the Bank strong implementing agency management and sus- requested, as a condition of effectiveness, a for- tainability of benefits. These projects were mal commitment from the provincial utilities to designed to mitigate the social costs of an eco- carry out all project activities in their area of nomic stabilization and adjustment program, responsibility. Largely because of this local lay the basis for a targeted food assistance pro- association with the federal government, the gram, a rd improve social service delivery. Most project met all its objectives. of the funding was channeled through the Fondo londureflo de Inversi6n Social (FHIS), a The delegation of supervision responsibility to semi-independent agency established in 1990 the Bank's regional mission in Abidjan was a with more freedom than traditional line minis- key ingredient in the success of the C6te d'Ivoire tries in the areas of procurement, contracting, Agricultural Sector Adjustment Loan. The task and hiring of personnel. FHIS got off to a rapid manager and other mission staff held weekly start, eventually approving more than 5,500 meetings with borrower officials. Because of this projects, with 4,500 completed by 1993. The intensive "supervision" and close collaboration, most promising of these have been 56 that pro- program objectives and the roles of the imple- vided credits to NGOs and community banks to menting agencies became better understood. lend to microenterprises. Thanks to careful This learning process was particularly impor- selecticn of NGOs by FHIS, enforcement of tant since loan conditionally had been negoti- detailed procedures, technical assistance, and ated in haste and was vague on major points. substantial monitoring and supervision by FHIS staff and donors, FHIS has kept bad debts A number of factors distinguish high-quality associated with these projects to under 5 per- appraisals and supervision of projects with 47 satisfactory outcomes (see Figure 2.1 and project risks were likewise anticipated. Cover- Annex Table 2.1, which list all the factors that age of relevant commercial and environmental are taken into account in assessing the appraisal risks, although high, showed some shortcom- quality; the numbers are based on a sample of ings. Sound appraisals were handled by highly projects evaluated in 1993-94). Well-appraised qualified staff, effectively incorporated the les- projects with satisfactory outcomes invariably sons of past experience with the sector and the identified economic and/or sociological risks borrower, and secured government commit- adequately and provided for their resolution. ment to the project's objectives. However, even Nearly all technical, financial, and institutional well-appraised projects occasionally included implementation plans that were less than ade- quate to achieve project objectives (Figure 2.2). FIGURE 2.1: QUALITY OF APPRAISAL BY MAJOR AREAS OF ANALYSIS: PROJECTS WITH SATISFACTORY OUTCOME, For supervision (Figures 2.3 and 2.4 and Annex LIKELY SUSTAINABILITY, AND SUBSTANTIAL ID IMPACT Table 2.2), evaluators found that providing the borrower with sound advice was the most Economic important characteristic, followed by skill in Sociological modifying the project scope and design as Technical appropriate to address implementation difficul- ties, and clarity and frankness in reporting Financial on project implementation. In general, well- Institutional supervised projects also benefited from ade- ia quate numbers of high-quality supervision Enionmental staff and good performance by consultants. Environmna 0 20 40 60 80 1oo Impact of macroeconomic instability Percent satisfactorily performed Source. Annex Table 2.1. Of the four macroeconomic variables tested, domestic inflation proved to be the most signifi- cant (as a proxy for macroeconomic instability) FIGURE 2.2: FACTORS AFFECTING APPRAISAL QUALITY, in the outcome equation. Although its impact PROJECTS WITH SATISFACTORY OUTCOME, LIKELY was much weaker than that of borrower and SUSTAINABILITY, AND SUBSTANTIAL ID IMPACT Bank process variables, macroeconomic instabil- ity does affect project performance negatively. Donor coordination Instrument suitability The Brazil Second Rural Development Project Commitment Economic and sector work (Bahia II) offers insights into how even a well- Design prepared project can be undermined by insta- Staff quantity bility. The project was to improve socioeco- Borrower implement. capacity nomic conditions of about 45000 small Staff quality farmers, Consultant performance and increase agricultural production by intro- Past experience ducing drought-resistant technology and area Risk identification e Adequacy of implement. plan e s Project activities were underway m 0 4 bearly 1983 when a stabilization program called 0 20 4[0 50 80 100 Percent witli positive iinpact for substantial budget cuts at the federal level. Source Annex Table 2.1. Bahia II suffered throughout implementation from deep cuts in counterpart funding and 48 erratic releases of funds. By project closure, the Bank had disbursed only 24 percent of the FIGURE 2.3: PERFORMANCE OF SUPERVISION BY MAJOR aprovd loan e a on. 2AREAS OF ATTENTION: PROJECTS WITH SATISFACTORY OUTCOMES, LIKELY SUSTAINABILITY, AND SUBSTANTIAL ID IMPACT The encouraging news is that quality Bank appraisal and supervision can reduce the risks Advice posed by macroeconomic and political instabil- Flexibility ity to project success (see Box 2.1). While these Bank interventions may be effective at the level of individual projects, one has to ask the ques- Identification of problems tion whether they can be replicated for large Attention to develop. impact numbers of projects within a country portfolio Follow-up and are the most cost-effective way of dealing with the risks associated with instability. This Enforcement is further discussed in the country-based 0 20 40 60 80 100 analytical context below. Source Annex Table 2.2 Percent satisfactorly performed Impact of governance In addition to borrower and Bank behavior, the FIcURE 2.4: FACTORS AFFECTING SUPERVISION QUAL- individual project-based econometric analysis ITY, PROJECTS WITH SATISFACTORY OUTCOMES, LIKELY identifies "freedom from corruption" (as a SUSTAINABILITY, AND SUBSTANTIAL ID IMPACT proxy for governance)9 as a statistically signifi- cant de-erminant of project sustainability. On Staff quality average, project performance tends to be less staff qIantty satisfactory in countries where the implementa- ta pfance tion environment is negatively affected by cor- ruption. Thus, the outcome of various projects Timing such as the Sixth Highway Project in Haiti or Time in field the Gas Technical Assistance Project in Nigeria Supervision_plans was se- erely hampered by lack of transparency, spurious accounting practices, and noncompli- Implementation reviews ance with Bank procurement rules. 0 20 40 60 80 100 Percent with positive mpact Sorte. Annex Table 2.2. Improving country performance The framework sharper focus-in addition to all the determi- nants of project performance that were identified The country portfolio-based econometric analy- through project-based econometric analysis. Bor- sis brings the importance of overall country eco- rower and Bank process performance, however, nomic performance, particularly inflation and are found to be powerful determinants of good per capita GDP growth (as proxies for efficient portfolio performance, with some importance resource allocation and a generally healthy eco- also attributed to the quality of Bank-borrower nomic policy environment that is conducive to dialogue (as proxied by the number of ESW greater welfare and reduced poverty), into reports and field representation of the Bank).'o 49 Box 2.1: QUALITY BANK PROCESSING OVERCOMES UNFAVORABLE MACROECONOMIC ENVIRONMENT: EXAMPLES FROM BRAZIL Two Brazilian projects illustrate how strong equipment and great dislocation of academic supervision can mitigate the impact of an plans during the early years of the project. unstable economy. The Brazil Science and Tech- Significant amounts were wasted because nology Project sought to strengthen research there were no mechanisms to protect project capacity in specific scientific and technologi- funds from inflationary erosion. Halfway cal areas deemed important to economic through project implementation, the Minis- growth in Brazil. The strategy for achieving try of Science and Technology (MCT) the overall performance objective involved diverted 8 percent of project resources to providing funds to ten program areas, such finance "emergency" projects. However, a as mineral technology and biotechnology. strongly negative Bank reaction helped to During the life of the project (1985-91), Brazil stop this diversion of funds from project underwent extraordinary governmental and objectives. financial instability and experienced high turnover of its science policy officials. Severe Despite the complexity of the project design, constraints on public resources brought the and the sometimes chaotic context in which it release of committed funds to a halt at several was implemented, the project did achieve points. The rate of inflation remained higher significant gains. Laboratories were substan- than 25 percent per month throughout most tially refurbished, raising their productivity of project implementation. Long delays in significantly; students, teachers, and tech- disbursement of funds for awarded grants nicians received support for technical train- caused delays in receiving crucial research ing and university study; a new law greatly The fact that country economic performance financing, this effect would be expected to (reflecting good policies and satisfactory out- become even more prominent. come in terms of inflation and per capita GDP growth) matters more in explaining portfolio This section will illustrate through selected performance as opposed to individual project country case experiences how the country eco- performance can be largely interpreted as nomic environment influences portfolio perfor- a "large-scale effect." This finding is consistent mance and what difference borrower and Bank with the endogenous growth literature, process performance is likely to make. The which suggests that economic policies matter in "standard expectation," that is, good (bad) addition to the growth effects derived from country economic performance being condu- investment in physical or human capital. If the cive to good (bad) portfolio performance, is rep- analysis were to cover not only Bank-supported resented in the NW and SE corners of a three- projects but countries' entire development by-three matrix representing the quality of the effort, irrespective of the source of external country's macroeconomic environment and of 50 (Box 2.*1 continued) nomic and financial instability. It achieved a facilitated imports of scientific equipment; satisfactory outcome in terms of both sub- and advisory groups learned how to deal stantive policy change and sectoral policy with conflict-of-interest problems endemic to stability. Despite the general turmoil, govern- relatively small scientific communities. ment controls over agricultural marketing and pricing were dismantled, contributing to How was this accomplished, despite the the recent expansion of output and exports. often-chaotic conditions in which the project Although lack of macroeconomic stabiliza- was implemented? It helped that project tion made implementation difficult, it did not objectives were sound and shared by the rel- derail the program. The quality of the Bank's evant Brazilian communities; Bank and local preparation and appraisal work played a sig- program staff dealt skillfully with the nondis- nificant role. Eight appraisal missions were bursernent of counterpart funds and the fre- mounted by the Bank to assess the risks and quent delays in receiving import licenses; fine-tune the action plan. On the borrower's and effective oversight by an independent, side the project was prepared by a team close international panel of distinguished experts to the Minister of Planning and based on gave the program legitimacy and generated broad consensus among all interested public pressure to solve problems. and private sector parties. The loan addressed important distortions in the sector, The second example, the Brazil Credit and that is, price controls and subsidies, which Marketing Reform Project, illustrates the were a large burden on the budget. Moreover, "1preventive powers" of high quality the policy reforms were nonthreatening, appraisal. This project was implemented improving the welfare of producers, traders, against a background of acute macroeco- and consumers. portfolio performance. On the other hand, good development) and macroeconomic perfor- portfolio performance resulting from extraordi- mance (inflation and per capita GDP growth), nary efforts by the borrower and/or the Bank as well as on the statistically significant aspects againmt the odds of a poor country economic of Bank and borrower process performance envircnment can be found in the SW corner, (such as appraisal, supervision, implementa- and tl e case of portfolio neglect by the bor- tion, management of implementation agency, rower and/or the Bank is found in the NE and so on.) The best and the worst performers corner (see Figure 2.5). on the different performance tallies were then identified, from which a few were chosen for To identify the "outliers" most representative of anecdotal illustration in the text. The selection the dli,,ferent matrix entries, all countries in the of country cases is shown in Figure 2.5. count:-y analytical sample were ranked by the combined ratings on portfolio performance The ranking exercise was also used to cross- (outcome, sustainability, and institutional check the relative strength of the key indepen- 51 FIGURE 2.5: PORTFOLIO AND COUNTRY ECONOMIC PERFORMANCE a Portfolio performance Country economic Good Median Poor performance (top 20 countries) (middle 20 countries) (bottom 20 countries) Good "Standard expectation," that Eight countries: Bangladesh, Neglect by borrowers and (top 20 is, good economic perfor- Burkina Faso, Burundi, Bank results in bad portfolio countries) mance is conducive to good India, Morocco, Nepal, performance in spite of portfolio performance. High- Pakistan, and favorable odds of good eco- est payoff from additional Papua New Guinea nomic performance. Payoff Bank process efforts. Quality from better borrower and of Bank-country Bank performance could be dialogue also matters. high and quick to move from NE to NW. Eleven countries: Benin, China,b Indonesia, Jordan, One country: Senegalb (with Korea, Malaysia, Mauritius,b important qualifications) Portugal, Sri Lanka, Thailand, and Tunisia Median Four countries: Chile, Six countries: Colombia,b Ten countries: Cameroon,c (middle 20 Ethiopia, Jamaica, and the Egypt, Mauritania, Central African Republic, countries) Philippines Myanmar, Paraguay, COte d'Ivoire, Dominican and Turkey Republic, Guinea-Bissau, Kenya, Mali, Niger, Panama,c and Rwanda Poor "Extraordinary efforts" Six countries: Argentina,c "Standard expectation," that (bottom 20 by borrowers and Bank Madagascar, Mali, Tanzania, is, bad economic perfor- countries) needed to overcome the Uganda, and Yugoslavia mance is likely to result in odds of bad country eco- bad portfolic performance. nomic performance and to Priority for efforts to move from SE to SW. improve country environ- ment to move from SE to NE Five countries: Ecuador, and eventually to NW. Ghana, Hungary,b MeXiCo,b and Zimbabweb Nine countries: Brazil, Guinea, Haiti, Nigeria,b Peru,b Somalia, Sudan, Zaire, and Zambia a. The matrix shows the number of countries that fall into the different combinations of economic and portfolio performance. b. Selected as illustrations in the text. c. Countries selected for discussion in the SE corner because of low ranking of the most critical variables include: Argentina tor low ranking on outcome; and Cameroon and Panama for low ranking on growth. dent variables in determining portfolio perfor- formance," followed by borrower and Bank mance. The rank correlations are basically in process performance. This underscores the line with the results of the economic analyses, validity of the hypotheses reflected that is, they are highest for macroeconomic per- in Figure 2.5. 52 Good versus bad portfolio performers mance in countries with bad portfolio performance than it is in countries with good The econometric analyses highlighted country performance. Similarly, GDP per capita growth macroeconomic performance as an important is more important for portfolio sustainability determinant of project and portfolio perfor- in countries with poor portfolio performance mance. A quick cross-check of the sample of than it is in countries with good performance. 1,339 projects evaluated during 1991-94 con- This underscores the validity of the "standard firms this finding: while overall the outcome of expectation" formulated above, according to 37 percent of projects was rated unsatisfactory, which improved portfolio performance along a this ratio was 56 percent in the ten countries line SE-NW in Figure 2.5 could be expected with the worst macroeconomic performance from improving country economic perfor- and 60 percent in countries in nonaccrual status mance, with borrower and Bank portfolio pro- and/or suffering from civil conflict.12 This also cess performance taking abackseat. By contrast, tends to be corroborated at the sector level: improvements in Bank process performance, for example, 31 percent of all population, particularly higher quality at entry, have a health, and nutrition projects evaluated during higher payoff in terms of better portfolio perfor- 1991-94 had unsatisfactory outcomes; by con- mance among countries that are already above- trast, this ratio was 75 percent for the worst average portfolio performers. The findings of macroeconomic performers. Similarly, this cor- this and the preceding paragraph have impor- relation is also found in the current Bank port- tant implications for Bank operational folio as monitored through the ARPP process: priorities (see below). the likely achievement of projects' development objectives is rated unsatisfactory for 11.5 per- In the country portfolio analysis, countries were cent of all projects in the FY95 IBRD and IDA ranked according to four major classes of vari- portfoho (9.5 percent for IBRD; 13.4 percent for ables, three independent (macroeconomic envi- IDA and blend). For the ten worst economic ronment, Bank performance, and borrower performers among IDA blend countries, this performance) and one dependent (outcome, ratio is 29.2 percent, and for the countries in sustainability, and institutional development, nonaccrual status and/or suffering from civil or ID). Among the 61 countries included in the conflict it is 47.6 percent. The correlation is also sample, three countries stand out in this exer- confirmed at the level of projects with a poverty cise in every respect: China as the largest coun- reduction content: overall, 11.9 percent are try and Mauritius as one of the smallest are the rated unsatisfactory; in the ten countries with only countries that rank among the top ten best the worst economic performance, the ratio performers in all four areas. Peru, at the other is 16.1 percent. extreme, was-during the period covered by the analysis-consistently among the bottom The country portfolio-based econometric ten countries (see Box 2.2). Other countries like analysis uses "quantile regression," which-in Argentina, Catneroon, Colombia, Hungary, Pan- addition to the general findings that hold for ana, and Zimbabwe fall in the areas in between. the entire sample of countries-allows differen- tiation among country groups according to the Country-specific illustrations: the level of portfolio performance. As one might significance of rapid growth expect, not all determinants of portfolio performance are equally important across the This part presents country illustrations along spectrim of countries. GDP per capita growth the SE-NW diagonal in Figure 2.5: China and is much more important for portfolio perfor- Mauritius as good macroeconomic and portfo- 53 Box 2.2: SHARPLY CONTRASTING PORTFOLIO PERFORMANCE: CHINA AND PERU Macroeconomic environment tion has significantly improved in the last few years.) A significant factor contributing to the diver- gence in portfolio quality has been the sharp The impact of the macroeconomic environ- contrast in the macroeconomic settings of the ment on the Bank portfolio is best illustrated two countries. In China, all the projects were by the divergent experiences of similar implemented in an environment of fast projects (development finance credit projects). growth and relative stability. Between 1981, In Peru, the Bank supported two industrial when China became a Bank member, and credit projects and a small-scale enterprise 1994, real per capita GDP grew at an annual project during 1977-88. Growing macroeco- rate of 8.8 percent, while inflation was largely nomic instability and recession severely under control averaging just 8.6 percent. By affected implementation of those projects. contrast, in Peru, the projects were under- Many subprojects became economically inef- taken in a period characterized by a deep ficient. By contrast, all similar operations in recession, run-away inflation, and severe China (three industrial credit projects and two distortions in the economy. During 1981-91, rural credit projects) were highly satisfactory. real per capita GDP fell by 2.2 percent per The Second Agricultural Credit Project pro- year, and inflation spiraled out of control, vides a good example. Few of the 3,503 sub- averaging about 200 percent per year. (With projects experienced significant problems. All the recent reform efforts, however, the situa- main components were implemented satisfac- lio performers; Argentina, Cameroon, Panama, sound macroeconomic environment, strong and Peru as poor performers at the opposite borrower commitment, and good appraisal and end; and Colombia right in the middle. supervision to portfolio quality (see Box 2.2). Of the 37 projects implemented in China, all had Panama. The Panamanian portfolio of nine a satisfactory outcome. Sustainability of project projects examined for this review ranked benefits was considered likely in 34 (94 percent) among the ten worst in performance. Less than of them, while 24 (61 percent) had a substantial half of the projects had satisfactory outcomes, ID impact. By contrast, in Peru, in the opposite none provided sustainable benefits, and none corner of the matrix, only three of the 18 achieved substantial institutional impact. This projects (17 percent) were rated as satisfactory is in spite of Bank performance, which was and of likely sustainability, while ID impact was among the strongest of all the portfolios sur- found to be negligible in all but two of those veyed. On average, well over nine in every ten projects. The sharply contrasting experiences of projects were found to have satisfactory Bank China and Peru highlight the importance of a processing at all project stages. 54 (Box 2.2 continued) projects, for example, the absence of govern- torily, with financial and economic rates of ment commitment to policy reforms and return substantially exceeding 12 percent. institutional improvement contributed to project failure. Government pressure to lend Borrower performance at strongly negative interest rates to failing public enterprises resulted in a quick decapi- The sharply contrasting portfolio outcomes talization of the implementing agency. between China and Peru reflect as much the impact of the macroeconomic environments Bank performance as borrower performance. Of particular importance is the degree of government com- Bank performance in appraisal and supervi- mitment. The seriousness that China attaches sion also affected portfolio outcomes signifi- to its relationship with the Bank ensures a cantly In China, appraisal and supervision high degree of commitment at all levels of were carried out satisfactorily in 87 percent government. In more than 85 percent of the and 91 percent of the projects, respectively. projects, strong commitment by the govern- In most cases, appraisal reports correctly ment as well as implementing agencies was identified potential risks, paid adequate cited by the performance audit reports as a attention to the technical and economic factor positively affecting project outcomes. issues, and made a good assessment of bor- In Peru, lack of such commitment is consid- rower's commitment and institutional capac- ered to have impeded project achievements. ity. Timely supervision enabled the Bank to In the above-mentioned industrial credit identify problems and take corrective actions. Macroeconomic shortcomings appear to be tion of the project was adversely affected by the the major determinant of poor portfolio perfor- allocation of insufficient counterpart funds and mance. Panama ran large fiscal deficits averag- lack of staff continuity As a result of austerity ing over 6 percent of GDP. With a decline of measures imposed in 1985, project investments internal savings, large amounts were borrowed were reduced and spread over six years, instead to finance rising deficits (debt/GDP ratio of of the four years originally anticipated at nearly 10)0 percent); investment was crowded appraisal. Bank financing increased from out, which depressed growth (real per capita 60 percent to 80 percent in an attempt to offset GDP declined by over 1 percent yearly on aver- the shortage of counterpart funds. Neverthe- age during the 1980s and early 1990s). less, this proved insufficient to insulate the project from the deteriorating financial situa- Among the investments that had to be curtailed tion. On November 30, 1987, disbursements on on account of the fiscal crisis were projects all Bank loans to Panama were suspended due financed by the Bank. The Panama Road Reha- to the borrower's failure to maintain service on bilitatiorn Project is representative. Implementa- outstanding indebtedness to the Bank. As a 55 Box 2.3: ARGENTINA'S EXPERIENCE WITH BANK-FINANCED PROJECTS The 17 recently evaluated Argentine projects Over most of this 15-year period, the govern- examined were implemented during 1979- ment failed to adopt credible fiscal measures 93, a period during which the country experi- or curb its reliance on the inflation tax to enced repeated macroeconomic and sectoral finance the budget. Yet, throughout this upheavals. At the beginning of this period, period, the Bank continued lending substan- Argentina had a protected, noncompetitive tial sums to Argentina. For example, during manufacturing sector and a government the late 1980s, the Bank provided new quick- overextended in the productive, service, and disbursing assistance (Second Trade Policy financial sectors. Back as far as the early Loan), even after Argentina was no longer 1950s, the economy had experienced erratic eligible to draw from an IMF Extended Facil- and low real growth, high inflation, public ity Arrangement because of its deteriorating sector deficits, declining investment-to-GDP macroeconomic situation. Bank financial sec- ratios, and episodic capital flight. Anti-infla- tor projects could not operate effectively in an tionary policies followed during 1978-80, environment of chronic inflation, weak relying on a fixed nominal exchange rate as supervision by the central bank, and market an anchor (tablita). This triggered a financial distortions. Reluctant to delay the release of a and economic collapse, massive external financial package negotiated by the govern- indebtedness, and a severe skewing of the ment with its external creditors, the Bank income distribution. Several subsequent agreed to disburse an agricultural sector loan attempts also failed. It has been only since despite noncompliance with a critical loan 1991 that a renewed effort to stabilize and condition requiring implementation of a fed- restructure the economy has achieved major eral land tax. gains, significantly lowering inflation, raising average rates of growth, and opening the To be successful, such Bank operations economy to external competition. required government commitment, particularly result of the prolonged suspension, the closing Box 2.3). It is interesting to note that in the date of the Road Rehabilitation Loan was not meantime significant progress has been made extended, and $4.3 million had to be canceled. in both Argentina and Peru. For the last three years, portfolio performance has been excellent In addition to Peru and Panama, Argentina in both countries. In the Bank's 1995 Annual offers another example where a macroeconomic Report on Portfolio Performance, neither coun- environment that was not conducive to success- try had any problem projects with regard to ful Bank operations was misread by the Bank at development objectives as compared to a the outset and could not be overcome in spite of Bankwide average of 11.5 percent. Moreover, great Bank efforts during implementation (see implementation progress has also been gener- 56 (Box 2.3 continued) inces to assume responsibility for the compo- in carrying forward the complex and politi- nents, and the failure to identify the cally sensitive reforms supported by the sec- incompatible fiscal and budgetary frame- toral adjustment loans of recent years. work. Even the partially canceled Housing However, during most of this period, sup- Project Loan, which was rated satisfactory for port was lukewarm at best. As a result, poli- appraisal, was singled out for not having cies were frequently at odds with loan addressed adequately in the project design objectives, and policy reversals were com- the risks it had correctly identified. mon. For example, the Highway Sector Pro- gram Loan encountered serious governance While the quantity of Bank supervision was problems involving problematic working more than adequate, its quality left much to relations between the federal and provincial be desired, particularly during a time when levels of government. the deteriorating economy was corroding the capacity of many implementing agencies. Seven of the 17 projects reviewed had defi- Project completion reports frequently noted cient appraisals. The appraisals of Trade Policy that "the most important concern of Bank Loans I and I were criticized by evaluators supervision missions was the rate of disburse- for their inappropriate assessments of the ment of the existing loan and the possibility of macroeconomic framework and the feasibil- a follow-up project." Supervision of a loan to ity of trade reforms. Both adjustment pro- the State Development Bank (BANADE) was grams eventually failed, and with them went judged inadequate because it failed to recog- a substantial part of the trade liberalization nize in a timely fashion the "marked institu- measures that had formed the central ratio- tional and financial decline of BANADE." nale for the two operations. The Highway Little attention was paid to the problem, Sector Project was faulted for flaws in the because the Bank was already focused on "the design of the technology transfer component, preparation of a follow-up operation, the errors in assessing the capacity of the prov- Small and Medium Industrial Credit Project." ally good, with only 10 percent problem cases in achieved substantial institutional impact. Peru and 13 percent in Argentina as compared Weak macroeconomic performance was one of to 17.3 percent Bankwide. several important factors depressing portfolio performance. Real per capita GDP growth Cameroon's portfolio performance places this declined by an average of 0.8 percent annually country among the poorest performers of the 61 during the 1980-92 period. The transformation countries sampled-outstripped only by Zaire, from a primarily agricultural exporter prior to the worst performer. Only 43 percent of projects 1975 to an oil exporter during the late 1970s in the sample had a satisfactory outcome; less explains some of the country's subsequent than a third had sustainable benefits, and none macroeconomic difficulties in the 1')80s. Oil rev- 57 enues brought transitory prosperity. Central was also regularly glossed over. It became a government finances recorded surpluses from major contributing factor to the extraordinarily 1979 to 1983 and small deficits until 1986. But poor portfolio performance. external borrowing to finance public projects led to dangerous over-indebtedness, especially Colombia is a case in the mid-field with close after oil prices collapsed in 1986. With declining to median macroeconomic performance, oil revenues, public finances changed dramati- above median for portfolio, and significantly cally; deficits became significant, and the debt above median for Bank and borrower process service was a serious burden. Consequently, performance. Colombia thus conforms to the local counterpart funding of projects was "standard expectation" with its middle position severely reduced, disrupting project implemen- on the SE-NW diagonal in Figure 2.5. Within the tation and explaining a significant part of the cohort of sample countries, Colombia-during poor portfolio performance. 1979-93-experienced average annual per cap- ita GDP growth of 1.4 percent and average Project completion reports also flag other short- inflation of 24.6 percent. This close to median comings in the Cameroon portfolio of projects. environment was conducive to satisfactory Because of inadequate design and appraisal, portfolio performance. However, portfolio per- projects frequently had to be redesigned after formance was significantly enhanced by well Bank approval, and results often fell well above median process performance by the bor- short of those projected at appraisal. In a num- rower and the Bank. This result also tends to ber of cases, there was an inaccurate assessment confirm the finding from the quantile regres- of the extent of the economic difficulties, sion that borrower and Bank performance is including overoptimism about the country's most rewarding in countries with historically external prospects, and underestimation of the above average portfolio performance. consequences of declining economic prospects on fiscal revenues, which led to recurrent diffi- Mauritius is-in addition to China-another culties in project funding. The magnitude of the example of good performance in the NW cor- governance problems was also underestimated. ner. The experience of Mauritius further rein- For example, the growing economic and politi- forces the positive correlation between portfolio cal tensions triggered by the economic crisis, quality, an enabling macroeconomic environ- with their special ethnic and tribal dimensions, ment, and high quality Bank and borrower per- led to civil unrest and military intervention formance. Among the 61 countries in the during 1991-92, which caught the Bank by sample, Mauritius ranks fourth in portfolio surprise. Draft legislation negotiated between quality. Of the eight projects implemented dur- the Executive Branch of the government and ing the period 1978-88, seven were rated the Bank frequently encountered grave difficul- satisfactory and of likely sustainability. The ties in Parliament, necessitating major and 80 percent ID rating represents the highest of unexpected alterations. And the weak imple- the whole country group. mentation capacity of public institutions-suf- fering frequently from inadequate coordination The successful macroeconomic adjustment with the work of other agencies, widespread efforts supported by the Bank's structural corruption, and inefficiency-was too often adjustment loans in the 1970s contributed to addressed through expatriate technical assis- Mauritius' impressive portfolio performance. tance, which failed to overcome difficulties Following a period of stagnation and price and build local capacity. The government's instability in the late 1970s, the Mauritian often weak commitment to project objectives economy strongly recovered. Between 1983 and 58 1994, per capita GDP grew at 5.3 percent annu- and vice versa. Mexico, Hungary, and Zimbabwe ally, and inflation averaged just 7.3 percent. are good illustrations for the SW corner, with A stable macroeconomic environment and poor macroeconomic performance during the improved policy framework helped to stimu- relevant period, yet satisfactory portfolio late a strong response from the private sector. performance as the result of good or even out- This was an important reason for the successful standing borrower and Bank process perfor- implementation of some of the projects, for mance. The NE corner, or good macroeconomic example, the Sugar Sector Project. and poor portfolio performance, is hard to illus- trate with any case in the country sample. The Good appraisal and supervision by the Bank, country that comes closest is Senegal, although in close collaboration with the borrower, also one has to make an important qualification on made for project success by fostering owner- the macroeconomic performance. ship and commitment, and ensuring appropri- ate project design and correct identification of Hungary. With all of its projects achieving satis- key issues. In the Sugar Sector Project, for exam- factory outcomes (86 percent achieving likely ple, through the involvement of both the gov- sustainability and 64 percent achieving sub- ernment and the private sector in project stantial institutional development), Bank- preparation and appraisal, the Bank helped the supported projects in Hungary have a stellar government establish "a constructive partner- performance record. Yet, this is a surprising ship between the government and the private result, given the persistent macroeconomic sector for sector development." Under the First difficulties the country faced during the Highway Project, the Bank collaborated with sample period (1978-94). the borrower in the design and supervision of all components of the project. Such collabora- The Hungarian economy performed relatively tion helped sensitize the government to the well until the mid-1970s, when it started to run importance of road maintenance for which large balance of payments deficits brought adequate financing has been provided. about by a series of external shocks. Hungary's Overall, in more that 70 percent of the cases, response to these shocks was to defer adjust- good appraisal and effective supervision were ment by borrowing externally, which led to a cited as significantly affecting project outcomes. rapid run-up of convertible currency debt, equivalent to 41 percent of GDP, by 1978. The Country-specific illustrations: the signifi- macroeconomic policy environment remained cance of borrower and Bank performance largely unpropitious through the mid-1980s, and the Hungarian policy of gradual reforms The above country examples in Boxes 2.2-2.3 did not succeed in transforming and dynamiz- and the text illustrate the NW and SE corners of ing the economy. the three-by-three matrix in Figure 2.5, that is, the combination of good macroeconomic per- How then did the portfolio of Bank projects formance with good portfolio performance, do so well? Part of the answer appears to be and the combination of bad macro and bad found in the exceptionally strong borrower and portfolio, with some variation regarding the rel- Bank performance, both of which were ranked ative strength of Bank and/or borrower perfor- among the top ten of the 61 country portfolios mance. The following examples illustrate the surveyed. Remarkably, all of Hungary's SW and NE corners of the matrix, that is, the projects were rated satisfactory on the bor- combination of inadequate macroeconomic rower's preparation, commitment to project performance with satisfactory portfolio results objectives, quality of project implementation, 59 and overall implementation. Similarly, all Bank supervision of the Zimbabwe portfolio, materi- processing inputs-identification, preparation, ally contributed to the unexpectedly favorable appraisal, and supervision-in the Hungarian development impact of these projects. Take, for projects surveyed were rated at least satisfac- instance, the case of the Zimbabwe Urban tory. Another factor may have been the empha- Development Project. The project's primary sis on physical benefits in most projects, which objective was to promote Zimbabwe's capacity experience has shown can be delivered more to supply affordable housing through reform readily than intangibles such as policy reform. of the housing delivery and mortgage market. This was to be achieved mainly by concentrat- An example of how a project with predomi- ing the low-income housing efforts of local nantly physical components could draw upon authorities on lots of serviced land and by quality borrower and Bank efforts to override introducing private sector financing. A key an unfavorable macroeconomic policy climate constraint was that the availability of mortgage is provided by the Hungary Petroleum Project finance was curtailed by the unfavorable mac- approved in 1984. At the time of appraisal in roeconomic situation. Nonetheless, the devel- 1983, Hungary faced dwindling oil reserves. opment of the original group of sites went Stopping this decline was the main objective of according to schedule and generally at or the government and the Bank in this project. below projected costs. And, as it turned out, development and rehabil- itation of three gas fields was the most reward- Moreover, the project created a new vehicle for ing physical component, fully achieving the delivering low-income housing, forging a link expected economic rates of return. Bank assis- between the public and private sectors. Local tance was timely in supporting the govern- governments shifted mortgage financing ment's effort to preserve a measure of energy responsibilities to local building societies, independence, and staff continuity on both releasing public funds for other uses and simul- sides contributed importantly to the smooth taneously reducing publicly borne fiduciary implementation of the project. risks. The volume of funds made available by local banks for low-income housing greatly Zimbabwe is another example where the Bank exceeded what the public housing banks had made a difference against the odds of a mediocre been able to provide. The project also proved to macroeconomic environment. 13 Large fiscal def- be sustainable, as evidenced by the good mort- icits averaging 9 percent of GDP drove inflation gage loan repayment history maintained by upward, and rigid exchange rate policies led to project beneficiaries. This latter result was a severe overvaluation of the currency. The by-product of the participatory approach growth policy climate in Zimbabwe suffered, integrated into the project design. with real per capita GDP declining at the rate of nearly 1 percent per year between 1978 and 1994. The experience of Mexico illustrates that good Still, all six Bank projects during this period appraisal and supervision and a high degree achieved satisfactory outcomes; 95 percent were of borrower commitment can still make some sustainable; and a remarkable 75 percent difference even in an adverse macroeconomic achieved substantial institutional impact-the environment. Out of the 38 projects imple- highest ratio among the 61 countries sampled. mented during 1982-93, 71 percent achieved satisfactory results, while 62 percent had Strong Bank contributions in identifying a rating for likely sustainability. All this projects appropriate to Zimbabwe's stage of was achieved despite a highly volatile development, combined with high quality macroeconomic situation, with the annual 60 inflation rate often exceeding 100 percent. Real- anomaly was driven entirely by Senegal's rela- istic project design based on extensive country tively low inflation rate. During the period and sector work, effective Bank supervision, 1979-93 in which the projects in the country and a government committed to project objec- portfolio sample were implemented, annual tives were the primary reasons for the reason- inflation averaged just 5.1 percent. But low ably good portfolio outcome. In more than 70 inflation largely reflected Senegal's special cur- percent of the cases, good supervision by the rency arrangement rather than effective macro- Bank and a high degree of borrower ownership economic management and a robust economy. contributed to project success. In fact, during the entire implementation period, Senegal's economy experienced severe The Second Trade Policy Loan (TPL) provided external imbalance. The government's inability the best example. The TPL was a simultaneous to institute fundamental structural reforms led attempt at trade liberalization and macroeco- to a sharp erosion of Senegal's international nomic stabilization, and succeeded on both competitiveness and a decline in economic fronts. Several factors accounted for the success activity. Particularly after 1982, real per capita of the program. Good and intensive policy GDP fell by an annual rate of 1 percent. dialogue sensitized the government to the structural issues and ensured a high degree of Economic deterioration significantly affected program ownership, while detailed economic portfolio performance, particularly portfolio and sector work helped in designing a realistic sustainability: while 70 percent of the reform package. In designing the program, the 22 projects implemented over the period Bank made sure that trade liberalization did not 1979-93 had a satisfactory outcome, sustain- undermine the stabilization effort. Nontariff ability was considered likely in just 23 percent. barriers, which were the most important form Economic recession threatened the expected of trade protection before reform, were trans- flow of benefits because of the negative impact formed into tarriffs. Tariffication actually on the government's budget and, hence, its abil- complemented the stabilization program by ity to provide adequate funding of operation contributing to an increase in fiscal revenues. and maintenance, as was the case with the Another good example is the Housing Finance Fourth and Fifth Highway Projects. Lagging Project's provision of finances for low-cost supply response also led to policy reversals that housing and significant policy reform, such as derailed the government's adjustment effort, as removing limitations on interest capitalization happened under SAL IV. Given the govern- in a high-inflation environment. The latter actu- ment's inability to adjust the exchange rate, an ally helped to expand credit supply signifi- export subsidy was needed to compensate for cantly by bringing in commercial banks. the overvaluation of the real exchange rate. However, as economic conditions deteriorated, Senegal. As previously noted, a good macroeco- the government found it increasingly difficult nomic environment is-as a rule-positively to make such payments. The result was a seri- associated with good portfolio performance. A ous erosion of Senegal's competitiveness and noticeable exception to this rule is Senegal. At an increase in external imbalance. In conclu- first glance, it fits the characteristics of the NE sion, Senegal had a stable, yet low-growth eco- corner of the matrix. Although it has a relatively nomic environment. While stability was high ranking of macroeconomic performance certainly a positive factor, the progressive eco- (20), Senegal's portfolio performance is quite nomic decline became the dominant factor over poor, ranking it at 43 of the 61 countries. Closer time. Thus, Senegal may actually be closer to scrutiny, however, shows that this apparent the SE corner than to the NE. Bank process per- 61 formance was rated low, while the borrower's great attention at the early stages of the project performance was better. Overall, the negative cycle, but also in the ex post analysis to learn factors prevailed and led to generally more from actual experience about their impor- poor portfolio performance. tance and the best way of taking them into account. To address these issues, risk manage- ment based on flexible responses to changing Broader issues of Bank-country events is crucial. An important step in the right assistance direction is the ongoing effort to strengthen monitoring by means of performance indicators The foregoing paragraphs discussed the key so as to get early feedback about the chances of a determinants of country portfolio performance project's success. Greater country selectivity and based on the findings of econometric analyses discrimination in the choice of support instru- and anecdotal evidence from country cases. ments are also important. But there are also broader issues related to Bank-country assistance such as those raised Identification of, and adequate response to, in connection with the recent CODE (Commit- these institutional factors is intimately linked to tee on Development Effectiveness) review of the skills mix of Bank staff and their counter- OED's Ghana country assistance review. These parts in the client countries. Often in the past, issues may either escape the type of analysis the Bank's traditional economics focus may undertaken in this chapter altogether or may have been too narrow and other expertise not be adequately highlighted because of data should have been brought to bear in the design limitations. Among the broader issues illumi- of projects and implementation strategies. nated by the 1994 cohort of evaluation projects Corrective action by Bank management has are the societal dimension of Bank operations; begun through recruitment and training. the institutional capacity; the international environment; the importance of nonlending The relevance of ESW for project and program work; and last, but certainly not least, the design was statistically significant in only one skills mix of Bank staff. simulation. But experience with Bank-country dialogue suggests that policy advice based on Institutional factors certainly play a major role rigorous analysis plays an important role in in determining the ultimate developmental shaping the country environment, the orienta- impact of Bank-supported interventions. They tion of individual interventions, and indeed the are also important for the timing of certain entire portfolio-as well as in donor coordina- actions and the time required for results to mate- tion. In this context, the progressive underpin- rialize. Certain factors like borrower commit- ning of country assistance strategies (CASs) ment were picked up in the econometric and individual projects with performance indi- analysis. Most others-like interest group cators sets the stage for more thorough ESW. dynamics in economic policy reform or involve- ment by beneficiaries or the civil society at large The econometric analyses performed for this in program/project design and implementation report did not identify global economic factors or the relative strength of a country's institu- as significant determinants of portfolio perfor- tions-were not fully captured. Rather, they mance. At the same time, it is clear that individ- entered the analysis in a somewhat tenuous way ual projects may be highly affected by an through the assessment of appraisal quality and adverse movement in the terms of trade or the underlying factors that determine appraisal other adverse developments in international quality These are areas that require not only markets. It is also clear that a country's policy 62 response to global economic developments policy improvement in, countries where the matters in determining the actual economic economic environment is inimical to lending. performance, which was found to be of critical importance for overall portfolio performance. Key operational implications derived from the Thus, these factors are not to be neglected and findings of this chapter are: need to be taken into account in the preparation and design of a project. Again, adequate risk * Bank involvement in countries with poor analysis and flexible response to adverse devel- portfolio performance and adverse policy opments are important for the success of a environments should focus on macroeco- project or an entire portfolio. Considering, for nomic stabilization and growth-promoting instance, the longer-term development of com- structural reforms. Successful and sustained modity prices-a notoriously difficult field of improvement of the policy environment is economic forecasting-some progress has been likely to result in significant improvement in made by projecting price ranges within a cer- the existing and future portfolios-in terms tain probability. The probabilistic forecast, in of both development impact during imple- turn, lends itself as an input into stochastic risk mentation and project and portfolio sustain- analysis, which should be more systematically ability. This underscores the importance of used in economic project analysis. At the same Bank support for getting the economic fun- time, it is also an area to be captured more care- damentals right-the prime rationale for fully in the design of CASs and their adaptation adjustment lending and a frequent objective in response to external developments affecting of Bank economic and sector work. a country. This said, a lot depends on a coun- try's policy response, for example, a currency * While good Bank performance at the devaluation in response to a lasting deteriora- appraisal and supervision stages is likely tion in the terms of trade, which could be to make a significant difference in outcome growth enhancing and also have a positive and sustainability, the effort has to be very impact on domestic prices and fiscal revenues. strong to overcome the odds of a poor coun- try economic and implementation environ- ment. Equally, Bank involvement with Implications for projects and the portfolio process is more Bank operational priorities successful in countries that give priority to effective portfolio performance, a good The findings from the econometric analyses and predictor of which is the past portfolio anecdotal evidence from project and country performance record. Given that borrower cases have important implications for setting performance is on average even more the Bank's operational priorities. These findings important than Bank performance for clearly support greater country selectivity with good results, lagging borrower commitment regard to lending operations. Of course, it is and poor management by the implementa- necessary that a careful balance be struck tion agency considerably raise the odds between selectivity (motivated by realistic pros- against success. Both findings suggest that pects for success and considerations of cost- the Bank should be more selective in its effectiveness) and legitimate service entitlement lending interventions. by member countries (springing from the coop- erative character of the Bank Group). Use of * The current Bank practice of maintaining nonlending services may be a cost-effective way minimum lending levels in countries with of maintaining contact with, and contributing to adverse policy environments should be 63 revisited in the light of this chapter's find- deficit relative to GDP, the rate of GDP per capita growth, the lev- ings. The odds of success are poor in the el of country GDP per capita, and the real domestic interest rate. absence of a cohesive strategy to address Social/political indices tested included those representing bu- economic fundamentals. Nonlending ser- reaucratic delay, respect for basic human rights, political stabili- vices directed at restoring adequate policies ty, the freedom to organize, the degree of democratic practices, the availability of civil liberties, and the degree of corruption. may be a better option. The data for these variables were drawn from the World Handbook of Political and Social Indicators, Jodice and Taylor, 1983; Freedom in Specifically, ESW-based Bank-country dia- the World Political Rights and Ciil Liberties, Freedom House Survey Team, R. Bruce McColm et al. (New York: Freedom House, various years); World Human Rights Guide, C. Humana, more cost-effective than lending in a gener- (London: Hodder and Stoughton, c 1986); "Measuring Pol- ally adverse country environment that is yarchy," Coppedge and Reineke, StudIes Io Comparative Interna- marked by poor policies and poor economic tional Dezelopment (Spring) 25, 1: 51-72; and "Governance and and portfolio performance. Moderate lend- the Returns to Investment: An Empirical Investigation." Isham, Kaufmann, and Pritchett, World Bank Policy Paper (Washington, ing for institution-capacity building may DC World Bank, publication pending). The Gastil index has also be warranted where a government is been updated and extended to 165 countries for the period of committed and where direct beneficiary 1972-94. It uses a seven-point scale to weight a 14-item checklist participation is feasible.14 tor civil liberties. In searching for a time period relevant to project and portfolio per- formance, a core period, starting two years prior to approval of the first evaluated loan in the portfolio and ending three years after Notes the completion of the last evaluated operation, was chosen. The decision to include two years before approval was based on the 1. The analysis uses OED's Annual Review Database for project premise that quality at entry is irrevocably set during this period, or portfolio-related variables; the International Financial Statistics and, theoretically, could be shaped significantly by non-Bank vari- Yearbook (Washington, DC: IMF), the World Debt Tables, (Washing- ables, such as the country's macroeconomic and political stability, ton, DC: World Bank, various years), the World Bank Economic external shocks, and so on. For example, the quality of the and Social Database, and the World Bank Development Econom- macropolicy environment (proxied by inflation and growth) dur- ics (DEC) Database for global and country economic variables; ing project preparation and appraisal might alter decisions about and the World Handbook of Political and Social Indicators, C.L. Tav- the project's size, the conditionality, the focus on particular sectors, lor and D.A. Jodice, Vols. 1 and 2 (New Haven, CT: Yale Univer- and so on, of, say, an adjustment operation. Appropriate or inap- sity Press, 1983), and others for the governance variables. propriate assessments of a country's political, social, and econom- ic envimnment during the early stages of project preparation In addition to variables for loan amount and the regional and sectoral onigin of an operation, the OED database furnished vari- obviously affect outcomes, sustainability, and institutional impact. ables for project Outcome, sustainability, institutional develop- The three years following completion are frequently determina- ment; the quality of project identification, preparation, appraisal, tive of the success of implementation. OED time series indicate implementation, and supervision; the degree of compliance with that outcome ratings tend to stabilize soon after this period. loan covenants and conditionality; and the degree of borrower 2. This chapter has two background papers. The first reports the commitment to project objectives. The criteria used in arriving at findings of a project-level econometric investigation of the deter- these ratings are discussed in Assessing Development Effectiveness: minants of project performance using data from project comple- Evaluation in the World BanA and the International Finance Corpora- tion and performance audit reports prepared between 1991 and tion, (Washington, DC: World Bank, May 1994). For the Bank- 1994. "Probit analysis" is used to identify the link between per- borrower dialogue, proxy variables included the number of Bank formance ratings and a collection of other variables measuring economic and sector reports annually, the presence or project characteristics, the quality of Bank processes, and the absence of a Bank resident mission, and the degree of govern- macroeconomic and social-political environment. Three mea- ment commitment. ue i xm sures are examined: the development outcome rating, the sus- Global economic factors were represented by the real LIBOR tainability rating, and the institutional development rating. In (London interbank offered) interest rate, the weighted average the second paper, the country, rather than the project, is the unit economic growth rate of the G7 countries, and changes in nonoil of analysis. The 61 countries selected had at least seven projects developing country terms of trade. Country economic variables evaluated during CY91-94, so that the project sample used was tested included domestic inflation, the black market exchange a subset of the database employed in the project-based study. rate premium, the ratio of public debt to GDP, the size of the fiscal 64 The model employs "quantile regression," which allows a differ- Bank projects, the level of government commitment to project entiation among countries according to the level of portfolio per- objectives, the provision and reliability of domestic funds for the formance. The model included proxies for the macroeconomic operation, the administrative procedures, and the quality of de- policy climate, external shocks, civil liberties, bank economic and cisionmaking. sector studies, the borrower-Bank policy dialogue, and evalua- 8. The last two factors were not highlighted above, as they are-- tors' ratings of Bank processes. statistically speaking-significant only at the 90 percent confi- 3. Governance was proxied by political instability and "freedom dence level, while the other variables are significant at the 95 per- from corruption," assessed at country level. Corruption has an cent confidence level. economic and social dimension, and thus falls within the purview 9. Different aspects of governance were also factored into the of the Bark's focus on development, when it entails rent-seeking portfolio-based econometric analysis. In most model estimates, behavior induced by bribery or other unlawful or improper these factors were not picked up as statistically significant since means. For a discussion of the relevance of corruption issues to the country sample size reduced the degrees of freedom and thus economic development, see Governance: The World Bank's Experi- imposed analytical limitations. The impact of governance on ence (Waslington, DC: World Bank, May 1994). In particular, country and portfolio performance deserves further analysis. Box 10, p. 19, highlights Bank support for economic reforms that Important work is currently under way in the Bank's vice presi- (a) reduce opportunities for rent-taking by simplification of rules dentia unit for Environmentally Sustainable Development in and replacement of administrative with market mechanisms, and connection with estimates of countries' social capital. (b) strengthen institutions so that they might, in turn, improve controls aid reduce incentives for corrupt behavior 10. It would be worthwhile repeating the portfolio-based econo- metric analysis in a few years' time as OED's database expands 4. Institutional development is captured only in the project- with new entries from implementation completion and perfor- based analysis. There are important reporting gaps in OED's da- manceaudit reports and makes it likely thatsomeof the findings tabase that made the country-specific econometric analysis im- would be more robust. possible because of reduced sample size and the associated loss of "degrees of freedom." 11. In this case, macroeconomic performance was measured by GDP per capita growth alone, which is the overriding determi- 5. Cover ance is "the manner in which power is exercised in nant for longer-term sustainability It inflation is also taken into the management of a country's economic and social resources for account, macroeconomic performance does not come out as the development." Governance and Developnient (Washington, DC: strongest determinant of portfolio performance. This reflects a World Bank, 1992, p. 58). major distortion in the country data related to the African coun- b. With multi-variate regression analysis there is often a prob- tries belonging to the CFA franc zone: these countries have been lem of strcng multi-collinearity, that is, the existence of a high de- doing relatively well on inflation but very poorly on growth as a gree of correlation among two or more independent variables. If result of the fixed parity with the French franc. Once this anom- multi-collinearity is strong, the respective independent variables aly is eliminated by considering growth alone, the results con- may jointly be found statistically insignificant and have high form more closely to the expectation derived from the standard errors. At the same time, taken together, the indepen- econometric analyses. dent variaoles in question would still improve the overall ex- 12. The countries that fall in this latter category are taken from planatory power of the equation as reflected in the R-. One the "FY95 Annual Report on Portfolio Performance" (Washing- would imagine that multi-collinearity might be strong between ton, DC: World Bank, forthcoming, Table 41. variables related to the international environment and those re- flecting dcmestic economic performance and-even more so- 13 One should note that, although neither Hungary nor Zimba- among variables measuring Bank and borrower process perfor- bwe are among the bottom ten macroeconomic performers, they mance. Hc,wever, the appropriate correlation tests suggest that were close enough to the bottom with ranks of 48th and 51st. multi-collinearity is not a problem in the equations estimated for 14. See The Contribution of People's Piirticipation: Ei'idence froni 121 the purpome of this analysis. Rural Water Supply Projects, Deepa Narayan, Environmentally 7. Borrow er performance during preparation and implementa- Sustainable Development Occasional Paper Series No I tion is evaluated in terms of the policy environment created for (Washington, DC: World Bank, 1995). 65 3. Poverty, environment, and private sector development The thematic content of recently Overall performance ratings for operations evaluated operations with thematic components, at 64 percent satis- factory, were about the same as for operations To what degree are the Bank's three thematic on average. These operations were somewhat objectives-poverty alleviation, environmen- more complex and risky, and their likely sus- tally sustainable development, and private sector tainability was slightly higher, than for the development-reflected in specific components group as whole (67 percent, as opposed to of project design? How successfully have opera- 65 percent, of the operations rated satisfactory tions met their stated thematic objectives, and were considered likely to sustain their benefits). what accounts for their varying performance? The 737 operations that OED evaluated in Poverty reduction 1993 and 1994, through performance audits and reviews of completion reports, were analyzed The Bank's strategy for reducing poverty sup- to identify best practices and lessons of experi- ports policies that increase the productivity of ence. As in last year's Annual Review, only the poor, provide better access to social services, those operations with components that specifi- and promote broadly based growth. The Bank cally targeted the thematic objectives were pursues this strategy through policy dialogue featured.' Many others may have had positive with governments and through lending opera- or negative effects on poverty, the environment, tions and nonlending services. Hence, although or private sector development. all operations financed by the Bank must be consistent with its poverty-alleviation strategy, Of the 737 operations, 58 percent, accounting every operation need not have an explicit pov- for $35 billion of commitments, had one or more erty-reduction component. The appropriate thematic components. Borrowers from low- mix of instruments depends on the country, its income countries (those with GNP per capita poverty problems, and the efforts being made less than $726) accounted for roughly half the to address these problems. operations with thematic components (Figure 3.1). The share of operations with thematic Types of poverty-reduction interventions often components varied considerably among sec- supported by the Bank include: tors. Policy reform operations had the highest incidence; 83 percent of these operations had * targeted operations intended to reach the thematic components. poor in rural and urban regions, such as 66 agriculture and rural development, educa- tion and population, and health and nutri- FCUE.1T MAICOPN TSBCUTR tion a popationh and INCOME GROUP, OPERATIONS EVALUATED IN 1993 tion operations; and AD19 AND 1994 * poverty-focused adjustment operations, 400 which support efforts to reallocate public expenditures to essential social services E3007 and infrastructure and to eliminate distor- tions that prevent the poor from participat- ing in growth. 200 Of the operations evaluated in 1993 and 1994, 100 most of the 157 that contained poverty-reduc- 2 tion components supported efforts to increase ] the income levels of relatively poor beneficia- $725 or less $726-$1,395 $1,396-$2,895 $2,896-$5,055 over $5,055 ries. They focused on improving farming prac- GNP per capita tices to increase the levels and efficiency of U Total number of projects Projects with poverty production; creating job opportunities; and pro- [ Projects with PSD Projects with environment viding access to credit, better shelter, and Source: Table 3 1. enhanced social services and social safety net programs. The largest share-47 percent-of the operations with poverty components was in countries with the lowest per capita income. no access to infrastructure, on-farm credit, or social services. The projects, which Best practices reached 24,000 families, are being repli- cated, and the cost per family is relatively In agriculture, most of the operations were low. Beneficiary participation in the design designed to improve the productivity and and implementation of the project remains income of the rural poor through land develop- outstanding. A large proportion of settlers ment and credit programs. participate in local organizations involved in productive, social, and marketing activi- * A land development project in Malaysia ties. Nearly all families have clear land titles rehabilitated smallholders' lands, which and are highly conscious of the need to were previously idle or low yielding, with maintain project works. high-yielding varieties. The beneficiaries, many of whom live in poverty, are expected U Thailand's Northern Agricultural Develop- to enjoy higher farm incomes and better liv- ment Project addressed the twin concerns of ing standards. The project's success can be household poverty of upland smallholders traced to the rehabilitation of the land and hill tribes and resource degradation through innovative group farming arrange- caused by poor cultivation practices. Under ments, which boosted farmers' confidence the project, titles and land-use certificates and encouraged entrepreneurship, allowing were given to beneficiaries to motivate for successful diversification into new crops. investment and conservation of suitable land. Access to roads and social services was * Three rural development projects in also improved. The project provides useful Ecuador targeted the rural poor, who had lessons for development interventions in 67 TABLE 3.1: THEMATIC COMPONENTS BY COUNTRY INCOME GROUP, OPERATIONS EVALUATED IN 1993 AND 1994 Total no. of Environmental GNP per capita" operations Poverty incidences PSD incidences incidences $725 or less 387 74 137 106 $726-$1,395 115 36 43 30 $1,396-$2,895 113 19 35 34 $2,896-$5,055 80 24 30 18 Over $5,055 10 1 2 1 Not classified 32 3 7 7 Total 737 157 254 196 Commitments ($mill.) 52,886 10,764 21,707 13,378 a. Country income groups are taken from Operational Policy 3.10, Annex D, issued June 1995, and are measured in constant 1994 US dollars. difficult social and biophysical environ- tors. The project was also innovative in ments. Of particular note, the project used a allowing beneficiaries to decide on how the "problem census" approach to develop- dwellings were to be built. It showed that ment, which relied on a high level of benefi- choosing options based on beneficiary inter- ciary involvement. This participatory mode ests and allowing diversity makes programs of project implementation also helped with more robust. the establishment of successful revolving funds designed by the villagers to support * An urban development project in Tunisia their own credit needs. was designed to provide better shelter and improved urban services for low-income In urban development, most of the satisfactory families, redressing past inequities in hous- operations that achieved their poverty allevia- ing policy. The project supported the reha- tion objectives were designed either to provide bilitation of substandard settlements, housing and shelter to the urban poor or to including revitalization of an old part reverse decay in urban services affecting the of the Medina of Tunis, the development of living conditions of low-income groups. The serviced lots, and the provision of credit most successful ones also incorporated self- for on-site construction. The project financing or cost-recovery provisions. improved the living conditions of 220,000 low-income beneficiaries, three times the * An urban development project in Zimba- number planned at appraisal. Innovative bwe aimed primarily at developing Zimba- financial measures were successfully intro- bwe's financial and institutional capacity to duced to promote self-financing of the revi- supply affordable low-income housing. The talization program. This was done by selling project used local financial institutions to the upgraded space at market prices and introduce new serviced-sites delivery sys- using the net revenue to replenish a fund to tems for low-income households. The allow additional rehabilitation works project showed the viability of involving to be carried out. local thrift institutions in meeting low- income households' needs and in forging a U An urban development program in Jordan strong link between public and private sec- improved conditions of low-income people 68 in the Amman region by providing shelter at N Innovative financial measures that promote affordable prices, encouraging urban devel- self-financing by beneficiaries. opment in appropriate locations at lower cost than under traditional practice, and a Social action programs, such as safety nets providing for full cost recovery. There is and the protection of basic services, to pro- widespread evidence of a proliferation of tect the poor from the adverse consequences good-quality shelter, with positive spill- of adjustment. over effects to other areas. At the root of the success was extensive community par- OED completed a comprehensive review of the ticipation and the inclusion of skills training impact of adjustment operations on the poor programs to strengthen the earning capacity (Box 3.1). The review showed that most coun- of households. tries that successfully implemented adjustment programs also reduced poverty. These coun- Of the three program and policy operations in tries achieved per capita income growth. They the group, the first structural adjustment loan maintained or increased social spending on to Hungary had specific components to mit- health and education. Their social security and igatE the negative effects of adjustment on welfare programs either rose during the adjust- the poor. It helped establish an unemploy- ment period or rebounded soon after. ment insurance scheme to cushion the impact of increasing unemployment after Implementing the lessons the restructuring of unprofitable enterprises (unemployment had reached 4 percent of Evidence is growing that the Bank is progres- the labor force in mid-I 991). Steps were sively implementing these lessons of experi- also taken to protect the real value of pen- ence, both through its program of targeted sion benefits for the poor, to improve the interventions (PTI) and through poverty- delivery of health care services, and to focused adjustment operations and contain health costs. other instruments.2 Lessons of experience In FY94, PTI lending accounted for 25 per- cent of the Bank's investment lending in Lessons of experience drawn from this group of 63 operations; in FY95, the share rose to 32 per- poverty-alleviation efforts highlight the impor- cent in 75 operations. Most PTI operations tance of the following: are intended to reach the poor in three sectors: agriculture and rural development, education, " Bottom-up participatory processes that and population, health, and nutrition. The enhance beneficiaries' role in the design and main goals of the agriculture and rural implementation of operations. development operations have been to raise agricultural productivity, increase market " Cooperative approaches that boost benefi- access, and improve basic infrastructure ciaries' confidence and encourage entrepre- and social services. neurship and innovation. Projects that reflect the lessons learned include: * Regularization of land-use titles (both in (1) an agricultural rehabilitation project in Viet agriculture and in housing projects) to Nam that aims to increase rural income of poor encourage beneficiaries' involvement and smallholders through the provision of rural commitment to development efforts. credit and services; (2) an irrigation technical 69 BOX 3.1: STRUCTURAL ADJUSTMENT, GROWTH, AND POVERTY REDUCTION A recent study by OED analyzes all the eval- growth must take place in activities in uated adjustment operations that the World which the poor participate. The principal Bank supported in 1980-93-144 operations sectors providing employment and income in 53 countries-and tracks what happened for the poor (agriculture, labor-intensive to poverty and income distribution. It finds exports, rural/urban informal economy) that countries that successfully implemented should be supported by appropriate policy the adjustment policies agreed with the Bank instruments and investment programs. have achieved growth in per capita income Strive for better allocated and more efficient and reduced the proportion of their popula- public spending, especially social spending. The tions in poverty. Bank should insist on more thorough pub- lic expenditure reviews than are now the For 23 countries that borrowed for adjust- norm, and should make greater use of pub- ment, enough poverty data are available for lic expenditure conditionality in its lending. reliable "before and after" comparisons. Two * Safeguard the welfare of people who stand to thirds of these countries successfully imple- lose during adjustment. At the design stage mented adjustment, achieved per capita of operations, the Bank should study the income growth, and all but one reduced the likely impact on winners and losers, and percentage of their populations living in pov- advise accordingly on food policy, social erty. The findings affirm that well-managed spending allocations, and safety nets to adjustment can help to remedy social dis- protect groups who are vulnerable. Dur- tress. This said, rates of poverty reduction ing the early phases of adjustment, when varied widely. And while income distribu- investment normally slows down, pro- tion improved in 60 percent of the countries grams should protect the consumption that achieved growth and poverty reduction, levels of the poorest parts of the popula- it remained constant or worsened in the rest. tion and help to sustain social services. Income distribution is of concern because the * Lend for adjustment where governments are more inequitable it is, the higher the growth committed to reforms and likely to sustain rate that is needed to reduce poverty. them. The most common reason why adjustment operations failed was that the Implications for program design agreed policies were only partly adopted or were soon abandoned. * Give deliberate attention to the sources of a Support countries' efforts to improve data growth. Restoration and acceleration of on poverty. growth still remains a major policy objec- Note: See C. Jayarajah, W. Branson, and B. Sen, The Social Dimen- tive for countries undergoing adjustment. sions of Adjustment: World Bank Experience, 1980-93, A World Bank Operations Evaluation Study (Washington, DC: World For the poor to benefit from growth, Bank, forthcoming). 70 assistance project in Ecuador designed to bene- * Pollution management and urban environ- fit poor farmers by giving them control of the ment projects, which include interventions irrigation systems; and (3) an area development for capacity building for environmental project in Tunisia designed to increase farming management, resources for onlending to productivity and income through better infra- enterprises and agencies for environmental structure and services. Similarly, in the human management, pollution prevention and resources area, the Bank has provided support treatment facilities, and support for research, to increase or improve basic health, primary new technologies, and policy design. education, and basic skills in projects in Brazil, India, and Pakistan. Living conditions of the * Natural resources and rural environmental poor have been addressed through urban projects, including forest management development, shelter, and water and sanitation and biodiversity conservation, pest projects. The Surabaya Urban Development management, soil conservation, and Project in Indonesia, for example, targets poor river basin management. households with loans to improve sanitation, drainage, and home improvements, and helps * Environmental institutions projects, beneficiaries obtain land certificates. Social which seek to strengthen national and development projects in Peru and Ecuador aim local environmental management capacity to protect the poor and vulnerable groups from and public involvement in environmental the negative impacts of adjustment through decisionmaking. income-generation activities and retraining for the unemployed. One hundred and ninety six (27 percent) of the 737 operations, accounting for $13 billion in Out of 50 adjustment operations approved in lending commitments, had environment-related 1994 and 1995, 62 percent include poverty components. Nearly three fourths of the water reduction components. Many of the projects and sanitation operations had environmental with these components provide support for components; industry, agriculture, and energy public expenditure programs in basic social also had above-average shares of operations services. Some are designed to remove distor- with such components. tions that especially affect the poor, for exam- ple, certain taxes on basic needs or labor market Best practices regulations. Recent operations in Chad, Niger, and the Slovak Republic also support safety-net Among the most successful ESD efforts were measures, including labor-intensive public those that provided effluent treatment facilities, works programs, targeted child nutrition and improved forest management, undertook social assistance programs, and unemployment biodiversity conservation and research, and social security benefits. and aided in pest control. * Control of industrial effluents: India's Madhya Environmental sustainability Pradesh Fertilizer Project expanded domestic capacity, taking advantage of the The Bank pursues targeted programs, including newly developed Bombay High natural gas specific projects, regional and global efforts, resources. Adequate effluent treatment and and strategies and policies in support of envi- discharge facilities were built in accordance ronmentally sustainable development (ESD). with standards in industrialized countries. Three types of projects are typical: The project's objectives were more than 71 fully met. A compulsory initial environmen- cial independence, with the phasing out of tal assessment of the plant at the design government subsidies. phase was followed by a comprehensive environmental impact assessment four * Avoiding environmental contamination: A key years after the plant was commissioned. The environmental goal of Sudan's Third Agri- studies recommended that the project incor- cultural Rehabilitation Project was to reduce porate the planting of a greenbelt of about environmental contamination from chemi- 100,000 trees using the treated effluent, and cal pesticides. The project introduced a pes- use various other emission controls. The ticide handling system that achieved long- environmental impact assessment con- lasting results. A key accomplishment was cluded that the pollution control measures to broaden awareness of the hazard of were effective and that all effluents were chemical pesticides. This component had within the prescribed standards. Detailed not been fully designed at the start of the disaster and emergency plans are in place project; its success can be attributed to active and their management is periodically project management and intensive Bank reviewed. The project was awarded first supervision during implementation. prize for excellence by the Ministry of Pro- gram Implementation. At the core of its * Preserving biodiversity: Malaysia's National success has been a dedicated high-quality Forestry Research and Development Project project team that from the start took charge was the first free-standing forestry research of project preparation and all aspects of project in the Bank. Project identification project management, especially the environ- reflected growing awareness of the need mental components. for a change in the direction of tropical for- estry research with special emphasis on pre- * Treating urban sewage: Tunisia's Third Urban serving biodiversity. The main goal of the Sewerage Project rehabilitated, upgraded, operation was to strengthen institutional and expanded sewerage systems in some capacity to carry out expanded research 30 secondary urban centers, including sew- in ecology, natural forest management, age treatment and promotion of the reuse of plantation forestry, timber processing, treated effluent for irrigation, while protect- forest economics, and policy. As a result ing the environment from the discharge of of the project the national institution has untreated sewage. It also supported institu- emerged as one of the world's leading tional strengthening to improve manage- tropical research institutions. ment of the subsector. As a result of the project, new legislation was drafted requir- * Pest control: Responding to the threat of ing pretreatment of sewage and compulsory a new cycle of locust invasions in the sewer hookups for industries. While there Maghreb, the Bank provided urgent assis- were some implementation delays, the tance to support Algeria's national locust project's institutional objectives were suc- control program. As part of this effort, it also cessful, and the agency in charge (ONAS) supported the reinforcement of comprehen- became the region's top performer. The sive surveillance and warning systems. Bank and ONAS have continued efforts to Though the invasion was smaller than promote the reuse of treated effluent for irri- expected-due to ecological and meteoro- gation, a delicate venture because only a logical factors, as well as successful locust limited number of crops can benefit. A chal- control efforts-the project achieved its lenge remains with regard to ONAS' finan- main goal: ensuring that Algeria would 72 have the institutional capacity and means to in 61 countries, for a total commitment of $10 control any impending locust invasion and billion. Of these, 55 address pollution manage- the ability to predict locust activity and ment and urban environment problems, thereby facilitate future control programs. 62 address rural environment and natural The speed with which the Bank responded resources problems, and 20 focus on strength- shows that support of this nature can be ening environmental institutions. Since the mobilized quickly. Rio summit, the Bank's entire portfolio is being adjusted in response to concerns about the envi- Lessons of experience ronment and to incorporate new knowledge. Experience points to the following lessons: A major lesson of experience being heeded is that all Bank-financed projects must be screened * Environmental assessments and action for their potential environmental impacts. Envi- plans must precede, and be reflected in, ronmental assessments are rooted in the Bank's project design. This has been the Bank's business activity; they influence project design policy since 1989. and investment alternatives, involve height- ened public consultation, and include built-in * Environmental impact assessments should monitoring and management plans to mitigate continue to be done after projects are com- unavoidable, negative environmental impacts. missioned. In response to lessons of experience, the Bank's " Building institutional capacity in environ- approach to projects addressing pollution and mental management can be the most impor- the urban environment has changed signifi- tant long-term Bank contribution. cantly over time. In the area of industrial pollu- tion, the Bank's assistance has shifted from * Where capacity building in environmental pollution control to pollution prevention management has been successfully through technical assistance for industrial pol- achieved, the institutional experience icy formulation and the promotion of cleaner acquired should be disseminated to help industrial technologies. Recently financed others in the country or region facing envi- projects reflect this approach. A project cover- ronmental problems. ing effluent treatment plants in India includes adopting a policy of pollution prevention, * Sensitizing target populations to environ- waste minimization, and private sector incen- mental hazards can have long-lasting tives. In Russia, a recently approved emergency beneficial results. loan to contain an oil spill involved extensive local participation. The portfolio of urban envi- * Enhancing preparedness to address envi- ronmental management projects is focusing ronmental emergencies, such as pests or increasingly on domestic sewage treatment and unhealthy emissions, requires early prepa- disposal and solid waste management. ration and often rapid response with finan- Recently approved projects in Korea, Lebanon, cial and technical support. and the Baltic reflect the new orientations. Implementing the lessons Forestry sector projects predominated among natural resource management projects financed At the end of FY95, the Bank's portfolio of envi- in 1994 and 1995. A typical approach is reflected ronmental operations contained 137 operations in the Madhya Pradesh Forestry Project, which 73 aims to improve the state forestry agency's absorbed nearly one fourth of the resources planning, implementing, and monitoring committed to PSD components. capacity; to adopt participatory processes, tak- ing into account the interests of indigenous peo- Best practices ples and other disadvantaged groups; to increase forest cover and productivity; and to Although the Bank's PSD strategy was not for- improve forest management and biodiversity mally articulated until 1991, some of the opera- conservation. In addition to forestry operations, tions approved in the mid-1980s show features recent projects were approved for pest manage- that are in the mainstream of current PSD think- ment, drainage, and watershed rehabilitation. ing and contain elements of best practice, offer- Strengthening public sector institutions respon- ing worthwhile lessons. sible for the environment in borrowing coun- tries has been another Bank priority. Many improving the business environment: The Sec- Bank-financed projects in support of this goal ond and Third Rehabilitation Credits to aim to produce national environmental action Mozambique supported a gradual program plans, and include the financing of studies, of stabilization and liberalization, to encour- technical assistance, and training. age smallholder production and private investment. Adjustments proceeded gradu- ally in each area so that the effects of policy Private sector development changes could be evaluated. The govern- ment privatized a significant number of The Bank supports private sector development small and medium-size parastatal compa- (PSD) by promoting efforts to improve the busi- nies and reduced the regulation of the pri- ness environment; promoting public enterprise vate sector. As a result of these efforts, the reform and privatization in cases where the economic climate has improved, stimulating private sector would clearly use the resources considerable private sector interest, and more efficiently; promoting private participa- clear progress has been made in shifting tion in infrastructure; and assisting with finan- toward a market economy. The benefits are cial sector reform. More broadly, satisfactory likely to be sustained because of growing adjustment operations contribute to private private sector investment. The success of sector development by helping to create an these operations is attributed to government enabling and competitive business environ- commitment to the reforms, the gradual ment for the private sector. process followed, and an adequate level of external assistance. Of the 737 operations reviewed, 254 (34 percent) had PSD components, which absorbed $22 bil- * Public enterprise reform and divestiture: The lion of lending commitments. PSD components Road Transport and Telecommunications featured in all sectors, but they were much more Sector Adjustment Loan to Mexico and the common in agriculture, finance, and program Transport Sector Adjustment Loan to and policy operations than in the utilities sec- Argentina both supported the privatization tors. Most of them supported sector or struc- of key public enterprises and the easing of tural adjustment, rural and industrial credit government regulation as a way to promote and finance, and export development. They PSD. In both cases the initiative came from were especially important in program and pol- the governments. The Bank provided finan- icy operations, more than 80 percent of which cial and technical support, based on a large had PSD components. Policy reform operations amount of relevant sector work, kept a low 74 profile, and never set the basic policy objec- and exemplary commitment by govern- tives. In both countries the results have been ment, sector institutions, and beneficiaries; highly successful. In Mexico, the telecom- coherent project design based on relevancy munications network has doubled relative and feasibility of project components; to the early 1980s and exceeded government effective borrower-Bank relationship; targets; TELMEX's (Telefonos de Mexico) and favorable macroeconomic environment internal efficiency has improved and its during implementation. workers have benefited. Private sector par- ticipation has increased in all areas of mod- * Providing sustainable access to credit: Indone- ern communications. The deregulation of sia's BRI/KUPEDES Small Credit Project, trucking has witnessed a large number of begun in 1987, complemented other initia- new entrants into the market, decline in tives to support the development of freight rates, modernization of trucking microenterprises and of nonbank financial fleets, and improved services. The impact of institutions to serve the poor. It helped the the reforms in Argentina has been similar. Bank Rakayat Indonesia to (1) establish a The benefits of the reforms are likely to be network of branches, known as KUPEDES, sustained, given the governments' strong to provide nonsubsidized credit to small commitment to privatization and deregula- borrowers, mobilize savings, and extend tion, the growing number of private sector banking services; (2) raise funds to place the firms, and the solid evidence of the benefi- KUPEDES on a self-financing basis; and (3) cial effects of the new policies. improve institutional capacity for program implementation. Completed in 1994, the Private provision of infrastructure: Chile's project extended market-based credit to Highway Reconstruction Project provides small, and especially female, borrowers, and an outstanding example for other countries banking services to rural areas. The experi- to follow in shaping road maintenance ence shows that in some circumstances, pro- operations and policy. The project helped viding greater access to nonsubsidized to reorient Vialidad, the government's main credit to small borrowers (by extending the road agency, toward increased reliance banking network to unserviced areas) can be on private sector contractors to carry out a more effective and sustainable strategy for the works, and engineering consultants to developing small enterprises than simply prepare and control the work, through com- increasing the volume of subsidized credit petitive bidding. A direct objective was to through an existing network. keep government employment in check and infuse private sector expertise and efficiency Lessons of experience into government business. Vialidad and the private sector displayed great compe- Evaluation evidence on private sector develop- tence in implementing the operation. Qual- ment highlights the following lessons: ity of work was high, routine and periodic maintenance of roads exceeded overall * The health of the macroeconomic environ- targets, and the projects were completed on ment matters most. schedule and within or below original budgets. Chile is now well ahead of most * In countries undergoing a dramatic Bank borrowers in shifting road mainte- transformation from a controlled to nance to the private sector. The excellent a market economy, pursuing a gradual project results are attributable to strong but steady program of stabilization and 75 liberalization may be the key to improving rose from nine in FY88 to 23 in FY94, and the economic climate and inducing private although this number fell in 1995, there was an sector interest, offsetting increase in the number of PSD invest- ment lending operations. " Government commitment must be expressed through unmistakable signals As evident from the lessons of recently corn- that the reforms introduced are irrevers- pleted operations, the Bank's PSD support strat- ible-for example, through changes in the egy should first and foremost improve the legal framework within which the private business environment, particularly through sector operates. reforms aimed at developing sound legal and regulatory frameworks, and institutional " Where government commitment to PSD arrangements that underpin the emergence of a reforms is strong, policy initiatives, mea- private sector. Knowledge about country-spe- sures, and actions precede Bank involvement cific business environments has been increased and are eventually reflected in less ex ante through sector studies and private sector assess- conditionality in Bank financial support. ments (PSA). PSAs have been carried out in more than 27 countries. The findings drawn from these " In successful PSD reforms, the Bank pro- assessments have provided specific information vided quality sector work and kept a low to buttress country assistance strategies and profile, while providing effective technical operations to aid private sector growth. Enter- and policy support. prise reform and privatization are now a central focus of PSD activities; more than 50 lending " Extending banking networks to unserviced operations featuring privatization as a key com- areas seems to be a sustainable way to ponent were approved by the Bank in FY94 and provide nonsubsidized credit to small FY95. The Bank has shifted its approach from borrowers, mobilize savings, and extend focusing on small and medium firms to empha- banking services. sizing privatization of larger public enterprises. This shift is a direct consequence of the lessons " Where capital markets are still developing, learned, particularly those from Argentina. financial intermediation loans--designed with minimum regulation and allowing Bank support for private participation in infra- market forces to dictate the terms-can be structure and in assisting financial sector effective tools for channeling credit to pri- reform has also benefited from the lessons of vate entrepreneurs. experience. Since 1988 there has been remark- able growth in this area, including a large role In the last two years OED has prepared compre- by the International Finance Corporation. The hensive reviews documenting various aspects Bank has financed more than 120 infrastructure of PSID experience and best practices; these are projects with significant private participation summarized in Box 3.2. components. Many of these operations have funded public investments and legal and regu- Implementing the lessons latory reforms designed to promote a private sector role. just as in earlier successful opera- The number of Bank lending operations with tions in Chile and Mexico, Bank projects in PSID components increased from 66 in FY88 to Albania, Ethiopia, and Poland include a private 89 in FY94, and 85 in FY95. The number of sector role in transport maintenance and repair, adjustment operations with PSD components and the deregulation and privatization of truck- 76 Box 3.2: SUPPORT FOR PRIVATE SECTOR DEVELOPMENT: LESSONS FROM OED STUDIES Industrial reorientation in East Africa associated with adjustment operations. The OED study finds that Bank policy in support During 1985-92, the Bank advised the gov- of privatization is adequate and well articu- ernments of Kenya, Malawi, Tanzania, and lated, but needs to be applied sensitively to Zambia to reorient their inward-looking specific conditions. Success has been hardest industrial development strategies, reform the to achieve in low-income countries, where policy environment to improve the countries' political commitment to privatization has international competitiveness, and foster the been weak and where the environment for development of private industry, especially private enterprise has often been difficult. small and medium-size enterprises. Tanzania Success depends heavily on: and Zambia have an active informal sector, but the formal private sector is still small. * Government commitment to privatization. Kenya, by comparison, has a large and viable a Borrower's administrative capacity. private sector with the potential to expand a Level of development of the country's and create jobs. In Malawi and Tanzania, par- private sector. tial economic liberalization has produced a Macroeconomic conditions, policies, and growth in the number of micro, small, and regulations that encourage competition. medium-size enterprises. Industrial restructuring: a review of The OED study concludes that the Bank World Bank operations should support private sector expansion in parallel with parastatal privatization by pro- The OED review analyzes Bank experience moting efforts that reduce investor uncer- with industrial restructuring in 46 countries tainty; initiate comprehensive reforms to to determine what strategies work and why. improve incentives, reduce controls, and pro- It argues that the Bank must extend its role mote domestic and external competition; beyond helping governments establish the remove trade barriers; introduce regulatory enabling environment for the private sector and institutional measures to improve access into support for: to information; reform the financial sector; and develop private sector institutions. 0 Technology development, technology transfer, licensing, and the like. World Bank assistance to privatization in * Improved information about market developing countries opportunities, particularly for exports. N Labor training, redeployment, and other Between 1982 and 1992, some 68 countries actions to mitigate the costs of received support for privatization, mostly restructuring. 77 ing. Support for a private sector role in the pub- tive and competitive business climate. Part of lic utilities sector has also seen sustained action, the strategy calls for stepping up delivery of including in projects in Argentina, India, Mali, nonlending services, which will help govern- the Philippines, and Sierra Leone, among many ments identify and correct policy distortions. others. In the financial sector, support has increased from 20 operations in FY93 to an The Bank also recognizes the increasing estimated 34 operations in FY94. importance in low-income countries of the informal sector and the role of microenter- In the poorest countries, the challenges of pro- prises, and is becoming involved in microfi- moting private sector development are many nance operations to improve microenterprise times harder than in higher-income countries access to credit and financial services, and Based on experience, developing an appropri- promote savings mobilization. ate business environment in these countries is a long-term process vulnerable to frequent rever- sals. Accordingly, the Bank approach calls for Notes encouraging strong political commitment through establishing or strengthening the legal 1. Thematic components were identified in operations whose and institutional systems and the required objectives included enhancement or protection of the environ- human resource base. The Bank has recognized ment, enhancement or strengthening of the private sector's role, that under such circumstances, when low- or targeting of specific social groups characterized by socioeco- income countries begin to embrace reforms, nomic status (that is, poverty). donors must be ready to provide the necessary 2. See Poverty Reduction and the World Bank: Progress in Fiscal support to sustain the effort to create an attrac- 1994 (Washington, DC: World Bank, 1995). 78 4. Improving development effectiveness Introduction project outcomes. Similarly, OED evaluations have provided ample evidence that quality at The development agenda is increasingly com- entry, and careful management of ongoing plex. The expectation that the Bank would operations, are as critical to ensuring satisfac- mainly support borrowing countries' physical tory outcomes. Sound portfolio management, infrastructure requirements has slowly been including the management of risks, makes a overtaken by an increasingly complex agenda substantial difference in achieving develop- with poverty alleviation, environmentally sus- ment results. tainable development, and private sector devel- opment at its core. This agenda pursues There is growing evidence that a more con- progress in economic management within a certed effort to address thematic emphases in fully participatory framework, including the Bank operations is positively linked to project views of all stakeholders and partners, and outcomes. A review of best practices points to enhanced attention to gender equity. The com- enhanced results due to adequate treatment of plexity of the Bank's agenda is reflected in the thematic concerns in project design. Of course, broad range of institutional and policy objec- addressing thematic concerns on their own is tives being pursued and the variety of instru- not sufficient to yield satisfactory results. It is ments deployed, as well as in the diversity and the confluence of these concerns and a favor- dispersion of its clients. able policy and institutional environment that provides the basis for satisfactory results. The evaluations completed in 1994 confirm this trend toward increased complexity. These oper- ations pursue more goals and have more com- Learning from Bank clients and staff ponents than those in earlier evaluation years. Two thirds of the operations evaluated are The performance record and an increasing deemed to substantially stretch the country's understanding about the factors that drive it implementation capacity. Nearly half involve a are providing an increasingly solid base on multiplicity of goals and institutions, and are which the Bank can learn and is learning. substantially vulnerable to risks. Evaluation experience has a privileged role in organizational learning. OED project evaluations have repeatedly emphasized the overarching importance of a Learning about the Bank's development effec- sound economic environment to satisfactory tiveness through operations evaluation is central 79 to the Bank's organizational learning process. growth is likely to be sustainable only when the So is learning directly from clients and staff. economic environment is propitious. An OED Increasingly the Bank is making efforts to elicit review of country assistance to Ghana2 but- views from these key stakeholders, that is, those tresses the same findings. who are responsible for identifying, designing, preparing, appraising, supervising, and evaluat- A sound economic environment is achieved ing specific operations. Recent findings by OED, through policy reform. Substantial experience based on a sample of borrowers in 13 countries confirms the conditions needed for effective and of Bank staff, indicate that there is room for reform. Lack of borrower ownership has been enhancing borrower and Bank effectiveness in cited as one of the principal reasons for unsatis- quality at entry and portfolio management. Bor- factory project results. Assessing borrower rowers' perception of Bank performance is commitment is not only a matter of verifying a largely favorable; they appreciate above all the government's acceptance of the need for Bank's technical competence and the quality of reform. Government commitment to reform its policy advice. However, borrowers feel the denotes willingness to legislate or amend laws Bank should do more in disseminating cross- and regulations, commit staff and resources, country experience, which they consider to be and abide by conditions agreed to with financ- one of the Bank's most important roles. Both ing partners. Best practice has shown that borrowers and Bank staff share the view that government commitment to reform should much remains to be done to improve the timely precede donor involvement. Conditionality is monitoring of expected development objectives not a substitute for borrower commitment. if these are to be achieved. The Bank can assess borrower ownership and commitment to reforms through economic and A sound policy environment matters sector work, in developing country assistance strategies, and, specifically, during the identifi- Evaluation experience confirms that a stable and cation stage in the financing of new operations. open economic environment positively affects Project design and appraisal must reflect this project outcome. The analyses carried out for this commitment, which must be reassessed contin- review on the determinants of project and portfo- ually through ongoing dialogue and during lio performance confirm the direct link between policy and project implementation. Prior OED the behavior of key macroeconomic indicators work has singled out four criteria that allow the and operational outcomes. Government deci- Bank to assess ownership and commitment: sions at the micro level, or at the level of individ- ual projects, must take into account fully the * The locus of initiative for the policy or economic environment in which the projects are project must be in the government. set if their expected objectives are to be attained. The 1994 cohort provides ample examples of this * The key policymakers responsible for imple- type of macro and microeconomic synergy. mentation must be intellectually convinced that the goals to be pursued are the right ones. Whether the Bank achieves its thematic objec- tives or not is also related to the health of the m There must evidence of public support from economic environment in the countries con- the top political and civic leadership. cerned. OED's recent study of the impact of adjustment on the poor shows that the poor * There must be evidence that the government benefit from economic growth, and economic is building consensus among the affected 80 BOX 4.1: GHANA COUNTRY ASSISTANCE REVIEW: GENERIC LESSONS Accuracy of basic assumptions and sectors, and must channel resources to promote participation. * The Bank must address its propensity to a Bank economic and sector work can be a underestimate institutional and other con- powerful tool to define problems, explain straints in implementing reform programs. their causes and consequences, and iden- " The Bank must increase the realism of tify solutions-and should be used to fos- its projections and help clients manage ter ownership, through dissemination risks of, for example, commodity price and debate of policy options. fluctuations. * Privatizations often involve a political process, and therefore must be grounded Appropriateness of priorities in appreciation of a country's political economy and borrower ownership. * The Bank's country assistance strategies (CASs), and underlying economic and Weaknesses in economic and sector work sector work, must recognize and reflect (ESW) political economy considerations and constraints. * The Bank must strengthen the explicit link * To ensure that viable enterprises are not between quality of projects at entry into a victimized by the speed of trade reforms, country's portfolio and the quality of the Bank should specifically assess the prior ESW. costs and benefits of their closure. 0 The Bank must develop the methodolo- * Bank advice and support in financial sec- gies to define and measure the costs and tor reform must balance improvements in benefits of ESW, and systematically assess the regulatory framework with measures the lessons of experience of ESW to foster greater competition. " The Bank must take steps to define best Bank staffing, skills mix, and location practice in institutional development of work strategy formulation, to take account of systemic institutional constraints and a The Bank must match staff expertise and underlying issues of economic governance. experience to the complex demands embedded in current country strategy. It Borrower ownership and political will must consider the potentially disruptive effect of frequent staff turnover on policy * The Bank must define and measure the dialogue and effective implementation, minimum workable degree of ownership and recognize that certain functions are in different projects and across countries better performed by field-based staff. 81 stakeholders and can rely on their support economic variables, external factors, and gover- and cooperation. nance considerations. These aspects also matter, but have a lesser impact on project outcomes. The Bank's country assistance strategy instru- Chapter 2 discusses these relationships in detail. ment is key to putting the lessons of experience to work. OED's Ghana country assistance The importance of quality at entry, as measured review highlighted a series of generic issues by the quality of project identification, pre- related to how the Bank can use this instrument paration, and appraisal, was reviewed for more effectively (Box 4.1). To enhance the effec- 1 ,125 projects evaluated between 1991 and 1994. tive use of the country assistance strategy and Annex Table 4.1 shows that projects with promote the wide use of generic lessons, the adequate or better identification, preparation, Bank is progressively introducing performance and appraisal had an 80 percent satisfactory indicators in country assistance strategies. rate versus 25 percent for projects that were deficient in all these aspects.3 Of the projects This Annual Review reaffirms the importance with deficient appraisal but adequate identifi- of the Bank helping borrowers build their eval- cation and preparation, 46 percent had satisfac- uation capacity. The Bank does not have and tory outcomes, nearly half the rate of those with should not seek a monopoly on learning from adequate appraisal. Only 34 percent of projects operational experience. Country ownership with deficient preparation and appraisal, but requires that the borrower be the intellectual adequate identification, were rated satisfactory. source for policy reforms and project selection. Evaluation capacity development provides the In addition to these analyses, evaluation work borrower with the instruments needed to ini- carried out by OED, jointly with the Bank's tiate its own development strategies and pro- Operations Policy Department, in the past year grams, to take account of government and civil suggests a set of key lessons to ensure best prac- society's views, and to express them in sound tice in Bank economic analysis of projects. development alternatives and choices. Evalua- Sound economic analysis of projects seems to tion capacity provides borrowers with a useful be a strong proxy for quality at entry (Box 4.2). instrument to better manage development, and its inherent risks, learn from experience, and take responsibility for results, as true owner- Portfolio management matters ship demands. The Bank has a role to play in fostering evaluation capacity building-a cru- The analysis of the factors that determine cial ingredient of good governance. project and country portfolio performance in Chapter 2 indicates that the quality of borrower implementation and Bank supervision is also Quality at entry matters crucial. After preparation and appraisal, how well supervision is carried out is the next best The analyses carried out for this Annual Review predictor of project performance. Adequate or point unequivocally to quality at entry as being better supervision improves the relative per- of key significance in explaining project and centage of satisfactory projects, even where portfolio performance. As Chapter 2 noted, the preparation and appraisal have been deficient. quality of project preparation, a borrower Deficient supervision hurts projects that have responsibility, and of appraisal, a Bank responsi- had adequate or better identification, prepara- bility, have a significantly stronger effect on tion, and appraisal (see Annex Table 3.1). The project performance than do key country macro- results reported in Chapter 2 also suggest that 82 BOx 4.2: QUALITY OF ECONOMIC ANALYSIS: CRITERIA The nine elements listed below describe the This analysis examines a project's financial analytical process required to judge the qual- impact, and determines recurrent cost fund- ity of project economic analysis. The tenth cri- ing and cost recovery requirements. The ade- terion ensures that the nine elements are quacy of the project's financial sustainability coherently analyzed and weighed in arriving is determined. at an overall judgment about the quality of the project economic analysis carried out. Some of * The soundness of the project's cost/bene- the judgments require that aspects beyond the fit analysis involves assessing the method- strictly economic ones be factored in: ological rigor of cost and benefit flows, the realism of the "without project" scenario, The project must be explicitly linked to the the realism of price and quantity assump- Bank's economic and sector work and tions, the use of appropriate opportunity must be a part of the country strategy. costs to correct for market distortions, the choice of discount rate, and the expected This criterion assures that the project is rele- net present value estimations. vant given country requirements and that its * Project risks must be identified and taken objectives are coherent with the macroeco- into account either in formal risk analy- nomic policy environment. sis, or through quantitative sensitivity analysis. * The project must be the best choice among * Institutional constraints that may pose a alternatives to achieve the desired objec- risk to the achievement of the project's tives. Project design must reflect the choice expected results must be fully identified that either produces the highest net present and factored into the analysis. value, for projects subject to cost/benefit * The poverty incidence of the project must analysis, or be the least-cost alternative, be examined to ensure that net benefits usually in the case of social sector projects. accrue to the target population. * Environmental externalities must be fac- Other aspects should be reflected in the tored into the analysis to ensure that net choice-for example, whether there has been benefits compensate for any unavoidable adequate participation by stakeholders in the diseconomies. choice of alternatives. * Performance indicators based on risk analysis must be identified so that the * The project's fiscal impact should be risks that most directly affect project pri- assessed to determine its effect on the ority objectives and expected outcomes public investment program and public can be monitored on a regular basis dur- expenditures. ing project implementation. 83 under some circumstances, good supervision of best practice where project success can be can act as a positive countervailing force, even attributed to participatory approaches followed where the country's macroeconomic perfor- in the design and implementation of projects mance is unfavorable. with thematic objectives. Similarly, participa- tion is at the core of enhancing government However, OED's review of the Bank's Annual ownership and commitment to sustainable Report on Portfolio Performance suggests that policy reforms. Indeed, the level of participa- current supervision practice does not focus suffi- tion is an important ingredient of ownership. ciently on determinants of outcomes that would While not new, this finding is worth emphasiz- induce the early restructurings or the mid- ing given that it is all too often honored in course corrections needed to achieve projects' the breach. development objectives. While progress has been made in recent years to improve rating It has been well established that development methodologies, there remains a gap between projects have become increasingly complex and supervision performance ratings, which often taxing, both on borrowers and the Bank. It is turn out to be too optimistic, and evaluation rat- also evident that the Bank, in partnership with ings given to completed projects. Bank manage- borrowers, can improve substantially the man- ment is aware of the need to close this gap, the agement of necessary risks, or take steps to min- so-called "disconnect." More forceful portfolio imize them. The lessons from the quality at management, focused on realistic assessments of entry reviews point to how this can be done development objectives, will be needed to do so. effectively. Best practice also points to the essen- tial place of pilot projects, where mainstream- Further evidence that this needs to be addressed ing them can be too risky if undertaken too soon by the Bank with some urgency comes from a and without sufficient evidence that the objec- recent OED review of Bank monitoring and eval- tives can be achieved. Within and cross-country uation practices in approved operations. The dissemination of development experience, an study, based on a review of 164 investment area where borrowers expect the Bank to play a projects, recognizes that significant progress is key role, invites dialogue and the construction being made at the working level, but concludes of partnerships that increase the likelihood that monitoring and evaluation is still a neglected that expected results can be attained. feature of quality assurance-thus confirming the views of borrowers and staff referred to Finally, effective development partnerships earlier in this chapter. The review notes that require mutually reinforcing and coordinated the quality of performance indicators used is still Bank action with other development agencies, not high, that there are insufficient Bank skills multilateral and bilateral, and with nongovern- deployed in this area, and that borrower mental organizations whenever appropriate. capacity needs to be addressed more effectively. Through such strategic alliances, the Bank and its partners can maximize their effective- ness and enhance the standard of living Development partnerships matter of the poor. A lesson that stands out from the 1994 cohort of evaluated operations is that stakeholder partic- Notes ipation has a significantly positive effect on project outcomes. The review of thematic per- 1. See C. layarajah, W. Branson, and B. Sen, The Social Dune- formance in Chapter 3 contains many examples sions o Adjustment: World Bank Experce, 1980-93, A World 84 Bank Operations Evaluation Study (Washington, DC: World 3. Appraisal performance is, of course, influenced by the quali- Bank, forthcoming). ty of identification and preparation. Seventy nine percent of 2. See Robert P Armstrong, Ghana Countn Assistance Review. A projects with satisfactory outcomes had adequate appraisals pre- Study in Development Effectiveness, A World Bank Operations ceded by adequate identification and preparation (50 percent ot Evaluation Study (Washington, DC World Bank, 1996). the total number of projects reviewed). 85 Annexes Annex 1 Figure 1.1 Disbursements of evaluated operations, by region, 1994 and 1974 cohorts Figure 1.2 Disbursements of evaluated operations, by sector, 1994 and 1974 cohorts Figure 1.3 Distribution of the 1994 evaluation cohort and the FY95 ongoing portfolio, by sector Table 1.1 Operations evaluated, by years of approval and closing, 1994 cohort Table 1.2 Distribution of evaluated operations, by years of approval and closing, 1974-94 cohorts Table 1.3 Operations evaluated, by sector and region, 1994, 1993, 1990-94, and 1974-94 cohorts Table 1.4 Disbursements of operations evaluated, by sector and region, 1994, 1993, 1990-94, and 1974-94 cohorts Table 1.5 Major objectives of operations evaluated, by sector, lending type, and region, 1994 cohort Table 1.6 Demandingness, complexity, and riskiness of project objectives, 1994 cohort Table 1.7 Performance of operations evaluated, by sector, lending type, and region, 1994, 1993, 1990-94, and 1974-94 cohorts Table 1.8 Performance weighted by disbursements, by sector, lending type, and region, 1994, 1993, 1990-94, and 1974-94 cohorts Table 1.9 Distribution and performance of operations evaluated, by lending type and year of evaluation, 1974-94 cohorts Table 1.10 Sustainability of operations evaluated, by sector, lending type, and region, 1994, 1993, and 1989-94 cohorts Table 1.11 Sustainability of operations evaluated, by year of approval, 1978-87 Table 1.12 Achievement of institutional development objectives, by sector and region, 1994, 1993, and 1989-94 cohorts Table 1.13 Influence of objectives on achieving institutional development Table 1.14 Relevance, efficacy, and efficiency and overall outcome, 1994 evaluation set Table 1.15 Achievement of major objectives, 1994 cohort Table 1.16 Average rates of return, by sector and region, 1994, 1993, 1990-94, and 1974-94 cohorts Table 1.17 Cost and implementation time variations, by sector and region, 1994, 1990-94, and 1974-94 cohorts Table 1.18 Outstanding operations in the 1994 evaluation cohort 89 ANNEX FIGURE 1.1: DISBURSEMENTS OF EVALUATED OPERATIONS, BY REGION, 1994 AND 1974 COHORTSa 1994 Cohort: 246 operations with $17,249 million disbursed South Asia (I 1.73%)- Africa ( 2.56%) Middle East and North Afnca (8 13%) ,I East Asia and Pacific (15.82%) Europe and Central Asia (8.95%) Latin America and Caribbean (42.80%)- 1974 Cohort: 50 operations with $4,040 million disbursed South Asia (7.40%) Middle East and North Africa (4.42%) Afna (14.85%) --East Asia and Pacific (9.81%) Latin America and Carthbean (33.42%) Europe and Central Asia (30.09%) a. Disbursements are net commitments measured in real terms (1994 constaint dollars). 90 ANNEx FIGURE 1.2: DISBURSEMENTS OF EVALUATED OPERATIONS, BY SECTOR, 1994 AND 1974 COH.ORTS' 1994 Cohort: 246 operations with $1 7,249 million disbursed Water and sanitation (2.66% Urban (6.83%) - Transport (9.43%) Agriculture (27.98%) Telecommunications (0.50%) Technical assistance ((0.29%) Program and policy (12.7, -Energy (4.54%) Powe (1.6% Finance (7.25%1 Human resource (5.47%) Indus"ry 111.12%) 1974 Cohort: 50 operations with $4,040 million disbursed Water and sanitation (0.35%)- Agriculture (5.48%) Energy (2.88%), Finance (9.87%,) -uan resource (2.86%) Transport (62.34%,)Poe 147% Telecomnnunicationts (I .45%~ 1 a. Disbursements are net commu-itments measured in real terms (1994 constant dollars). 91 ANNEX FIGURE 1.3: DISTRIBUTION OF THE 1994 EVALUATION COHORT AND THE FY95 ONGOING PORTFOLIO, BY SECTOR 1994 Cohort: 246 operations Other 1 1.63%) Water and sanitation (3 66%)- Urban (7.32%)-1 Transport (10.16%.) Agriculture (32.93%) Telecommunications (1.22%) Program and policy (7.32%i Power (8.94%)- \ Energy (2.44%) - Finance 18.94% I Human resource (9.76%j FY95 Ongoing Portfolio: 1,742 operations Other (1.78%) 1 Water and sanitation (5.74%) Urban (6.54%)- Agriculture (23 13%) Transport (I 1.029) - Telecommunications (1.66%i- - -Energy (3.33%) Finance (5.17%) Program and policy (1 1.83%) Power (6 43%) - Human resource (20.611,,) Industry 12.76%j 92 ANNEX TABLE 1.1: OPERATIONS EVALUATED, BY YEARS OF APPROVAL AND CLOSING, 1994 COHORT Year Number of operations approved Number of operations closeda 1963 1 0 1964 0 0 1965 0 0 1966 0 0 1967 0 0 1968 0 0 1969 1 0 1970 1 0 1971 0 0 1972 0 0 1973 0 0 1974 0 2 1975 0 0 1976 0 0 1977 2 0 1978 0 1 1979 4 0 1980 1 1 1981 1 0 1982 8 0 1983 12 0 1984 34 2 1985 59 0 1986 42 1 1987 22 1 1988 24 1 1989 17 2 1990 8 11 1991 6 15 1992 2 78 1993 1 110 1994 0 11 1995 0 6 1996 0 2 1997 0 1 1998 0 0 1999 0 1 Total 246 246 a. The formal closing date normally occurs shortly after the final disbursement of the loan or credit. However, there are cases of projects where disbursements are either canceled or disbursed early where legally established closing dates are not adjusted. This explains the ten evaluated projects with closing dates after 1994. 93 ANNEx TABLE 1.2: DISTRIBUTION OF EVALUATED OPERATIONS, BY YEARS OF APPROVAL AND CLOSING, 1974-94 COHORTS Year of closing Year of 1995 approt,al 1963 1965 1967 1968 1970) 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980) 1981 1982 198.3 1984 1985 1986 1987 1988 1989 1990 1991 7992 1993 1994 or later Total 1958 1 1 1961 1 2 2 1 1 7 1962 1 1963 1 1 1 1 2 6 1964 2 1 4 2 2 1 1 1 1 15 1965 3 5 1 3 1 1 14 1966 1 2 1 5 11 4 1 227 1967 9 15 4 4 2 1 35 1968 3 11 19 11 8 6 5 2 1 2 1 69 1969 1 4 15 18 22 14 10 7 5 2 98 1970 1 1 6 16 24 25 22 11 94 5 120 1971 1 1 4 13 20 23 18 18 10 9 117 1972 2 2 4 21 21 22 27 24 12 4 1 1 141 1973 3 2 I 17 29 39 36 22 11 6 1 167 1974 3 2 6 9 23 34 45 33 19 4 1 1 180 1975 2 4 3 4 12 33 38 41 25 18 7 1 188 1976 3 2 3 15 42 51 52 30 12 1 211 1977 3 1 8 24 29 59 45 36 8 3 1 217 1978 1 4 5 2347 54 5334 12 3 3 3 242 1979 2 1 5 31 32 51 57 34 16 4 1 2 1 237 1980 1 3 6 5 224555 5538 12 7 3 2 254 1981 6 5 1324 3346 5724 11 11 2 232 1982 5 3 8 29 4858 4822 21 6 3 1 252 1983 6 6 14 12 35 45 46 36 30 3 233 1984 2 4 3 21 18 28 25 45 20 166 1985 3 6 11 10 38 47 45 32 3 195 1986 7 8 9 14 18 31 27 2 1 117 1987 1 6 11 10 12 16 12 4 1 73 1988 5 9 7 12 10 7 4 54 1989 2 8 12 6 3 4 35 1990 1 7 3 2 1 0 14 1991 2 3 I 2 8 1992 1 1 2 1993 1 1 Total evaluated 1 1 2 6 3 7 39 78 66 86 95 116 108 142 177 201 210 255 227 247 239 228 254 189 192 197 208 118 23 14 3,729 ANNEX TABLE 1.3: OPERATIONS EVALUATED, BY SECTOR AND REGION, 1994, 1993, 199094, AND 1974-94 COHORTS 1994 1993 1990-94 1974-94 No. % No. % No. % No. % Sector Agriculture 81 33 86 27 465 31 1,141 31 Energy 6 2 18 6 80 5 128 3 Finance 22 9 32 10 136 9 344 9 Human resource 24 10 30 9 141 10 363 10 Industry 14 6 19 6 77 5 168 5 Pollution control 0 0 0 0 0 0 5 0 and solid waste Power 22 9 28 9 117 8 313 8 Program and policy 18 7 16 5 94 6 168 5 Technical assistance 4 2 15 5 53 4 91 2 Telecommunications 3 1 4 1 23 2 90 2 Tourism 0 0 0 0 6 0 25 1 Transport 25 10 44 14 151 10 576 15 Urban 18 7 13 4 71 5 127 3 Water and sanitation 9 4 13 4 70 5 190 5 Lending type Adjustment 30 12 36 11 153 10 228 6 Investment 216 88 282 89 1,331 90 3,501 94 Region Africa 75 30 114 36 534 36 1,177 32 East Asia and Pacific 40 16 54 17 244 16 629 17 Europe and 13 5 18 6 88 6 290 8 Central Asia Latin America and 59 24 58 18 297 20 773 21 Caribbean Middle East and 28 11 29 9 133 9 382 10 North Africa South Asia 31 13 45 14 188 13 478 13 Total 246 100 318 100 1,484 100 3,729 100 Note Totals may not sum, due to rounding error. 95 ANNEX TABLE 1.4: DISBURSEMENTS OF OPERATIONS EVALUATED, BY SECTOR AND REGION, 1994, 1993, 1990-94, AND 1974-94 COHORTS (1994 US dollars) 1994 1993 1990-94 1974-94 $m1ill. % $mill. % $mill. % $mill. % Sector Agriculture 4,826 28 4,150 16 27,295 25 60,162 24 Energy 782 5 1,262 5 5,611 5 8,664 3 Finance 1,250 7 3,856 15 12,234 11 24,089 10 Human resource 943 5 1,351 5 5,502 5 12,014 5 Industry 1,917 11 2,763 11 8,348 8 16,783 7 Pollution control 0 0 0 0 0 0 193 0 and solid waste Power 2,014 12 3,500 13 14,337 13 33,929 14 Program and policy 2,116 12 3,712 14 14,854 13 28,065 11 Technical assistance 50 0 139 1 559 1 879 0 Telecommunications 87 1 231 1 718 1 5,404 2 Tourism 0 0 0 0 317 0 993 0 Transport 1,626 9 3,513 13 11,811 11 39,773 16 Urban 1,178 7 730 3 5,278 5 7,859 3 Water and sanitation 459 3 990 4 4,308 4 9,713 4 Lending type Adjustment 5,621 33 8,601 33 28,228 25 41,686 17 Investment 11,628 67 17,596 67 82,946 75 206,836 83 Region Africa 2,167 13 5,155 20 20,154 18 38,885 16 East Asia and Pacific 2,728 16 4,680 18 21,787 20 51,018 21 Europe and 1,544 9 2,097 8 9,394 8 27,990 11 Central Asia Latin America and 7,383 43 7,693 29 34,937 31 67,485 27 Caribbean Middle East and 1,403 8 1,306 5 7,130 6 19,611 8 North Africa South Asia 2,024 12 5,265 20 17,772 16 43,533 18 Total 17,249 100 26,197 100 111,173 100 248,522 100 Note: Totals may not sum, due to rounding error. 96 ANNEX TABLE 1.5: MAJOR OBJECTIVES OF OPERATIONS EVALUATED, BY SECTOR, LENDING TYPE, AND REGION, 1994 COHORT Total no. of Number of projects With these major objectives 1994 cohort Institutional operations Macro Sector Physical Financial development Sector Agriculture 81 0 14 28 9 36 Energy 6 0 1 5 1 2 Finance 22 2 5 4 4 17 Human resource 24 1 0 14 2 19 Industry 14 2 4 8 4 9 Power 22 0 7 16 5 12 Program and policy 18 9 9 0 6 11 Technical assistance 4 1 1 0 0 2 Telecommunications 3 0 3 1 2 3 Transport 25 1 4 16 3 13 Urban 18 1 7 11 4 10 Water and sanitation 9 0 3 7 4 8 Lending type Adjustment 30 12 14 1 10 13 Investment 216 5 44 109 34 129 Region Africa 75 3 17 27 9 42 East Asia and Pacific 40 2 5 25 6 26 Europe and Central Asia 13 0 5 7 4 7 Latin America and Caribbean 59 11 18 24 10 37 Middle East and North Africa 28 1 10 16 11 17 South Asia 31 0 3 11 4 13 Total 246 17 58 110 44 142 Note: Totals may not sum, due to rounding error. ANNEX TABLE 1.6: DEMANDINGNESS, COMPLEXITY, AND RISKINESS OF PROJECT OBJECTIVES, 1994 COHORT Percent high or substantial Out of Out of Highly Substantially Modestly Negligibly Total 1994 cohort objective type Demanding on borrower 57 108 68 7 240 67 69 Demanding on Bank 26 73 122 19 240 40 41 Complex 37 88 98 16 239 51 52 Risky 31 85 105 15 236 47 49 97 ANNEX TABLE 1.7: PERFORMANCE OF OPERATIONS EVALUATED, BY SECTOR, LENDING TYPE, AND REGION, 1994, 1993, 1990-94, AND 1974-94 COHORTS Satisfactory operations 1994 1993 1990-94 1974-94 No. % No. % No. % No. % Sector Agriculture 46 60 54 67 252 56 703 63 Energy 4 - 10 59 60 76 104 82 Finance 13 59 14 47 76 57 246 72 Human resource 19 83 23 79 99 71 279 78 Industry 8 67 9 56 38 53 107 66 Pollution control and solid waste 0 0 0 3 - Power 14 64 21 88 82 73 259 84 Program and policy 12 67 11 79 59 64 120 73 Technical assistance I - 5 36 27 53 54 61 Telecommunications 2 - 3 - 16 70 80 89 Tourism 0 0 5 - 16 64 Transport 16 67 27 63 103 69 451 79 Urban 14 78 6 50 52 74 99 79 Water and sanitation 7 - 8 62 39 56 139 74 Lending type Adjustment 21 70 22 65 96 64 155 69 Investment 135 65 169 64 812 63 2,505 73 Region 112 112 Africa 42 57 59 53 293 55 735 63 East Asia and Pacific 31 78 43 80 187 77 519 83 Europe and Central Asia 9 69 10 83 50 62 225 80 Latin America and Caribbean 37 64 34 63 167 57 527 69 Middle East and North Africa 17 63 17 65 89 69 293 78 South Asia 20 77 28 70 122 70 361 78 Total/average 156 66 191 64 908 63 2,660 72 -Not applicable, Notes: Percentage calculations exclude projects not rated and are only shown tor groups with ten or more projects. Totals may not sum, due to rounding error. 98 ANNEX TABLE 1.8: PERFORMANCE WEIGHTED BY DISBURSEMENTS, BY SECTOR, LENDING TYPE, AND REGION, 1994, 1993, 1990-94, AND 1974-94 COHORTS (1994 US dollars) Satisfactory 1994 1993 1990-94 1974-94 $mill. % $mill. % $mill. % $mill. % Sector Agriculture 3,254 67 2,964 71 15,728 58 38,900 65 Energy 422 - 527 42 3,326 59 6,347 73 Finance 905 72 2,327 61 7,801 64 17,547 73 Human resource 842 89 1,217 90 4,477 81 10,122 84 Industry 1,310 68 1,684 62 5,099 61 12,254 73 Pollution control and solid waste 0 0 0 103 - Power 863 43 3,128 90 10,193 71 27,932 82 Program and policy 1,045 49 3,243 92 10,568 72 21,842 78 Technical assistance 20 - 58 42 312 56 561 64 Telecommunications 50 - 85 - 429 60 5,012 93 Tourism 0 0 294 - 731 74 Transport 1,026 63 2,648 76 8,985 76 32,088 81 Urban 1,134 96 106 15 4,246 80 6,391 81 Water and sanitation 401 - 404 41 2,140 50 6,624 68 Lending type Adjustment 3,540 63 6,139 71 18,985 67 29,889 72 Investment 7,733 67 12,251 71 54,614 66 156,567 76 Region Africa 1,562 72 2,884 56 10,858 54 24,576 63 East Asia and Pacific 2,403 88 3,935 84 18,656 86 44,525 87 Europe and Central Asia 951 62 1,301 62 5,550 59 21,537 77 Latin America and Caribbean 3,936 53 5,730 75 21,292 61 46,159 69 Middle East and North Africa 872 62 695 58 5,053 72 15,373 79 South Asia 1,548 77 3,845 76 12,189 70 34,286 79 Total/average 11,273 65 18,390 71 73,599 67 186,455 75 -Not applicable. Notes: Percentage calculations excluded projects not rated and are only shown for groups with ten or more projects. Totals may not sum, due to rounding error. 99 ANNEX TABLE 1.9: DISTRIBUTION AND PERFORMANCE OF OPERATIONS EVALUATED, BY LENDING TYPE AND YEAR OF EVALUATION, 1974-94 COHORTS Investment Adjustment Total Evaluation year No. % sat. No. % sat. No. % sat. 1974 50 86 0 - 50 86 1975 53 87 4 100 57 88 1976 69 84 1 100 70 84 1977 102 88 7 100 109 89 1978 89 90 8 100 97 91 1979 129 88 1 100 130 88 1980 85 87 2 100 87 87 1981 106 85 2 100 108 85 1982 119 76 8 100 127 77 1983 178 83 0 - 178 83 1984 169 73 4 75 173 73 1985 184 70 8 50 192 69 1986 243 81 5 80 248 81 1987 180 71 7 86 187 71 1988 164 73 5 60 169 73 1989 250 70 13 50 263 69 1990 329 60 30 63 359 61 1991 261 61 18 56 279 61 1992 243 63 39 62 282 63 1993 282 64 36 65 318 64 1994 216 65 30 70 246 66 Total/average 3,501 73 228 69 3,729 72 -Not applicable. Note: Percentage calculations exclude projects not rated. 100 ANNEX TABLE 1.10: SUSTAINABILITY OF OPERATIONS EVALUATED, BY SECTOR, LENDING TYPE, AND REGION, 1994, 1993, 1989-94 COHORTSa 1994 1993 1989-94 Number of Likely Uncertain Unlikely Number of Likely Uncertain Unlikcly Number of Likely Uncertain Inlikely operations b ( (%) (%) operations b operations b Sector Agriculture 77 35 44 21 81 42 32 26 534 36 37 27 Energy 6 50 17 33 17 35 41 24 101 52 35 13 Finance 22 41 50 9 30 33 40 27 161 35 41 24 Human resource 23 74 13 13 29 52 34 14 158 61 25 14 Industry 12 50 42 8 18 50 28 22 83 46 35 19 Pollution control 0 - - - 0 - - - 1 100 0 0 and solid waste Power 22 55 27 18 26 62 19 19 130 65 25 11 Program and policy 18 39 28 33 14 50 29 21 96 43 35 22 Technical assistance 4 25 50 25 14 21 36 43 54 30 46 24 Telecommunications 3 67 33 0 4 75 25 0 32 84 13 3 Tourism 0 - - - 0 - - - 8 75 0 25 Transport 24 50 17 33 44 48 30 23 183 47 35 18 Urban 18 33 50 17 12 17 42 42 83 47 29 24 Water and sanitation 9 33 44 22 13 46 31 23 79 41 42 18 Lending type Adjustment 30 43 33 23 34 47 32 21 158 43 36 21 Investment 208 44 36 20 268 43 32 25 1,545 45 34 21 Region Africa 74 23 51 26 114 26 36 38 588 33 36 31 East Asia and Pacific 40 73 18 10 54 69 22 9 309 66 23 11 Europe and 13 46 38 15 12 67 17 17 100 50 42 8 Central Asia Latin America 58 50 31 19 55 49 29 22 336 43 35 22 and Caribbean Middle East and 27 52 22 26 27 52 37 11 153 55 31 14 North Africa South Asia 26 38 42 19 40 40 40 20 217 42 42 16 Total/average 238 44 36 20 302 44 32 24 1,703 45 34 21 -Not applicable. a. Data for this variable were not available prior to 1989. b. Some operations for which sustainability was not considered an issue were excluded. ANNEX TABLE 1.11: SUSTAINABILITY OF OPERATIONS EVALUATED, BY YEAR OF APPROVAL, 1978-87 Number of Likely Uncertain Unlikely Approval year operations (%) (%) (%) 1978 98 42 36 22 1979 144 42 37 21 1980 182 45 35 20 1981 184 42 38 21 1982 230 43 33 24 1983 214 44 37 19 1984 160 42 40 18 1985 186 49 33 17 1986 114 46 29 25 1987 72 51 35 14 Total/average 1,584 45 35 20 102 ANNEX TABLE 1.12: ACHIEVEMENT OF INSTITUTIONAL DEVELOPMENT OBJECTIVES, BY SECTOR AND REGION, 1994, 1993, AND 19894 COHORTS a 1994 1993 1989-94 Substantial Modest Negligible Substantial Modest Negligible Subsiantial Modest Negligible No.b o Nob M No.b bNo. Sector Agriculture 73 37 42 21 79 25 57 18 500 27 51 22 Energy 6 17 33 50 17 53 29 18 101 46 38 17 Finance 21 29 62 10 30 23 60 17 159 28 51 21 Human resource 22 59 41 0 29 24 55 21 156 34 46 20 Industry 12 58 25 17 18 28 56 17 84 33 49 18 Pollution control 0 - - - 0 - - 1 100 0 0 and solid waste Power 22 45 9 45 25 52 20 28 123 40 35 25 Program and policy 17 41 41 18 13 38 46 15 90 28 48 24 Technical assistance 4 25 25 50 14 21 29 50 54 22 37 41 Telecommunications 3 33 67 0 4 25 50 25 30 33 47 20 Tourism 0 - - - 0 - - - 8 38 38 25 Transport 24 29 38 33 42 21 33 45 178 25 43 33 Urban 18 33 44 22 12 17 33 50 82 28 41 30 Water and sanitation 9 33 56 11 13 54 23 23 77 45 34 21 Lending type Adjustment 29 48 34 17 32 38 44 19 150 30 48 22 Investment 202 37 41 22 264 29 45 27 1,493 31 45 24 Region Africa 72 28 50 22 111 20 47 33 576 23 48 28 East Asia and Pacific 39 54 36 10 54 39 56 6 300 45 42 13 Europe and Central Asia 13 23 38 38 12 67 33 0 93 34 48 17 Latin America 56 48 36 16 55 33 33 35 326 31 41 28 and Caribbean Middle East and 27 41 30 30 26 35 46 19 150 37 43 19 North Africa South Asia 24 29 38 33 38 26 42 32 198 24 52 24 Total/average 231 39 40 22 296 30 45 26 1,643 31 45 24 - Not applicable. a. Data for this variable are not available for projects evaluated prior to 1989. b. Some operations for which institutional development was not considered an issue were excluded. ANNEX TABLE 1.13: INFLUENCE OF OBJECTIVES ON ACHIEVING INSTITUTIONAL DEVELOPMENT Substantial Moderate Negligible Number % Number % Number % Total Borrower commitment Positive 23 64 10 28 3 8 36 Negative 0 0 18 60 12 40 30 Quality of preparation Positive 12 71 4 24 1 6 17 Negative 0 0 18 56 14 44 32 Design Positive 12 80 2 13 1 7 15 Negative 2 6 18 55 13 39 33 Supervision Positive 19 66 9 31 1 3 29 Negative 2 8 14 56 9 36 25 Establishment of new organization Positive 7 70 3 30 0 0 10 Negative 0 0 1 17 5 83 6 Elimination of organization Positive 0 0 2 100 0 0 2 Negative 0 0 0 0 2 100 2 Restructuring an organization Positive 9 47 10 53 0 0 19 Negative 1 13 2 25 5 63 8 Regulatory change Positive 2 33 4 67 0 0 6 Negative 1 17 0 0 5 83 6 Number of financing arrangements Positive 0 0 0 0 0 0 0 Negative 0 0 2 33 4 67 6 Monitoring and evaluation Positive 6 100 0 0 0 0 6 Negative 3 13 15 63 6 25 24 Exogenous factors Positive 0 0 0 0 0 0 0 Negative 4 27 6 40 5 33 15 104 ANNEX TABLE 1.14: RELEVANCE, EFFICACY, AND EFFICIENCY AND OVERALL OUTCOME, 1994 EVALUATION SET Relevance Overall outcome High Substantial Moderate Negligible Not rated Total Highly satisfactory 22 5 1 28 Satisfactory 57 110 28 3 4 202 Unsatisfactory 12 38 37 14 3 104 Highly unsatisfactory 2 3 5 1 11 Not rated 9 9 Total 93 153 69 22 17 354 Percent satisfactory 85 75 42 14 50 67 Efficacy Overall outcome High Substantial Moderate Negligible Not rated Total Highly satisfactory 22 6 28 Satisfactory 19 104 68 5 6 202 Unsatisfactory 1 5 52 43 3 104 Highly unsatisfactory 1 2 7 1 11 Not rated 9 9 Total 42 116 122 55 19 354 Percent satisfactory 98 95 56 9 60 67 Efficiency Overall outcome High Substantial Moderate Negligible Not rated Total Highly satisfactory 19 6 3 28 Satisfactory 11 99 75 9 8 202 Unsatisfactory 2 1 7 1 11 Highly unsatisfactory 1 7 37 55 4 104 Not rated 9 9 Total 31 114 113 71 25 354 Perceni satisfactory 97 92 66 13 69 67 ANNEX TABLE 1.15: ACHIEVEMENT OF MAJOR OBJECTIVES, 1994 COHORT Efficacy of objective types Percent high or substantial High Substantial Moderate Negligible Total Out of 2994 cohort Out of objective type Macro 4 7 6 0 17 4 65 Sector 12 23 11 12 58 14 60 Physical 35 50 20 5 110 35 77 Financial 14 19 6 5 44 13 75 105 ANNEX TABLE 1.16: AVERAGE RATES OF RETURN, BY SECTOR AND REGION, 1994, 1993, 1990-94, AND 1974-94 COHORTS 1994 1993 Economic rates of return Economic rates of return No. of Estimated Actual Gap No. of Estimated Actual Gap operations (%) (%) (% pts.) operations (%) (%) (% pts.) Sector Agriculture 33 22 17 5 41 25 15 10 Energy 3 47 83 -36 4 55 34 22 Finance 1 26 20 6 0 - - - Human resource 0 - - - 0 - - - Industry 6 43 14 29 3 20 18 2 Pollution control and 0 - - - 0 - - - solid waste Power 10 15 17 -2 15 14 15 -1 Program and policy 0 - - - 0 - - - Technical assistance 0 - - - 0 - - Telecommunications 3 28 16 12 2 37 31 6 Tourism 0 - - - 0 - - - Transport 16 31 28 3 26 37 30 7 Urban 8 30 25 4 7 23 22 1 Water and sanitation 2 8 5 3 2 14 15 -1 Region Africa 19 24 18 6 31 30 19 11 East Asia and Pacific 19 25 25 -1 21 25 20 5 Europe and Central Asia 6 34 11 24 3 20 25 -5 Latin America and Caribbean 13 25 17 7 15 30 27 4 Middle East and North Africa 12 23 21 2 9 29 20 10 South Asia 13 32 33 -1 21 24 18 5 Total 82 26 22 4 100 27 20 7 106 (Annex Table 1.16 continued) 1990-94 1974-94 Economic rates of return Economic rates of return No. of Estimated Actual Gap No. of Estimated Actual Gap operations (%) (%) (% pts.) operations (%) (%) (% pts.) Sector 12 11 Agriculture 219 23 12 11 697 23 39 12 Energy 23 51 39 12 43 46 18 2 Finance 2 19 18 2 2 19 - - Human resource 0 - - - 0 - 12 17 Industry 32 29 12 17 100 24 - - Pollution control and solid 0 - - - 1 30 13 2 waste Power 70 15 13 2 221 15 - - Program and policy 0 - - - 0 - - - Technical assistance 0 - - - 0 - 21 7 Telecommunications 14 28 21 7 72 20 9 7 Tourism 4 16 9 7 17 18 28 5 Transport 103 32 28 5 458 25 24 1 Urban 34 25 24 1 75 25 9 2 Water and sanitation 22 11 9 2 83 12 Region 14 11 Africa 170 25 14 11 542 23 20 4 East Asia and Pacific 107 24 20 4 341 23 18 7 Europe and Central Asia 31 26 18 7 143 22 17 9 Latin America and Caribbean 77 26 17 9 340 21 24 1 Middle East and North Africa 54 25 24 1 183 21 18 9 South Asia 84 26 18 9 220 25 Total 523 25 17 8 1,769 23 12 11 -Not applicable. 107 ANNEX TABLE 1.17: COST AND IMPLEMENTATION TIME VARIATIONS, BY SECTOR AND REGION, 1994, 1990-94, AND 1974-94 COHORTS Time variationsa Cost variationsb 1994 1990-94 1974-94 1994 1990-94 1974-94 Average Average Average Average Average Average variation variation variation variation variation variation No. (%) No. (%) No. (%) No. (%) No. (%) No. (%) Sector Agriculture 77 40 452 47 1,124 44 66 3 399 -10 1,041 3 Energy 6 47 79 86 127 78 5 -42 68 -6 112 -8 Finance 22 23 134 37 342 30 11 0 33 1 122 12 Human resource 23 28 139 42 356 65 19 -13 124 -9 337 6 Industry 12 55 72 48 161 51 7 -11 52 -4 137 13 Pollution control and solid waste 0 - 0 - 4 30 0 - 0 - 4 -9 Power 22 39 113 55 309 54 20 -3 100 -2 291 15 Program and policy 18 38 92 38 165 26 2 -16 20 3 45 1 Technical assistance 4 21 51 61 89 58 2 26 28 -9 55 -2 Telecommunications 3 44 23 45 90 68 2 25 20 12 84 23 Tourism 0 - 6 88 25 79 0 - 6 45 24 26 Transport 24 43 149 55 573 67 16 5 131 -5 535 11 Urban 18 26 70 39 126 56 11 -24 59 -17 112 -6 Water and sanitation 9 24 70 47 188 64 6 -23 64 -10 181 10 Region Africa 74 32 530 50 1,169 50 49 16 387 1 965 7 East Asia and Pacific 40 29 243 35 628 44 33 -13 200 -12 552 4 Europe and Central Asia 13 29 81 38 280 51 9 -10 62 -8 232 12 Latin America and Caribbean 58 42 292 57 763 59 40 -11 216 -14 641 9 Middle East and North Africa 27 39 129 53 374 58 15 -15 98 -13 303 11 South Asia 26 49 175 54 465 56 21 -12 141 -8 387 2 Average 238 36 1,450 49 3,679 53 167 -4 1,104 -7 3,080 7 -Not applicable. a. Excludes operations not rated for outcome. The loan closing date was used for operations for which physical completion dates were not available. b. Excludes operations for which cost data were not available. ANNEX TABLE 1.18: OUTSTANDING OPERATIONS IN THE 1994 EVALUATION COHORT Commit- Net con- OED Report AR Adjustment/ ment mitment Approval Closing Sustain- ID ID type cohort Country Project description AR sector investment ($mill.) ($mill.) year year Outcome ability impact L1927 PAR 87 Chile Highway Reconstruction Transport INV 42 42 80 84 Highly satisfactory Likely SUB Project L2231 PAR 91 China Daqing Oilfield-Gaotaizi Energy INV 162 162 83 90 Highly satisfactory Likely SUB Reservoir Development L2757 PAR 91 Indonesia Telecommunications TA Telecommu- INV 15 14 86 90 Highly satisfactory Likely SUB nications L2252 PAR 92 China Zhongyuan-Wenliu Energy INV 101 97 83 90 Highly satisfactory Likely NEG Petroleum Project L2297 PAR 92 Chile Second Highway Transport INV 128 128 83 90 Highly satisfactory Likely SUB Reconstruction Project L2379 PAR 92 Colombia Popayan Region Earthquake Urban INV 40 40 84 88 Satisfactory Likely SUB Reconstruction L2426 PAR 92 China Karamay Petroleum Project Energy INV 100 90 84 91 Highly satisfactory Likely SUB L2401 PAR 93 Colombia Power Development Power INV 198 198 84 87 Satisfactory Likely SUB Finance Project L2589 PAR 93 Chile Road Sector Project Transport INV 290 290 85 90 Highly satisfactory Likely SUB L3087 PAR 93 Mexico Industrial Sector Industry ADJ 500 498 89 91 Highly satisfactory Likely SUB Policy Loan C2091 PCR 94 China North China Earthquake Urban INV 30 30 90 92 Highly satisfactory Likely MOD Reconstruction C2212 PAR 94 Honduras First Social Investment Fund Human INV 20 20 91 94 Highly satisfactory Likely SUB Project resources C2461 PAR 94 Albania Rural Poverty Alleviation Agriculture INV 2 2 93 95 Satisfactory Likely SUB Pilot Project L2334 PCR 94 Jordan Amman Transport and Urban INV 30 24 83 93 Highly satisfactory Likely NEC Municipal Development L2800 PCR 94 Indonesia BRI/KUPEDES Small Finance INV 102 101 87 92 Highly satisfactory Likely SUB Credit Project L2832 PCR 94 Chile Pehuenche Hydroelectric Power INV 95 40 87 94 Highly satisfactory Likely SUB Project L2947 PCR 94 Mexico Housing Finance Project Urban INV 300 300 88 94 Satisfactory Likely SUB L3306 PCR 94 Jordan Emergency Recovery Human INV 10 10 91 93 Highly satisfactory Likely SUB Project resources Notes: PAR=performance audit report; PCR=project completion report; INV=investment; ADJ=adjustment; SUB=substantial; NFG=negligible; and MOD=moderate. Annex 2 Table 2.1 Performance of appraisal in World Bank projects, 1979-94 Table 2.2 Performance of supervision in World Bank projects, 1979-94 110 ANNEX TABLE 2.1: PERFORMANCE OF APPRAISAL IN WORLD BANK PROJECTS, 1979-94 Projects with poor appraisal and poor performance a Projects with good appraisal and good performance b Performance of appraisal by major areas Satisfactory Unsatisfactory Unsat. as % of lota Satisfactory Unsatisfactory Sat. as % of total Technical 17 41 70.7 113 2 98.3 Financial 7 49 87.5 90 3 96.8 Economic 11 44 80.0 98 0 100.0 Commercial 3 22 88.0 33 7 82.5 Institutional 10 63 86.3 120 5 96.0 Environmental 7 8 53.3 21 5 80.8 Sociological 5 11 68.8 34 0 100.0 Projects with poor appraisal and poor performance Projects with good appraisal and good performance Factors that influence quality of appraisal Positive Negative Neg. as % of total Positive Negative Pos. as % of total Commitment 11 69 86.3 123 4 96.9 Borrower capacity 16 65 80.2 122 8 93.8 Design 16 60 78.9 124 5 96.1 Risks identification 11 70 86.4 103 11 90.4 Adequacy of implementation plan 17 60 77.9 95 16 85.6 Instrument suitability 50 23 31.5 114 0 100.0 Past experience 8 56 87.5 83 7 92.2 Economic and sector work 10 17 63.0 29 1 96.7 Staff quality 6 37 86.0 82 6 93.2 Staff quantity 7 11 61.1 62 3 95.4 Performance of consultants 4 10 71.4 37 3 92.5 Donor coordination 4 9 69.2 24 0 100.0 a. Projects with unsatisfactory appraisal, unsatisfactory outcome, unlikely or uncertain sustainability, and negligible or modest institutional development impact. b. Projects with a satisfactory appraisal rating and satisfactory performance in all three areas. ANNEX TABLE 2.2: PERFORMANCE OF SUPERVISION IN WORLD BANK PROJECTS, 1979-94 Projects with poor supervision and poor performance a Projects with good supervision and good performance b Performance of appraisal by major areas Satisfactory Unsatisfactory Unsat. as % of total Satisfactory Unsatisfactory Sat. as % of total Reporting 29 41 58.6 109 5 95.6 Identification of problems 15 61 80.3 116 7 94.3 Attention to development impact 5 58 92.1 92 6 93.9 Advice 15 46 75.4 116 1 99.1 Follow-up 4 53 93.0 96 8 92.3 Enforcement 18 50 73.5 84 14 85.7 Modifications 24 18 42.9 98 4 96.1 Projects with poor supervision and poor performance Projects with good supervision and good performance Factors that influence quality of supervision Positive Negative Neg. as % of total Positive Negative Pos. as % of total Plans 3 25 89.3 25 4 86.2 Timing 9 28 75.7 61 9 87.1 Sufficiency of time in field 5 28 84.8 58 9 86.6 Staff quantity 11 17 60.7 56 6 90.3 Staff quality 4 44 91.7 77 8 90.6 Performance of consultants 4 8 66.7 37 4 90.2 Implementation reviews 1 16 94.1 8 2 80.0 a. Projects with unsatisfactory supervision, unsatisfactory outcome, unlikely or uncertain sustainability, and negligible or modest institutional development impact. b. Projects with a satisfactory supervision rating and satisfactory performance in all three areas. Annex 3 Table 3.1 Project processing and outcome, 1991-94 evaluation sets 113 ANNEx TABLE 3.1: PROJECT PROCESSING AND OUTCOME, 1991-94 EVALUATION SETS'a Identification Preparation Appraisal Supervision Adequateldeficient No. % sat. Adequate/deficient No. % sat. Adequate/deficient No. % sat. Adequate/deficient No. % sat. Adequate 623 84 Adequate 705 80 Deficient 82 56 Adequate 777 77 Adequate 45 64 Deficient 72 46 Deficient 27 15 Adequate 988 69 Adequate 16 45 Adequate 29 71 Deficient 13 17 Deficient 211 39 Adequate 107 45 Deficient 182 34 Total number of Deficient 75 17 projects: 1,125 (64% satisfactory) Adequate 8 43 Adequate 11 40 Deficient 3 33 Adequate 21 25 Adequate 2 50 Deficient 10 10 Deficient 8 0 Deficient 137 25 Adequate 2 50 Adequate 4 25 Deficient 2 0 Deficient 116 25 Adequate 52 37 Deficient 112 0 Deficient 60 15 a. Includes only projects rated for all four stages. Percentage satisfactory calculations exclude projects not rated for outcome. Annex 4 Table 4.1 1994 cohort (246 projects newly evaluated in 1994), by region, country, and sector 115 ANNEX TABLE 4.1: 1994 COHORT (246 PROJECTS NEWLY EVALUATED IN 1994), BY REGION, COUNTRY, AND SECTOR Connit- Cancel- Institutional Loan/ ?nent lation Key project dates Report development Region/country Sector Project title credit no. ($mnill.) ($nmill.) Approval Closing no. Type Overall rating Sustainability impact Africa Benin Agriculture Second Borgou Rural C1877 21 0 Feb-88 Sep-93 13444 PCR Highly satisfactory Likely Substantial Development Project Botswana Agriculture National Land L2566 11 2 Jun-85 Mar-93 12981 PCR Unsatisfactory Uncertain Moderate Management and Livestock Project Burkina Faso Agriculture Fertilizer Project C1550 14 6 Feb-85 Jun-93 13817 PCR Satisfactory Likely Moderate Burundi Agriculture Second Forestry Project C1620 13 2 Jul-85 Mar-93 13179 PCR Satisfactory Unlikely Substantial Cameroon Agriculture National Agricultural L2766 18 0 Nov-86 Jun-93 13631 PCR Unsatisfactory Uncertain Moderate Research Project Cape Verde Finance Industrial Finance and C1579 4 0 Apr-85 Dec-92 12705 PCR Satisfactory Likely Substantial Promotion Project Transport Infrastructure C1954 5 0 Sep-88 Dec-93 13749 PCR Satisfactory Likely Moderate Rehabilitation and Technical Assistance Central Africa Agriculture National Livestock C1681 12 0 Apr-86 Jun-92 13219 PCR Satisfactory Uncertain Moderate Republic Project C6te d'lvoire Agriculture Agricultural Extension L2765 34 2 Nov-86 Jun-92 13118 PCR Unsatisfactory Uncertain Moderate Project Agriculture Agricultural Sector L3127 150 0 Oct-89 Jun-91 13120 PCR Satisfactory Uncertain Moderate Adjustment Loan Finance Industrial Finance and L2617 30 11 Sep-85 Jun-93 13268 PCR Unsatisfactory Unlikely Moderate Development Project Telecommunications Second Telecommunica- L2696 25 0 May-86 Jun-93 13202 PCR Satisfactory Uncertain Moderate tions Project Equatorial Technical assistance Technical Assistance C1489 6 0 May-84 Mar-93 12917 PCR Highly Unlikely Negligible Guinea Project unsatisfactory Ethiopia Technical assistance Technical Assistance C1522 4 0 Sep-84 Jun-93 13704 PCR Unsatisfactory Likely Negligible Project Gambia Agriculture Second Agricultural C1476 9 2 May-84 Jun-93 13509 PCR Unsatisfactory Uncertain Moderate Development Project Program and policy Second Structural C2032 23 0 Jun-89 Jun-92 12936 PCR Satisfactory Likely Moderate Adjustment Credit Transport Second Highway C1682 6 0 Apr-86 Mar-92 13490 PCR Unsatisfactory Unlikely Negligible Maintenance Project Urban Urban Management and C1443 12 0 Mar-84 Feb-93 13734 PCR Satisfactory Uncertain Moderate Development Project Ghana Agriculture Oil Palm Development C1498 2S 2 Jun-84 Dec-93 13818 PCR Satisfactory Likely Substantial Project-Second Phase Transport Port Rehabilitation C1674 25 0 Mar-86 Apr-93 13149 PCR Satisfactory Likely High Project Guinea Agriculture First Agricultural C1636 8 2 Nov-85 Jun-92 13148 PCR Unsatisfactory Unlikely Negligible Services Project Finance Private Sector Promotion C2148 50 0 May-90 Dec-92 13302 PCR Satisfactory Uncertain Moderate Credit Technical assistance Second Economic Man- C1963 15 0 Nov-88 Jun-94 13261 PAR Unsatisfactory Uncertain Moderate agement Support Project Kenya Agriculture Agricultural Sector C1718 12 0 Jun-86 Dec-92 13178 PCR Satisfactory Uncertain Substantial Management Pmject Agriculture Animal Health Services C1758 15 5 Feb-87 Dec-93 13836 PCR Unsatisfactory Unlikely Moderate Project Lesotho Human resources Health and Population C1585 4 1 Apr-85 Dec-92 13055 PCR Satisfactory Likely Moderate Project Madagascar Industry Ilmenite Mining C1928 9 7 Jun-88 Jun-93 12790 PCR Unsatisfactory Uncertain Substantial Engineering Project Urban Urban Development C1497 13 2 Jun-84 Jun-93 12932 PCR Satisfactory Uncertain Moderate Project Malawi Agriculture National Agricultural Cl549 24 0 Feb-85 Oct-93 13634 PCR Satisfactory Uncertain Moderate Research Project Agriculture Agricultural Extension C1626 12 0 Sep-85 Jun-93 13633 PCR Satisfactory Uncertain Moderate and Planning Support Project Agriculture Agricultural Sector C2121 70 0 Apr-90 Dec-92 13603 PCR Unsatisfactory Likely Moderate Adjustment Credit Human resources Second Family Health C1768 11 0 Mar-87 Jun-93 13508 PCR Satisfactory Likely Moderate Project Urban First Urban Project C1528 15 4 Nov-84 Jun-93 13626 PCR Satisfactory Uncertain Moderate Water and sanitation Second Lilongwe Water C1742 20 0 Nov-86 Jun-93 13744 PCR Satisfactory Uncertain Moderate Supply Project Mali Agriculture Mopti Area C1597 20 6 May-85 Dec-92 13445 PCR Unsatisfactory Uncertain Negligible Development Project Agriculture Second Forestry Project C1654 6 0 Jan-86 Dec-92 13160 PCR Unsatisfactory Likely Moderate Mauritania Agriculture Small-scale Irrigation C1571 8 2 Mar-85 Jun-93 13786 PCR Unsatisfactory Uncertain Negligible Project (continued on next page) (Annex Table 4.1 continued) Commit- Cancel- Institutional LoanI ment lation Key project dates Report development Region/country Sector Project title credit no. ($rnill.) ($nillI.) Approval Closing no. Type Overall rating Sustainability impact Finance Industrial and Artisan C1572 5 3 Mar-85 Dec-91 12770 PCR Highly Unlikely Not rated Development Project unsatisfactory Program and policy Public Enterprise C1567 16 1 Mar-85 Dec-92 13102 PCR Satisfactory Uncertain Moderate Technical Assistance and Rehabilitation Mauritius Agriculture Sugar Industry Project L2728 30 2 Jun-86 Jun-93 13161 PCR Satisfactory Likely Substantial Finance Industrial Finance L2927 10 1 Mar-88 Dec-95 12713 PCR Unsatisfactory Uncertain Moderate Project Niger Agriculture Irrigation Rehabilitation C1618 9 0 Jun-85 Dec-91 12980 PCR Satisfactory Uncertain Negligible Project Transport Transportation Sector C1706 30 0 May-86 Jun-93 13773 PCR Unsatisfactory Unlikely Negligible Credit Nigeria Human resources Sokoto Health Project L2503 34 8 Mar-85 May-93 13730 PCR Satisfactory Uncertain Moderate Industry National Petroleum L3104 28 19 Jun-89 Jun-93 13807 PCR Unsatisfactory Unlikely Negligible Corporation Refineries Rehabilitation Urban Second Urban L2607 53 8 Jul-85 Jun-93 13279 PCR Satisfactory Uncertain Moderate Development Project Water and sanitation Lagos Solid Waste and L2620 72 3 Sep-85 Sep-93 13636 PCR Satisfactory Unlikely Moderate Storm Drainage Project Rwanda Finance Fourth Rwandese C1650 9 0 Jan-86 Dec-93 12830 PCR Unsatisfactory Uncertain Negligible Development Bank (BRD) Project Human resources Family Health Project C1678 11 0 Apr-86 Dec-93 13136 PCR Satisfactory Unlikely Moderate Human resources Third Education Project C1683 16 0 Apr-86 Dec-93 13641 PCR Unsatisfactory Unlikely Moderate Urban Urban Institutions C2041 32 31 Jun-89 Dec-96 13635 PCR Highly Unlikely Negligible Sectoral Development unsatisfactory Project Sao Tome & Program and policy First Structural C1825 7 0 Jun-87 Mar-92 13303 PCR Satisfactory Uncertain Negligible Principe Adjustment Credit Senegal Finance Financial Sector C2077 45 0 Dec-89 Jun-92 13724 PCR Satisfactory Uncertain Moderate Adjustment Program Program and policy Fourth Structural C2090 96 24 Feb-90 Jun-93 13723 PCR Unsatisfactory Unlikely Moderate Adjustment Credit Telecommunications Second Telecommunica- C1714 22 5 Jun-86 Jun-93 13623 PCR Satisfactory Likely Substantial tions Project Watei and sanitation Eleven Centers Water C1554 24 0 Mar-85 lun-93 13833 PCR Satisfactory Uncertain Moderate Supply Project Sierra Leone Power Power Engineering and C1265 5 0 Jun-82 Sep-93 13681 PCR Unsatisfactory Uncertain Negligible Technical Assistance Project Somalia Agriculture Livestock Health C1647 4 0 Jan-86 Jun-93 13159 PCR Highly Unlikely Negligible Services Project unsatisfactory Sudan Agriculture Sugar Rehabilitation C1506 60 7 Jun-84 Dec-93 13649 PCR Unsatisfactory Unlikely Moderate Project Agriculture Emergency Flood C2011 75 7 May-89 Oct-93 13427 PCR Satisfactory Uncertain Substantial Reconstruction Project Power Power Rehabilitation C1624 30 0 Aug-85 Jun-93 13627 PCR Unsatisfactory Uncertain Negligible Project Power Fourth Power Project C1788 38 0 May-87 Jun-93 13767 PCR Satisfactory Uncertain Negligible Tanzania Agriculture Agricultural Sector C2116 227 0 Mar-90 Jun-94 13741 PCR Highly satisfactory Likely Substantial Adjustment Credit Transport Port Rehabilitation C1536 27 0 Dec-84 Dec-92 13763 PCR Satisfactory Unlikely Negligible Project Togo Human resources Educational C1568 12 0 Mar-85 Jun-92 13640 PCR Satisfactory Uncertain Moderate Improvement Project Uganda Agriculture Agricultural C1539 10 1 Jan-85 Sep-93 13812 PCR Satisfactory Uncertain High Development Project Program and policy Second Economic C2087 129 0 Feb-90 Jun-93 13687 PCR Satisfactory Uncertain Negligible Recovery Credit Technical assistance Third Technical C1951 18 0 Aug-88 Dec-95 13590 PCR Satisfactory Uncertain Substantial Assistance Project Transport Railways Project C1986 7 1 Feb-89 Jun-93 13748 PCR Unsatisfactory Unlikely Moderate Zaire Agriculture National Agricultural C2292 17 16 Aug-91 Jun-97 13602 PCN Unsatisfactory Uncertain Moderate Research Project Human resources Education Sector C2213 21 20 Mar-91 Jun-96 13788 PCN Not rated Not rated Not rated Rehabilitation Project Urban Higher Education C1839 11 2 Jul-87 Jun-93 13138 PCR Unsatisfactory Uncertain Moderate Rationalization Project Zambia Agriculture Industrial Forestry C1437 22 4 Jan-84 Sep-93 13700 PCR Unsatisfactory Unlikely Negligible Project-Third Phase Zimbabwe Human resources Family Health Project L2744 10 0 Jul-86 Sep-93 13784 PCR Satisfactory Likely Substantial (continued on next page) (Annex Table 4.1 continued) Commit- Cancel- Institutional Loan/ ment lation Key project dates Report development Region/country Sector Project title credit no. ($mill.) ($nill-) Approval Closing no. Type Overall rating Sustainability impact Urban Urban Development L2445 43 7 Jun-84 Sep-93 13832 PCR Satisfactory Likely Substantial Project East Asia and Pacific China Agriculture Daxinganling Forest C1918 57 1 Jun-88 Mar-94 13564 PCR Satisfactory Likely Substantial Fire Rehabilitation Project Energy Changcun (Luan) L2501 126 51 Mar-85 Dec-93 13417 PCR Satisfactory Likely Moderate Coal Mining Project Human resources Provincial Universities C1671 120 0 Mar-86 Dec-92 13146 PCR Satisfactory Likely Substantial Project Industry Shanghai Machine L2784 100 4 Mar-87 Dec-93 13595 PCR Satisfactory Likely Substantial Tool Project Transport Dalian Port Project C1875 96 1 Feb-88 Dec-93 13775 PCR Satisfactory Likely Moderate Urban North China C2091 30 0 Feb-90 Dec-92 13510 PCR Highly satisfactory Likely Moderate Earthquake Reconstruction Project Indonesia Agriculture Second Swamp L2431 65 4 May-84 Dec-93 13837 PCR Unsatisfactory Uncertain Moderate Reclamation Project Agriculture West Tarum Canal L2560 43 3 May-85 Apr-93 12818 PCR Highly satisfactory Likely Moderate Improvement Project Agriculture Fifth Transmigration L2578 160 66 Jun-85 Dec-92 13507 PCR Satisfactory Uncertain Moderate Project Agriculture Smallholder Cattle L2628 32 6 Oct-85 Mar-93 13035 PCR Satisfactory Likely Substantial Development Project Agriculture Central &West Java L2649 166 2 Jan-86 Jun-93 13630 PCR Satisfactory Likely Substantial Provincial Irrigation Development Agriculture Third National Agricul- L2748 70 17 Jul-86 Jun-93 13628 PCR Unsatisfactory Unlikely Moderate tural Extension Project Finance BRI/KUPEDES Small L2800 102 0 Apr-87 Dec-92 12973 PCR Highly satisfactory Likely Substantial Credit Project Human resources Second Health L2542 39 2 May-85 Mar-93 13745 PCR Unsatisfactory Likely Negligible (Manpower Development) Project Human resources Second University L2547 147 1 May-85 Dec-93 13639 PCR Satisfactory Likely Substantial Development Project Industry Industrial Energy L2879 21 2 Oct-87 Jun-93 13060 PCR Unsatisfactory Uncertain Moderate Conservation Project Urban Housing Sector Loan L2725 275 77 Jun-86 Dec-92 13296 PCR Satisfactory Unlikely Moderate Water and sanitation Second East Java Water L2632 43 2 Nov-85 Dec-93 13637 PCR Satisfactory Uncertain Moderate Supply Project Korea, Human resources Technology L3037 16 0 Apr-89 Dec-92 13130 PCR Highly satisfactory Likely Negligible Republic Advancement Project Industrv Third Technology L2913 50 1 Feb-88 Dec-92 13269 PCR Satisfactory Likely Substantial Development Project Laos Program and policy Structural Adjustment C2037 40 0 Jun-89 Jan-92 12664 PCR Highly satisfactory Likely Moderate Credit Malaysia Agriculture National Forestry L2828 9 2 Jun-87 Jun-92 13697 PCR Satisfactory Likely Substantial Research and Development Project Human resources First Primary and L2685 127 18 Apr-86 Dec-92 13145 PCR Satisfactory Likely Substantial Secondary Education Sector Project Human resources Second Primary and L2987 59 0 Aug-88 Dec-92 13145 PCR Satisfactory Likely Substantial Secondary Education Sector Project Power Sarawak Power Project L2942 56 1 May-88 Jun-94 13774 PCR Highly satisfactory Likely Substantial Myanmar Agriculture First Groundwater C1381 14 0 Jun-83 Jun-92 12891 PCR Satisfactory Likely Moderate Irrigation Project Agriculture Second Rubber C1385 9 0 Jun-83 Mar-93 12873 PCR Satisfactory Likely Substantial Rehabilitation Project Agriculture Second Wood C1444 25 0 Mar-84 Mar-93 13829 PAR Highly Unlikely Moderate Industries Project unsatisfactory Papua Agriculture West Sepik Provincial L2475 10 0 Oct-84 Dec-92 13088 PCR Unsatisfactory Uncertain Negligible New Guinea Development Project Power Yonki Hydroelectric L2722 29 2 Jun-86 Dec-92 13599 PCR Satisfactory Likely Negligible Project Philippines Agriculture Irrigation Operations L2948 45 22 Jun-88 Jun-93 13826 PCR Satisfactory Likely Substantial Support Project Solomon Human resources Secondary Education C1686 5 0 Apr-86 Dec-92 13129 PCR Highly satisfactory Likely Substantial Islands Project Thailand Agriculture National Agricultural L1922 30 0 Nov-80 Jun-92 13119 PCR Satisfactory Likely Substantial Research Project (continued on next page) (Annex Table 4.1 continued) Commit- Cancel- Institutional Loan/ ment lation Key project dates Report development Region/country Sector Project title credit no. ($mill.) ($mil.) Approval Closing no. Type Overall rating Sustainability impact Industry Bangchak Oil Refinery L2548 85 0 May-85 Jun-92 13041 PCR Highly satisfactory Likely Substantial Restructuring Project Power Power System Develop- L3027 90 0 Mar-89 Mar-92 12935 PCR Satisfactory Likely Substantial ment Project Transport Railway Efficiency L2872 13 2 Sep-87 Dec-93 13764 PCR Satisfactory Uncertain Moderate Improvement Project Transport Highway Sector Project L2894 50 0 Dec-87 Dec-92 13768 PCR Satisfactory Likely Substantial Transport Second Highway Sector L3008 87 0 Dec-88 Dec-93 13768 PCR Satisfactory Likely Moderate Project Urban Third Shelter Project L2795 21 14 Apr-87 Dec-92 13733 PCR Unsatisfactory Uncertain Negligible Vanuatu Agriculture Multiproject C666 2 0 Mar-86 Jun-93 13823 PCR Unsatisfactory Unlikely Negligible Europe and Central Asia Albania Agriculture Rural Poverty C2461 2 0 Feb-93 Jun-95 13819 PAR Satisfactory Likely Substantial Alleviation Pilot Project Cyprus Agriculture Southern Conveyor L2386 27 3 Mar-84 Dec-91 13066 PCR Satisfactory Likely Negligible Projects for Water Resources Development Hungary Energy Second Industrial L2709 25 5 May-86 Jun-93 13834 PCR Satisfactory Likely Substantial Energy Conservation Project industry Fine Chemicals Project L2511 73 0 Apr-85 Dec-90 13255 PAR Satisfactory Likely Substantial Industry First Industrial Restruc- L2700 100 12 May-86 Jun-93 13732 PCR Satisfactory Uncertain Moderate turing Project Industry Second Industrial L2834 150 55 Jun-87 Jun-94 13732 PCR Satisfactory Uncertain Moderate Restructuring Project Poland Program and policy Structural Adjustment L3247 300 0 Jul-90 Dec-91 13001 PCR Satisfactory Likely Moderate Loan Portugal Human resources Manpower Training and L2456 31 21 Jun-84 Dec-92 13137 PCR Unsatisfactory Likely Moderate Development Project Turkey Agriculture Agricultural Extension L2405 72 10 Apr-84 Oct-93 13614 PCR Satisfactory Uncertain Moderate and Applied Research Project Energy Energy Sector L2856 325 13 Jun-87 Sep-93 13766 PCR Unsatisfactory Unlikely Negligible Adjustment Loan Power Fourth Tek Transmission L25R6 142 5 fun-85 Dec-91 12679 PCR Satisfactory Uncertain Negligible Project Power Power System Opera- L2602 140 25 Jul-85 Dec-93 13684 PCR Unsatisfactory Uncertain Negligible tions Assistance Project Transport Third Ports Project L2535 135 78 May-85 Dec-92 13647 PCR Unsatisfactory Unlikely Negligible Latin America and Caribbean Argentina Industry Trade Policy and Export 12815 500 4 May-87 Dec-90 13223 PAR Unsatisfactory Likely Moderate Diversification Loan Industry Second Trade Policy L2996 300 0 Oct-88 Dec-90 13223 PAR Satisfactory Likely Moderate Loan Power Power Engineering L2751 14 0 Sep-86 Jun-93 13682 PCR Highly satisfactory Likely High Project Transport Public Enterprise L3291 300 0 Feb-91 Dec-93 13457 PCR Highly satisfactory Likely High Adjustment Loan Urban First Housing Sector L2997 300 278 Oct-88 Jun-93 13194 PCR Unsatisfactory Unlikely Negligible Brazil Agriculture Northwest Region L2353 65 28 Oct-83 Mar-92 13197 PCR Unsatisfactory Uncertain Substantial Development Program- Third Phase Agriculture Northeast Irrigation L2680 48 5 Apr-86 Jun-93 13743 PCR Unsatisfactory Unlikely Moderate Engineering and Technical Assistance Agriculture Fourth Agro-Industries L2960 300 58 Jun-88 Dec-92 13828 PCR Unsatisfactory Uncertain Negligible Credit Project Agriculture Agricultural Credit L2971 300 32 Jun-88 Dec-93 13827 PCR Unsatisfactory Likely Negligible Project Human resources Science and Technology L2489 72 0 Jan-85 Dec-91 13144 PCR Satisfactory Likely Substantial Project Power First Eletrobras Power L2138 183 11 May-82 Dec-89 13173 PCR Unsatisfactory Unlikely Negligible Distribution Project Power Second Eletrobras Power L2364 251 14 Dec-83 Dec-91 13173 PCR Unsatisfactory Unlikely Negligible Distribution Project Power CHESF-FURNAS Power L2564 400 155 Jun-85 Dec-92 13772 PCR Unsatisfactory Unlikely Negligible Transmission Project Caribbean Finance Third Caribbean Devel- C1364 7 1 May-83 Jun-92 13256 PAR Satisfactory Likely High Region opment Bank Project (continued on next page) (Annex Table 4.1 continued) Commit- Cancel- Institutional Loan/ ment lation Key project dates Report development Region/country Sector Project title credit no. ($mill.) ($mill.) Approval Closing no. Type Overall rating Sustainability impact Chile Agriculture Agricultural Services L2481 56 0 Dec-84 Oct-91 13227 PCR Satisfactory Uncertain Moderate and Credit Project Finance Small and Medium L2613 40 0 Aug-85 Jun-92 13181 PCR Satisfactory Likely Moderate Industry Project Finance Financial Markets Loan L3143 130 40 Dec-89 Jun-95 13707 PCR Satisfactory Likely Substantial Power Pehuenche L2832 95 55 Jun-87 Jun-94 13624 PCR Highly satisfactory Likely Moderate Hydroelectric Project Power Alto Jahuel-Polpaico L2833 22 0 Jun-87 Jun-94 13625 PCR Highly satisfactory Likely Moderate Transmission Project Urban Second Housing Sector L3030 200 0 Mar-89 Jun-93 13769 PCR Satisfactory Uncertain Negligible Project Water and sanitation Valparaiso Water L2652 6 0 Feb-86 Dec-90 13277 PCR Satisfactory Likely Substantial Supply Reconstruction Project Colombia Human resources Health Services L2611 37 20 Jul-85 Jun-93 13574 PCR Highly Unlikely Moderate Integration Project unsatisfactory Power Power Sector Adjust- L2889 300 75 Dec-87 Sep-90 13453 PCR Satisfactory Uncertain Negligible ment Loan Transport Rural Transport Sector L2668 62 0 Mar-86 Dec-92 13452 PCR Satisfactory Uncertain Substantial Project Water and sanitation Fourth Bogota Water [2512 129 25 Apr-85 Jun-93 13473 PCR Satisfactory Uncertain Negligible Supply and Sewerage Project Costa Rica Agriculture Agricultural Sector L3447 41 41 Mar-92 Jun-99 13787 PCN Not rated Not rated Not rated Investment and Institu- tional Development Program and policy First Structural Adjust- L2518 80 0 Apr-85 Jun-86 13263 PAR Satisfactory Uncertain Moderate ment Loan Program and policy Second Structural L3005 100 0 Dec-88 Jul-92 13263 PAR Satisfactory Uncertain Moderate Adjustment Loan Dominica Program and policy Structural Adjustment Cl817 3 1 Jun-87 Jun-92 13312 PCR Unsatisfactory Unlikely Moderate Credit Dominican Transport Third Road L2609 36 0 Jul-85 Dec-93 13451 PCR Satisfactory Uncertain Moderate Republic Reconstruction Project Ecuador Finance Industrial Finance L2672 115 3 Mar-86 Dec-93 12845 PCR Unsatisfactory Uncertain Moderate Project inance Third Small-scale L2673 30 0 Mar-86 Dec-92 12847 PCR Unsatisfactory Uncertain Moderate Enterprise Credit Project Program and policy Public Sector L2516 6 0 Apr-85 Mar-93 13235 PAR Satisfactory Likely Substantial Management Project Guatemala Telecommunications Third Telecommunica- L2385 30 3 Mar-84 Jun-92 13049 PCR Unsatisfactory Likely Moderate tions Project Water and sanitation Water Supply L2759 23 19 Oct-86 Dec-95 13448 PCR Highly Unlikely Negligible Rehabilitation Project unsatisfactory Honduras Human resources First Social Investment C2212 20 0 Feb-91 Mar-94 13839 PAR Highly satisfactory Likely Substantial Fund Project Human resources Second Social C2401 10 0 Jun-92 Dec-94 13839 PAR Satisfactory Likely Substantial Investment Fund Program and policy Structural Adjustment L2990 50 0 Sep-88 Dec-89 12951 PCR Unsatisfactory Unlikely Negligible Loan Jamaica Finance Kingston Free Zone L2107 14 6 Mar-82 Dec-90 13265 PCR Satisfactory Likely Moderate Project Program and policy Second Trade and L3303 30 0 Mar-91 Dec-92 13237 PAR Satisfactory Likely Substantial Financial Sector Adjustment Loan Mexico Agriculture Livestock and L0610 65 0 May-69 Jul-74 13002 PCM Highly satisfactory Likely High Agricultural Development Project Agriculture Rio Fuerte/Rio Sinaloa L1706 92 10 May-79 Jun-91 13090 PCR Satisfactory Likely Moderate Irrigation Project Agriculture Chiapas Agricultural L2526 90 32 Apr-85 Dec-93 13824 PCR Satisfactory Uncertain Negligible Development Project Agriculture Agricultural L2859 20 0 Jun-87 Jun-93 13157 PCR Satisfactory Uncertain Moderate Extension Project Agriculture Agricultural Sector L2918 300 0 Mar-88 Nov-90 12834 PCR Satisfactory Likely Substantial Adjustment Loan Finance Second Small and L2546 105 0 May-85 Dec-92 13259 PCR Satisfactory Uncertain Moderate Medium-scale Mining Development Finance Industrial Technology L2747 48 0 Jul-86 Dec-93 13271 PCR Unsatisfactory Uncertain Moderate Development Pmject Transport Second Highway L2428 200 0 May-84 Jun-91 12719 PCR Unsatisfactory Likely Substantial Sector Project (continued on next page) (Annex Table 4.1 continued) Commit- Cancel- Institutional Loan/ ment lation Key project dates Report development Region/country Sector Project title credit no. ($mill.) ($mill.) Approval Closing no. Type Overall rating Sustainability impact Transport Lazaro Cardenas L2450 76 38 Jun-84 Dec-92 13280 PCR Satisfactory Likely Moderate Industrial Ports Project Urban Housing Finance Project L2947 300 0 Jun-88 Jun-94 13142 PCR Satisfactory Likely Substantial Paraguay Human resources Second Vocational L2373 5 0 Jan-84 Jan-93 13147 PCR Satisfactory Likely Substantial Training Project Peru Power ELECTROPERU Power L2018 25 6 Jun-81 Jun-87 13572 PCR Satisfactory Likely Substantial Engineering Project Power Sixth Power Project L2179 81 56 Jun-82 Jun-88 13571 PCR Unsatisfactory Likely Negligible St. Vincent Power Power Project C1479 5 0 May-84 Jun-93 13747 PCR Satisfactory Likely Substantial Uruguay Power Power Engineering L2484 4 0 Jan-85 Jun-90 12837 PCR Satisfactory Likely Substantial Project Transport Third Highway Project L2238 45 3 Feb-83 Dec-90 13295 PCR Satisfactory Uncertain Moderate Venezuela Industry Trade Policy Loan L3092 353 2 Jun-89 Jun-93 13270 PAR Satisfactory Uncertain Moderate Program and policy Structural Adjustment L3091 402 1 Jun-89 Jun-93 13270 PAR Unsatisfactory Unlikely Negligible Loan Program and policy Interest Support Loan L3279 150 0 Dec-90 Feb-91 13270 PAR Highly Unlikely Moderate unsatisfactory Middle East and North Africa Algeria Agriculture Agricultural Credit L3009 110 32 Dec-88 Sep-92 13196 PCR Unsatisfactory Uncertain Moderate Project Program and policy Economic Reform L3117 300 1 Aug-89 Sep-92 13301 PCR Unsatisfactory Unlikely Substantial Support Loan Transport Second Railway Project L2976 143 120 Jun-88 Feb-93 13645 PCR Unsatisfactory Unlikely Negligible Jordan Human resources Primary Health Care L2531 14 5 May-85 May-93 13139 PCR Satisfactory Likely Moderate Project Human resources Emergency Recovery L3306 10 0 Mar-91 Oct-93 13742 PCR Highly satisfactory Likely Substantial Project Power Sixth Power Project L2710 28 1 May-86 Nov-92 12853 PCR Satisfactory Likely Moderate Program and policy Industry and Trade L3142 150 0 Dec-89 Dec-92 12918 PCR Satisfactory Likely Moderate Policy Adjustment Loan Transport Multi-mode Transport L2463 30 0 Jul-84 Dec-92 13082 PCR Unsatisfactory Unlikely Negligible Project Urban Amman Transport and L2334 30 6 Jul-83 Jun-93 13300 PCR Highly satisfactory Likely Substantial Municipal Development Project Urban Second Urban L2587 28 12 Jun-85 Jun-93 13288 PCR Satisfactory Likely Negligible Development Project Morocco Agriculture Oulmes-Rommani L2217 30 17 Dec-82 Dec-92 12701 PAR Satisfactory Uncertain Moderate Agricultural Development Project Agriculture Agricultural Sector L2885 225 0 Nov-87 Apr-92 13511 PCR Satisfactory Likely Not rated Adjustment Loan Transport Port of Casablanca and L2657 22 0 Feb-86 Jun-93 13594 PCR Satisfactory Likely Substantial Mohammedia Project Syria Agriculture National Agricultural L2621 8 8 Sep-85 Oct-90 13177 PCN Not rated Not rated Not rated Extension Project Tunisia Agriculture Northwest Agricultural L2502 15 9 Mar-85 Dec-91 12817 PCR Unsatisfactory Unlikely Negligible Production Project Finance Export Industries Project L2522 50 24 Apr-85 Jun-92 12b65 PCR Satisfactory Likely Moderate Power Fourth Power Project L2455 39 17 Jun-84 Dec-90 13172 PCR Satisfactory Likely Negligible Urban Third Urban L2223 25 0 Dec-82 Jun-93 13514 PCR Satisfactory Uncertain Moderate Development Project Urban Second Urban L2429 33 0 May-84 Jun-93 13222 PCR Satisfactory Likely Moderate Transport Project Water and sanitation Third Urban Sewerage L2255 34 1 Mar-83 Dec-92 13419 PCR Satisfactory Likely High Project Water and sanitation Seventh Water L2368 50 0 Dec-83 Jun-93 13418 PCR Satisfactory Likely High Supply Project Yemen Agriculture Central Highlands C1453 8 2 Mar-84 Dec-92 13825 PCR Unsatisfactory Unlikely Negligible Agricultural Development Project Agriculture WadiAl-Jawf C1584 10 8 Apr-85 Dec-92 13822 PCR Unsatisfactory Unlikely Negligible Agricultural Development Project Energy Technical Assistance to C1702 12 1 May-86 Dec-93 13479 PCR Satisfactory Uncertain Substantial the Petroleum Sector Finance Second Industrial C1547 8 3 Feb-85 Dec-91 12789 PCR Unsatisfactory Uncertain Negligible Development Project Power Fourth Power Project C1701 12 3 May-86 Jun-93 13770 PCR Unsatisfactory Unlikely Negligible Agriculture Second Wadi Hadra- C1346 9 1 Apr-83 Jun-92 13065 PCR Unsatisfactory Uncertain Negligible mawt Agricultural Development Project (continued on next page) (Annex Table 4.1 continued) Commit- Cancel- Institutional Loan/ nent lation Key project dates Report development Region/country Sector Project title credit no. ($mill.) (SmillJ Approval Closing no. Type Overall rating Sustainability impact Transport Fourth Highway Project C1617 14 0 Jun-85 Jun-93 13593 PCR Satisfactory Likely Substantial South Asia Afghanistan Transport Third Highway Project C0927 18 18 Jun-79 Jun-84 13282 PCN Not rated Not rated Negligible Bangladesh Agriculture Second Forestry Project C1634 28 10 Nov-85 Jun-92 13043 PCR Satisfactory Likely Substantial Agriculture Shrimp Culture Project C1651 22 8 Jan-86 Jun-93 13679 PCR Satisfactory Uncertain Moderate Energy Industrial Energy C1942 11 11 Jun-88 Jun-95 13765 PCN Unsatisfactory Unlikely Negligible Efficiency Project Transport Chittagong Port Project C1247 67 24 May-82 Dec-92 12960 PCR Satisfactory Likely Negligible India Agriculture Agricultural C0230 6 6 Oct-70 Dec-74 13217 PCN Not rated Not rated Not rated Aviation Project Agriculture Third National Coopera- C1502 220 31 Jun-84 Jun-92 13140 PCR Unsatisfactory Unlikely Moderate tive Development Corporation Project Agriculture Kerala Social Forestry C1514 32 6 Jul-84 Mar-93 13588 PCR Satisfactory Uncertain Moderate Project Agriculture National Agricultural C1523 39 6 Oct-84 Mar-93 13696 PCR Satisfactory Uncertain Substantial Extension Project Agriculture Second National C1569 49 12 Mar-85 Mar-93 13632 PCR Satisfactory Uncertain Moderate Agricultural Extension Project Agriculture National Social C1611 165 11 Jun-85 Mar-93 13698 PCR Satisfactory Uncertain Moderate Forestry Project Agriculture Himalayan Watershed [2295 46 18 May-83 Sep-92 12875 PCR Satisfactory Uncertain Moderate Management Project Energy Cambay Basin L2403 243 29 Mar-84 Oct-92 13683 PCR Highly satisfactory Likely Negligible Petroleum Project Finance Fifth Industrial Credit L0340 30 3 Jun-63 Apr-78 13253 PCN Satisfactory Likely Not rated Human resources Fourth Population C1623 51 13 Jul-85 Mar-94 13785 PCR Satisfactory Uncertain Moderate Project Industry Third Fertilizer Project L1743 250 250 Jun-79 Nov-84 13059 PCN Not rated Not rated Not rated Transport National Highway L2534 200 97 May-85 Dec-93 13644 PCR Unsatisfactory Unlikely Negligible Project Nepal Agriculture Third Rural C1727 19 14 Jul-86 Jul-93 13629 PCR Unsatisfactory Unlikely Negligible Development Project Program and policy Second Structural C2046 60 0 Jun-89 Jul-92 12871 PCR Satisfactory Likely Substantial Adjustment Credit Pakistan Agriculture Sixth Salinity Control C0754 70 4 Dec-77 Jun-92 12889 PCR Satisfactory Unlikely Negligible and Reclamation Project Agriculture Salinity Control and C0877 60 0 Jan-79 Jun-92 12890 PCR Satisfactory Likely Negligible Reclamation Project Agriculture Baluchistan Minor Irri- C1243 14 0 May-82 Dec-93 13494 PCR Satisfactory Likely Moderate gation and Agricultural Development Project Agriculture Command Water C1487 47 1 May-84 Dec-92 13428 PCR Satisfactory Likely Moderate Management Project Agriculture Baluchistan Agricultural C1533 8 1 Dec-84 Jun-92 12893 PCR Unsatisfactory Uncertain Moderate Extension and Adaptive Research Finance Second Small Industries C1499 50 3 Jun-84 Jun-92 13182 PCR Unsatisfactory Uncertain Negligible Project Finance Second Industrial Invest- C1646 150 2 Jan-86 Dec-92 13124 PCR Satisfactory Likely Moderate ment Credit Project Industry Fauji Agrico Fertilizer L1400 55 55 Apr-77 Dec-80 13198 PCN Not rated Not rated Not rated Project Urban Lahore Urban C1348 16 3 Apr-83 Dec-92 13420 PCR Satisfactory Uncertain Substantial Development Project Sri Lanka Agriculture Second Dairy L2576 38 38 Jun-85 Dec-91 12894 PCN Not rated Not rated Not rated Development Project Finance Third Small and C1860 20 1 Dec-87 Jun-93 13199 PCR Satisfactory Uncertain Moderate Medium Industries Project Transport Second Roads Project L2517 24 0 Apr-85 Dec-92 13276 PCR Satisfactory Likely Moderate Notes: PCR=project completion report; PAR=performance audit report; and PCN=project completion note. * r*o CHINA FRANCE ISRAEL NEW ZEALAND RUSSIAN FEDERATION Wennergren-Williams AD Distributors of China Financial & Economic World Bank Publications Yozmot Literature Ltd. EBSCO NZ Ltd. ISdatelStvo P. 0. Box 1305 World Bank Publishing House 66, avenue d'i6na P.O. Box 56055 Private Mail Bag 99914 9a, Lolpachrul pereulok S-171 25 Solna 8, Da Fo Si Dong JIe 75116 Paris Tel Aviv 61560 New Market Moscow 101831 Tel: (8) 705-97-50 Publications Beijing Tel: (1) 40-69-30-55 Tel: (3) 5285-397 Auckland Tel: (95) 917 67 49 Fax: (8) 27-00-71 Tel: (1) 333-8257 Fax: (1) 40-69-30-68 Fax: (3) 5285-397 Tel: (9) 524-8119 Fax: (95) 917 92 59 Prices and credit terms vary Fax: (1) 401-7365 Fax: (9) 524-8067 SWITZERLAND fom county t0 countr. GERMANY R.O.Y. International SAUDI ARABIA, QATAR Librairie Payot COLOMBIA UNO-Vertag PO Box 13056 NIGERIA Jarir Book Store Service lastitutionnet Consult your local distributor Infoenlace Ltda. Poppelsdorfer Allee 55 Tel Aviv 61130 University Press Limited P.O. Box 3196 Cites de Montbenon 30 before placing an order. Apartado Aereo 34270 53115 Bonn Tel: (3) 5461423 Three Crowns Building Jericho Riyadh 11471 1002 Lausanne Bogolt D.E. Tel: (228) 212940 Fax (3) 5461442 Private Mail Bag 5095 Tel: (1) 477-3140 Tel: (021)-320-2511 Tel: (1) 285-2798 Fax: (228) 217492 lbadan Fax: (1) 477-2940 Fax. (021)-320-2514 ALBANIA Fax (1) 285-2798 Palestinian Authority/Middle East Tel: (22) 41-1356 Adrion Ltd. GREECE Index informatlon Services Fax: (22) 41-2056 SINGAPORE, TAIWAN, Van Diermen Editions Technique Perlal Rexiepl SIr COTE DWIVOIRE Papasotirlou S.A. PO.B. 19502 Jerusalem MYANMAR, BRUNEI Ch. de Lacuez 41 Pall. 9. Shk. 1, Ap. 4 Centre d'Edition ot de Diffusion 35. Stournara Sir. Tel: (2) 271219 NORWAY Asahgate Publishing Asia CI11807 Blanay Tirana Africalnes (CEDA) 106 82 Athens ITALY Narvesen Information Center Pacific Pie. Ltd. Tel: (021) 943 2673 Tel: (42) 274 19; 221 72 04 B.P. 541 Tel: (1) 364-1826 Llcosa Commissionaria Sansoni SPA Book Department 41 Kallang Pudding Road 04-03 Fax: (021) 943 3605 Far (42) 274 19 Abidjan 04 Plateau Fax: (1) 364-8254 Via Duca DI Calabria, 11 P.O. Box 6125 Etlerstad Golden Wheel Building Tel: 225-24-6510 Casella Postale 552 N-0602 Oslo 6 Singapore 349316 TANZANIA ARGENTINA Fax 225-25-0567 HONG KONG. MACAO 50125 Firnnze Tel: (22) 57-3300 Tel: (65) 741-5186 Oxford University Press OIcina del Libro Inlernaclonal Asia 2000 Ltd. Tel (55) 645-415 Fax: (22) 68-1901 Fax: (65) 742-9356 Maktaba Street Av. Cordoba 1877 CYPRUS Sales & Circulation Department Fax: (55) 641-257 PO Box 5299 1120 Buenos Aires Center of Applied Research Seabird House, unli 1101-02 PAKISTAN SLOVAK REPUBLIC Dar es Salaam Tel: (1) 815-8156 Cyprus College 22-28 Wyndham Street, Central JAMAICA Mirza Book Agency Slovart G.T.G. Ltd Tel: (51) 29209 Fax: (1) 815-8354 6, Dogenes Street, Engoni Hong Kong lan Randle Publishers Ltd. 65, Shahrah-e-Quaid-e-Azam Krupinska 4 Fax (51) 46822 P.O. Box 2006 Tel: 852 2530-1409 206 Old Hope Road PD. Box No. 729 PO Box 152 AUSTRALIA, FLII, PAPUA NEW GUINEA, Nicosia Fax: 852 2526-1107 Kingston 6 Lahore 54000 852 99 Bralislava 5 THAILAND SOLOMON ISLANDS, VANUATU, AND Tel: 244-1730 URL: htp/www.sales@asia2000.com.hk Tel: 809-927-2085 Tel: (42) 7353601 Tel: (7) 839472 Central Books Distribution WESTERN SAMOA Fax 246-2051 Fax: 809-977-0243 Fax: (42) 7585283 Fax: (7) 839485 306 Silom Road D.A Information Services HUNGARY Bangkok 648 Whitehorse Road CZECH REPUBLIC Foundation for Market JAPAN Oxford University Press SOUTH AFRICA, BOTSWANA Tel: (2) 235-5400 Mitcham 3132 National inormation Center Economy Eastern Book Service 5 Bangalore Town For single titles: Fax: (2) 237-8321 Victoria prodejna, Kornvilska 5 Dombovarl Ut 17-19 Hongo 3-Chome, Sharae Faisal Oxford University Press Tel: (61)392107777 CS - 113 57 Prague 1 H-1117 Budapest Bunkyo-ku 113 PO Box 13033 Southern Africa TRINIDAD & TOBAGO, JAMAICI Fax: (61) 3 9210 7788 Tel: (2) 2422-9433 Tel: 36 1204 2951 or Tokyo Karachi-75350 PO. Box 1141 Systematics Studies Unit URL: htp.wwwdadirect.com.au Fax (2) 2422-1484 36 1 204 2948 Tet: (03) 3818-0861 Tel: (21) 446307 Cape Town 8000 #9 Watts Street URL: http://www.nIs.cz/ Fax: 361 204 2953 Fax: (03) 3818-0864 Fax: (21) 454-7640 Tel: (21) 45-7266 Curepe AUSTRIA URL: httpJ/www bekkoame.or.jp/-svI-ebs Fax: (21) 45-7265 Trinidad, West Indies GeroI and Co. DENMARK INDIA PERU Tel: 809-662-5654 Graben 31 Samfundsliteratur Allied Publishers Ltd. KENYA Editorial Desarrollo SA For subscription orders: Fax: 809-662-5654 A-1011 Wien Rosenoerns A1t6 11 751 Mount Road Africa Book Service (EA) Ltd. Aparlado 3824 Internaltonal Subscription Service Tel: (1) 533-50-14-0 DK-1970 Frederiksberg C Madras - 600 002 Quaran House, Mtangano Street Lima I PO. Box 41095 UGANDA Fax: (1) 512-47-31-29 Tel: (31)-351942 Tel: (44) 852-3938 P.O. Box 45245 Tel: (14) 285380 Craighall Gustro Ltd Fax (31)-357822 Fax: (44) 852-0649 Nairobi Fax: (14) 286828 Johannesburg 2024 Madhvani Building BANGLADESH Tel: (2) 23641 Tel: (11) 880-1448 PO Box 9997 Micro Industries Development ECUADOR INDONESIA Fax: (2) 330272 PHILIPPINES Fax: (11) 880-6248 Plot 16/4 Jinia Rd. Assistance Society (MIDAS) Facultad Latinoamericana de PI. Indira Limited International Booksource Center Inc. Kampala House 5, Road 16 Ciencias Sociales Jalan Borobudur 20 KOREA, REPUBLIC OF Suite 720, Cityland 10 SPAIN Tel/Fax: (41) 254763 Obanmond! R/Area FLASCO-SEDE Ecuador PO. Box 181 Daejon Trading Co. Ltd. Condominium Tower 2 Mundl-Prensa Libros, S.A. Dhaka 1209 Calle Ulplano Pae 118 Jakarta 10320 P.O. Box 34 H V dela Costa, corner Castello 37 UNITED KINGDOM Tel: (2) 326427 y Av. Patria Tel: (21) 390-4290 Yeielda Valero St. 28001 Madrid Microinto Ltd. Far (2)811188 Quto, Ecuador Fax: (21) 421-4289 Seoul Makall, Metro Manila Tel: (1) 431-3399 PO. Dox 3 Tet (2) 542 714; 542 716 528 200 Tel: (2) 785-1631/4 Tel: (2) 817-9676 Fax: (1) 575-3998 Alton, Hampshire GU34 2PG BELGIUM Fax: (2) 566 139 IRAN Fax (2) 784-0315 Fax: (2) 817-1741 httpj/www.tsai.es/mprensa England Jean De Lannoy Kowkab Publishers Tel: (1420) 86848 Av. du Rol 202 EGYPT, ARAB REPUBLIC OF P.O. Box 19575-511 MALAYSIA POLAND Librerla Inlernacional AEDOS Fax: (1420) 89889 1060 Brussels Al Abram Tehran University of Malaya Cooperative International Publishing Service Conseil de Cent, 391 Tel (2) 538-5169 At Gain Street Tel: (21) 258-3723 Bookshop, Limited UI. Piekna 31/37 08009 Barcelona ZAMBIA Fax (2) 538-0841 Cairo Fax: 98 (21) 258-3723 P.O. Box 1127 00-577 Warzawa Tel: (3) 488-3009 University Bookshop Tel: (2) 578-6083 Jalan Pantal Saru Tel: (2) 628-6089 Fax: (3) 487-7659 Great East Road Campus BRAZIL Fax (2) 578-6833 Ketab Sara Co. Publishers 59700 Kuala Lumpur Fax: (2) 621-7255 PO. Box 32379 Pubicactes Tecticas Internacionals Khaled Eslamboli Ave., Tel: (3) 756-5000 SRI LANKA, THE MALDIVES Lusaka Ltda. The Middle East Observer 6th Street Fax: (3) 755-4424 PORTUGAL Lake House Bookshop Tel: (1) 213221 Ext. 482 Rua Polxoto Gomle, 209 41, Sherif Street Kushe Delafrooz No. 8 Livraria Portugal PO. Box 244 01409 Sao Paulo, SP Cairo Tehran MEXICO Rua Do Carmo 70-74 100, Sir Chitlampalam A. ZIMBABWE Tet (11) 259-6644 Tel: (2) 393-9732 Tel: 8717819 or 8716104 INFOTEC 1200 Lisbon Gardiner Mawatha Longman Zimbabwe (Pie )Ltd. Fax (11) 258-6990 Fax: (2) 393-9732 Fax: 8862479 Aparlado Postal 22-860 Tel: (1) 347-4982 Colombo 2 Toude Road, Ardbennie 14060 Tlilpan, Fax: (1) 347-0264 Tel: (1) 32105 PO. Box ST125 CANADA FINLAND IRELAND Mexico D.F. Fax: (1) 432104 Southerton Renoul Pubtlshing Co. Ltd. Akateeminen Kirakauppa Government Supplies Agency Tel: (5) 606-0011 ROMANIA Harare 1294 Algoma Road PO. Box 23 Olig an tSoidthair Fax: (5) 606-0386 Compani De Libraril Bucuresti SA SWEDEN Tel: (4) 662711 Ottawa, Ontario KIB 3W8 FIN-00371 Helsinki 4-5 Harcourt Road Sir. Lipscani no. 26, sector 3 Fritzes Customer Service Fax: (4) 662716 Tel: 613-741-4333 Tet (0) 12141 Dublin 2 NETHERLANDS Bucharest Regeringsgalon 12 Fax: 613-741-5439 Fax: (0) 121-4441 Tel: (1) 461-3111 De LindebooVInOr-Publikaties Tel: (1) 613 9645 S-106 47 Stockholm URL: hltpibooknet.culiet.ftakal Fax: (1) 475-2670 PO. Box 202 Fax: (1) 312 4000 Tel: (8) 690 90 90 7480 AE Haaksbergen Fax: (8) 2147 77 THE WORLD BANK A partner in strengthening economies and expanding markets to improve the quality of life for people everywhere, especially the poorest The World Bank Headquarters 1818 H Street, N.W Washington, D.C. 20433, U.S.A. Telephone: (202) 477-1234 Facsimile: (202) 477-6391 Telex: MCI 64145 WORLDBANK MCI 248423 WORLDBANK Cable Address: INTBAFRAD WASHINGTONDC European Office 66, avenue d'lena 75116 Paris, France Telephone: (1) 40.69.30.00 Facsimile: (1) 40.69.30.66 Telex: 640651 Tokyo Office Kokusai Building 1-1, Marunouchi 3-chome Chiyoda-ku, Tokyo 100, Japan Telephone: (3) 3214-5001 Facsimile: (3) 3214-3657 Telex: 26838 13607 ISBN 0-8713-3607-X Cover design by Joyce C. Petruzzelli 9 7808211 36076