Report No. 12403 Annual Review of Evaluation Results 1992 October 13,1993 Operations Evaluation Department FOR OFFICIAL USE ONLY MICROGRAPHICS Report No: 12403 Type: OER Document of the World Bank This document has a restricted distribution and may be used by recipients only in the performanceof their official duties. Its contents may not otherwise be disclosed without World Bank authorization Abbreviations and Acronyms ARIS - Annual Review of Implementation and Supervision ARPP - Annual Report of Project Performance CECPS - Country Economic Policy Department of the World Bank DFC - Development Finance Company EAP - East Asia & Pacific Region ECA - Europe and Central Asian Region EDI - Economic Development Institute ERR - Economic Rate of Return ESAL - Energy Sector Adjustment Loan ODP - Gross Domestic Product GNP - Gross National Product IDA - International Development Association IFC - International Finance Corporation IMF - International Monetary Fund LAC - Latin America and the Caribbean Region MENA - Middle East and North Africa Region NGO - Nongovernmental Organization NTPC - National Thermal Power Corporation OD - Operational Directive OED - Operations Evaluaion Department OMS - Operational Manual Statement PCR - Project Completion Report PE - Public Enterprise PHN - Population, Health, and Nutrition PMU - Project Management Unit PPAR - Project Performance Audit Report PPF - Project Preparation Facility PIT - Posts, Telegraph and Telecommunications PUB - Public Utilities Board SAL - Structural Adjustment Loan SAS - South Asia Region SECAL - Sector Adjustment Loan SME - Small and Medium Scale Enterprise SMI - Small and Medium Industry TA - Technical Assistance UNDP - United Nations Development Program UNEP - United Nations Environment Program WDR - World Development Report FOR OFFICIAL USE ONLY * ' THE wom 6AN Weehington, 0.c. 2043s oisse of ObetaM3enera Oertosaluesen October 13, 1993 . MMOANUMTO THE EXECTIV D=ECMR ANDz THEPRSIEN SUDECT: AnmuaL R f Ea uation Rul 1992 Tis year's review-the nineteenh in an annual serles-bas three =ma objcsves: (1) to udate operadona trends based on the evaluation findlngs of the 1992 cohort, consisting of 277 completed Bank financed operadons; (Ii) to mmni key operadonal features and outcomes associated with Bank fnancing of agriculture, the largest single reciplent of Bank commitments; (1ii) to provide a preffim~nay assessment of the Bank's roe In Insdtutioal development, a growing component of country assistance progras. The review highlghts the complementarity of the three main crltera used by the Operadons Evaladon Department (OED) In its evaltion (i) project outcomes; (l) sus finabity; and iii) indtudo~a developmenL Together, these indices are used by OED to track the contribuon of Bank operations to development effec~veness. The annal review reaffirms the important role of exogenom shocks and country condidons in determining development outcomes. While som levelling off In trends Is apparent, the review cautions against concluding that the declne In project performance noted in past revlews bas been arrested. A substantial gap between the en=nmic returns projected at appraisal and those reestmated at compledon reman while a study joindy carried out by the Operations Polcy Development Department and OED confirms that suff radngs of the development prospeca of ongoing projects are overoptimistic. Together these fndings reinforce the urgency of sm~engtening "quality at entry as wei as portfolio management safeguards, as outined in the Next Steps Program. For the rst time, die anmal review presens the impact of Bank operations in ters of the three major themadc emphases of the Bank's current development agenda (poverty reducdon; environmen~a protecon and privat. sector development). While interesdng illustrations wem identied, systematic aggregatIon of results by themadc emphases was hindered by lack of relevant information in project documens, reflecting the recency of some of diese emphases as wel as the uneven qualty of monitoring and evaluation components. his documnthau a testicted disribution and mary be uasdby recipients only in the perf b....ncc ?their ofEicial daus. Is contents u.yf odmwla.be =-o woéo~t wordd wank au*mraion. 2 In search of increasing relevance, the Bank's operations have become more complex and demanding. This Is amply illustrated by the agriculture and institutional develorment chapters. It may not be appropriate for the Bank to seek a higher rate of "satisfactory" operations by reducing the scope and ambition of its development agenda. Instead, the "best practices" identified in this annual review suggest that greater clarity, realism and rigor In assessing and managing the risks of the Bank's loan and credit portfolio would go a long way in enhancing the development impact of Bank operations-provided portfolio management processes incorporate greater participation with borrowers, development partners and beneficiaries. Specifically, based on the record of outstanding operations selected from the 1992 cohort-spanning a wide variety of sectors and instruments-the review identifies five interrelated factors which would help to deliver better results "on the ground": (I) effective management of the project cycle; (ii) borrower ownership of project objectives; (iii) active involvement of beneficiaries; (iv) explicit institutional strengthening; and (v) visible benefits traceable to the development intervention. Attachment This donumtas a eaticd diaItuion and say be usnd by scipleats only in the peformanceoftheir of * cial dules. Its coatents my nat oewle be disolased wlaut World Bank authoridu on. ANNUAL REVIEW 1992 CONTENTS EXECUT S UMMARY .................................... SA. 1. EVALUATING OPERATIONAL PERFORMANCE .................. 1 A. THE 992 REV ................................ 1 B.THE1992COHORT ................................ 1 ProjectOutcomes ................................... 1 A d justment Operations ............... ................ 13 InstitutionalD elopmen .............................. 13 Sustainabiity ...................................... 16 Project Outcomes, Institutional Development, andSustainability ................................ 18 Other Aspects of Project Peformance ...................... 19 PhysicalAch evements ................................ 19 TmeandCostVaratons .............................. 20 C. TEhATIC AREAS ................................ 22 Poverty Reduction................................... 22 Eirona alImpac ................................ 24 Private Sector Development (PSD) ........................ 26 D.CONCLUSK 3NS ................................... 30 Project Outcomes, Institutional Development, andSu ana ity ................................ 31 Other Aspects of P ject Perormance .................o..... 31 2. PERFORMANCE TN5AN)FACTORS ...................... 33 A. PERFORMANCETRENDS............................ 33 Regional, Country, and SorolPefranme.................. 37 Portfolio Com position ................................ 39 B. PERFORMANCE FACTORS ........................... 40 External Environment and ExogenouShoks.................. 40 CountryConditons .................................. 43 ProcesFac gFa4 .................................. 4 Identifi in .................................... 46 Preparn on ..................................... 48 Appra. ....................................... 48 Supervion ..................................... 49 Cancellations .... .............................. C. LINKAGES WITH REVIEW OF ONGOING PROJECTS ......... 54 ANNUAL REVIEW 1M9 CONTENTS PaB2 No. D. CONCLUSIONS................... .. .... ... 57 TrendsinProject Performance ............ 57 Factors,Afmecting Project Performance ... . .................. 58 Linkages with Review of Ongoing Projects ................... 59 3. THE AGRICULTURAL PORTFOLIO .......................... 61 A. IN T RODUC ON ................................. 61 B. THESEITING .................................... 61 Agriculture In Development ............................. 61 Achieving Agricultural Growth........................... 62 C. EVOLUTION OF WORLD BANK AGRICULTURAL LENDING .... 63 D. PERFORMANCE OF THE AGRICULTURAL PORTFOLIO, 1970-85 68 GeneralTrends ... 68 Borrower InceDives. 68 The Project Factor and the Bank's Role ..................... 71 Agriculture and OtherLanding ........................... 73 E. SUBSECTOR LENDINGXPERIENCB.................... 75 Irrigation ........................................ 75 Farm Credit ...................................... 76 Area Development .................................. 77 PlantationCrops .................................... 78 F. CONCLUSIONS ................................... 79 Evaluation of the Agricultural Portfolio, 1970-85 ............... 79 Subsector Lending Experience ........................... 79 4. INSTITUTIONAL DEVELOPMENT ........................... 81 A. INTRODUCTION .................................. 81 B. CONCEPS...................................... 81 C. THE BANK'S INSTITUTIONAL DEVELOPMENT ROLE ........ 84 D. INSTITUTIONAL DEVELOPMENT PERFORMICE .......... 90 ANNUAL REVEW 1992 CONTENTS E. ISSUES......................................... 94 SectorWork ...................................... 94 Design .......................................... 95 Indicatoro ......................... ............. 101 Supervision .................... .................. 102 Borrower Cohnment ................................ 103 New versus ExistingOrganizations ........... 106 Instruments ............... ......... ...... . 108 F. CONCLUSIONS ........................................ 112 LIST OF ANNEES........................................ 115 1.1 Performance of Operations Evaluated, by Evaluation Year (1974-1992) .... 8 2.1 Country Portfolio Performance and Terms of Trade, 1980s ........... 42 2.2 Relationship Between Ratings and Problems with Highly Canceled Projects ............................... 52 2.3 Comparison of ARPP and OED Evaluation Ratings ............... 54 3.1 Agriculture's Share in National Income, Employment and Export Barnings by Country Income Groups, 1989 ................ 62 3.2 Percentages of Total Agricutural Leading 1970-85, by Region and Subsector ........................... 65 3.3 Performance Ratings of Agriculture and Non-Agricultur Projects 1970-77 and 197845 Compared ....................... 68 3.4 Indices of Agricultural Commodity Prices 1970-92 ................ 70 3.5 Difference Between Appraisal and Completion Rates of Retur ......... 71 3.6 Performance Ratings, Agriculture and Other Projects, 197-85, with and without Africa and Area Development Projects ........ 75 4.1 Distribution of Sectoral Institutional Reforms in SECALSs ........... 86 4.2 Funding Sources for the 1992 Project Cohort .................... 88 4.3 Achievement of ID Objectives, 1989-92 Cohorts Rating by Sector ....... 91 4.4 Achievement of ID Objectives, 1989-92 Cohorts Rating by Region ...... 92 4.5 ID Objectives, 1992 Cohort ............................... 98 4.6 Distribution of ID Components and Frequency, 1992 Cohort .......... 98 4.7 Borrower Ownership Index ............................... 106 4.8 ID Methods Used In 1992Cohort ........................... 109 ANNUAL REVIEW 1992 CONTENTS PaR2No% 1.1 Economic Rates of Return, Sectoral RERRS, 1992 Evaluation Year vs. 1974-91 Cohorts .............................. 3 1.2 Distribution of Operations Evaluated in 1992, by Year of ApprovalandClosing ............................. 4 1.3 Distributionof by Sector ....................... 5 1.4 DistributionofCommItmetsgbyRegion ....................... 1.5 Performance of Operations Evaluated In 1974-92 ................. 7 1.6 Project Performance Weighted by Commitment Amount ............. 9 1.7 Achievement of Institutional Objectives by Sector, Region (1992,1989-91cohorts)............................ 14 1.8 Project Sustainability of 1992 and 1989-91 Cohorts ................ 17 1.9 Interrelationship of Project Sustainabilty, Institutional Development and Performance Ratings, 1989-92 Evaluation Cohorts 19 1.10 Physical Investment 1992 Cohort by Region, Sector ................ 20 1.11 Time Overruns, 1992 and 1974-91 Cohorts ..................... 21 1.12 Cost Overrun, 1 and 1974-91 Cohorts ...................... 22 2.1 Performance of Operations Approved (1968-84) .................. 34 2.2- Performance of Operations Approved (1968-84), Weighted by Commitment 35 2.3 Cohort Appraisal and Re-Estimated ERRs, by Year of Approval(1968-84) .............................. 36 2.4 Comparison of ARPP and OED - Evaluation Ratings, 1975-92 ........ 55 25 PCRIPAR Ratings & OEDIARPP - Disconnect, by Region ........... 56 2.6 PCR/PAR Ratings & OEDIARPP - Disconnect, by Sector ........... 57 3.1 Bank Lending, Total, Agriculture, and Adjustment ................ 6 3.2 Agriculture Performance and Leding ........................ 69 3.3 Agriculture and Non-Agriculture Project Performance Ratings ......... 74 J.03ES 1.1 Five Successful AgricultUrA Projects in the 1992 Cohort ............. 11 1.2 Adjustment in Sub-Saharan Africa ........................... 15 1.3 Early Experience with Involuntary Resettlement .................. 25 1.4 NaturalResucManagement ............................. 27 1.5 Bank Lending for Industrial Technology Development (ITD) .......... 29 2.1 The Developmental Costs of Flawed Quality at Entry: The Tanzania Mufindi Project ........................ 53 4.1 InstituionaDevelopment ................................ 83 4.2 World Bank'sMandate.................................. 85 4.3 Technical Assistan Delivery Methods .............D...v....... 87 4.4 Principles of New Orientation in Technic Cooperation ............. 89 4.5 Rural and Area Development Projects in Northeast Brazil ............ 93 4.6 Sector Work and Strategy Formulation in Indonesia's Power Sector ...... 97 ANNUAL REVIEW 1992 CONTENTS PaSe No. 4.7 Chile - Promoting Design Changes through the Process Approach to Institutional Development . .......................... 100 4.8 Obtaining the Borrower's Commitment ..... . . . . . . . . . . . . . .... 105 4.9 Project Management and Staffing ......................... 110 4.10 ImprovingtheEffectivnessofIDTA ........................ 111 CHART 2.1 Project Processing & Outcome - 1991 and 1992 Evaluation Cohorts ..... 47 This study was prepared by a team led by John Johnson (Task Manager). Contributors included Sulaiman Wasty (Consultant), Dudley Wallace (Consultant), Gerardo Sicat, Hans Adler (Consultant), Samantha Kappagoda (Consultant), and Thais Franco (Research Analyst) on Chapters I and 2; Jacob Meerman, Graham Donaldson, Javier Tellez, Charles Sheerin (Consultant), Ty Mitchell (Consultant), and Poonsook Mahatanankoon (Research Assistant) on Chapter 3; and Kudlapur Krishna (Consultant), Alain Barbu, and David Thomas (Consultant) on Chapter 4. Background work on sectors, used throughout the review, was provided by Hans Thias (Human Resources), Abmad Ahsan (Structural Adjustment Operations), Nicolas Mathieu (Finance), Farokh Namabadi and George Maniatis (Industry), Robert Armstrong (Technical Assistance), Richard Berney (Energy), Rene Ribi (Power), Jan de Weille (Transport), Tauno Skytta (Urban), Jozsef Buky (Water and Sanitation), and Ronald Rldker and George Baldwin (Population). Thomas Balley (Consultant) provided overall statistical support and database management Copy editing was done by Angela Gentile-Blackwell. Administrative support was provided by Alejandra Sarmiento, Eneshi Davis, Carmen Mispireta, Jasmine Mason-Anderson, Norma Namisato, Carla Sarmiento, and Lorna Sibblies. ANNUAL REVIEW OF EVALUATION RESULTS FOR 1992 EXECUTIVE SUMMARY . This Is the nineteenth Annual Review of Evaluation Results. Like Its predecessors, it presents salient aspects of recent Bank evaluation work, based on Project Completion Reports (PCRs) prepared by Bank operational staff and Performance Audit Reports (PARs) undertaken by the Operations Evaluation Department (OED). Besides highlighting the performance of i.ank-supported development projects and adjustment programs, this year's review assesses selected aspects of Bank and borrower experiences in promoting irstitutional development. ii. Chapter 1 assesses the cohort of projects evaluated during CY92 from the standpoint of how well It measures against the principal criteria of completed project performance-outcomes, Institutional development, and sustainability-and discusses how these dimensions of performance interrelate. The assessment is later broadened to present some relevant findings from the cohort related to the core objectives of the Bank's mission, notably poverty reduction, environmental impact, and private sector development. Chapter 2 evaluates long-term trends in portfolio performance and discusses how project performance is influenced by external, country, and project factors. Chapter 3 takes a closer look at secular project performance in the single most Important sector in the Bank's lending portfolio-agriculture-and the factors lying behind the changing trends. The review concludes with a fourth chapter focused on a particularly significant criterion of project performance-institutional development-which must be addressed more successfully if operational outcomes are to be significantly improved. Overview of Erman (Chapter 1) iI. The 1992 evaluation cohort consists of 277 projects involving total investments of more than US$44 billion and Bank/WA commitments of nearly US$18 billion. Most of the projects in this cohort were approved by the Board during the period 1982-85, and closed during 1990-91. iv. Projects in any given evaluation cohort are usually assessed within a year or two after physical completion, and anywhere from two to twenty years after Board approval. Hence, the conclusions presented below about project outcomes cannot be extrapolated directly to an assessment of the Bank's un portfolio of projects. The outcome of each operation is compared with its objectives at the time of Board approval, taking into account major changes that may have been agreed during execution. The objectives are then assessed for their relevance to the borrower's development requirements. OED audits occur at a later date. While the analytical emphases of PCRs and audits often vary, differences that exist between PAR and PCR performance ratings at the time of audit are small, and often reflect a more recent assessment of project outcomes. Impact evaluations, which are generally performed five to ten years after completion for a much smaller number of operations, permit a re- examination of project results over a longer time horizon. Such evaluations focus on the social, environmental, and institutional impacts of Bank operations. v. For about 30 percent of the cohort projects, whose principal costs and benefits were readily quantifiable, an economic rate of return was calculated and compared to the opportunity cost of capital, normally set at or above 10 percent. In addition, nonquantifiable costs and benefits, such as the relevance of the project to country and sector priorities, were taken into account. vili vi. For projects where It was not feasible to calculate an economic rate of return, evaluators had recourse to cost-effectiveness Indicators andlor a comparison of the benefits of reform in relation to the costs incurred (e.g., the social cost of labor retrenchment In the public sector). vii. Overall A m ent. For the first time since 1988, the proportion of projects rated zafactory Improved over the previous year's evaluation cohort. Sixty-seven percent of the projects ewluated during CY92 were rated 'satisfactory." While significantly above the ratio of 62 percent registered In the 1990-91 cohorts, this was still the third lowest share recorded since the series began In 1974. The *commitment success" rate-or the proportion of total Bank and IDA commitments allocated to satisfactory projects-at 75 percent was substantially higher than the rates of 66 percent In 1991 and 58 percent in 1990, and broadly In line with the historical average. The performance of larger operations (over US$100 million) showed systematic Improvement for the second consecutive year, achieving an average satisfactory ratio of 81 percent. vill. The proportion of satisfactory project outcomes varied considerably by Ag=in and egion. The main differences when compared with earlier evaluation cohorts were: * Agriculture, human resources, and industry projects evaluated during CY92 performed significantly better than those evaluated in 1990-91, but performance in the financial sector declined sharply. * Projects evaluated during CY92 in all regions except South Asia showed significant improvement relative to the performance of projects evaluated during 1990-91. ix. In keeping with the upward trend in project performance, this year's cohort contained a significantly higher number of outstanding projects-24-than did the 1990 and 1991 cohorts. These projects were distinguished by quality project processing, borrower ownership of basic project aims, beneficiary participation, institutional strengthening, and visible benefits. x. Roughly one-seventh of the operations evaluated during CY92 fell into the category of adjustment lending. Performance ratings for this Important lending Instrument improved modestly to 62 percent satisfactory in 1992, compared to 56 percent in 1991. However, both fell below their cohort averages. Thirteen of the 15 unsatisfactory adjustment operations took place in Sub-Saharan Africa. xi. Economic Rates of Return (ERRs). ERRs were estimated on completion for 87 cohort projects. These rates ef return need to be interpreted cautiously, since benefit estimates are still largely projections. The average re-estimated ERR (RERR) in the 1992 cohort, weighted by project costs, fell by 3 points, relative to the 1991 cohort, to 13 percent. It was also below the historical average RERR of 15 percent. Diminished returns on industrial and transport projects in the 1992 cohort accounted for most of the decline. The historically-close relationship between a project's yield and its overall rating was maintained, although significant divergences were noted in the water and sanitation and in the power sectors. These divergencies appear to reftect the financial, but not economic, impact of often-repressed utility rates, as well as difficulties in fully-measuring consumer surpluses. xil. Institutional Development (ID). Evaluation is not solely concerned with balancing project costs and benefits, but also with a country's ability to make effective use of its human, organizational, and financial resources to achieve lasting development. Approximately 95 percent of the projects in the 1992 cohort had Institutional development objectives. Of these, nearly one-third substantially achieved these goals, while close to half did so partially. This was a slight improvement over the comparable figures for the 1989-91 cohorts. ix xii. Sustainability. Another crucial dimension of project performance is the exzent to which an operation is likely to maintain an acceptable level of net benefits throughout its economic life. It requires judgments about the likely responses to challenges and risks to which an operation may be subjected. The sustainability of project benefits in the 1992 cohort was rated, on average, broadly in line with that of the 1989-91 cohorts. Projects of "likely" sustainability constituted 54 percent of the totwl. However, at the sectoral level, average sustainability ratings displayed substantial variability, with industry and telecommunications doing significantly better, while power, transport, finance, program and policy, urban, and energy projects did relatively worse. Regionally, sustainability of projects improved everywhere except in Latin America and Africa. xlv. Project Outcomes, Institutional Development, and Sustainability. Where a project fails to achieve its institutional objectives, the evaluator must assess whether this will prevent achievement of a satisfactory project outcome or pose a serious obstacle to sustainability. This is not to say that all projects rated satisfactory are automatically regarded as sustainable and as having achieved substantial Institutional development. Yet, there Is a strong interrelationship among the three ratings. In a sample of over a thousand completed projects, 97 percent of the projects rated highly on both sustainability and ID grounds also achieved a satisfactory outcome, whereas only 5 percent of the projects deficient in sustainability and ID were rated as satisfactory. xv. Other Aspects of Project Performance. There was modest overall improvement in the achievement of physical goals. Seventy-six percent of the projects evaluated during CY92 substantially achieved their physical objectives, while 21 percent partially achieved them. This compares with corresponding ratios of 67 percent and 27 percent in the 1991 cohort. In regional terms, the achievement of physical objectives was high in Europe and Central Asia (ECA), South Asia, East Asia and Pacific (EAP), and Middle East and North Africa (MENA); intermediate in Africa; and lowest in Latin America and the Caribbean (LAC). Counterpart funding difficulties and deterioration in the borrower's macroeconomic situation appear to be the main reasons for less-than-satisfactory physical performance. xvi. The 1992 cohort provided further evidence that delays in project completion continue their slow growth. Project implementation took about 50 percent longer (6.7 years) than had been projected at appraisal for numerous reasons, including political and other disturbances in the host country, insufficient institutional capacity, counterpart funding shortages, and flaws in project design. Time overruns were highest for projects in the finance, energy, and telecommunications sectors, and for projects in the LAC region. xvii. Project costs in US dollar terms were slightly lower (4 percent) at completion than had been expected at appraisal. In most cases, these cost underruns were a by-product of the significant currency devaluations undertaken by many borrowers during the 1980s when most of these projects were implemented. xviii. Thematic Areas. A sample of 55 projects from the 1992 cohort was selected to extract relevant findings with respect to poverty reduction, environmental impact, and private sector development. xix. Poverty Reduction. The 1992 cohort provides a number of examples of "best practice" in improving living conditions, enhancing emplo)ment opportunities, and improving the delivery of social services to the poor, along with some instances of conspicuous failure. Shortcomings in achieving poverty reduction were often associated with project designs which either failed to take full account of the repercussions on the poor, or failed to take full account of the repercussions of policy reforms on poverty. x xx. Environmental Impact. Sound environmental management Is another key objective of Bank assistance. Of growing concern in recent years has been the impact of resettlement. In this context, a recent OED study concluded that, although Bank guidelines are broadly appropriate, they were poorly applied in practice. In future resettlement work, there is a need for: * rigorous attention to minimizing the number of people affected; * greater effort to restore incomes, including more accurate measurement; * accurate costing and adequate funding; and * improved management. xxi. There were examples of dramatic improvement in pollution control, land restoration, and natural resource management among several projects in the 1992 cohort. But other projects were characterized by regrettable shortcomings in project design and preparation, with correspondingly adverse environmental effects. Recent OED studies have suggested a framework for looking at natural resource management (NRM), and applied it in two countries: Nepal and Bolivia. These studies found that: * Neither ESW nor project preparation and appraisal work provided an adequate analytical framework for judging long-term trade-offs in the natural resource area between efficiency, equity, and sustainability. * In Nepal, the Bank focused on NRM issues later than other donors and failed, initially, to zero in on the fundamental issue of managing the country's natural resource endowment for the long term. Coordinated Consultative Group action could have improved NRM by linking it to macroeconomic policy through structural adjustment lending, public investment and expenditure reviews, and the national environmental action plan. Evaluation of more recently-approved projects will be needed to gauge how National Environment Action Plans the Bank is currently promoting have succeeded in remedying these deficiencies. * Important insights gained from Bank-borrower interactions were often not reflected in formal internal reports, with the result that lessons of experience had not been systematically applied to subsequent Bank operations, and opportunities to influence borrowers had been missed. In Bolivia, for example, the readiness of local institutions to implement NRM programs had been generally overestimated by the donor community, and the Bank's instruments of intervention in NRM had, consequently, proved to be weaker than expected at appraisal. * The Bank knew too little about the complex and sensitive relationships between socioeconomic and natural systems. * Improving NRM on a national scale would have required concessionary funding and software approaches which the Bank was not equipped to provide. * A lead role in improving NRM would have required comprehensive country-level overviews of NRM issues, donor-country coordination, utilization of additional financial and technical resources, and a greater borrower commitment to the resolution of these issues. xi xxii. Private Sector Development. A satisfactory framework for private sector development requires removing impediments to the free flow of labor and capital through such measures as the lowering of entrylexit barriers and the easing of controls on investment, financial, and labor markets. Parallel reforms in public enterprises and In the delivery of essential public services enhance the enabling environment while creating more space for private initiative. xxiii. The lessons of experience culled from the 1992 cohort suggest that headway has been greatest in refbrming incentives systems to promote competitive, efficient behavior in both put . and private enterprises through the liberalization of prices, the reduction of trade barriers, and the reduction of public subsidies. On the other hand, progress toward privatization has been less impressive (with the notable exception of the Mexico Public Enterprise Reform Loan). The cohort sample also illustrates how a project design which overlooks a key constraint on the public sector capacity to deliver essential public services is not likely to succeed. xxiv. As a general observation, shortcomings in meeting the Bank's thematic objectives among the sampled projects were most often associated with a narrowness of scope in project design, failure to monitor progress adequately and undertake timely mid-course corrections, and exogenous country events, such as political unrest, which undermined the borrower's commitment and/or the borrower's capacity to achieve thematic goals. Performance -Trends and Factors (Chapter 2) xxv. To ensure continuity with past Annual Reviews, Chapter I focuses on the performance of the projects evaluated during 1992. However, the time spans during which these projects were Implemented vary considerably and overlap heavily with the implementation of projects in past evaluation cohorts. These overlaps may obscure the influence of certain factors which vary systematically with the year in which loans were aproed. Among these are: * changes in Bank policies; * global economic trends; and * the growing number of "problem" projects which have entered recent evaluation cohorts after experiencing long delays in implementation. To better account for these factors, trends in portfolio performance are assessed in Chapter 2 by examining all 3,137 rated projects in OED's database, organizing them by the year in which they were approved. xxvi. The aggregate data on project outcomes and on trends in the RERR-ERR gap suggest that portfolio performance tracked according to the year of project approval has experienced three distinct phases: * an initial phase lasting from the late 1960s until 1973, during which the average share of evaluated projects rated satisfactory exceeded 80 percent, reaching a peak of 89 percent in 1969; * a second phase from 1974-79, during which a sharp deterioration in portfolio performance lowered the percentage share of "satisfactory" projects to the mid-to-upper 60s; and xii * the third and most recent phase, consisting of projects approved during the 1980s, during which "satisfactory" ratios have oscillated around 70 percent. xxvii. What factors might account for these variations in portfolio performance trends? Three potential causative factors are offered as hypotheses which require further research: * changes in the global environment and In country economic conditions. Given that all of the projects approved during the middle phase of serious portfolio design (1974-79) were appraised prior to the onset of the international oil price, interest rate, and recession shocks, their design was not able to anticipate the sharp fluctuations in relative prices and in aggregate demand which ensued. * the increased complexity of objectives incorporated into Bank projects approved since the early 1970s. Assuming a learning process, Implementation may have begun catching up with the innovative requirements of new-style lending by the early 1980s. * the sharp expansion in Bank lending during the 1970s which may have lowered project quality at entry. xxviii. Is it now fair to affirm that the declining performance trend of the late 1970s has "bottomed out," or is even in a phase of mild recovery? Considering that only about one-fifth of the projects approved after 1983 have been evaluated, and that, judging from past experience, the better-performing projects in a given approval year are likely to enter OED's database first, it would be premature to conclude that project performance has "turned the corner." Indeed, several OED simulations using the performance ratings of the first 10-25 percent of the projects evaluated for each approval year seem to point in a different direction, suggesting a possible decline in final average ratings for the cohorts of projects approved in 1984 and 1985 relative to the 1980-83 period, followed by a modest recovery. xxix. Regional, Country, and Sectoral Performance. Performance trends have varied significantly across regions, countries, and sectors. ECA was unique in that it was the only region in the Bank which maintained a stable and high level of project performance on loans approved throughout the 1970s. However, its performance has since deteriorated significantly. All other regions shared the common experience of a deteriorating portfolio of loans approved during the latter half of the 1970s, but divergent experiences with loans approved in the 1980s. In LAC and South Asia, portfolio performance continued to deteriorate; in EAP and MENA, it stabilized; and in Africa, it improved. xxx. Differences among countries are even more dramatic. For the 70 active borrowing member countries which had at least 15 completed Bank loans or credits through December 1992, average "satisfactory" ratios ranged from 100 percent in China to 45 percent in Peru and Sudan. Other high performing countries included Botswana, Chile, Costa Rica, Greece, Honduras, Jordan, South Korea, Mauritius, Myanmar, Pakistan, Paraguay, Sri Lanka, and Thailand. Other low performers included Algeria, Argentina, Haiti, Jamaica, Kenya, Liberia, Nigeria, Panama, Syria, Tanzania, and Zaire. xxxi. Another way of judging country performance is to measure changes in country portfolio perfcrmance over time. For this purpose, the performance of 46 countries which obtained at least five Bank/IDA loans in each of the three time periods discussed above was analyzed: * Countries with steady or improving performance throughout the three periods include Botswana, Chile, Indonesia, Korea, Malawi, Morocco, Sri Lanka, and Thailand. X111 * Countries withdeclining performance during the 1970s, but stableor improving performance in the 1980s include Burkina Faso, Colombia, COte d'Ivoire, Ecuador, Ethiopia, Ghana, Honduras, India, Jamatca, Jordan, Kenya, Madagascar, Malaysia, Mali, Nepal, the Philippines, Senegal, Tanzania, Tunisia, Turkey, and Yemen. * Countries with steady or improving performance during the 1970s, but declining performance during the 1980s include Bangladesh, Cyprus, Pakistan, Papua New Guinea, and the former Yugoslavia. * Finally, countries with declining performance throughout are Argentina, Brazil, Cameroon, Liberia, Mexico, Nigeria, Panama, Peru, Sudan, Zaire, and Zambia. Of course, comparisons of country performance over time do not fully reflect the extent to which countries also operate at different Imalb of performance. Both measures of performance are relevant. xxxii. Sectors exhibiting above-average levels of performance historically include telecommunications (90 percent satisfactory), power and energy (87 percent), transport and urban development (82 percent), and human resources (79 percent). Agriculture (64 percent), tourism (64 percent), free-standing technical assistance (67 percent), and industry (68 percent) were below-average. In terms of trends, among sectors having five or more projects in each of the subperiods used, only urban development maintained a stable and high level of performance throughout. Only transport and energy succeeded in raising performance ratings significantly on projects approved during the early 1980s, relative to projects from the late 1970s. The finance and human resource sectors managed to stabilize project performance in the early 1980s, after a decline in the 1974-79 approval year cohorts. The remaining six sectors exhibited a continuous decline in performance ratings. xxxill. Portfolio Composition. Starting in the early 1970s, the Bank's lending portfolio underwent progressive diversification. The focus shifted away from physical infrastructure and large Industrial projects toward inter alia poverty alleviation and coping with the debt and energy shocks. The question this raises is whether the changing country and sectoral composition could have conceivably accounted for the changes in portfolio performance over time. To address this query, overall portfolio trends were compared with 'normalized" trends. The results indicate that neither changes in the country nor in the sectoral composition have had any material long-term impact on portfolio performance trends. In other words, performance ratings moved as they did largely as a consequence of shifts in performance within, rather than across, regions and sectors. xxxiv. Performance Factors. Project performance is a finction of: * tie external environment and exogenous shocks; * country conditions; and * project processing. xxxv. External Environment and Exogenous Shocks. External shocks appear to play a much larger role in explaining disparities in long-term, developing country growth rates than had been previously acknowledged. Usually, the magnitude of external shocks is not as critical as the timeliness and the appropriateness of the policy responses to them. xiv xxxvi. Among the most important external shocks during the period of 1973-90 was the terms of trade shock. Favorable or stable terms of trade were associated with above-average performance in over 40 percent of the country portfolios tracked in OED's database. On the other hand, the portfolios of Burkina Faso, Cameroon, Congo, Nigeria, and Zaire were adversely affected by their deteriorating terms of trade. The lack of diversity and flexibility in the structure of their economies may partially explain their greater vulnerability to terms of trade shocks. Conversely, despite a deterioration in their terms of trade, the portfolios of Chile, Colombia, Ghana, Malawi, Malaysia, and Thailand achieved above-average performance. Finally, some portfolios, such as those in Bangladesh, Brazil, Haiti, Kenya, and Niger, did worse, despite improving or stable terms of trade. xxxvil. Country Conditions. The importance of a stable international economic environment notwithstanding, the predominant stimulus for good project performance is generated from within the country in the form of sound macroeconomic management and structural adjustment. Successful structural/sectoral adjustment has the coll4teral effect of markedly improving the performance of Investment projects, a result confirmed by the high degree of correlation between ratings of investment and adjustment operations OED found in a sample of 51 countries. xxxviii. Further analysis identified the rates of economic output growth and domestic inflation, and the degree of human resource development, as highly-significant variables associated with portfolio performance. Benin, Burundi, Cdte d'Ivoire, Ethiopia, Ghana, Lesotho, Rwanda, and Zimbabwe were all identified as countries which achieved above-average portfolio performance, despite low indices of human development. At the other extreme, Congo and Kenya experienced belov'-average portfolio performances, despite moderately-high GDP growth, relatively stable inflation, and better-than-regional average levels of human development. These findings also applied to adjustment operations. For example, the overall human development index of the ten successful adjusting countries in the 1992 evaluation cohort was 0.54, compared to 0.23 for the eight unsuccessful adjusters. xxxix. Borrower commitment/ownership of projects and programs is often cited as vital to policy credibility and project sustainability. In a review of over one thousand investment projects approved during the 1980s, OED found that virtually all cases of highly-satisfactory outcomes (such as in Chile, Indonesia, Korea, Mauritius, and Morocco) were marked by a strong commitment to project objectives, including to their policy and institutional goals. Another OED study of 100 completed adjustment operations also found that borrower ownership was strongly associated with overall program outcomes in 73 percent of the cases. xl. Finally, a stable political environment provides long-range planning horizons for core ministries, thereby ensuring continuity. During various periods of the past, political instability in countries such as Bolivia, Chad, Nigeria, the Philippines, and Uganda proved to be a major impediment to successful project implementation. xiI. Processing Factors. Empirical evidence drawn from 545 projects in OED's 1991 and 1992 evaluation cohorts reinforces the importance of quality processing. Specifically, the proportion of satisfactory outcomes is shown to increase consistently with the adequacy of identification, preparation, appraisal, and supervision. Equally important, when early stages of project processing are deficient, quality monitoring and supervision can make a crucial difference in project outcomes. The role of high- quality and deficient project processing in determining project outcomes is illustrated with examples from the 1992 evaluation cohort. xli. Unkages with Review of Ongoing Projects. There has been a widening gap between the ARPP and PCR-PAR ratings since the early 1980s. However, since the projects in the current ARPP and xv OED cohorts are not strictly comparable, a joint OPRIOED study was undertaken to compare ratings on 926 completed investment projects and 104 completed adjustment operations approved since FY79. xilli. The study revealed that 21 percent of the projects had ratings "disconnects." Seventeen percent were rated nonproblem projects by ARPP during the final year of supervision, but unsatisfactory by the PCR-PAR at completion; 4 percent were rated as problem projects by ARPP but satisfactory by the PCR-PAR. By region, the largest "disconnect" was in LAC and Africa. For countries with ten or more rated projects, the largest "disconnect" occurred on projects in Nigeria, Zaire, Brazil, Madagascar, and Bangladesh. By sector, the "disconnect" was largest for technical assistance, agriculture, industry, and water supply and sanitation. For adjustment operations, the "disconnect" was 31 percent. xliv. Major conclusions of the study are: * There are wide variations in the "disconnect" between ARPP and PCR-PAR ratings for the same countries and sectors, and the "disconnect" for adjustment operations is larger than for investment operations. * The ARPP overall status ratings seem to be dominated by short-term concerns about project implementation, while ARPP ratings on development objectives reflect longer-term optimism that objectives will be achieved once implementation problems are resolved. * Excessive ARPP ratings optimism during the early years of implementation indicates that risks at project entry are not being realistically identified and appraised. The AgricuuIral Portfolilo: 1970-8 (Chapter 3) xlv. The assessment of agricultural projects approved during 1970-85 is of particular interest since these were simultaneously the years of most rapid growth in agricultural lending and of serious deterioration in agricultural project outcomes. xlvi. Evolution of World Bank Agricultural Lending. Early lending was almost exclusively production-oriented. For example, mechanization, land reclamation, and irrigation accounted for over 90 percent of agricultural lending in 1952. During the period 1961-66, a single category-irrigation and flood control-accounted for 68 percent of Bank/IDA lending. In 1969, the Board was still approving large credits to India and Pakistan to import tractors and smallholders were far from central to the Bank's strategy fbr agricultural development. By contrast, today agricultural projects lie at the intersection of the thematic priorities of the Bank, be it poverty reduction, beneficiary participation, environmental protection, or private sector development. xlvii. The McNamara Presidency (1968-80) brought far-reaching changes, including a strong emphasis on reducing mass poverty. Lending expanded very rapidly, particularly agricultural lending whose share in total Bank/IDA lending increased from 11 percent during the years 1961-65 to a peak at over 37 percent in 1978. The worldwide attack on poverty also brought new approaches to the fore, notably integrated rural and area development projects. These did not exist as lending categories in 1965, but, for the 1970-85 period, they accounted for more than a quarter of all agricultural projects. xlviii. By the mid-1980s, however, performance audits were signaling that the majority of area development projects was not succeeding. Several years later, Irrigation projects also began manifesting alarming difficulties. Through 1987, OED had rated over 80 percent of completed irrigation and drainage operations as satisfactory. But in the 1989 cohort, the satisfactory rating fell to 46 percent. xvi Unresolved institutional problems-and problems of rural governance-were at the core of the failure to adequately maintain and operate irrigation schemes. Sustainability came increasingly into question, as projects rated satisfactory at completion broke down a few years after. The enabling environment was also subjected to scrutiny, as experts worldwide began to question the impact of administered pricing, governmental regulation, and the reliance on parastatal organizations for inputs and other marketing support. xlix. Following reorganization in 1987, there was a pronounced decrease in the volume of Bank leading to agriculture, partly as a reaction to its poor track record. By the end of the eighties, the Bank's agricultural staff was searching for new lending approaches. In an effort to improve project outcomes, the "check list" of items to be considered in the typical agricultural project grew longer. These included the soundness of the macroeconomic framework, the degree of government commitment, weaknesses in public support services and in sectoral policies, realistic approaches for dealing with price and weather risks, beneficiary participation at preparation, and environmental protection. I. Performance of the Agricultural Portfolio, 1970-85. The quantitative record reflects this history and shows that performance deteriorated dramatically for agricultural projects approved after 1977. What explains this deterioration? Two reasons stand out: unfavorable borrower incentives, project factors like low borrower commitment, and what the Wapenhans Task Force described as the Bank's "lending culture" which adversely affected the quality of project preparation and implementation. I. Borrower Incentives. To invest, farmers need favorable and dependable output-input price relationships. Unfortunately, agricultural prices peaked in 1977, and fell more or less continuously through 1992. The success of many commodity projects (cotton, palm oil, rice) was dependent upon realizing producer prices around the levels the Bank had forecast, levels which usually proved to be over- optimistic. The shortfall in realized prices not only discouraged farmer investment, but also reduced government revenues in many low-income agrarian countries. This, in turn, led to cuts in recurrent funding across the board, e.g., for area development projects. In addition, in many countries, the macroeconomic policy framework exhibited an increasing bias against agriculture over time, with the result that their domestic terms of trade moved decisively against agriculture in the late 1970s. lii. The Project Factor and the Bank's Role. In addition to the factors identified in the Portfolio Management Task Force Report as contributing to a secular deterioration in overall portfolio performance, agriculture was burdened with peculiar weaknesses, including excessive complexity of project design, inadequate technical packages, and inattention to climatic risks. The area development projects epitomized these shortcomings. Their performance ratings fell from 61 percent satisfactory during 1970-77, to only 40 percent in 1978-85. Since area development projects constituted a quarter of all projects in agriculture during these periods, their deterioration explains half of the overall decline in agricultural performance. liii. Agriculture and Other Lending. Over 1970-85, agriculture project performance also lagged behind that of other sectors by an average of 14 percentage points. Several factors help explain this difference: * Because of the importance of agro-technical problems and social and cultural variables, agricultural projects may have been more difficult to "get right" than the average non- agricultural project; xvii * The *public production model" proved to be a larger failure for agriculture, which is market-based and incentives-driven, than for certain kinds of infrastructure (e.g., highways, water supply) and the social services; * Because of their reliance on the market, agricultural projects were more vulnerable to poor macroeconomic management than many other sectors. liv. The combined impact of these factors can be illustrated by considering the differential impact of lending to Africa on ratings for agriculture relative to ratings on all projects. In particular, Sub- Saharan Africa stands out as the region whose economies (a) are most heavily dependent on a few commodity export crops (and on their world prices), (b) have the weakest economic policies, and, (c) exhibited a pervasive dedication to public production approaches during this period. If Africa projects were excluded from the calculation of average performance ratings for all lending, not just for agriculture, then the shortfall in agriculture's performance relative to other sectors would shrink from 14 percentage points to 10. Eliminating the impact of ratings for area development projects as well would reduce the difference to only 6 points. lV. Sector Lending Experience. Four subsectors accounted for three-quarters of Bank/IDA lending in agriculture over 1970-85: irrigation (29 percent), farm credit (21 percent), area development (17 percent), and plantation crops (7 percent). The conclusions from recent OED studies on each of these subsectors are synthesized below. Ivi. Irrigation. The Bank's activities in irrigation have been shaped by making loans rather than by formulating explicit policies. Until recently, no irrigation policy paper had been published, and few operational directives existed. Beyond project documents, Bank literature on irrigation is sparse and does not include in-depth discussions of the thorny problems which affect the subsector. Ivii. Project evaluations report pervasive design problems. Typically, irrigation projects have aimed at developing systems with high delivery capacities, capable of on-off operation that can move water on demand throughout the canals. In the humid tropics, these highly "reticulated" systems have been plagued by vandalism (destruction of gates) and "water-hogging" by irrigators at the canal heads, with disastrous consequences for those at the tail-end of the network. What to do about this has not been completely resolved, but, undoubtedly, more realistic assumptions at the design stage are needed. Iviii. A second problem involves failure to maintain adequate drainage. The ensuing waterlogging and salinization have forced as much as a fifth of arable irrigation land out of production in many areas. Finally, operation and maintenance (O&M) of the canals and head-works are often seriously inadequate. While one of the reasons for inadequate O&M is insufficient finance, poor management is also common. The Bank's initial enthusiasm for irrigation cost recovery stemmed from the presumed link between cost recovery and better O&M. But OED evaluations showed there was often little relationship, because water use fees were channelled into general budgetary revenues for which O&M had to compete with other claims. Therefore, many recent studies have concluded that financial and managerial autonomy could greatly improve irrigation O&M. Consequently, the Bank now promotes water-user groups and the hand- over of systems to them. But the institutional requirements of this approach are exacting, which may help to explain why the results to date have been mixed. lix. Nevertheless, the evaluation record shows that Bank-supported irrigation developments have made an impressive contribution to equity, greatly reducing malnutrition and spurring economic growth. Yet, dealing adequately with the problems identified above will be even more critical in the future, since xviii rural populations continue to grow and it is no longer possible to respoLd primarily by developing new irrigation schemes, because the best sites have already been developed. lx. Farm Credit. Completed agricultural credit and rural finance projects achieved an 81 percent satisfactory rating over the review period. Although the central rationale for credit remains the financing of new technologies for farmers capable of using them profitably, newer projects have accorded greater emphasis to ensuring the viability of financing intermediaries, generating rural savings, and, most recently, promoting beneficiary participation. lxi. These changes respond to criticisms that Bank agricultural credit projects have not dealt adequately with nonviable Intermediaries, have acquiesced in government pressures to lend at subsidized and negative real interest rates, and have tolerated lax loan repayment practices. Because of these changes, the number of new credit operations declined sharply after 1983. Subsequently, Operational Directive 8.30, issued in 1992, institutionalized restrictive conditions, which few agricultural credit operations can fully meet. Yet, agricultural credit projects have generally met their farm production targets, have helped millions of farmers, have improved intermediaries via Bank-supported programs to strengthen management, have played an important role In funding technological change, and have promoted long-run food security. While a return to "business as usual" practices would not be warranted given the need to improve the quality of overall financial sector management, increased Bank support for agricultural credit should be encouraged, including appropriate institutional development support. lxii. Area Development. OED studies found that area development projects were excessively complex, hastily prepared, and frequently located in fragile agro-ecological areas. Poor farmer incentives, inappropriate production technology, inadequate institutional capacity, and weak governmental commitment were also identified as major factors in their poor performance. lxiii. Notwithstanding the poor overall track record, area development projects have worked very effectively when there is strong local management and careful preparation. Recent OED evaluations in Africa reveal a significant improvement in performance for the simpler, second-generation projects initiated in the early 1980s. Indeed, with several African countries now adopting more positive economic policies, and the Bank's new approaches emphasizing beneficiary participation and work with nongovernmental organizations (NGOs), it might be appropriate to review opportunities to apply the lessons of experience and, with improved quality, increase lending for area development in Africa and in other regions with similar needs. lxiv. Plantation Crops. In the 1970s, the Bank greatly restricted lending for the beverage crops (coffee, cocoa, tea) because of world-market conditions: highly inelastic price and income demands. But as the Bank withdrew from lending, other lenders replaced it. Prices continued to fall, as production and exports increased. But very little increase came from the Bank's weakest borrowers, notably in Sub- Saharan Africa, whose production of beverage crops has largely stagnated. lxv. OED's recent report recommends that the Bank abandon its policy of general restrictions on plantation crop lending. This does not imply that there should be a sharp increment of Bank lending for these commodities, given the pessimistic projections for future commodity prices. But such a change could prove helpful to some smaller countries whose export production has declined in volume and quality, notwithstanding their desperate need for foreign exchange. lxvi. Such changes in the Bank's approach to agricultural lending, if conscientiously implemented, should lead to more successful project outcomes than in the past. It is imperative that the Bank devote xix the administrative and managerial resources needed to resolve the specific subsector issues raised above so as to carry out its basic development mandate more effectively. Institutional Development (Chapter 4) lxvii. The Bank's Role in ID. Since its inception, the Bank has been concerned about the need to ensure the efficient management of the investments it assists. Whereas Its initial technical assistance (TA) support was largely investment oriented, the Bank now considers that investment sustainability is seriously jeopardized without institutional effectiveness. Therefore, broader institutional development- related issues such as national economic management and civil service reform have become the focus of the Bank's operational involvement. lxviii. Decisions concerning policy and institutional reform rest with the borrowers themselves, and their willingness and capacity to undertake such reforms is influenced as much by cultural and socio- political factors and commitment to change, as their own appreciation ef the need to strengthen the institutional foundations of their economies. The Bank can assist through its country dialogue, economic and sector work and such operational interventions as policy-based lending, but its involvement is constrained by its limited knowledge of cultural and socio-economic factors and by considerations of the borrowers' sovereignty. While the Bank does not interfere with the functioning of governments, it nevertheless seeks to reassure itself that the overall country environment is conducive to effective development in terms of institutions, policies, rules, and procedures; where it is not, it becomes incumbent on the Bank to engage the borrower on broader ID issues. Adjustment operations have provided the opportunity to review selective governance aspects related to key macroeconomic and structural obstacles to development, but the longer timeframe needed for ID reform is not well suited to fast disbursing operations, and parallel TA loans were therefore initiated as a means of enhancing borrower capacity. The policy reform and institutional capacity-building features of these loans-which translate into improved sector and national economic management-are also now becoming a part of investment projects and sector loans. As the Bank broadens its sectoral agenda with emphasis on poverty reduction, environmental management, and an enhanced role for the private sector, it appears likely that it will be called upon to sustain and expand its TA support for ID. However, relative to organizational improvements which the Bank has effectively assisted, ID is more complex, and much dependent on the borrowers' responsiveness. lxix. The changing mix of ID objectives contributes to the difficulty of interpreting aggregate ID ratings. Unlike investment related TA, there are no simple, measurable indicators, and hence judgments frequently have to be qualitative and subjective. The growing complexity and the wide sectoral range of institutional objectives also contribute to the unpredictability of their eventual outcomes. Two other factors-the multiplicity of instruments used and the large numbers of bilateral and multilateral donors involved in assisting ID-add their own complications. lxx. ID Performance. Data relating to the achievement of institutional objectives in the Bank's operations approved during the 1970s and early 1980s have been assembled systematically since 1989. There is a high correlation between ID achievements and ultimate project outcomes, and a similar correlation is noted between ID and project sustainability. No significant trend in ID achievements has emerged among the projects evaluated during 1989-92, but in none of these years have "substantial" ID achievements been recorded by more than a third of them. The principal reason for this is the increase in the share of projects whose objectives go well beyond organizational improvements at the project level. Given the inherent complexity of projects with a dominant ID thrust and the learning process the Bank is currently going through in this relatively new area of involvement, it is not surprising to witness a rise in the risks of intervention. Even so, the absolute level of ratings offers no grounds for complacency. xx The findings suggest that a greater degree of realism is needed in fixing the parameters of the projects with particular attention to project design, and also to borrower capacity and priorities. lxxi. Issues. The continuing problems associated with ID projects stem in part from capacity limitations within the Bank, but also from inadequacies on the side of the borrower. Among the less satisfactory aspects within the Bank are the uneven quality of the institutional analysis, the lack of a strategic context for ID, deficiencies in design including inadequate recognition of the limited implementation capacities of some borrowers, too many components, an approach which in the past tended to be rigid, lack of Indicators to measure progress and shortcomings in supervision. The more common problems among borrowers are an inadequate appreciation of the importance of ID for sustained development, little or no sense of ownership of ID operations, reluctance to accept what is perceived as an outside agency conceptualizing and shaping the institutional agenda, and a belief that Bank staff may be insufficiently sensitive to local cultural factors. lxxii. Sector Work. There is increasing evidence that operations that support ID are rarely based on adequate sector work, and often lack a strategic context. While the great majority of the 1992 cohort were preceded by sector work, it is the uneven quality of the institutional analysis that is a source of concern. Despite the recognition that institutional analysis is a key element, the resources deployed to support and expand it operationally have been modest. While the number of staff with ID skills has increased somewhat and staff awareness of ID issues has grown, progress is still inadequate relative to the growing need for ID assistance. lxxiii. Design. Three distinct elements of project design have a bearing on the ultimate outcome: the number and complexity of the components, their sequence and phasing, and the borrower's absorptive capacity. It is axiomatic that the full range of components be acceptable to the borrower and that project design is consistent with efforts already underway :.ad with initiatives supported by others. Appropriate specificity of objectives, end products, inputs, and sequencing is essential. lxxiv. Specificity versus flexibility in project design are not mutually exclusive. The former promotes clarity of objectives and introduces greater discipline. On the other hand, flexibility permits constructive change and allows all parties to be responsive to emerging requirements in a changing environment. The "blueprint* approach to design, while particularly appropriate for physical investment projects, is not equally suitable for projects in which the principal focus is ID, because of, possible political factors, changing priorities, instability of institutions, arbitrary and unpredictable personnel changes. In such situations, the flexible "process" approach is more appropriate. On balance, a need for greater Bank flexibility to adjust or redesign ID components after project approval has been an important lesson of experience, but the final choice should be influenced by the kind of activity and the borrower's capacity. It is, therefore, necessary to validate a conscious institutional design strategy during the early stages of project identification and preparation, which should be elaborated and confirmed during appraisal and negotiations. lxxv. Indicators. The systematic monitoring of progress against explicit performance criteria or concrete or dated targets has rarely been provided for in ID projects or for ID components, making it difficult to form judgments on whether objectives have been reached or individual components succeeded or failed. As a result, many ID issues repeatedly identified in successive projects remain essentially unresolved. However, new ground is being broken in this regard and some projects now incorporate imaginative performance indicators, features that will probably impart greater credibility to the ID components from the borrowers' perspective. xxi lxxvI. Supervision. The supervision of projects or components directed at ID requires staff skills and numbers which, because of the rapid increase in such projects under Implementation, are in short supply. ID projects require close monitoring to facilitate early recognition when a component is faltering or when an objective is becoming unrealistic with regard to content or timing. Appropriate, achievable alternatives need to be developed in close consultation with the borrower, and the projects' consultant expertise and equipment needs to be realigned and redeployed within the framework of the original legal agreement and financial provision. A review of operations in the 1992 cohort principally directed at ID shows that the time devoted to their supervision, the skills deployed, and the frequency of supervision were subject to considerable variation. The limited amount of information available on skills employed or needed makes it difficult to conclude that missions lacked skills that were sufficiently representative; however, nearly half the missions consisted of only one person which, given the multiple components and ID focus, may not have been adequate. lxxvii. Borrower Commitment. Among the key requirements for the success of Bank-assisted ID is the borrow2r's unequivocal recognition that the ID is needed to meet an existing deficiency, that it serves high priority objectives, and that local capabilities are inadequate. Where this is the case, TA is demand driven, the initiative for seeking assistance being clearly with the borrower and hence more likely to be successful. Empirical criteria developed by OED for assessing borrower ownership, when applied to adjustment projects in the 1992 cohort, demonstrated that a very high ownership index correlated closely with projects with a satisfactory rating, and that where the ownership rating was very lo%, the performance was unsatisfactory. lxxviii. New versus Existing Organizations. New organizations have been created in projects assisted by the Bank when new objectives and functions could overload an existing line agency. In these cases, the Bank's usual response in the past was to create project management units (PMU), often outside the ministry concerned. However, the desirability of PMUs has long been questioned when they were established outside of ministries, were not closely linked to line agencies or required considerable numbers of expatriate staff. Therefore, the objective has increasingly become the long-term strengthening of a line agency rather than the successful implementation of the particular project on hand. In the broader context of ID, the Bank has assisted in the creation of new organizations, the mandates of which Pre much wider than that of a PMU in an Investment project. Some of these have been successful, and others not. When the decision to create a new organization has not in itself been the cause of success or failure, it is clear that success depends on the borrower's willingness to provide it with a clear mandate, and also financial and managerial autonomy. lxxix. Instruments. The choice between a freestanding ID project and ID components in an investment project is linked to the range of objectives and whether a multiplicity of institutions is involved. The former is usually preferred in broad-based ID operations which involve many agencies with divergent responsibilities. Conditionality, when employed in the context of adjustment operations, needs to be clearly formulated, genuinely accepted by the borrower, closely monitored, have a realistic timeframe, and carry an unambiguous understanding that noncompliance could lead to the use of remedies. Because of the lack of precision in some conditicaality, the ID outcome has often been unsatisfactory. lxxx. When it is determined in consultation with the borrower that ID requires a TA input, the Instruments and precise methods of TA to be used should be decided in advance through a consideration of alternatives. In the 1992 cohort, expatriate consultants were used three times more frequently than local consultants, and long-term consultancies were more than twice those of short-term consultancies. Training components were an intensively used instrument of ID and very considerable numbers of individuals were trained, although the quality of the training could not always be ascertained. Numerous xxil studies were included, but information on their utilization was inadequate. Twinning and the participation of NGOs was surprisingly limited. Borrower acceptance of the choice of instruments Is critical for the success of ID. lxxxi. Conclusions. The Bank's response to continued weaknesses in public sector organizations, and the hitherto limited scope of the private sector in relation to its potential, has been a new series of free-standing loans aimed at capacity building, restructuring and divestiture of public enterprises, and facilitating a bigger role for the private sector. The major thrust of ID lending is the Improvement in the management of public sector organizations in its broadest form, in which notable progress has been achieved. However, the Bank has had less success in addressing some of the culturally Important determinants of governments' performance, in part because its institutional analysis has lacked incisiveness owing to the limited access to those areas of the borrower's domain which are considered privileged, and which raise governance issues over which the Bank has no jurisdiction. While respecting the borrower's sovereignty, the Bank is concerned that, unless borrowers become more closely involved in addressing ID-related issues, the sustainability of the projects it supports may be jeopardized. lxxxii. OED's evaluation of ID shows a disquietingly poor impact. However, there is a growing reservoir of experience and knowledge in the Bank which Is available for improving the performance of ID operations, and several new instruments-the Institutional Development Fund, a revised Operational Directive and a Handbook on Technical Assistance-are now in place. A conscious utilization of these instruments along with progress in other areas-such as research and training-could contribute to an increasing effectiveness in the Banks assistance for ID. 1. EVALUATING OPERATIONAL PERFORMANCE A. TME 1992 REVIEW 1.01 Tids Is the nineteenth Annual Review of Evaluation Results. Like its predecessors, it is based on Project Completion Reports (PCRs) prepared by Bank operational staff and Performance Audit Reports (PARs) undertaken by the Operations Evaluation Department (OED). Fldings from recent OED studies are also tapped. Following past practice, this review begins by (a) reporting on the performance and key lessons learned from the investment projects and adjustment operations evaluated during CY92. This is followed by (b) an assessment of long-term trends in portfolIo performance and of selected portfolio management issues (Chapter 2) and (c) by an overview of agriculture project performance trends (Chapter 3).1 The review concludes with an examination of a particularly significant criterion of performance-lastitutional development-which must be addressed more successfully if operational outcomes are to be significantly improved (Chapter 4). 1.02 This chapter discusses major project rating-outcomes, institutional development, and sustainability-nd how they interrelate. Other aspects of project Implementation, such as physical achievement and time and cost variations, are reported in Section B. In Section C, the n of cohort performance is broadened to cover the thematic perspectives of paverty reduction, environmental impact, and private sector development. Conclusions are provided in Section D. B. THE 1992 COHORT Project Outcomes 1.03 The Bank evaluates all lending operations by examining their implementation and initial results usually within a year or two after physical completion, and anywhere from two to twenty years after Board approval. Hence, the conclusions presented below about project outcomes cannot be extrapolated directly to an assessment of the Bank's a=e portfolio of projects unless account is taken of intervening changes In global, country, and project factors which may have substantially modified the potential development Impact of more-recently approved projects. 1.04 Whenever possible, an economic rate of return (ERR) is calculated. It is a major criterion for projects whose principal costs and benefits can be readily quantified. However, ERRs I See, in particular, the Operations Policy Depatment 'Portfolio Management Task Face (PMTP) Repost, Effective Implmentaion: Key to Development Impact, Repott No. 11.6, September 1992, and 'Portfolio Management Net Steps-A Program of Actions, Repot No. 5228, June 15, 1993. 2 are not calculated for policy-based operations, technical assistance, financial iatermediation, or for a majority of human resource operations, but are done for most operations in the remaining sectors. When ERRs are calculated, they are compared with the opportunity cost of capital, which Is normally set at or above 10 percent, the minimum threshold embedded In the Bank's appraisal methodology. 1.05 Besides re-estimating rates of return, the evaluator takes into account nonquantiflable benefits and costs. This includes gauging the relevance of the project to country and sector priorities, the effect of policy distortions, the degree of achievement of physical objectives, the actual and expected capital and recurrent costs, and the economic, financial, social, environmental, and institutional justification. 1.06 When it is not feasible to estimate a rate of return-the case for nearly 70 percent of all projects evaluated-cost-effectiveness Indicators are considered to determine whether there were more efficient ways to meet a validated level of effective demand (e.g., least-cost assessment for a utility project). For a policy-based loan or an institutional development project, the benefits of the reform are assessed in relation to the costs Incurred (e.g., the social cost of labor retrenchment). 1.07 Based on the above approach, the outcome of each operation is compared with Its objectives at the time of Board approval, taking into account major changes that may have been agreed on during execution. OED audits occur at a later date. While the analytical emphases of PCRs and audits often vary significantly, overall differences between PAR and PCR performance ratings at the time of audit are not large, and often simply reflect a more recent assessment of project outcomes. Impact evaluations which are generally performed five to ten years after completion for a much smaller number of operations permit a re-examination of project results over a longer time horizon. Such evaluations focus on the environmental and social impacts of Bank operations. 1.08 There were 87 cohort operations In sectors for which economic rates of return were available at the time of project completion (Figure 1.1 and Annex Table 1.1). The average re- etmated ERR (RSRR), weighted by actual project cost to indicate the return on investment, was 13 percent, well below the average RERR of 16 percent on the 1991 cohort, and the average RERR of 15 percent for all cohorts combined. This decline in RERRs reflected in part the greater weight of comparatively low-yielding industrial projects (average RERR of 5 percent) plus the 6 percentage point decline In the average RERR for normally high-yielding transport projects, relative to the 1991 cohort.3 1.09 The historically close relationship between project yield and project outcome was maintained (see Annex Table 1.2). For example, in the 1992 evaluation cohort, only 8 percent of all 2 OED aediled 251 pajects in 1988-92 whose PCRa were included in earlier Annual Review cohorts. In 86 pereat of then eases, PAR-based atings were the same as those deived from the PCR. Among the remaining projects, 26 percentofthe ating changed from unatisfactory to satisfactory on the basis of the PAR, while 75 percent moved in the other direction. The net shit was 17 projects, or Just 7 percent of the total. Inimany of the cases where changes in ratings have eourvd, ro-evaluated projects were included in Ocluster audits several-in some instances, seven to eight-years ater the PCRs wee -rnted. For a dissussion of th treads in th RERR-ERR gap on an aproval = basis, see Chapter 2, Figure 2.3 and the salated 3 low-yieldfg projects (RERR under the 10 percent benchmark) received a satisfactory rating. Conversely, only 5 percent of the high-yielding projects (RERR over 10 percent) were rated corY. These percentages are very much in line with the comparable percentages in the historical cohort (9 and 3 percen, respectively). In terms of sectoral distribution, the highest occurrence of divergence between the RERR and the overall performance rating In the historical cohort has been in power, where one-quarter of 44 low-yielding projects achieved satisfactory ratings, and water and sanitation, where over half of 40 low-yielding projects were similarly rated. In general, such divergence reflects the impact of repressed utility rates and difficulties in measuring consumer surpluses. In regional terms, the highest ratings divergences were in Latin America and the Caribbean (LAC) and Middle East and North Africa (MENA) (18 percent), while the lowest were In East Asia and Pacific (AP) and South Asia (SAS) (8 percent). Figure 1.1: Economic Rates of Return, Sectoral RERRs, 1992 Evaluation Year vs. 1974-91 Cohorts Power AVERAGE Urban Telco iatnsp Energy 0 10 20 30 40 Percent Rate of Return Source: Anna Table 1.1 1974-91 1992 1.10 The operations evaluated during 1992 (277) were nearly equal in number to last year's cohort (278), but substantially larger in the value of commitments (US$17.6 billion in 1992, US$14.0 billion in 1991, Annex Table 1.3). Most of the loans were implemented during the 1980s (Figure 1.2 and Annex Table 1.4). The following assessment is based on PCRs for 237 operations, project completion notes (PCN) for two, and PARs for 38. During 1992, OED also audited 78 operations, 4 the PCRs of which had been submitted to the Board In earlier years, and, thus, were included in previous Annual Review cohorts.' Figure 1.2: Distribution of Operations Evaluated in 1992, by Year of Approval and Closing so 60 140 1 20 20 0 1 * 197 * 199 181 1983 1985 1987 1989 19 1 1976 1978 1980 1982 1984 1986 1988 1990 1992 Year of Approval/Closing Source:n= WdxTe 1.4 -0- Year ofApproval eso Yewr of Closing 1.11 Figures 1.3 and 1.4 display the sectoral and regional distribution of all operations in the 1992 cohort and in the universe of evaluated projects. The main sectoral differences are in the larger shares of program and policy, finance, energy, urban, and water and sanitation in the 1992 cohort relative to the universe. In ttrms of regions, projects in Africa and In East Asia and Pacific are significantly larger in this cohort. For 239 projects, the ratings were based entirely on the findings of the PCR or PCN. For an additional 38 projects, a PAR was submitted jointly with the PCR to the Board during CY92. Ratings of the PAR superseded dse of tho PCR for these latter projects. In addition, PARs were completed in 1992 for 78 projects submitted to tho Board during previous calendar years. Ratings of these projects from older evaluation cohorts were revised, as necessary, to rafloc" the dings of OEDs audits. The updates of ratings made for this review have not materially changed the conclusions discussed below about trends in project performance. .: � � �'" л $ �� � �� � �^ � F, L^� � � � � � � ��� � � � �� � � � � ���� r и „'�s Pi` д 1 � �� � и � ь4i ` У� � �+ ^;", � � � �5�� � � ��i 3 . f7f и � и У � � � ��� ..f ' �� � � � ��� � �� � и �l ,�у �L�- r+ д+ � � ��� н `t � � � , �� и +� �' � ,. � 'р � � „ й . .. � � �'3 � - � ы �, � � � � � � � � � � � � . � 6 Figure 1.4: Distribution of Commitments by Region r-mam& ad Naffi ~ (5. 1%) 2~=~~2%) Latå A~ ud Ca~ (27.5%) ........... 'w" co ............. 1992 Ltap^aiomdcu~cxo%) ma (1&0%) -- ------------------ EMAsåmdp~(2z1%)-ý-- So= A~ TaMc 1.3 1974-91 7 1.12 For the first time since 1988, the proportion of projects rated as having a satisfactory outcome improved over the previous year's cohort (Figure .1.5 and Table 1.1). Altogether, 183 projects (67 percent of the rated cohort) are judged to have had satisfactory outcomes. While sigalficantly above the ratio of 62 percent registered in the 1990-91 evaluation years, It Is the third lowest share recorded since the series began In 1974. On the other hand, the "commitment success' rate-or the proportion of total Bank and International Development Association (IDA) commitments allocated to satisfactory projects-at 75 percent is substantially higher than the rates of 66 percent in 1991, and 58 percent In 1990. It Is broadly In line with the historical average of 74 percent for the 1974-91 period (Annex Table 1.5). Figure 1.5: Performance of Operations Evaluated in 1974-92 100 60 40 20 0 1974 1976 1979 1980 1982 1984 1986 1988 1990 19M 1975 1977 1979 1981 1983 198S 1987 1989 1991 Evakuatlon Year Source: Table 1.1 1.13 Performance of larger operations showed systematic Improvement for the second consecutive year (Annex Table 1.6). The satisfactory ratio for 57 rated large-scale projects (US$100 million or more) was 81 percent. In the four largest sectors, which accounted for two-thirds of all large-scale lending, performance was above the average for the 1992 cohort as a whole: In agriculture, where 15 projects achieved an average satisfactory ratio of 86 percent; In program and policy, where 10 projects achieved an 80 percent rating; in finance, whose seven large-scale projects were all rated satisfactory; and in power, where seven projects achieved a 71 percent ratio. 8 1.14 Geographically. large-scale projects were distributed more evenly among the reglons, with LAC bavlag 28 percent, BAP 26 percent, and Africa (APR) 19 percent. hIs was In contrast to past experence, when half or more of all large project -lmmitmeni were concentrated in LAC, whure satdn~nry out~mes tended to fal somewhat below cohort averages, as was the ca in 1992 (6 percent). In the 1992 cohort, four regons did well above average, Including BAP (100 percent), MENA (100 percent), APR (82 percent), and Europe and Central Asia (ECA) (80 percent). Other than this shift la the regional distributon of large-scale loams, the size of the loan commitment per se has not proved to be a reoabl predltor of projeot perf"rmance, based on past OED research. Table 1.1: Performance of Operatons Evatuated, by Evaluation Year (1974-1992) Percent Rated as Having Sat~sfactory Performance Nuber of er vhulo oot Coborts (no.) (%) (%) 1974 50 86 86 1975 57 88 88 1976 70 86 86 1977 109 90 90 1978 97 91 91 1979 130 88 88 1980 87 89 89 1981 108 85 85 1982 127 80 79 1983 178 85 84 1984 173 74 73 1985 192 70 70 1986 248 82 82 1987 187 72 72 1988 169 74 74 1989 263 70 70 1990 359 64 62 1991 278 63 62 1992 277 67 67 Av~rgs 3159 75% 75% Ntes OWD'a tmal meodotaeauminimfaory bpufann=s bbaued on achivnof at lasta l0e me ER, or ser sgnd a~benefe if th ERR a lowa , or an valuator's quambåve judg ~ #«abau9erfomf. If o ER wese *ated. * 8ased a~ te r ~gsgiv by ~R, PAR ad PCNs mitad to dh Boama duuing å. evaluadon yar. se also Faotaot. 4. b Updated fotratlage ofrPAR. comphled a year or.r aerå s iginal PCI or PCN. S Soe, for examplo, the disussion in Chaper 4, section C of 0BD 1990 Annual Re~w f Eluaro Rub, 1990 RepotiNo. 9870, August 1991. 9 1.15 Project performance varies considerably by aga and ngign (Figure 1.6 and Annex 1.5). The main differences when compared with earlier evaluation cohorts are: (a) Agriculture, human resources, and industry projects evaluated during CY92 performed significantly better than those evaluated in 1990-91. In contrast, performance in the financial sector declined sharply. Compared with the universe of evaluated projects since 1974, agriculture, transport, human resources, industry, urban, and telecommunications all achieved higher ratings. Figure 1.6: Project Performance Weighted by Commitment Amount Middle East and North Africa, East Asia and Pacific u e and Central Asia Latin America and Caribbean South Asia Africa IIu Telkmmninpationas - - --- --- Transport w Industry Human Resource Program & Policy Energy __ _ _ _ _ _ _ _ _ AVEltAGN Agriculture Power Technical Assistance ---------------------- Water & Sanitation F in nce------ --------- Tourism Pollution Control & Solid Waste 0 10 20 30 40 50 60 70 80 90 100 Percent Satisfactory Source: Annex Table 1.5 P 1974-91 * 1992 (b) The proportion of satisfactory operations in all regions except South Asia showed significant improvement relative to the figures recorded in the 1990-91 evaluation cohorts. Compared with all evaluated cohorts, East Asia and Pacific, Europe and Central Asia, the Middle East and North Africa, and Latin America and the Caribbean all recorded a higher percentage of satisfactory projects in this year's cohort. The reasons why project performance has varied over time and among regions, project types, and sectors is a principle theme of Chapter 2. 10 1.16 In keeping with the upward trend in project performance, this year's cohort contains a significantly higher number of outataigg2n=L24--than did the 1990-91 cohorts (Annex 1.7). An examination of these projects reveals significant lessons with respect to the importance of: Quality Project Processing. All of these projects and programs were well-identified, prepared, and appraised. Bank supervision was deficient in only two of the cases (Grenada's Agricultural Rehabilitation and Brazil's Municipal Development Projects). Ghana's Structural Adjustment Credits (SACs I and II), in particular, manifested a high quality of identification and appraisal work. 1I addition, -hands-on" supervision by expert Bank staff buttressed Ghanaian efforts In critical areas and permitted flexibility in resolving unforeseen problems in implementation. The high production response witnessed In Benin's Area Development Project was attributable to appropriately-designed incentives, a supportive framework for cotton production, and the introduction of higher-yielding varieties of maize. Similarly, the smooth implementation of the Earthquake Rehabilitation and Reconstruction Project in Mexico resulted from the flexibility demonstrated both by the government and the Bank to adjust to the evolving needs of Mexico's post-disaster recovery. The various components of Equatorial Guinea's Exploration Promotion Project were marked by a fUl understanding among all concerned donor agencies. * Borrower Ownership. In virtually all cases, there was strong borrower commitment to the basic aims of the projects, including key policy and institutional objectives. As the example of Ghana's SACs I and II shows, the inclusion of the private sector, unions, and other interest groups in brainstorming sessions about the adjustment program fostered a greater understanding for, and acceptance of, the necessary policy measures. Similarly, Grenada's Credit Finance Project was fully consistent with the government's macroeconomic and political goals of transferring economic activities from the public to the private sector and of increasing agricultural GDP. Tunisia's structural adjustment loan (SAL) had its greatest impact in areas, such as fiscal reforms, where a political consensus had already been reached. In Bolivia, the Emergency Social Fund enjoyed unflagging governmental support at the highest level, while remaining free from political interference in its daily operations. * Beneficiary Participation. Executing agencies and, in some cases, final beneficiaries, participated actively and effectively in project piu paration and implementation. The Communal Irrigation Development Project in the Philippines pioneered a new relationship between irrigators and the national irrigation administration. With the participation of major irrigators along with financing, advice, training, and contracting services provided by the national administration, 144 small, scattered, irrigator-owned rice systems were upgraded, extended, or built. Likewise, Brazil's Parana Market Towns Improvement Project was underpinned by an existing local initiative which was instrumental in maintaining strong political support, even when control of the state government shifted from one party to another. ' A project is considered outstanding if it (a) has exceeded all its major objectives, (b) is highly innovative in objectives, design, or implementation, andlor (c) has successful features which are highly replicable. 11 T.,heiransi toa Maret Eco0oMY The Grad Agric~lua R~hbilitation and Crop Diversification Project (rdt1558-GRD for <1US$5.96 million equivulent) successfully helped transform the Greada agricultural setor kom a gentrally-planned to a market-based eonomy. Ifplementation prved morm difficult dn exp~cted du to political tieavals wbich 'causd uncrtatinty, low morale, and slow Implemetaton. Tie prcject anaat office was not offective until, after strng mncouragemen by IDA, the Minister of Agrc~hr becam peronll-involved. The ultimat succ of the opwation cofirs that stbacka ca be overons thrugb .fficdve k.-borowroopcration. - At completion, the impsct on employm~nt genoution excoeded expuc~ationh o.4w~tin.nd.Outsidø griculture. Agro-industry expanded and supportwserice,including,extnsion d inpøts, Improved. The project is considered sustainable, inne smal! part, beuue the appropiat ;~Infrastructure, -technical, and psychological base has been established in the private sector. However, ¼ffctivo fqllow-up will be meeded. The governent stil! neøds to provide an appropriate macro-plicy $ nnmnMnt ånd the private sector must lern~ to be moro sensitive to varos =narket ~ange. . Recét acern focuses on the oed for sol conservation practics agricultur is pushed onto mor fragile land w ~ trban epanslon d tourism. å Stroq Bauk-~orrower Relad~mip and a Favorable Policy Envrnment ThIb.China Rubber Develo~pmnt Project (Credit 1417/Spei Fund 005-C.A for US$108.i >~lo°equivalen) was desiged to increse the production of naul rubber and improvo the øfficionoyof htins nubber production system. - Implementation was thorough, despite some Initial besitancy by tenoficiar-n adopting Øew teclnologies. A combination af g land prepamtion, and appropriao and;soil practices led to considerable physical and socio-economic benefits. lhe timber proemssing 6«ilities were flincially and economically viablø. Five features ensured project succes: strong borrwur (miltment, Utaghtffoward and realistic objectiveas, good relations between the Bank and bormwer, a vorabl poliøy ¢di~ato, and an effective raining program. Partlcipaton aud Capable Project Mnageient Mulsuia'a MaWcca Aguicultual Døvelopment Project (La 2141-MA for US$17.75 million) &mns e paticipaton by project beneficiariea in its original design but a bass of happort for te project wihich mnsured isuatninability, even foilowing sharp changes in tho externul environment which forced bie changes in its design. In this ca, the original project aims were to promote development of mn~ber-producing areas, irrigtin and drainage works, and dairy and fishpond enterprises in Malacca. stIte. However, a change n the ditection -of macroeconomic policy led to sharp reductions In agricultural sbsidies which rendered the fispond and dairy development components unprofitable. With the cnsent of eneficlari~s, a capable, effective, and flexible project management team replaced te fishpond and daisy-componsnts with a fincially and economicaly-viable plan to promote oil pam development With thes. sbifts in design, overal project implementadon continued to be successful, thus demonstrating th ned for flexibility in 4usting project design to emerging circummances. 12 ~N A Communal Trigation Development Prejecg in the Philippines (Loan 2173-PH for US$38.1 aMllion) supports the concept that low-cost irrigation prjects coupled with farmer involvement in operatons and maintenance (O&M) van generate successful outcomes. This project was designed to lacreas rice production and incomes of households currntly growing sainaed crops through conversion to kdgation, tengthen the capabilities of the National Irrigation Association (NIA), and support a program naor which farmers plan, constact, operate and Wintain communal Irrigation systems. The project, pr"puwed u=t appraised with a signicant amount of fmer participation, achieved its objectives with Ioseased yields and incomes. Farmers were orSaized into IrrigatIon AssocIations which effectively opeate to krigation systw, and tbi MA and Its provincial offces strengthened their institutional capabiRties The foundation for a participatory approach to irrigation management was established tisfactorily. The adoption of least-cost construction techniques, combined with users assuming rsponsibility for O&M resulted in a low-cost irrigation project. 7Pgricipaio at Project Inception and During Execution 'Th $enl Boiqou Province Rural Development Project (Credits 1127-BEN/1127-l-BEN for US$ o.o9 million. equivalent) deonstrates the importance of farers' involvement throughout the project cyel. The: aain objectivs of the project were to improve sural incomes and promote exports through incresed productio of cotin and food crops. Project implementation and supervision went well but .dftored fomt delays in coastmuction, diabursements of funds, and weak fimancial management.. Farmers rspionded well. to. di incentives and support services provided by the project, with production targets , eppissed, and incomies growing beyond appraisal expectations. Much of this improvement was due to use of lmproved ep varieties. Institutional Strengthening. Several projects fostered competent and motivated Institutions. Kenya's Geothermal Exploration Project upgraded the capabilities of the institutions responsible for geothermal exploration and development while successfully establishing a competent department within KPC, the Kenyan power utility. Similarly, the Economic Recovery Program Loan to the Philippines was instrumental in improving the country's financial institutions. The Housing Development Finance Corporation (HDFC) Project in India supported the continued growth of HDFC and helped establish a sound framework for market-oriented housing finance in the country. * Visible Benefits. All the projects were characterized by significant and clearly- identifiable benefits. China's Perennial Crops Project generated productive tree plantings and technological inputs. In Myanmar's Irrigation and Drainage Project, excellent progress was achieved in the implementation of the dams and power generation facilities. China's Rubber Development Project, which aimed at improving and expanding the national rubber production system, exceeded the nft benefits projected at appraisal. Jordan's Second Arab Potash Project substantially 13 Increased production of a natural resource with an enormous potential for earaing foreign exchange, devised a long-term research and development (R&D) program to assess potential output, and Introduced measures to restore the national potash company's manclal structure and broaden its capital base. The Economic Recovery Loan to the Philippines was Instrumental in bringing about marked Improvement in the country's economic situation. Stimulated by capital inflows tied to a recovery of investor confidence, imports and private investment revived and economic growth accelerated. 1.17 It is instructive to trace the interaction of these lessons of experience in a key sector like agriculture, which, historically, has had the lowest percentage of satisfactory projects in all evaluated cohorts. The sharp upturn In the 1992 agricultural cohort to 72 percent satisfactory, compared to 49 percent in 1991 and an average of 61 percent during 1974-92, derived primarily from Improvements in the performance of the fisheries, irrigation/drainage, and credit/finance subsectors, and in adjustment lending (SECALs). Eight of the 87 agricultural projects evaluated were rated outstanding. (See Box 1.1 for selected examples of agricultural projects which succeeded in widely divergent, and challenging, circumstances.) Adjustment Operations 1.18 Evaluations of adjustment lnding are growing in number. Roughly oi. eventh of the operations included in the 1992 cohort (39) fell into the category of adjustment lending. Performance ratings for this important lending instrument improved modestly to 62 percent satisfactory In 1992, compared to 56 percent in 1991, I.e., below the cohort averages in both years. This sub-par performance is best understood from a regional perspective. Thirteen of the 15 adjustment operations in the 1992 cohort deemed unsatisfactory involved adjustment operations in Sub-Saharan Africa. Viewed from a different perspective, of the 22 operations in the cohort which took piace in Sub-Saharan Africa, 13 were rated unsatisfactory, whereas, of the 17 operations In other regions, only two were so rated. A recent OED study on adjustment in Sub-Saharan Africa provides some lessons of experience regarding program design, implementation, and borrower ownership (see Box 1.2). Institutional Development (ID) 1.19 Evaluation is not solely concerned with balancing project costs and benefits. Another major consideration is the extent to which a project improves a country's ability to make effective use of its human, organizational, and financial resources in order to achieve successful and lasting economic development. This is called institutional development. OED evaluators assess the institutional development impact expected to be achieved as a result, for example, of redesigning ministry functions and interministerial linkages, identifying institutional alternatives for service delivery, organizing beneficiary groups, redistributing authority and responsibility betwen levels of 8 the factors underlying trends in the permance of conpitd agricukural peas aeV eplorad in dail In chapr 3. 14 government, and/or reorienting the bureaucracy following deregulation as part of a Bank-financed operation.' 1.20 Approximately 95 percent of the projects in the 1992 cohort had institutional development objectives. Of these, nearly one-third substantially achieved these goals, while close to half did so partially (Figure 1.7 and Annex Table 1.8). These results are slightly better than the comparable figures tr the 1991 cohort and the averages for 1989-91. Figure 1.7: Achievement of Institutional Objectives by Sector, Region (1992, 1989-91 Cohorts) East Asia and Pacific Middle East and North Africa South Asa Afric Lati America and Caribbean Europe and Central Asia War & Sanitation HumanAneTale. PeRensourcel 0 : Pollution Contr unol icat - Aguriulur 0 10 L L 40 5 ; 7 so 9 10 Solctd lUssons of exereme about implementation of inshiwMinal. development objechmve am dwocussed in Chapw4. 15 B0or L2'AMjustnent ht Sub-Sharan Africa STe dateos U baluatb* 4s a4jvstment opeationskMgia,etwldeé mnly flpercent wereq at4 .# atisfaQtory. All too often> poUcy reforms we»re n mmnted SW 4i4 cot ted to thedir outcmes.A recent 0ED atndy-'Ajutmn in Shb-abaan Afdca:'Sueataed Findng om" E babwna June 30, 1993-dmntifted some of 6h key ieusons 8té rlreforms are vatiriy not nffieent tt> geeaeaamg pgytsme esf a4jUstmnt programsreuteåro crocconouziö stability, including tustalnable tevelt of exen4dée64 n complmentry mesure to dvelo the pivat sector (basic infrastucture, business support systerns, legl-rfor) - weil as long-term inistituitional reform. It is.difficult to derive general lessons on tho sequmnclag of structural reform. not linked to the. *political and economic reulities of tho country couteit. However, sonme of tho oxperience, cossetwith: the findings of earlir ORD studie., may be relevant to other countuios. Tis suggests that. (s) export promotion lakes prinno over Import liberalization per se; (b) trade reform.-ars unlikely to be eftectlve unless accomopanied (or preceded) by internal competition palices.; and (c) rehl sctor adjustment abauld, n ornmally precede finncial sector liberalization. Policy reform programs succeed more aften when tho objeetdves and te line .frast for implementation are realistic and conditions are clear and pioritized. For oxample, in Mozambjne,. tho Second and Third Rehabilitation Credits incorporaed a program of partial steps phased ln over sevora .yars in deference to the Government's wishes. The program was then uccessfully completed.t Wll.repared koanu, based on.higb-quality SSW, ed to improved design, bta rmtho*1 e:est .and les o k It .hsette to spend mre t gettieg the prqoet- right than: to -lund w'pomaturely. Successfl.progme are based an a roalistic and carefleern of expected oW,n the potential effects of externsi thocka, and (he maijor risk» faing tho program.. Supeiifon aod mosiforiug play a key rolo in keeping program.-on track and "respondiug to anexpected.developmeadt heu satiactory outcomo of several acfustment operations in Ghana.awesmuch to thorough preparatry work, realistic assumptions and projections, and good supervision. Without mustained progress on the policy front, tho provision of balanco of payments support can bo counterproductive, leading to higher debt, weakened policy credibility, und a imuch more difficult fask of adjustment in the future. Under these conditions, efforts should be directed toward devoloping tho institutional capacity and borrower ownershlp needed to sustain reform. Ujntil those are in place, lower leve!. of adjustment assistance are advisablo, with more funda targeted to specific sectorél progrm. ar investnment projects. Cöte d'ivoire is a goo example of m country where iarge inflowa of relatively *y xptsive bulance of payments support .were given befor. an appropriate asount of policy reform was. ~undertaken, use maring tht a4ustment uttack c marc difOicult and costly.·..· 16 (Mox 1.2 continued) Borrower Ownership and Political Economy Issues The greater the participation of the borrower in design and isp'-mentation of thU progm, the. greater the ownership. and, hence, the greater the likelihood of a successfil outcome. The very low degree of borrower ownership of the adjustment operations in Cte d'1voke explains much of to unsatisfMtory outcome of the program. But even when borrowers' commitment appears strng, d ta Ghana, it can still be weakened by mistrust between government and the private sector, and by th heavy costs of soform for powerful political groups. Frequent policy changes will qudakly underaie policy credibility and private sector confidence. The Bank needs to strengthen its understanding of political economy issues. Economic and geolow work should make a regular assessment of the impact of adjustment policies on various constit and the effect this might have on the momentum for reform. The Bank also needs to support efforts to. bring the private sector and other sociallpolitical groups into the process. Caution abould be exerciAe4 where significant doubts exist regarding the degree of ownership. Where there is scopo for disetion efforts should focus on policy areas where some degree of consensus can . be elicited: among implementing agencies, key interest groups in the country, and the Bank. . .. Sustainability 1.21 Yet another crucial dimension of project performance is sstainability, which measures the extent to which an operation is likely to maintain an acceptable level of not benefits throughout the economic life projected for it at appraisal. Arriving at a sustainability rating requires examination of many of the same basic factors which underlie the appraisal of an operation from an economic, social, financial, institutional, and environmental perspective. But it is distinct from the economic justification in that sustainability focuses on features which contribute to the durability of a development operation and its likely resilience to external shocks and changing circumstances. To illustrate, the evaluator will be interested in learning whether the borrower has made adequate provision for long-term support of any infrastructural, service, or Institutional investments made under the project. And the evaluator may inquire into whether any follow-on projects were implemented which continued or exoanded activities in the project under review. 1.22 Hence, assessment of sustainability requires judgments about the likely responses to challenges and rics an operation may be subjected to over its useful life. For example, an operation that is dependent on government subsidies, and which offers limited prospects of financial self- sufficiency, may be rated unsustainable, especially if the economic environment is characterized by severe fiscal constraints. Equally, an operation dependent on the extraction of a scarce, nonreplenishable resource, is, by definition, unsustainable. 17 Figure 1.8: Project Sustainability 1992, and 1989-91 Cohorts 1&iddle East and North Afica _ East Asia and Pacific Europe and Central Asia South Asia Afica MAW Latin America and Caribbean Telecomnmunications _________ _________ 'gTechicai Assistance------ --------E HumaaResource Power Urban AVERAGE Transport Energy_ Agriculture Program & Policy Water & Sanitation Finance Pollution Control & Solid Waste T o urism ----------------------_---------_ - 4 0 20 40 60 80 100 Percent Rated of Likely Sustainability Source. Annex Tale 19 1989-91 1992 1.23 The sustainability of project benefits0 in the 1992 cohort was rated, on average, broadly in line with that of the 1989-91 cohorts. Projects of *likely' sustainability constituted 54 percent of the total in 1992, compared to an average of 55 percent during the three previous years (Figure 1.8 and Annex Table 1.9). However, at the sectoral level, average sustainability ratings displayed substantial variability, with industry and telecomanications achieving improved significantly ratings in 1992 relative to 1980-91, while power, transport, fimance, program and policy, urban, and energy displayed significant deterioration. Regionally, sustainability of projects Improved significantly everywhere except in Latin America and the Caribbean and Africa, both of which had declines in likely sustainability ratings of 15 percent or more in the 1992 cohort, relative to the 1989-91 cohorts. * Sustainability is rated as being either "likely," *unlikely," or 'uncertain.* The last category includes (a) projects for which there is insufficient information at the time of whe the PCR/PAR were prepared to reach a judgment as to the likelihood of suatainability, and (b) projects whose fAture sustainability is truly uncertain. The 'marginal" category previously used in OED ratings was eliminated as of January 1993 in tho course of revising and amplifying the Project Information Fora. 'Marginal' projects were merged into the 'likely' category for this and all previous evaluation cohorts (1989-91) for which sustainability ratings are avalabL 18 Project Outcomes, Institutional Development, and Sustalnability 1.24 Underlying the concept of sustainability is a concern about how robust the various assumptio behind the economle justification really are. Where a project falls to achieve its Institutional development objectives, the evaluator assesses whether this will prevent achievement of a satisfactory outcome and/or pose a serious obstacle to sustalnability. For example, a roads project completed without having secured adequate maintenance and repair may be rated as unsustainable. Environmental aspects may also come into play. Thus, a high-return project, such as logging in a tropical forest, may be judged unsustainable-as well as unsatisfactory-if it involves unacceptable environmental consequences or risks. 1.25 This is not to say that all projects rated satisfactory are automatically rated as sustainable and as having achieved substantial institutional development. Yet, there is a strong Interrelationship between the three raings. For example, for 1,094 projects drawn from the evaluation cohorts of the past four years (1989-92), 84 percent of all projects judged as being likely sustainable also achieved substantial institutional development, while less than one-quarter of the projects rated as being unlikely sustainable were similarly rated. The relationship is somewhat weaker in the opposite direction. Approximately 45 percent of projects which achieved substantial institutional development were rated as likely to be sustainable; another equally large proportion was considered of uncertain sustainability. Only 10 percent is actually considered unlikely to be sustainable. On the other hand, when a project essentially failed to achieve its ID objectives, it was rarely considered as likely sustainable, indicating that ID is often a necessary, but not a sufficient, condition for project sustainability (Annex Figure 1.1 and Annex Table 1.10). 1.26 The relationship is much stronger when looking at either ID or sustainability in relatin to project outcomes (Anne Figure 1.2 and Annex Table 1.11). An overwhelmingly large proportion (90 percent) of sustainable projects was also rated satisfactory, while hardly any of unlikely sustainability (8 percent) were so rated. Similarly, 95 percent of projects which achieved substantial ID were also rated satisfactory, while only a little over one-fifth of the negligible ID projects achieved overall satisfactory performance. 1.27 Finally, if a project Is both sustainable g achieved substantial ID, it is almost always rated as satisfactory, whereas if the opposite occurs, it is virtually a guarantee of unsatisfactory overall performance (Pigure 1.9 and Annex Table 1.12). Ninety-seven percent of all projects rated as achieving "substantial" ID and being of likely" sustainability were also rated satisfactory on overall performance, whereas a mere 5 percent of projects rated as achieving *negligible* ID and being of "unlikely" sustainability obtained an overall satisfactory rating. 19 Figure 1.9: Interrelationship of Project Sustainability, Institutional Development and Performance Ratings, 1989-92 Evaluation Cohorts 80 R Project Sustaabillty UlikelY igible rnstitutional Development Socm AaeuR Table L12 Other Aspects of Project Performancen Physical Achievements 1.28 Seventy-eight percent of the projects in the cohort had physical investment goals-the balance being mainly adjustment and technical assistance operations. Over three-quarters sustantially achieved, and another 21 percent partially achieved, their physical goals (Figure 1.10 and Annex Table 1.13). Once again, this was a modest improvement over the 1991 cohort, for which the corresponding ratios were 67 percent and 27 percent.' In regional terms, the achievement of physical objectives was highest in ECA, South Asia, EAP, and MENA, all of which achieved Obstantial" physical achievement in 86 percent or more of their projects; intermediate in Africa (71 percent Osubstantial"); and lowest in LAC (59 percent "substantial"). Counterpart funding difficulties and a deterioration In the borrower's macroeconomic situation were two important reasons for less- than-sats ry physical performance. n Mw loan cancellatio resls for the 1992 cohort are reported in Chapter 2 in the context of a broader discussion oftto iaorical causes ofcancellations. u Data for the physical achievement variable are not available for projects evaluated prior to 1991. 20 Figure 1.10: Physical Investment 1992 Cohort by Region, Sector Europe and Central Asia h-f h South Asia uU----uuu--- East Asia and PacificE Ididdle East and North Africa Afica Latin America and Caribbean Telecamrounications Technical Assistance Finance -- - - - I I I Urban HusanResource Power Industry ----------- o Transport Energy AVERAGE Water & Sanitation Agriuture 0 20 40 60 80 100 Percent Substantial Source: Amex Table 1.13 Time and Cost Variations 1.29 Figures 1.11 and 1.12 depict cohort performance with respect to Implementation time and cost variations. Figures on time variations in most investment projects are based on physical completion dates, while those for development finance companies, agricultural credit, and policy- based operations normally refer to expected-versus-actual loan/credit closing dates. Projects in these latter categories, however, are typically excluded from ex ante and ex post cost coMpaios becaus figures on total project-related expenditures are generally not available. 1.30 In general, the 1992 cohort provides further evidence that delays In project completion continue their slow growth. On average, actual implementation of cohort projects took about half again as long (6.7 years) as had been projected at appraisal (4.5 years). The average time variation of 51 percent was slightly above the figure for 1991 (49 percent), eight points above the three-year average (1989-91) of 43 percent, and five points above the long-term average (1974-92) of 46 percent (Annex Table 1.14). Time overruns were highest for projects in finance (121 percent), energy (73 percent), and telecommunications (69 percent). By region, the highest time overruns were in Latin America (68 percent). 21 1.31 A number of factors account for project time overruns, In:1.Jg natural disasters, political upheaval, civil disorder, inadequate project design and appraisal, insufficient institutional capacity, inadequate contractor or consultant performance, procurement delays, and counterpart funding shortages, the latter often associated with deteriorating macroeconomic conditions. Figure 1.11: Time Overruns 1992, and 1974-91 Cohorts Latin America and Caribbean Africa _ South Asia Middle East and North Africa Europe and Central Asia East Asia and Pacific Finance Energy _______ __ Transport AVERAGEC AgricUture In_t_y Water& Sanitation Program & Policy Techmcal Asistance Urban Pollution Control & Solid Waste Tourism 0 20 40 60 80 100 120 140 Percent Overrun Source: Annex Table 1.14 1974-91 1992 132 This year's cohort reinforces the pattern of small cost underruns which were reported in recent years. The underrun averaged 4 percent in 1992, compared with 8 percent in 1991, and an average of 7 percent during 1989-91. In most cases, the significant currency devaluations experienced by many borrowers during the lifetime of these projects (mainly the 1980s) translated into savings of foreign exchange relative to project cost estimates at appraisal. Less frequently, loan cancellations and mid-stream reductions in project scope were a factor. Altogether, more than twice as many projects experienced cost underruns relative to those experiencing cost overruns, a pattern consistent with findings for the 1988-91 cohorts. 22 Figure 1.12: Cost Overrun 1992, and 1974-91 Cohorts Africa South Asia Latin America and Caribbean East Asia and Pacific Middle East and North Africa - Burope and Central Asia - Telecommunications Finance Iimen Resource Power Pollution Control & Solid Waste . Tourism . Urban . Program&Plc Industry . Energy . Water & Sanitation . Agriculture - Techmical Assistance ________ -40 -20 0 20 40 60 Percent Under Percent Over Source: Annex Table 1.14 992 C. THEMATIC AREAS 1.33 We now turn to a discussion of thematic aspects of the development impact of the 1992 cohort. The three themes addressed are central to the Bank's mission: poverty reduction, environmental impact, and private sector development. A sample of 55 projects from the 1992 evaluation cohort was selected to highlight the experience and to extract relevant findings in these areas. Thirty-six (or 65 percent) of these projectalprograms were rated satisfactory. Poverty Reduction 1.34 Sustained alleviation of poverty is the overarching objective of the Bank's country assistance strategies. An integrated strategy of poverty alleviation consists of: (a) broad-based The breakdown of the sample is as follows: 1 policy-based loans (SALISECAL); 14 agriClue; eight public utilities; six each energy and industryldevelopment finance companies; four education; and two urban sectors. In tems of regional distribution of the 32 countries in the sample, 16 countries were in Africa, nine in Latin America and tbe Caribbean, three in Middle East and North Africa, and two each in South and East Asia. The sample consists mostly of projects and programs for which PCRs and PARs were available at the middle of the calendar year. 23 economic growth that generates efficient income-earning opportunities for the poor; (b) Improved access to education, health care, and other social services that help the poor take advantage of these opportunities; and, (c) provision of a "safety net" for the most vulnerable. These aspects have been covered in the context of policy-based lending (SALs/SECALs)," as well as across sector loans/credits. The 1992 cohort provides a number of examples of "best practice" In reducing poverty, along with a few Instances of conspicuous failure. 1.35 mrvY Ling Cndi. Many urban development projects (e.g., Ethiopia and Bolivia) aimed at (a) implementing resource-efficient housing policies through self-help and Individual initiative; (b) improving health and environmental conditions through upgraded services and sanitation; (c) Increasing the access of low-income populations to shelter and other essential urban services; and (d) Improving urban Income distribution and environmental conditions. Both projects achieved substantial results In meeting their envisaged objectives. In Bolivia's project, notably, over 20,000 families throughout an area of 1,500 hectares were provided with drinking water services, approximately 700 beneficiaries received artisan loans, and over 1,500 new retail stalls were constructed. Overall, it is estimated that about 80 percent of the project benefits accrued to beneficiaries in the lowest quartile of household income in the project areas. 1.36 Similarly, projects in the water sector (Rwanda) and In irrigation (Sudan) improved living conditions in the target areas through rehabilitation, reinforcement, and extension of production and distributica facilities for potable water, as well as through field health centers and malaria control program 1.37 Enhanced Emplym Opotitie. Several agriculture sector projects (Tunisia and Tanzania), which aimed at Increases in cultivated areas and production yields, also had a substantial impact on incomes of the poor through enhanced wages for family labor, increased access to transportation, and Increased employment opportunities. Thailand's energy project, targeted also at the resettlement of villagers occupying the mining areas, had a significant impact on what was previously a remote rural area; the mine/power station complex is now providing direct employment for over 2,000 persons and indirect employment for many others. 1.38 The technical assistance component In Panama's SAL I financed studies of social security programs in the context of (a) social security and health; and (b) unit costs In the health sector-with the objective of a comprehensive diagnosis of the social security system, considering alternative ways for compliance, and recommending changes in the legislative framework. 1.39 The primary education project In Papua New Guinea assisted the National Department of Education and the provincial authorities in improving educational access and quality especially in underserved areas, and helped to reduce enrollment ratio disparities; the project also met its objectives of establishing new, and strengthening existing, procedures to Improve the quality of primary education, particularly via raising the professional qualifications of the teaching staff. H For th social impact of adjuwtmt, see also "World Bank Strucual and Sectoral Adjustment Operations: no Second OED Overview," Report No. 10870, June 30, 1992. 24 1.40 AdJUstment L&nding and the Poor. The uneven social impact of adjustment lending was evident In several assatisfactory operations. In Central African Republic's (CAR) SAL It, for example, while liberalization of the production of foodstuffs produced a redistribution of incomes In favor of rural areas, the provision of social services (especially education and health) to urban centers declined as a result of cuts in nonpersonnel expenditures. Panama's SAL H aimed at reforming the social security system, but poor administration and short-term liquidity problems associated with deficits in special pension programs resulted in the failure of these efforts. 1.41 Shortcomins in Some Investment Operations. Amed at Heling the Poor. Poor project management, erratic financing, and a deficient Incentives environment proved to be Insuperable obstacles in investment operations like Nigeria's Uorin Agricultural Development Project and Brazil's Amazonas Agricultural Development Project, both of which bad attempted to change traditional farming systems and thereby improve the living standards of small farmers. Environmental Impact 1.42 Sound environmental management is another key objective of Bank assistance.' The Bank Is now actively encouraging the preparation of environmental action plans by Its borrowing countries. These plans are Intended to analyze environmental issues in a comprehensive, multisectoral framework as well as to set forth a long-term strategy for maintaining the country's natural environment, the health and safety of its population, and its cultural heritage. Of growing concern in recent years has been the impact of resettlement associated with agricultural, urban, and infrastructural projects. The early experience of the Bank with involuntary resettlement was recently the subject of an OED study (see Box 1.3). 1.43 Pollution Control and Land Restoration. These objectives were at the core of many projects (e.g., Bangladesh and Thailand). In Bangladesh, the physical environment surrounding a refinery was dramatically improved via the reduction of flared hydrocarbon gases and of hydrocarbon sludge and waste effluents. In Thailand's Lignite Project, sulfur dioxide and other stack emissions were held within prescribed safety guidelines. Also, mine dust disturbance was minimized by water- spraying on haul roads and at conveyor transfer points. As areas of permanent overburden dumps were completed, they were graded, contoured, and planted with trees according to a final restoration master plan. Other areas were being restored to agriculture with recovered topsoil. Pakistan's Reservoir Maintenance Facility Project improved the safety of the Tarbela, Mangla, and Chashma dams through the replacement of obsolete maintenance equipment and the financing of additional equipment for seismic monitoring, dam instrumentation, testing, sampling, and surveying for dam safety. 1.44 Naua Resour MaeM Isue. Tanzania's Sao Hill Forestry Project - Phase U achieved substantial results in terms of a reduction in wind movements, improved retention and slower release of precipitation, and a return of wildlife. The project also prompted spontaneous private local planting of pine and eucalyptus. Also, Ethiopia's Urban Development Project achieved considerable improvements in environmental conditions through upgraded services and sanitation- roads, drainage, water supply, and solid waste systems-as well as from direct loans for this purpose Sco OD 4.02, Environmental Action Plans. 25 to the target co~mmnity. Recent OED studies have provided a framowork for looking at natural mourme managoment laos, and applied It in two countries: Nepal and Bolivia. The maJor findings are summarized in Box 1.4. LaS4 acqutibnnIs ~pervaIv. feuturs of Bank.pported operations. As of June 30. £993, .å. >. 131,prqt ~e inl tatinith mt -components, principally in the agriculturo, segy;wbm, and tspoftation fetous. Ths pouects will affect a total of 1.9 milon people. O.Wf t fint international o¶ml ta to- introduc guidlnes intended to lay down * é W.procedus by whIg uank staff ca sit borws dischuago »air rc~ponibility for tho t t:1d uu tietatla of people uffeced by Bhnk-finae ptqjecta. These guldolins wcr I ad~ ofd and e . Nevcrthle, public cicism of in kÌ prd be ae m r le^ vociferous over the following P~Otly, O U governumant. 6 e ióginilas.8tingag07; (c> tho legislative frswework; (d> competent plamuing; (o> . tt*WÖ!YloiUtp-( <>30 dequate cosing and fundig lhe atwdy so natedthat, although the d e spias b0ð¢dly appropluto, thy were poolly applied in practice, Pour generamilsson of iov osafttion io-mi nl tho nurobous of people affectod;. giunfärefort tornoae ipn, Icudng oreaccurate iaurnn ? t # I~ats.ooÉing aWd,=loqusto finding; and. * UprOVd aagnat Sourceis: OED 'roport:·on 'Bady Bxporenco with Involuntary Resmtlem m o,* Report No. ~. ]2j42, li~ 30, 1993; pa Evauatinn bana öpog Hydrolectric Project,` Report No. 12141, Jluái 30. 1993; "Itpaet Byaluution ån India.Zarnataka Irrigatlon Projecat," Report No. 12132, Juns 29,. i ; Wimn8 ttic w kidia Må~artr lk~gation i Project* Rport No. 12133, Juna 29, 1993; å&Imtt:Bvahuatios on TailT~d Kha LUwa Hydroolecti~ project,`Report No; 12131, lune 29. 1.45 Deficiencies in Project Prepartion. and Mitiating Factors. Some of the cohort projecs were characterzed by regrettable shortcomings in project design and preparation, with correspondingly advorse environmental effects. In Mauritanla's Guelbs Iron Ore Project, the objoctivo was to guard against generation of dust at the crusher station, whenever the water spraying 26 system was out of order. However, the volume of dust has been greater than expected. According to the PCR, "a huge cloud of dust can be seen from 10-15 km away.0 Evidently, project designers, equipment suppliers, and the owner grossly underestimated the magnitude of the pollution problems associated with the handling and processing of very dry ores. 1.46 Among the environmental components of Niger's unsatisfactory Second Forestry Project were natural resource management schemes (including erosion control and natural forest management). However, because of low rainfall and poor soils, the project did not produce the expected yields. The forestry development component of Brazil's Maranhao Rural Development Project had as principal objectives the enforcement of a forestry code and the financing of environmental education facilities. It also aimed at the creation of a forest reserve. But before the reserve could be established, the area was invaded by settlers who were allowed to remove most of the forest cover. A final example is Nigeria's Oyo North Agricultural Development Project. Its mixed environmental impact was greatly influenced by the incentives framework then being applied to the rural sector. Favorable farm price ratios, better access of the markets through project feeder roads, and subsidized tractor hire services encouraged and expanded land clearing-with adverse implications for the environment. Private Sector Development (PSD) 1.47 A common theme in reform programs supported by the Bank is the development of a competitive private sector. A satisfactory framework for private sector development requires removing the impediments to the free flow of labor and capital across sectors and activities through such measures as the removal of entrylexit barriers, Investment and capacity licensing, restrictions on foreign private investments, and controls on financial and labor markets. Simultaneously, reform of public enterprises should be encouraged to (a) reduce or eliminate the large budgetary subsidies such enterprises receive, and (b) remove the allocative inefficiencies such enterprises introduce through uneconomic pricing and investment policies, excessive employment, and weak management. When these elements are in place, forward-looking technology development programs may be appropriate (see Box 1.5). 1.48 Creatin an Enabling Environment and More Space for Private Initiative. CAR's Cotton Sector Adjustment Credit aimed at reducing rigidities and inefficiencies in the cotton industry through reforms of the incentive system and of the Institutional framework. Reform of the incentives system consisted of: (a) the elimination of import subsidies; (b) introduction of a flexible pricing system responsive to world prices; and (c) creation of a cotton stabilization fund managed by the industry. The goal of the institutional reform was to privatize the commercial and industrial activities of the cotton industry. The program achieved its objectives with respect to more flexible pricing, thereby alleviating the onus of cotton subsidies on the public budget. Greater transparency In the incentives system was also achieved through separation of the commercial and industrial activities in the cotton zone. However, privatization of the commercial and industrial activities was not achieved -partly because of disagreements between the sector's two major donors on how to implement the privatization. 27 Box 1.4: Natural Resource Management In 1989, ORD published a framework repart. Rnewable Resoue Manamen in Agut, Report No. 7345, Jume 1988, which analysed natural resonsce M M) in 335 cmPled agriculture and forestry projects. This has been followed by two iuadepth country reports on NRM In Nepal (1992) and Bolivia (1993). One or more additional country reports will be initiated In 1994. The framework report concluded that NRM information for dociulon-making to approve or rqject a project was generally adequate, but additional, important questions were not asked at appraisal, with the result that they were neglected in subsequent monitoring and evaluation. This, in turn, meant that performance audits were less useful than they might have been. Management was not provided with an adequate framework to judge the long-trm trade-offs between efficiency, equity, and mastainability. Procedural issues also affected th usefulness of the Bank's database and how it was used. Economle. and sector work (ESW) did not provide a sufficient strategic framework to identify and deal with the in NRM issues: project design was influenced more by policy papers and internal operational instructions. than by ESW. Important insights gained via Bank-borrower interactions were often not reflected in formal. internal reports. This helped to explain why a systematic effort had not been made to draw the lessons of NRM experience and apply them to subsequent Bank operations, and why opportunities to Ifluence. borrowers had been missed. The report recognized that NRM issues varied widely among borrowers and were governed by complex and sensitive social and institutional arrangemet. The Bank knew too little about the relationships and interactions between socio-economic and natural systems. To improve NRM on a. national scale might have required concessionary funding and software approaches which the Bank was not equipped to provide. 'To take a leading role in improving NRM would have required a cotprehenve. country-level overview of NRM issues, donor-country coordination, utilization of additional fnanial and technical resources, and a greater borrower commitment to the resolution of thoese Issues. The Nepal case study concluded that the Bank focused on NRM issues later than other donors. Its initial focus was on sound technical work and ways to increase land productivity, but not the long-term management of natural capital. Until recently, Bank ESW for Nepal had not given prominence to this: fundamental issue. More needed to be done to improve NRM through coordinated action at the Consultative Group level, which linked NRM to macroeconomic policy through structural adjustment lending, public investment and expenditure reviews, and the national environmental action plan. In Bolivia, the case study concludes that sustainable development for the country as a whole was dependent on how well NRM issues were Integrated into decision-making at all levels. Development policy which saw the trade-off between highland and lowland development as inevitable might not be effective. The donor community had overestIated the readiness of institutions in Bolivia to implement NRM programs, and the Bank's instruments of intervention had proven weaker than expected at appraisal. Decisions about NRM had been made without sufficient information, and the Bank needed to work with regional, national, and international bodies to correct this deficiency. Policy and institutional development for improved NRM needed to be designed and implemented. 1.49 The reforms supported by the Mexico Public Enterprise Reform Loan involved two major components: (a) continuation of the process of divestiture which the government had initiated In 1983 through the sale, liquidation, and merger of publicly-owned enterprises; and (b) improvements In the efficiency of public enterprises. Specifically, these measures included: (a) Improvement In the competitive environment, notably, through price liberalization; and (b) measures to foster managerial autonomy and accountability, including fundamental reforms in the function of the Board of Directors. Substantial divestiture reduced the number of publicly-owned enterprises from 1,555 to around 200 in nine years. The privatization program generated US$14 billion in revenues to the public sector during 1991-92 alone, thereby fulfilling a key role in the economic stabilization program. Public prices were adjusted significantly, albeit not always on the timetable originally envisioned. Additionally, a new constitutional regulatory law opened the way to private sector participation in activities such as downstream production of petroleum and gas derivatives which had been previously limited to the state. 1.50 Though the consolidated accounts of 45 public enterprises and decentralized agencies had shown a surplus in Panama during the early 1980s, some of these agencies required large public subsidies to cover their losses, notably the public entities in housing, development finance, and the agricultural sector. Panama's SAL I intended to eliminate subsidies over a five-year period through the sale, restructuring, and closure of some of the enterprises. Most of the objectives were met: closure of a major sugar mill, sale of a "loss-making* hotel, stricter control over public expenditures, and a major review of all public sector entitles. 1.51 Shortcominms in Achieving PSD ObJectives. The cohort sample also illustrates how project designs that take a partial view of the problem, or exogenous country factors transcending the scope of the operation, can block efforts to improve the supply of essential services to the private sector. ZambWs Water Supply (Rural Water) Project sought to strengthen the Department of Water Affairn so that it could more efficiently implement its program of water services to the private sector. However, an acute shortage of qualified and trained personnel, not addressed in the project design, negated most of the intended benefits of the project. Panama's SAL U included reforms in the public sector such as: (a) the closure or sale of several public enterprises; (b) a reduction of public employment; (c) preparation of a public investment program for the period 1986-87 geared to support an expansion of private activity; and (d) reform of the social security system. However, political instability and civil disturbances led to a fiscal crisis and an erosion in the country's commitment to these objectives. 29 Вох 1S: $aak Lendiag for Induslrla! Technology Oevelopanent pTD) : .. ...:.. . . :.: .. ,..,:..:д:2,:.;хг � :.. �:.• .. ч}:}::.т ..... -: ' . . , : ,:, .. . <•:.}:::::'::>`:��+.`�i�!+��@Iёx� всtа'.; 8�Сц!8!1 `11kЯ8@ iц .t�:: 3'�iŭ0tlЧk:, � <<.}�..:>:>:;�:<:•..,,� . . Р� , Р..;: у :Р: �#�8s.'��!8`��.цг���"<�iс�в.:��:�:?: '' .: .. ..: . . : , :., ;�б)i��,�t�fel� ��v►d'��d�voo)Pp�Bi��::+di��,thё� 1��.�;���::�t�;�:�ad� ��?�!�ах�� bs�v��`°io� : 3:.,: . .:: ..: :.:,:., .:, . . ' : . . . ' • >: ,,. . ;: :.::. ',:' .: ;:: � ;':,. :',.,'.. : :: -•::. :::<>ь: . . . ::.: : :.�:�;�oг:ф�:г�к.�asiat�::�i:%�r�ai;к��)�iolo`�::';:<:�'�i���ёo�;:.�::.'..���:,::4�!:.t�iso�::i1i�:.,,. ё�;:�`< ,;�•:;;��i�c�+` . ехрв�I ,.у:ч.:.r.: i'}::r? .,; � . . . ;л;:с;.; .ги . : . , . . , ... '; •..,,,,,,рΡ }:5л :�..3�''ilSFV�R..�' : . . , . ........ )!�=:)1,)Г.:°� �. "i�1:''���'D1?t' ��'':� � :�*,1�.��э��it:' :�it''�',Q��`��t1�д� :.. :.: .. . ..... ..... :. ..: : ..;, . . �, .; .,•:ь: .:: • .,.:.:: .:г.ч .. . : . ьч:.... • .... ... . �вг. . . .. -�.�:уΡ .., . �. .: : : .... . {�n� �а/ц��уΡ м }��{/� �'. . :: .; .. . �j , : .г:. :..::. ,. .. . . ..., . г:::гУ��Qk!lМ1�� : �'YN :vY`iгг.::С �:2': �.#SR� 6C�F � .wп!{'вw�'.�� f..' �:Й:�. гг}i: . "'1М� , . {• � :. . . ... ":� .. �.:.::. ;. ..._.s .;у.,;... ..ч ::,+у:.; ' `.К. . Х . .. s.... ч:. ь:. .o-.> . ..: r..: . .:: ....... '° , .: г ...: .:::: ....:; .. . . ::. . .. . : ... ... . . - . . .:..:: .... . .. . . . . ,. ........ . ... :.: . : .: . .... . .... .:........:.:.. .. ....... . ..: ......+...г::г _.:_ ..::.: • ,>:r+ , ...: . ... .... ...... .. .. г .. .. .г.. . . . ..: : ..... : . :: �. '�?� ..:ь:+. ... ..... . :..:, ь .-:.. . . ., n .. ., . ч::.}. .х„ ,. .. ... . ..:..:. ;: �м� ....: w:,.::: ...}ь.... . ... - :. .:...; �::: ..::3:::ь•;..л:.:.: .:..:... :л,�,.:лн: ..;.::,:.. .:.. .. . ......::.:.., . - '$r:`; ;�'ц : .; ь ,- .;' ...:... .: ,:.,.... •. . . ч:,. , . . .: . .: .: ...и:.•::.:.,:..:r ... ...:.::::.::.:.:.... .::...:....•:.....::.:.•...:.:.�.. _ .: ч : .., �.:.: л .; ., ': ':� ;:t%:: ....: ..} :.-:.: ':..:'. .'ч...:..,..... .. .... ....... гУл• }.:: -:., п,.--.: .г�!л:: vм . .ч"�\.:;:}•:'-гг'г'чь,г.:::�. . .. �. :.. ♦:4 i:4:.,:•.•: v:. � ' -'. '�г.У: :. . . .. ... . ^i\:.ti л•п-: :.ti`, �.: : :::. ....::. .:. .:.:'..,i!:.,:'%�+: � :. . . ..:; . .. ,;.} .. ;:::_ь:: ��- . . :К} `" ,.уΡ� `ь+iв ,<вМR�.�� : tiг�%� � *n� i{!:� �+ �ц�� ь;,ь:::.:�:п��:�У`:�Q)11t��::� .. �'�i:'�й.:: ''г.�i�.�i�'':; . ...:�i�f'����'�k�гitг:'. 4:'�1'р.�'�:�`i'..;� .}54:.v. ч\н�-}:.нi:..; .: .:. , г, , ... . . , , . �Т^7 � , -.н ь.г: :. • :. .... ...:. . ... • .. . . . . .' .,ь.л . �� ".'F' '`'':�t�8)':�� . �Я� �' .�. G�.. :��� . . @1'it2`►АА!�:.`�`�У)ВУ8. `:�йt3�l �� i:z�:.`.:.'.'ь '_;�,:>.' ;�:�:::�;::, .:.. .:...,-:ь . �:,:;: t ���? .. !с!1tY ь ч�у : :� .:л. ь :�М�, л.'.�:R'��::�:s�б С�`::.:;`. „ . ... . . :.: . ..: .. . :: . . , и.ь.. �.:ь. . : ;: ::.... ,: : . . ,.�,�:,,,г:......•`�ц�:::��t`:��:'t��:�"�ёia�in�'ь�i�i�*: ttiв`iгю�пti�ire����maa�гiarl�' . ..гt�s'1�`����oii :.а •<�::�ait:c:: .:..вtlciv4:.-;:'; :4в�..:::.У..:....:... :. . �!., ,:..:... ь..., , :. .... �.;., � :�:'�:::: �.:..: ... :.:. '::..>: .,... . . гч,},:::<}: �. .. « : ... ... _ .:,........<. ... ,..._; ... :� ..:..:.:.:. ..' .::: :. �� а?�+� .;:' �:�>.�п ся1:: ��::�t.:#��:::е7с�ви�t�:: � <�i�ь>�i�::::: .. :..):::: �'�..::``< ;: :� .:�:,.:'..: ,..<�: , .} :� . 3.,..,. ,: .<�'�Са��.: ::�:�. .t� �.. ..�.1.t�ю,�.:. :.г�!�Х?!::�.'�►.!ь?�п �,..:.�;.�: . . .:::.:.. . :.: . ' �l.?,�. ..и..:...;...ь:•':w:•л.:: �:::..::::.ч.}:лц.::.:.:4.:.. .:-' •� :::ь:Р..г.:. ,. , iv .:. { . .л . ..-.��i. ::s•,;h.::.,.;i�: ,}.:�.. � :{... .г ;{.. . :.:.л... :..:.�,г :.',::, гv. .>:�..� �Io:.��lУ�� � в. 8ilot�.:;.::. ., . ... . .. . z;н;.>...•... .,.::.. .... .....,.. .. .:. . .,:я�:,��}:;� '.�':�t@1о�.>�t1d:;;�:. в:.k�di�''>CQ11&���te�?)9�':,.'���� �::г:i�`:a'�. .;:,,,....:..;.• �9�! .. ......:.' -.:: ... �4... . . ..г. . .:,. : <... : .: ...,ч,::..,.ч::. ..:.::.�.}..:.:ч:>:,... �� . : ..}.. .. ,.. ... :: -. .. ,,..::.:�,;:.:::.: .ь. . .... . . .�� . .:ь. :..:.. �.:<:r.'.::;',ч...'..: ...: . .:. �' .. : ..,.:. . . ;: .. :..:: .:': :'.: . .. :. ... • .. .. :.� ..- : .tt9�:.��A.:�i�s':�i1.��>: >.` �XI��:���:�1:QH88<�=�`018.}.::::�UCI1цtiI0• �'':�``'�`'г :3.:8j1�':г...р�z; '`�:�'1�t�i'~..r .4�?W.......I� .. . . . .. .. �':�2.;.)! .. .. .�...: : . . �.., ;:::<.: �.�:.,.:: к,:�. �;;:�:'.: .,, �•w::, .:::., :••.:..,:.;:.:;:.,.,:_ :.3, ::... .:..: .. : ....; :..:.. . ..: -: .. ... ..... . >....:. ..... „..... : , „,.�:;�„_:+' г � .:. f,..:: г.;}..:2.:с� .:. .:: �..: ' : •:.>ч.: - . . , :. .:: �, -`::' }; -,...' .:.. .. . �. : -.,:::.� ::. ... ... . ::.. .. �. ; w��l�'�; .>�;...,...:..:ш1?.... .:..:�....ч.. ��!.::.:р.:. . г.:«::':.ч:::.'..::�:. г::�-.:::::. '�:��. �яд�::да� o��ii� :о�:: �;�т #�3zв:thGa<ко�в:?aEP�ir�;: ь�гаtв'�сц�r;;> : :.:;.'<.:'::�:: :: Г г`:^t: :;г'r'; •: .г.,., .,... ,. ., : ч:;:i:}ь:::::.гл':;......:;:::_..}:..}:....... ,.,, . ; ....,:}у :....;. .. ... , v:,w . . . . . , �' �' :• � : . v' .. . ...и•:: .: . ::: �.,-:: ..: .. . .:. .. . , . . - , ь:. :.. ь.+. . . . . .. . л , . .:-.:г,� ..: ...... . . .. . : ' ,.. .. ... .% ,.:,.. . . .. . .. .. .. . , .. . . .. . . . , .... .. . ..:.... ... . .. . : ... .. .... . . .. лл.ч ...ь..4ч..... ...... .ь .... .. .. .. ... .. ... .... ... . .... .. . . . . .. .. .. . . . . ... ........ . .::�. .•:<:•: :::.�..�: ;.:..::. :.:'.-i.г л.: : � . �ь :..'�.': :. �..,�,г:::: :п' ; . .у.. .ьг ::: •.ц. •. члчг.ч::.vч.:::: ': ь. ::.:ч ..:. .::: .. .. ..: г. ....ч::.::.:: г.?::. }. ... ,К. . 1 . . . .... .. .: . .. ... . •ь. . ........ :::::......:.: . :..:л :•.. ...г:^: . Ч . .. .. \ .. .. .. . . .. .. . .. ..... . .. . . . .... . 'ь ь , , . . ... . ..... . ..... .k ... ... .. .. ..v.. , ... . . :..... .:: ... .: . ч . .л .n . }........ ь..:...и.... .....ь .......:.. . .::.... .п..:. .- . ..ч .::,:л,,..}.} . ':::энi::'::v'л-г::�:::;::гS.;::i.:}.:: }::::':v� И �. .: . . ;гhь}.\..г....Jг.ч......}.., ,л..... .... 1... .... ....,в:.: . . . . ......... .: ... . ... ... ...:: •;.: . . , : :i :.}.:.. -: .: '��..:� `�j . .. . .. •�8.:.�.:$tT i �#I� al�.� fi�3 .. ... ... :} . ... . ... ..:. .: .:. :.: .Кг:'}.::,.<г::э�ч:�:.. . . ,:.: . .. �`г... �::L<'►.:1+��17.� �Я!G�`�:�Уд� _ .1.,..: : . .......ч .:.... . . .. . ...э� .... . .. .,. . .... .. . . . . .... .. �I��::::'�'�::�:.:+���?��'�'�::<;< • и,ии..v.,v.}}::.ь..:..л4�::.:..6'�;� .:.:.:......:: .:.::::.#.,.!,;,J,,..:::. .:.'.. .�„ , ..i'.ь ;. �:г?: -..�Е.•:: ... ::.,чг.,,.., ' • . . . ....... ... . •�: .? .: ... ..::г .:;а:: аь....ь„�. и ... ..,:. :.ь.....:.: ,,.._::,,..... }: .}�,.-r.r:.: . .r-... ....:.:': г`.r:г >�ц�}. 1{ �.. w�в w ,.4 . "�\'::'.}'�'4::г}w:3г:/.. • � � `Yfы�v <а'�t�1'z:�б�(?ц��:fIE�YBa,O ��i;pl�8. :`��в:<;уΡ/t t►�B�pYi::`: ::: � уΡ��� и ц �м.ж . ... .. .. .. . ... . . . . . .. . . .. �@.. ... : i?. .... ... : г�:'.�. �:::81�x�iт<::�:�?��?'�з".<';' .'.. •. .t v.:3•:.....:.:.г.м.г.;:й!!..::?::.::...,.:.�.;у..х.г.::n......::.:л.....F..: г .... ....:.:.... лл:... .:. - .чй\ль\. :.1: л.. К.:. Е.,...,.. с .. . .. ... ... ...... ..... . ... ....... . . ......... .. . .л. . ..... . ..: Yi'��3h::i•F Т.•.}м 1. ... v..:..: ... . . : :.:.. ......:.: : .: • . -л : :•.. :. ...:г.4v:. ::.... . . - � ь . ... .:.: .:ч.: :•:.вь:3 .:.i'б.ч 2::}i5: н.л: ti� :�Е ?Е1бе.g' �' `.:�f:i<►�.. Ч`п):���:�.... �г�::l�1�`:} ::#��>. `.� . :� � �� :i�e tвв �-:� � �$� . .,. ��,...::.:..�!.,.:...: :> :..::.: ....:...�!��!:::.::,ч.ч.:.., . . I�ды?�е?$.! . :�<`<: :;';t�t ��'.>гз>> : г ... .. .. .,. ..... ..... ... ... ..... ... .. ... .. .. .... ... .......: .::'. _::::-.' .. ...... . ., W „. :а... ... _... . ....:.ч:..::..;.. � г ,:..:..ь<:.: ь: :..: ,,.. „...-:}... . . .5. .ьь ..ь :�:::. ,�i... ::ч:: ч:.::ь:. ч::.,п,гл...:.::.гч.:.гr:: - .,... ...ч.гя..т.: `iн... :.i'`� ; '.:iхч •;3 }�:_ . . . . -. . . ., .: � . ь... �,,s;,,,,,уΡ,, . • �уΡ •�}уΡ� �Q � �.::� .R`У:��:д'�о. ..вlг,'.1S''�~�л �:1�;�� .Wi4+RtM1:,. r: . . �F�i�4 .�'.�<.r>:; .�.:• fi� .......... .. _.. �лС'�' :....� •' ..�.. л . ь•ич: ;:г.н,., •.. ' ..:.. .. ь..��::: � ..:�`�::. ';�r!::.::.�'!�':"�!.ч: � .,. ...... .ь. ..: . .: .:. ,:':.,,-.,-..: ::: ;-., - :. : .}}г. ...�}:.ь.3.:. :.... .::.:..::::�.::<. ;..:. . ... � •. .. .. ':}. . .; ....ч .::: � • ... ;.: . л::: . : .. . ..аг�.::а}r �:.:',... :.:::::....:.... {� � `'8, :::ti.:: ;v� ,; , } ,:,:: .... ..,: . .... с:};.:�. .::'.;' : гг�. .:� . �� �> .:�?Р�; . . . . .. .i��:Q�д:� . . . :�i; - ..:. . . . . .(i...��: ..: � � �' � ` 1t ь :`ггs'г:}.,. г . . � . ::�.... :. ... ...:.... ..... ... .. . . . . ......... . .�. ��:. ., . . .:.::. :;��#[ь �.:��•�1х�. �� .1�:<::;'':л�:.. �•: <}�>.3. � ч.....ч.: ,..... . . . �•г. ,}.ь ..�!�`:.. ..��� .ч}.:. .�;.�л •ц?�- л г..:. :.,н }:: ._....ь...:. .ь..ь..:.ч::нл:.•.,п:.,.:...:ь.v:1.г .ча:: г^:ю:м:.,.:.:.у: }ж.s,} : .}: v.:'�,'<:;. :.:.y,ri. ., w:. iv:,: ....и..........С:ггi:пv, : к}'' :::л::; f1ч'�+,:`' уΡ :_��:: :':i:.v:!.}:л:: ;: ..::::..::::.: .: . .. ...:: ... . . :: :ьи.. .... •},:3 i?.: и.: л}}: .г::w::..ь•}л: i:::.:•iл: .. . . . :.:ti :; е.\•:: ь}.. :,п., г:у.;. 2ii:'г: ..:... ь. ...: •. .::. : .: . . . : .. , =•г: i: г ..}'.� : ... .. . ., г�:ц:.i: :::.:.г:::.лL�� :'::2'•гь.�'::iч+::i:.'�::.гf:',L: �t:��'� .:SЁ::i�.~:i}::: } .}. . �.� vг::}:,:..:•::3ё г:::У:}::i•'.:"i:Yл:ii::in",-•::.::'.мК:.: �:� 1' � . v : 'ч.. . ьК:. }:i.'iл; +>г\:4 .. ., :. ... • - :?F.! f . . . . . . ь.ь. л.:...::..v.:}Л?.:С� ..... .... . ... ..... .... ...д..:. . , ....г.:л ,-..v�:.:i:iiз.:-чч:.::::-:гi ti.'i'ч•:ti�:С:г ь:ггггv �:3}.iёг:�}:${¢::}:iл:лв:; . в \ь .Zп. ...,л... .л.о.: ..:..::.ч::ч:ч:.::..ч:::у::..ь}.:ч4:.::. s.:.ri:.. ' ..i: .i ,:гг.•:гv::iv'г.::..г....л.гг.. .-........v... ..л . .:. ..... ...:...... ; l:i?:iг�н`лг~{.�7..',У,�.лвг.ilмЧг;::�; ..г.:.::.:}.?:::::.:::.циг.>v;:..:чг::г•:v::::::. •::•:::::ч.ь-:.:.а..}... • -:.8г� ..4л. :. л..:.л h . . .... .......:::. . ...... ... :•.:: •. . .:... :.ь. S . `й:.. "R t�л..ь:•:.... :ь:. г . ..:...: -.. .,. и.,.г ., :��. г:`.::�г.:::.ь}°:,.::-:,..,-:.. .. �:..: .а:....:..-,;г-.. ..,.:..�:..,. ..;7� ........:. ., .:....:: .:.�. ..,: . ..г .. ..:... .:. :: : .. ..:::.s.:..::т:3•::r'.С.:.;х>;^..::,;. ^._;,гг:rац'у':>;;::}r:_.;�iг;^.:.};.';7;:.;,. x.,. a� : -ц�� �{,, �.�уΡ�� �:л.� 1s.,� ги � �:^iВНВО�й: � 4-�j '. .: . - . . .. г`.ь.,; i у р,!? а ':нг.. +�l� �i4Ч� т,Wо'::М� ..fFRMЧё.'.. г r.. . . }л. }\ � s' �..: ��А. . ��?1.�`� � � �4а�:'��$:3�� �LiT �t' �8� • ...,; ...: . 4 М+4 .v,.....:...;:..: л . :.........: :..ч: :.:.: .: .. . . ... ..::. . . .:: : •. : . г=}:��. , : .: .:-..;�:::.лv:.л ... . .:...... ...: ..:: . . ........:,:гУ..г.г'*:tr:�?':. .л. ... л::::.::. 1�ь`• . г.. } ....::.. :..:::.:г: " � �' : ♦ . .л: •...1'.:i „t '�' а:2: '.::.M1: . .::.: . ..:лцv.: : ...:ч.. ::::: .. •,\л .л. . . м . ....г.: ::. . в. .:: ;}i''�':.'-}" :k.. .� , � кt��"<.,.:.::.};•:,;.>i.::: `�'. ' . )t::�:��' .�.�f10��:�::�:.�81д�`�цС8У1t'1V+�8"':�C�Q�C С�::. :.<•..��''`�.:..'� .1i1 0 н.�г%; �'�:�.,�.,......�..�:.}:;г;,.:3гг;.::.,..�...:.�}::;:•,?:..:::::::::::'::'.:::..;,},:....:.. .:г:...,..а.,:..ч::::.:�.::&�:,,'..:�b:«::...�г . ..� . .::�..�.....::.,.:,... �ьгh{... . г.. "��'� .. ,..:. .. .ч. С. м.:;..ч . °.:.�::.:;...г .. ,.. .. :.3 :•-�� �:2iг:::::ii� . �� ..,ь�::::•гл ..ч ., :.г. , и.:.:':. , пw.} .:. - .ь .- .. . . -. ..' .: .л .iгг:?гг л �уΡуΡ. �д �,,��j уΡ, ��; : :, IY18" . .�,` . .;;. ц ~-w_.� °� �tV ��*{����°�� °рΡц� �j{�� • .. �lг�vЧ' ��?�ам�::?�.,rм.�оо�• ��..•,.-.....:,�. ;;�г. �ии(f�`� w:w„�.'��::м, г �:���:��У.< ..s.:...... .:.:....аг�:}ч:ч:.., г ....:....... . . .:.:; .:: .......:•....ff.:i� .L.� f��S.:. ::ti<:ь :�)'",f,�;;M:. :::![:•;2^:.;;::,: �.}., л3i�у }пг,.:}�-.:}х'3::.:,г..ма}}.w.,:.\.:...гг::..у::. ..:гч.�:::. ..,:..г., .. �. хл: \. .х.п:.г:.л.}".л. ' ' ::4. ::t:iь:: �:.::... уΡ '.:;:+и :............zv . ..': ..... г.:: '': ... ;,..... ;.• .:.:.:." i�.;: : - .iг:::г::.},�:гN�л: '��p. �ц �г�г :.л:•. �.к"� т?�1 :: .) ;':7�8�:i�� ��:.���et1��::�112tLi)0 � :. о `.'�... : :::.>. . г�: ч: л . . : ....:<:...,. ��?�'..Х... ..: ...... . . . ........ . ..... .. . . ...... ... ..... . ��. .;�:`�а!к���`.� :::�iг�_'t�` ::�г.� . мь..,s`�,,.,.ь\�7.'т::::;;.::.ч..:,.. ч: ,ч.:.ь..,ч.::г.:..:..:..: ,ч: ..:....ч..::.�:: •:.: :.:.:.:: ...:R�: �?.� ! ��i� а..: ,.;. :.. :: �..,гч..:: ь...,:ь., .. .::: :ы: , ,. ,;,, . .. .. ..:. :...г:: � :уΡ��_.� Ъ i .�' -., ,}. г"s' :. ; � ;: рΡ =,tтt 3:.р. ',;:��'•!Е �.<;;са ;}•.v.. � .`�,... ::�1�:•'� � .<� црΡ ц� :<� :::��� � "�ч...,,,..г�•.:<'.•. ,.;�х .. . ° . • �' � �v)iiii� .... , ,. 3� .. .:.::::.:..��: х.: ..: ,.:. .: л::.:...>�. 1� _ �У У . �:�с�ег;:?в(�t�r лч�tь3t}. ,.�.. .r. ..г.: .:{'.:;';". �.а��� -:гг ':::г:� ...:,.. . .... v �:::,.......н::. ....ь.::......::::.:,.,.:....:.:::ч':, г.:w;:�:. :?У ' :: ... . . .г.Е. н}:н . . mii\вь:,vи: ?ьiл:^3:г::..ц.�...ь.ь�, ..... }}',..:....... .....г . .,.... .": • уΡ лtiг`}г<}`:� :г:k���4,У-ы, :4 :}��• рΡрΡ�q� ��ь� ['" : �' �� ��уΡ.уΡ��F��jв �(� ({ . !!!`!!'г . .1�:��:�'. .� . .F'P1+�Nw'...�.'!в\iM... . •{*�лg, �'_:�уΡ ..:_�•: г,. г}. .:дRW:.г..•.,:.:S,,•М.'М!'� : �• ��(� : в .: ц� � ц{�� .У::�.�.:.... : .wf.. :.�'.. Т!Ч•Т , .1' +.•л\�"�'гйгЧ:.ч}.':'�3t;:.:°г,л:г'г.'г;У.}•.ц}}:'чг`}:1.;.н..;.};:.г:::.};:.::�.:•г::ц:�:?`Т.:. W?.Т.}'�:�..-^:::..;•;г...л:iг',:,•.:7'^"`.;...:г�t:{•:::v ..К У• �?�.°. 1 3;?�,г.: :.:'.;$:}/'�.�:.:•С{. }�ч,:�•.ti •.qк• ;.} :. .:....:.:::: . :л;гi;:�. .:.3..А}:�г'•ij;:::}�:уΡ ?�':г,�:iу�.:г: ль��\� ц�ь,Ул...+: чгiw ,ч _ }� �уΡуΡ� '• �} •: /��� '� }�,�+�,� [([@� �(� : h�' ц}�.,3 . �Т � ��j 'г.� �j У�-` }�!1l1�.?�ь ':Tf�•' �.:�..�?:. }i1��+T": уΡц�i .тТ)г[![:? :}1HL•�ц ����ь'AR'. . •��F": . �.•:::::.�� .i•�S: Рг?:г. :: гь: .:..: ., .: .... .. �лr.с�;г:: ;:ь,:. ::� :r��x. }'.•`•!}Rгг _ ,::�?`"•. `�.,.•.'ь\ч.\ • ,.•.�:.... `.M1... .}гz: :':С... t:.'r.? �r�}+� :':Sг:`:.г. ...чг. г4,. ,гb.г }}}.х.;: .}:�'л4. . ... :л, .�уΡ::,:., .., :. ....,;.r:: ;с`лг::3-`:..�;.,�:?'.. :.г л,.:л: �::: лг.. . t..`г.;г..г M1:с: `ч, . 1\ ..R:Y. Q'г :�JГ.'рг :Х�ь=�. :i:'У.г� ::{1г.:. • Hf �?MN' :,'��' 'Cti}$ }} .}: • µ ...::лч•:- .. :::.:г.�.;•;�гг � ` ..h.. . ч.'� ..;;1,.. :i}!ч' :.3гS}:Чё�}} F;.h:fi. ..М!. . . ь. •:. .v::::.v:.: �:i": v:.ve::{}:.: •. . г:..•.г:�:..•..,v.:..,::.v:......... ..ь•xnv.л..Lл:.i.:v.v' .м. '`г''�К<глi::гг :г4�': i.:h' .^::3i. 4 ц г,.�., ..Л! . .ч}г1 "н}.:i?:ч ..}. г }:г :GЧ} .�j�.�} •.... .лг:::.::::л:..:ч:лг...:: цч.: :•:л.: :•. . . ..л...зь :. ...ч: ь:v:' .:ьг:•'. i::.нv. _... г.гС::г._;i.\чгч:4}..... i ' ё � .\г.}. . ...3:}:л•лг�и :г:3.. •'гу•у:3г :} i1W.гi4лu глУ.vv}ti}ь.•}•F.:�$:� t:й$•:3 iг}}:У.:.?л::�'ул. �.ж:г}. hK.h•:1fi�;}:.:.ч::гл x.SV::}..ч,г..v..: r,..:..v:4:�ifCг.h...... д}хг ;.,г::СС.iг`^' v.�S.-. . ь v..} ., , } .wг>rsч.�S` �i '� -'-"з:'.:г:•г'.' ..л-,..._•..v .........:........1..::ьь;х:.г.r-:ь,:-:. ...:.'t•:':У:с::�г-.хь:.-.....:':::.'�....:.....,:.:}:�.:}чs-хг..:':гi'i;:?:::н:}?н.л?}ц.::..п::.. ::.:.iG::�.....::,.гл'ц.k � . Т1о5 �у fW'ФвР rВСоIIlФвцд8 •�8t �д $�С10$ Р� 8lцl deYelb1ll48of ($,�) �I1)г �UBh'iЯ1 �ТlqBr thв ' �. `' :�п1С�е1аоnlд 3ook f�nr fntermediariвa that uadвrstand RBt;D, �6ave tвchaical expвttiae, ацд аt+в й�{ow]edgeabla ., aiгout vа�гiвоцы iк�ивцIвв atLd агв comanittвd аад wiцiлg W taJcв riвks. Although дхв govвcдment'а prese�cв � .: .iрдцу 1ie цеовsвагу to get �дtед. виQЪ fдtermediarias вЬоиlд Ъаvв вtroug private авсtот. pacticipatiaa аад ;�'�ирроп аьs wв1] иs complete autonomy ва► that bwвaucratio iцtssventiona are avoidвd аад tt�в Ъ3gl: qnallty of � ' � �tдв• staif вnsured. �£�, Iu �regard tc .RdtЛ iдвцhitions �nd their iinkв ta iudustry, thв shcdq wагдв agaiaat тiврlасед optimism. �С . '� �ч`�pюpgty suggвats �at tDe represeпtaцoa of Шв privata sector in thв poцcymatdng bodies of pu6lic R�D �. •• г�3tl�iti:tione wIl1 ьв]р tYte 1i�ge, while tbв R&D iGnstiwtions that тау 1:в веt ир throa$b tЬв collective �:patticipatiop of pdvata вactor &щв rвqulre gpverament вnppo�tt аяд 8nяncia3 osaiвtaпoe. T�te best арргоасЬ � .: ��poipr�ucd'вtteulgRh,epiug tввеигсh iit�tgнg евзtкtа W 1ю оив whвгв 3t ia сlватlу teo0gnixed Шаt 0�1у t�ою тeaesc+ch ' ,, .�иtвв witlt 'аетиiсв т snd�try cuhute соихд bв iastrгtmeutaf i�д вtrвt►gtb,ening tlye liu[�е адд wl:ere кtиу' : ��. { s��l�вid+�atlau of tbe RBcD.activtdes i� fiaanciaцy matched by the Ipdustriai вдtвгртiввв. . . ' а. �ап]r Leпdfпg for l�us�t Technвology ,Developmeпt, QED Rерыс No. 12138. Jипв 3Q, 1993. 30 D. CONCLURONS 1.52 IMe principal findings of Chapter I can be summarized as follows: "ect outcomes 277 operations were evaluated in 1992. For the first time since 1988, the proportion of projects rated as having a satisfactory outcome improved over the previous year's cohort-ftom 62 percent In 1990-91 to 67 percent in 1992. Still, this was the third lowest share ever recorded. On a commitments basis, performance improved even more In relative terms, as the share of Bank and WA commitments allocated to successful projects Increased to 75 percent, i.e., far above the rates of 1990 (58 percent) and 1991 (66 percent), and broadly In line with the historical average for completed projects. All regions except South Asia registered significantly higher rates of satisfactory project performance. From a sectoral perspective, agriculture, human resources, and industry performed significantly better than in 1990-91. Relative to their averages since 1974, six sectors achieved significantly higher ratings in 1992--agiculture, transport, human resources, industry, urban development, and tel Common characteristics of outstanding projects were quality project processing, strong borrower ownership, active beneficiary participation, and effective institutional strengthening. The performance of adjustment operations declined from 69 percent satisfactory in 1991 to 59 percent in IM. Eleven of the 12 unsatisfactory adjustment operations took place in Sub-Saham Africa. Developumd Nearly one-third of cohort operations substantially achieved their institutional development objectives, while dose to half did so partially. Sustainability 7%e sustainability of project benefits in the 1992 cohort (about 54 percent) was broadly in line with that of the 1989-91 cohorm. However, at the sectoral level, ratings showed substantial variability, with industry and telecommunications improving significantly, while power, transport, finance, program and policy, urban development, and energy declined. 31 Project Outcomes, Institutional Development, and Sustainabillty * There is both a strong conceptual and statistical relationship between these three parameters of project performance. Approximately five out of every six projects judged to be likely sustainable also achieved substantial institutional development. Other Aspects of Project Performance * There was a modest improvement In the achievement of physical goals, as over three- quarters of the operations in the 1992 cohort substantially achieved their physical objectives, compared to two-thirds in 1991. In regional terms, physical achievement was high in ECA, South Asia, EAP, and MENA; intermediate In Africa; and low in LAC. * On average, delays in implementation have continued to grow. Implementation of cohort projects took half again as long as had been projected at appraisal. The average time overrun of 51 percent was substantially above the three-year average (1989-91) of 43 percent. * Cost underruns continued to be the rule in the 1992 cohort, averaging about 4 percent compared to 8 percent in 1991 and 7 percent during 1989-91. More than twice as many projects experienced cost underruns relative to those with overruns. Thematic Areas * While many of the projects surveyed were designed and approved during the early 1980s, and, thus may not be altogether representative of current Bank practice in the thematic areas, they offer, nonetheless, selected examples of a positive longer-term impact in reducing poverty, improving the environment, and supporting private sector development. * Shortcomings in meeting the Bank's thematic objectives are most often related to a narrowness of scope in project design, the failure to monitor progress adequately and undertake timely mid-course corrections, and exogenous country events, such as political unrest, which undermine borrower commitment andlor the borrower's capacity to implement. 2. PERFORMANCE TRENDS AND FACTORS A. PERFORMANCE TRENDS 2.01 Chapter 1 focused on the key performance indicators emerging from the 277 projects evaluated during 1992. This fonMat highlights results and findigs emerging from new evaluation products. It also provides continuity with past Annual Reviews. 2.02 On the other hand, assessment of portfolio performance based on annual evaluation cohorts has Its limitations. The 277 loans evaluated in 1992 were approved over a period stretching from 1975 to 1990, with peak approvals In 1982-83. Hence, the implementation periods of these loans vary greatly within any single evaluation year, and overlap heavily from cohort to cohort. Yet, performance factors which vary systematically among cohorts organized by year of approval are obscured within cohorts organized by year of evaluation. Among these are: (a) changes in Bank policies, such as the shift away from operations having predominantly physical components toward projects having a greater institutional and policy focus; (b) the increasing numbers of 'problem' projects which have entered recent evaluation cohorts, after experiencing long delays in implementation;' and, (c) global economic trends. 2.03 To better account for these factors, trends In portfolio performance have been assessed by ecamining all 3,137 M l projects in OED's database, organized by the year in which they were approved. Comparatively few loans approved before 1969 were evaluated-170 to be exact. So, these loans were grouped together. Similarly, the 235 rated loans approved after 1983 are sparsely IFor example, 11 projects in the 1992 evaluation cohort took ten or more years to implement, against the historical cohort average of 6.2 years. Both the 1990 and 1991 Anmual Reviews noted that older operations performed 4Vndwy worse than operations aroved in recent years, or than previously-rated projects approved during the same years 3 The analysis contained in the 1990 Annual Review highlighed a smjor deerioain in project performance measured by year of approvaL The projects in question wer all kncluded in the 1990 cohoxt, but had been approved between 1976 and 1979. Noting that less than half of these projects had roeived a satisfatory rating, the review attributed this in large part to the effects of the second oil price shook of 197940 and to the onset of the wedd economic crisis of 198142, both of which occurred after these loans had been appraised, but during critical stages of their implementation. In this review, considrntion was also given to looking at projecls on a year-of-closing basis, an approach which would also give due weight to the period during which the loan was implemeAd. In the end, it was felt that the year-of-approval basis was preferable to the extent that it captured the effects of implementation period as well as year-f-closing, and was superior in capturing the effects of changes in Bank project design standards. 34 distributed across approval years, and, therefore, for analytical convenience, have also been treated as a sitgle group (Annex Table 2.1 and Figure 2.1). Figure 2.1: Performance of Operations Approved (196844) 100 80 60 40 20 0 Through 1968 1970 1972 1974 1976 1978 1980 192 1984 On 1969 1971 1973 1975 1977 1979 1981 1983 som: Amus 2.1 Approval Year 2.04 The aggregate data suggest that portfolio performance has experienced three distinct phases. The first lasted from the late 1960s until 1973. During this phase, the unweighted share of evaluated projects which received a rating of "satisfactory" remained significantly above 80 percent, reaching a peak of nearly 89 percent in 1969. The same pattern of relatively high and stable rates of satisfactory performance is reconfirmed when ratings are weighted by the size of loan commitments (Annex Table 2.2 and Figure 2.2). During the second phase, from 1974-79, a sharp deterioration in portfolio performance took place, with the percentage of projects rated "satisfactory" declining to the mid-to-pper 60s. During the third and final phase, for projects approved in the 1980s, the trend in performance shows the "satisfactory" ratios oscillating around the 70 percent level, with some Improvement in the performance of relatively recent projects (those approved after 1983 and initiated In the mid-1980s). Given this apparent levelling off in performance ratings, quesOns arise regarding t permanence of this trend, on the possibility of a *bottoming out," or of an incipient recovery in the satisfactory proportion of the ratings. 35 Figure 2.2: Performance of Operations Approved (1968-84), Weighted by Commitment 100 8o 60 40 20 0 Through 1968 1970 1972 1974 1976 1978 1980 1982 1984 Onward 1969 1971 1973 1975 1977 1979 1981 1983 Source: Anne Table 2.2 Approval Year 2.05 In examining performance by approval year, it must be remembered that in terms of approval year, the share of projects completed decreases significantly after 1983. Hence, these more recent projects may not be representative of the universe of operations, either in sectoral or geographic terms. As of December 1992, completion reports had been submitted to the Board for just over 20 percent of the loans approved after 1983, and only 12 percent of those approved after 1985. In addition, there may be a possible bias arising from delays In problem projects, as better projects may be completed sooner; for example, evidence shows that during the late 1970s and the early 1980s, a slight improvement in portfolio performance could be attributed to the on-time completion of several projects. In this context, preliminary projections, based on that small sample of projects approved after 1983 (which may be not be representative of the total population), suggest a possible deterioration in the ratings for at least two or three subsequent approval years before showing signs of a modest improvement.' Several simulations, usig the performance ratings of the rst 10 percent and 25 percent of the projects evaluated for each approval year, sUppot this direetion of the trend. 36 2.06 The three-phased pattern of project performance described above is supported by evidence provided by an analysis of the *gapa between ERRs projected at the time of project appraisal, which, historically, have been higher than the ERRs re-estimated at the time of project completion (Annex Table 2.3 and Figure 2.3). Although the break points between phases appear to differ by several years, compared to the pattern for rating of overall performance, a two- or possibly three-phased pattern is supported by the data. For projects approved up to 1972, the ERR-RERR gap In any single approval year never exceeded 4 percentage points. However, after 1972, the gap widened considerably, oscillating from a minimum of 6 to a maximum of 11 percentage points through 1982. Since 1982, the gap has narrowed once again to 6 percentage points.' Figure 2.3: Cohort Appraisal and Re-Estimated ERRs, by Year of Approval (1968-84) 30a 27. -24, 21* S18 15 12 a 94 6* 3 ThroM* 196 1;710 19'72 1974 1976 1978 19 li 11~ 1981 On 1969 1971 1973 1975 1977 1979 1981 1983 Year of Approval Source: Annex Table 2.3 0 Appmisal ERR 0 Re-estimated ERR 2.07 What factors might account for these variations in and for the recent levelling off in portfolio performance trends? Three potential causative factors are listed here as hypotheses that require follow-up work: 4 Note that there has been a distinct rise over time in the estimated ERR at time of approval; this is true irrespective of whether the data are organized by year of evaluation or year of approval. Disaggrsgating the data by time period and sector shows that changes in sectoral composiion over time have very little impact on the average resulos. For some of the reasons underlying this upward bias in ERR cetimates, see p.15 of the PMTP Repor. 37 (a) Changing global environment and country economic conditions. Given that all of the projects during the middle phase of serious portfolio decline (1974-79) were appraised prior to the onset of the oil price, interest rate, and recession shocks, their design was not able to anticipate the sharp fluctuations in relative prices and in aggregate demand which ensued. (b) Increased complexity of objectives incorporated into Bank projects since the early 1970s. Assuming a learning process by both Bank and borrowers has been underway for a number of years, implementation may have begun catching up with the often-innovative requirements of new-style development lending by the early 1980s.s (c) Sharp expansion In lending levels which, when it occurred, might have caused lower quality at entry.' Regional, Country, and Sectoral Performance 2.08 In relation to the overall performance trends, there are significant regional, country, and sectoral differences (Annex Table 2.4). Of course, common to all regions has been the adverse impact of global economic trends prevailing in the late 1970s and early 1980s. Sub-Saharan Africa and much of Latin America-economies highly dependent on imported fuels-suffered most from the rising oil prices that kindled domestic inflation and severe balance of payments problems; further, many debt- burdened countries, especially in Latin America, faced declining net capital flows from commercial banks. Regionally, project performance in Africa fell sharply among projects approved in the 1974-79 period relative to those approved in the earlier years, but improved among operations approved during the 1980s. In EAP and MENA, project performance deteriorated during the latter half of the 1970s and stabilized thereafter. By contrast, in LAC especially and in South Asia, there was a sharp decline in the performance of projects approved in 1974-79 compared to those approved prior to 1974, and a continued sharp drop in performance among projects approved in the 1980s. In ECA, performance was stable among projects approved in the 1970s, then fell substantially among those approved in the 1980s. This was clearly the case with respect to freestanding technical assistance (TA) projects, for example. Here, the proportion of projects having primarily a straightforward project support (PS) design declined steadily from the early 1970s, whereas the proportions of projects with more difficult policy development (PD) and institutional development (ID) objectives rose substantially. Further analysis showed that the highest ratings went, on average, to the *easier* PS- oriented prwject&-73 percent-compared to a *satisfactory* rate of 66 percent for ID- and 57 percent for PD-oriented projects. An OED background paper hypothesizes that the improvement identified in TA project performance after 1980 might be related to a learning process, drawing appropriate lessons from the failure of the first generation of PD and ID projects approved during the 1970s. However, due caution is made for the small size of the cohort of loans approved during the 1980s and possible biases in the sample attributable, inter alia, to the fact that with their shorter average implementation periods, successful TA loans tend to appear in OED's database earlier than unsuccessful loans. As discussed in Chapter 3, an overly rapid expansion in agricultural sector leading in the 1970s may be associated with declining project performance in that sector. Whether the Bank's capacity to design and appraise projects, or the countries' capacity to absorb more lending, did not expand commensurate with the volume of lending is difficult to determine. 38 2.09 Differences among countries are even more dramatic. Project performance was assessed for all of the 70 active borrowing member countries that had at least 15 completed Bank loans or credits through Dtcember 1992. These countries account for 90 percent of all completed Bank operations reviewed by OED through that date. Considering the entire period, average performance levels ranged from a high of 100 percent satisfactory in China to a low of 45 percent In Peru and Sudan. Other countries with performance ratings of 85 percent or more were: Botswana, Chile, Costa Rica, Greece, Honduras, Jordan, Korea, Mauritius, Myanmar, Pakistan, Paraguay, Sri Lanka, and Thailand. At the other extreme, in addition to Peru and Sudan, countries with ratings below 60 percent were: Algeria, Argentina, Haiti, Jamaica, Kenya, Liberia, Nigeria, Panama, Syria, Tanzania, and Zaire. The full list is presented in Annex Table 2.5. 2.10 Over time, the satisfactory ratings of project performance by country depicts four general tendencies, reflecting a combination of: (a) the uneven impact of changes in the external environment; (b) the extent to which these countries managed to buffer domestic economic activity from exogenous shocks through appropriate economic management; and (c) portfolio-specific characteristics. The portfolio performance trends for countries with at least five projects in each of the three time periods are presented below: * Steady or improving performance throughout the period under review: Botswana, Chile, Indonesia, Korea, Malawi, Morocco, Sri Lanka, and Thailand. * Declining performance during the 1970s, but stable or improving performance in the 1980s: Burkina Faso, Colombia, COte d'voire, Ecuador, Ethiopia, Ghana, Honduras, India, Jamaica, Jordan, Kenya, Madagascar, Malaysia, Mali, Nepal, the Philippines, Senegal, Tanzania, Tunisia, Turkey, and Yemen. * Steady or improving performance during the 1970s, but declining performance in the 1980s: Bangladesh, Cyprus, Egypt, Pakistan, Papua New Guinea, and former Yugoslavia. * Declining performance throughout: Argentina, Brazil, Cameroon, Liberia, Mexico, Nigeria, Panama, Peru, Sudan, Zaire, and Zambia. 2.11 Of course, comparisons of country performance overtime do not fully reflect the extent to which countries also operate at different 19293 of performance. Both measures of performance are relevant. 2.12 As in the case of regional and country performances, there are substantial variations in the percentage of operations judged satisfactory across sectors. The sectors at the upper end of the range have been telecommunications (90 percent), power and energy (87 percent each), transport and urban development (82 percent each), and human resources (79 percent). The sectors with below-average trends include agriculture (64 percent), tourism (64 percent), technical assistance (67 percent), and industry (68 percent). The percentage of satisfactory operations in the remaining sectors have ranged from 72 percent to 77 percent. (Annex Table 2.4.) 39 2.13 In terms of changes in sectoral performance ratings (for sectors with at least five projects in each of the three time periods): (a) the only sector which has maintained a stable satisfactory rating throughout has been urban development; (b) the only sectors which performed worse In the 1974-79 period but improved performance ratings in the subsequent period have been energy and transport; (c) the sectors which performed worse In the 1974-79 period but stabilized their performance ratings subsequently include finance and human resource; and (d) all the remaining sectors witnessed a continuous decline In performance ratings. Sector-specific issues that could possibly Illustrate variations in sectoral performance treads include: (a) Institutional development, and the extent to which It impacts on project management (virtually across all sectors); (b) staff development (particularly in the human resources sector); (c) borrower's absorptive capacity for expansion in Bank lending (as exemplified by some agricultural sector projects); (d) timeliness in project implementation and building up to capacity (as highlighted by the experience of several Industrial sector projects); and (e) degree of project complexity and specificity of objectives (as witnessed in the changing focus of program -,A policy lending-from simple balance of payments support with litfe policy content initially to highly- conditional adjustment lending In subsequent years). Portfolio Composition 2.14 Starting in the early 1970s, the Bank's lending portfolio underwent a progressive diversification. The focus shifted away from the provision of physical infrastructure and support of large industrial projects-first through inclusion of projects directly concerned with poverty alleviation and meeting basic human needs, and then through efforts to help borrowers contend with the energy and debt crises set off by the global shocks of the early and late 1970s. In subsequent years, the destabilizing impact of the recession of the early 1980s led the Bank to develop new ways of transferring resources to ease growing balance of payments problems and assist developing countries adapt their policies to falling commodity prices and other disequilibria. Bank support, in the form of SALs and SECALs, wab provided in exchange for the countries' agreements and willingness to undertake the needed policy reforms. The changes that have taken place over time in the relative shares of the Bank's portfolio by reglon/country and sectors are shown in Annex Tables 2.4 and 2.5. 2.15 The changing country and sectoral composition of the portfolio could conceivably account for changes in the Bank's portfolio performance ratings over time. Are overall performance trends pushed downwards as some poor-performing projects in Sub-Saharan African countries or projects in the newly-emerging market economies in Eastern Europe are substituted for the comparatively better- performing projects in East Asian countries? Could entry of new sectors-such as policy-based loans or in the areas reflecting Bank's emphasis on poverty alleviation-in the portfolio favorably or adversely influence overall performance trends? 2.16 To address these queries, overall portfolio performance trends were compared with "normalized-? trends. For the regicnal/country mix, the results Indicate that the changing country composition led initially (in the latter half of the 1970s) to a marginal deterioration (by an average of 7The *fixed* group consists of countries that had received live or more loans prior to 1974, and portrays the trend had the Bank continued to lend to the same countries after 1974 at the same proportionate levels of lending as in the years prior to 1974. The same procedure was applied to the sectoral mix of the portfolio. 40 about 3 percentage points) in the actual performance ratings but, subsequently, there has been no systematic divergence between the "normalized- and the actual trends. 2.17 For the sectoral mix, the actual trend depicts some improvement (by an average of 4 percentage points) in the performance ratings compared with the 'ormalized tread. In the period 197543, a relatively higher share of Bank leading (in terms of number of operations) favored sectors, such as energy and program loans, that eventually performed better than others. However, in recent years, the difference between the two trends Is again minimal. Thus, the changing regional/country and sectoral mix of the portfolio does not appear to be a critical element in variations over time in the overall performance ratings. This indicates that the differences in overall performance ratings over time were due for the most part to shifts in performance levels within, rather than across, regions and sectors. B. PERFORMANCE FAC'ORS 2.18 Project performance Is a function of three closely-related sets of factors:` * external economic environment and exogenous shocks: e.g., terms of trade, interest rates, and such circumstances as natural disasters or wars; * country conditions: e.g., macroeconomic framework and policies, growth performance, level of human development, trade restrictiveness, borrower ownership, and political stability; and * project processing: e.g., project selection, technical design, and supervision. The objective of this section is to delineate-on the basis of OED evaluation experience--the influence of these factors on project performance. External Environnment and Exogenous Shocks 2.19 External shocks-particularly those related to the terms of trade, International interest rates, and variations in global demand-pear to play a much larger role In explaining disparities among long-run growth rates of developing countries than previously acknowledged.' Further, the external enviroment can affect countries in widely-differing ways; at the same time, countries can OBD is currently formulating a research proposal to identif appropriate data sources, suggest suitable anslytical techniques, and develop testable hypotheses on the relationships between global, country, and project-related factors and project perforuance. In reviewing project-related factors, special attention will be devoted to the issue of whether high- quality procesing of a project (delineating also the Bank's role) has any measurable impact in terms of improved project outcomes. 9 See, for example, William Easterly, Michael Kremer, Lant Pwathet, and Lawrence Summers, -Good Policy or Good Luck? Country Growth Performance and Temporary Shocks,' paper presented at the World Bank Conference on How do National Policies Afte Long-nn Growth? P&ruary 8, 1993. 41 respond to It through a variety of approaches-such as export promotion, Import substitution, economic compression, and additional not external financing.w Usually, the magnitude of external shocks, per so, is not as critical as the timeliness and form of policy reform/responses to them; in this context, some countries may be better equipped than others depending essentially on the structure of their economies. 2.20 The late 1970s and the early 1980s were characterized by considerable economic Instability for many of the borrowing countries. Rising inflation and unemployment in the industrialized countries in the late 1970s led to a slowdown in the expansion of international trade and to falling prices for many primary commodities. Simultaneously, rising oil prices led to inflation and severe balance of payments problems in economies that were dependent on imported fuels. These trends persisted during the early 1980s as the industrialized countries experienced a further contraction of economic activity, continued trade sluggishness, and rapidly rising interest rates. With the precipitous drop In oil prices after 1981, even some oil-exporting developing countries found their economies in disarray. At the same time, the net flow of external capital to developing countries slowed substantially as commercial banks became reluctant to extend further credit to debt-burdened nations. In the period 1980-90: the G-7 ODP grew at an annual average rate of 3 percent (compared to 3.7 percent during 1965-80); the volume of world trade increased only at an annual average rate of 3.7 percent (compared to 5 percent during 1965-80); the real price of non-oil commodities continued to decline at an annual rate of 5 percent (compared to -1.1 percent during 1965-80); and the 0-3 real London Inter-Bank Offered Rate witnessed an annual increase of 4.5 percent (compared with 2 percent in the preceding period)."' 2.21 In this setting, much of the developing world experienced a fall in domestic investment, declining output growth rates, and increasing short-term macroeconomic management problems which extended throughout the decade. Trade imbalances were resolved-only temporarily in many instances -at the cost of curtaliment of essential imports and an accumulation of substantial foreign debt. Also, a generalized deterioration of public sector finances-coupled with the limited ability of many borrowers to fund planned new investments and/or provide for associated recurrent operation and maintenance costs-adversely affected the implementation environment for many country project portfolios. 2.22 The impact of external shocks has nonetheless varied across countries. First, oil price shocks have been both positive and negative for developing countries, reflecting the presence of both consumers and producers. Second, countres with access to concessional finance were less vulnerable to higher world market interest rates. The most critical factor, doubtless, has been the nature of a country's policy reforms. In this context, OED's evaluation experiencen shows that the central components of an appropriate macroeconomic adjustment were: (a) tightening the overall fiscal situation so as to restrain inflation and reduce debt to a sustainable level; (b) implementing real t See Desmond P. McCafthy and Ashok Dhareshwar, Econoni Mocks and the Global Environment, Policy Resourch WPS 870, March 1992. World Bank, Global Economid Prospects and de Developing Coures, various years. Sco *World Bank Structural and Sectoral Adjustaoot Operations: The Second OED Overview." 42 devaluations with the objective of reducing current account deficits; and (c) reducing negative Interest rate differentials so as to stem capital outflows and thereby increase net foreign exchange reserves. 2.23 Among the most important external shocks in the 1973-90 period was the terms of trade, reflecting not only the oil crises of 1973-74 and 1978-79 and the collapse of oil prices in 1986, but also the variability of the price of many primary commodities. The terms of trade developments were reflected In the portfolio performance in many borrowing countries.a Table-1: Countr Portfolo Performance and Tenms of Tradet 1980s Portfolo Perfornance of avestment Profects (No. of Country Portfolios) Above Below Average Average Total Tems of Trade: Improving or Stable Terms of Trade 27 12 39 (1980=100) Deteriorating Terms of Trade 3 13 26 Total 40 25 65 Notes: Countries which received Bank lending for five or more projects and/or programs approved since 1980. The eat-off point for aboveeverage performance is the overall cohort average of 72.5 percent. Source: OBD. Terms of Trade Indoe 1980= 100; period average. Source: World Bank. 2.24 A classification of the differing changes in the terms of trade and the countries' average portfolio performance of Investment projects during the 1980s is presented in Table 2.1. Favorable or stable terms of trade were associated with above-average portfolio performance in over 40 percent of the cases. The adverse impact of the deteriorating terms of trade on below-average performance of portfolios was most evident in the following countries: Burkina Faso, Cameroon, Congo, Nigeria, and Zaire. The lack of diversity and flexibility In the structure of their economies may partially explain their greater vulnerability to terms of trade shocks. Among the countries whose portfolios performed worse despite improved or stable terms of trade were: Bangladesh, Brazil, Haiti, Kenya, and Niger. Conversely, despite a deterioration in their terms of trade, several country portfolios achieved above- average performance; these included, notably: Chile, Colombia, Ghana, Malawi, Malaysia, and Thailand. 2.25 The disparate performances of country portfolios under varying impact of changes in the terms of trade reiterate the importance of appropriate policy responses and structural reforms. Adjustment programs Impact upon the economy primarily through changing relative prices and costs: a For example, according to an earlier Bank Report, an improvement of 15 percentage points in the terms of trade Index Increases the probability of a satisfactory project rating by 1.8 points. Conversely, increases in international interest and inflation rates wero found to have negative implications for Bank-financed investment projects. See Task Force on *The Relationship of Loan Processing to Project Quality," March 27, 1992, p. S. Also, se Chapter 3, Section D, on the subject of how changes in the global environment altered the performance of the Bank's agriculural portfolio. 43 these changes in price and cost variables in turn affect real variables such as output and the allocation of resources. The effect on resource allocation is reflected, in turn, In changes in the relative size of different parts of the economy: agriculture, Industry, and services; investment, savings, and consumption; and the relative shares of the public and private sectors. The data on most of these variables quite strikingly indicate the differences between successful and unsuccessful adjusters; a review of the evidence on supply response and output restructuring makes the difference between the two groups even clearer. 2.26 Natural disasters and wars (to the extent that these are beyond the control of the policymakers and project administrators) are also classified as external shocks. The performance of several unsatisfactory projects was further strained by such calamitles as: droughts in Burkina Faso, Malawi, and Mauritania; floods in Bangladesh and Peru; earthquakes in El Salvador and Mexico; and wars in Ethiopia and Somalia. Country Conditions 2.27 The Importance of a stable international economic environment notwithstanding, the predominant stimulus for good project performance is to be generated from within. In general, the presence or absence of a sound macroeconomic framework in and of itself does not 'explaina project performance. However, the elements that underlie a country's sound macroeconomic management and structural adjustment are also conducive to its good project performance-thus highlighting the mutually-reinforcing nexus among macroeconomic performance, policies, human resource development, borrower ownership, political stability, and project performance. 2.28 Successful structural/sectoral adjustment improves public sector performance and private sector investments by reducing distortions, improving the response to changes in relative prices, and promoting integration of developing countries la international markets. Empirical evidence-gathered from evaluations of 1,200 Bank- and IFC-financed projects implemented during the past 20 years-has confirmed that a policy climate that promotes entrepreneurial Initiative can make a significant difference in the productivity of investment projects." Statistical analysis conducted for this Annual Review confirms those findings. Of the 99 countries in OED's cohort of projects approved since 1980, 51 received lending for both investment and adjustment operations. (Annex Table 2.6 provides a list of countries with five or more Bank loans). The results show a significant correlation (at the 99 percent confidence level) between the performance of Investment projects and adjustment operations.5 " See World Development Report 1991: The Chalenge of Development Daniel Kaufmann, "The Porgowne Rationals for Policy Reform: The Productivity of Investment Prqjects, 1991, Processed; Danist Kaufnann and Jon Iam, *Does Policy-Based Lending Affect the Productivity of Investment Projects," Background Paper for World Development Report, 1991; and Daniel Kaufmann and Yan Wang, "Now Macroeconomic Policies Affect Project Performance in the Social Sectors,' Policy Rcsearch WPS 939, September, 1992. 0 The Annual Report of Portfolio Performance (ARPP) also notes that portfolios in countries carrying out a(Uustment programs also performed better than those in nonadjusting countries. See 'Annual Report of Portfolio Performance 1992," Table 10L 44 2.29 Likewise, steady output growth accompanied by fiscal stability Is vital for maintaining the requisite enabling environment for successful Implementation of investment projects. Prior OED work validated the positive association between economic growth and project performance across sectors at the country level.1 The ARPP FY92 provides additional evidence by showing that portfolios In relatively fast growing, open, and low Inflation economies tend to do better than those in low growth, closed economies with high Inflation." 2.30 For the purposes of analyzing this relationship further, OED's ratings of investment projects were regressed on a series of country factors serving as possible explanatory variables. (A matrix of the Investment portfolio performances, explanatory variables, and technical notes together with regression results are shown in Annex Table 2.7). The aggregate analysis confirms that higher GDP growth rate and lower inflation combined with a high index of human development were highly- significant factors associated with a country's above-average portfolio performance (notable examples include Korea, Malaysia, Mauritius, and Thailand). 2.31 The correlation of the three significant variables (GDP growth, inflation, and human development) with investment project outcomes was .797." Other variables included in the series (e.g., per capita income levels, current account balance, black market premium) were found to be either insignificant or marginally significant; particularly, ARPP's emphasis on "opennesso is not confirmed by OED's analysis: there are different measures of *openness" which are likely to yield different results.1' 2.32 These aggregate estimations identify several outliern country portfolios nonetheless. Benin, Burundi, COte d'Ivore, Ethiopia, Ghana, Lesotho, Rwanda, and Zimbabwe all achieved above- average portfolio performance despite their low Indices of human development. Particularly exceptional are portfolio performances in COte d'voire, Ethiopia, and Rwanda which also experienced I Examining the project performance of 359 operations in 86 countries, the AnnualReview of Evaluion Resuke, 190 traced the influence of country econonsio conditions during the 1980s (a indicated by GNP per capita growth rates between 1980 and 1988) on the performance of the operations (satisfactory projects in each country) in the 1990 evaluation cohort. On average, countries having negative per capita GNP growth rates in the 1980s were also characterized by below- average project performance, and those with positive economic growth generally enjoyed project performance ratings exceding that for the evaluation cohort as a whole. The validity of these results was corroborated by the experience of the majority of completed Bank projects surveyed in the previous Annual Reviews. a Christopher Kilby, 'Statistical Determinants of Project Performance,* Background Paper for FY92 ARPP, January 2, 1993. SWeighing each variable by the square root of the number of projects improves both the t-statistics and the overall fit of the regression; correcting the standard errors for holeroskedasticity improves the t-statistics even more. ARPP's inde of openness is defined by external trade as a proportion of GDP, whereas OED's analysis stresses the extent of restrictiveness in a country's trado regime (albeit with a reduced set of observations). The divergence in findings is attributable also to the difference in the unit of analysis and statistical techniques employed: ARPP and earlier analyses used project as a unit of analysis and probit techniques, while OED has used country portfolio as a unit of analysis and linear regression as the analytical method. 2a Using a plus or minus 20 percent cut-off point for those exceeding or falling short of predictions. 45 low output growth during the period and in Ghana (high Inflation). On the other extreme, both Congo and Kenya, especially, witnessed below-average portfolio performances despite moderately-high GDP growth, relatively stable inflation, and better-than-regional average of human development index. A large part of these divergences can be attributed to exogenous shocks, other country characteristics not readily-quantifiable (e.g., borrower commitment, and political stability), and project-specific factors. Shortcomings in these explanations or the absence of a comprehensive analytical framework aside, the three variables (high growth rate, low inflation, and a high level of human development) adequately capture the country factors important in explaining the aggregate data series Included in this analysis. 2.33 OED's findings specifically highlight the importance of a country's level of human developmentF1 for project success (as exemplified by the cases of Chile, Colombia, Korea, Malaysia, Mauritius, Paraguay, Sri Lanka, Thailand, and Uruguay). The three dimensions of the human development index (HDI--in its aggregated form-relate to one or many capabilities that they are expected to capture: of leading a long and healthy life; of acquiring knowledge, and communicating and participating in the life of the community; and of guaranteeing physical and social mobility. This seamless web of relationships between health, nutrition, and education was emphasized more than a decade ago2 and has been reinforced by evidence reported in subsequent reports. OED's current findings with respect to the performance of investment projects are also applicable to its cohort of adjustment operations: in the 1992 evaluation cohort, for example, the overall human development index for the ten successful adjusting countries was 0.54 as opposed to 0.23 for the eight unsuccessful adjusters. In sum, growing international linkages of information technology, marketing, and production-as well as innovations in management and production techniques-make the challenges of an increasingly skilled and adaptable labor force all the more urgent. 2.34 Borrower commitment/ownership of projects and programs is often cited as vital for credibility of policies, safeguarding against policy reversals, and ensuring sustainability of benefits. In this context, an OED study? of 100 completed adjustment operations found that the typical cases, where borrower ownership (as measured by country initiative, conviction of key policymakers, political will of top leadership, and efforts toward consensus building among various constituencies) was strongly associated with overall program outcome, constituted 73 percent of the total. Virtually all highly-successful cases among a sample of 1,000 projects (such as in Chile, Indonesia, Korea, Mauritius, and Morocco) approved in the 1980s, were marked by a strong commitment to project objectives including their policy and institutional goals. 2.35 Finally, a relatively stable political environment (irrespective of the nature of the political regime) provides long-range planning horizons for core ministries, thereby ensuring the continuity of S The yardstick of human progress used here is the UNDPs human development index (HDI) which-by combining indicators of real purchasing power, education, and halh-offers a measure of development much more comprehensive than GNP alone. Source: UNDP, Hwnan Development Report 1993, New York Oxford University Pres, 1993. The World Bank, World Development Report 1980. See in particular Chapter 5, 'Human Development Issues and Policies.0 Part I of that chapter was also published as World Bank, Poverty and lwnan Development. Z "World Bank Structural and Sectoral Adjustment Operations: The Second OED Overview.' 46 project and program objectives. Political Instability In many countries (e.g., Bolivia, Chad, Nigeria, the Philippines, and Uganda) proved to be a major impediment in project implementation. Processing Factors 2.36 The Bank's role manifests itself primarily in an appropriate assessment of the country- specific requirements through its ESW and policy dialogue, which simultaneously guides its involvement in the various stages of the project cycle: identification, preparation, appraisal, supervision, and evaluation. Among the reasons commonly cited for the Bank's declining portfolio performance is the insufficient emphasis on identification and assessment of major risks, and on Implementation planning. In numerous cases, inadequate preparation and appraisal have been faulted for subsequent execution delays, cost overruns, or other implementation problems; in others, inadequate project supervision has been held responsible for a project's unsatisfactory results. 2.37 Empirical evidence drawn from the 545 projects and programs In OED's 1991 and 1992 evaluation cohorts, for which complete information on project processing is available, reinforces the Importance of the quality of operations when they enter the Bank's portfolio (Annex Table 2.8). Further, each stage of project processing provides a value-added to the outcome of the operation, as Illustrated in Chart 2.1. Specifically, the proportion of satisfactory outcomes increases consistently with the adequacy of identification, preparation, appraisal, and supervision. Equally important, when the earlier stages of project processing are marked by deficiencies, monitoring and supervision can play a crucial role in salvaging project outcomes. 2.38 Statistical analysis confirms that sound appraisal and supervision are significant factors in determining OED's rating upon completion; also, the incremental contributions of the various stages of project processing are shown in Annex Table 2.9. 2.39 As highlighted in Chapter 1, all the outstanding projects In the 1992 evaluation cohort were well Identified, prepared, and appraised. The nature of some of the general shortcomings as well as effectiveness In the major stages of the project processing cycle, based on the principal process- related conclusions of the PMTF Report, are illustrated below. (See also Figure 2.1). Identification 2.40 A project is appropriately identified when it conforms to a well-conceived country assistance strategy, i.e., when (a) it represents a potentially worthwhile contribution to priority development objectives; (b) It has the support of policymakers within the country; and (c) it is a suitable and timely vehicle for Bank assistance. s Based on the information on identification, preparation, appraisal, and supervision as reported in PCRs-PARs for each project in the 1991-92 evaluation cohosts. Chart 2.1: Proect ProcssIng & Outcom 1991 and 1992 Evaluation Cohorts 53% Sat- Deficient 18% Sat (70 164% Sa Defien 45% Sat Deficdent (113) 37%Sat DDefken (34) ygå n.a. 0% sat (1) n.a. Dercen 29% Sat 0 - D~Scent --- (72) De~in 30 sat (0) ..- Der~int .- (70) N0%6 30% sat A - Deict (70) 21% Sat Deficen (34) 48 2.41 Adequacy of the ESW Needed for the Identification of Lending Priorities. The thoroughness of upstream Bank analysis (especially In sectors such as finance) was decidedly uneven. In the Ecuador Financial Sector Adjustment Loan and the Korea Industrial Finance Loan, the sector analysis available to project staff was relatively limited in scope and somewhat outdated by the time of appraisal. The Small and Medium-Scale Industry (SMI) leading to Korea and Mexico was undertaken without substantial or updated sector work. In the case of Mexico SMI Ill, good sector work, had it been available, would have helped design a better technical assistance component. On the other hand, the high production response witnessed in Benin's Area Development Project can be attributed to an appropriately-designed incentives and supportive framework for cotton production. Likewise, Ghana's SAC I and I were supported by high quality of identification and appraisal work. Preparation 2.42 Defining appropriate objectives for the preparation of the project and setting up arrangements to carry out the surveys, analyses, and consultations required often hold the key to the design of a quality operation. 2.43 Borrower Involvement and Commitment to Project Success. In the water supply and sanitation cohort, seven of the 17 projects were rated deficient in preparation. The deficiencies mainly related to inadequate assessment of government commitment, the exclusion of beneficiaries from project planning, and an overestimation of local institutional capacity. In contrast, the Ethiopia Urban Development Project is a good example of the positive impact of beneficiary participation on project results. This project was a success story in more than one way. Inter alia, physical targets were surpassed, access to services considerably improved, insitutional and policy changes implemented, and mortgage repayment maintained at excellent rates. The factor which contributed most to this success was the intensive participation of the community in project planning, design, and management via housing cooperatives based on place of employment. Construction of new houses and rehabilitation of existing houses were carried out through beneficiary self-help. In the Mexico Earthquake Rehabilitation Project, the quick response and high degree of flexibility were made possible by the already-strong institutional capacity and high-level political support for the project. Appraisal 2.44 Appraisal is the crucial stage of the quality assurance process. As the exclusive responsibility of the Bank, appraisal seeks to provide a neutral, objective, and rigorous assessment of the operation Intended to ensure its conformity with Bank standards and greatly increase the likelihood of effective and efficient Implementation. 2.45 Assessment of Institutional. Mana rial and Oranizational Canacities. In the financial sector, a major institutional issue has been strengthening the management of development finance institutions (DFIs), many of whom have subsequently failed, in part for lack of such management. The 1992 cohort includes several repeat DFI operations which should have focused on improving managerial performance, as distinct from providing more resource transfers. In many cases, this was done, but not without major design problems. For example, the Barbados Industrial Credit Project never instituted eligibility requirements for participating institutions based on financial and managerial soundness. In Panama Development Banking I, the Bank was unaware of mismanagement by the 49 Corporacida Financiera Nacional and, consequently, did not encourage timely restructuring. In the Mexico Small and Medium-Scale Industrial Development III Project, Inadequate attention was paid to coordination Issues among a large number of participants In what was probably an excessively-complex Institutional framework. In the Kenya Secondary Towns Project, an Inadequate risk analysis with regard to the existing implementation capacity contributed to eventual failure, even though the experience of two preceding projects should have offered ample material for completing such an analysis. In contrast, the Colombia Development Bank Project provided real incentives for participating intermediaries to adopt capacity-building measures by creating two distinct categories based on the degree of intermediary capitalization and applying different lending criteria accordingly. 2.46 Assessment of Country Commitment and Local Support. In the Tanzania Urban Water Supply Project, a divergence between the strategic goals pursued by the Bank, on one side, and the government, on the other, meant that the policy and institutional issues assessed during appraisal had no real hope of resolution. The Mexico Water Supply and Sewerage and Water, Women and Development Projects were both rated deficient at appraisal, largely because the Bank did not adequately test the borrower commitment to project objectives. Conversely, water supply projects in Mauritius, Rwanda, and Uganda succeeded, in no small part due to the fact that the Bank obtained firm governmental commitments to project goals and design in advance. Indeed, in the case of the Korea Nangang and Taego Water Supply Project, governmental and sector institutions retained firm control of project objectives and design from the outset, with the Bank's role narrowed to giving highly specific advice on special issues. Similarly, adjustment operations in Ghana, Mexico, Mauritius, and Madagascar included substantial input from the borrower governments. In Ghana, for instance, the adjustment program was preceded by many brainstorming sessions between government officials and their Bank counterparts on the design, sequencing, and implementation of operations. In Mexico, important background materials, studies, and papers were prepared by Mexican economists inside and outside the government. These materials formed the basis for the design of Trade Policy Loans I and II which substantially opened the real sector of the economy. The initiative for the Madagascar program clearly belonged to the government. And the success of the Benin operation was owed in part to the important seven-year policy dialogue between the Bank and the authorities prior to the adjustment program. 2.47 Identification of Critical Performance Indicators to Be Tracked During Sunervision and Evaluation This was, indeed, one of the principal lessons of experience which emerged from the review of the 30 loans in the 1992 transport evaluation cohort. Similarly, in the Mali Health Development Project, the absence of agreed performance indicators was identified as one of the major flaws in the operational design. Supervision 2.48 A clear distinction needs to be made between supervision and implementation. The borrower is responsible for project Implementation and for arranging for the technical assistance which may be required. However, the Bank's supervision function includes the periodic assessment of Implementation support requirements and the facilitation of such support. 2.49 The PCR for the Bangladesh Public Administration Project noted that, 'despite the regular and substantial monitoring, IDA had limited leverage in eliciting government support for 50 certain critical and agreed project activities.* IDA supervisory staff were urged to "go beyond the visits to project faciltes and staff, the formal presentations of the Alde-Memoire, and all other diplomatically-correct interactions with the government, and try to become more actively involved in aisting the government toward more effective project coordination/action.' In the Haiti Urban Development Project, the Bank did not step up supervision efforts, even though the project had a complex, frst-of-ts-kind design, the country experienced severe social and political dislocation during Impl n on, and the local capacity to implement the project was In doubt from the outset. In the Ghana Health and Education Rehabilitation Project, the task manager was based in the field and provided continous supervision and oversight for both componant, with periodic support from headquarters. However, oversight of the health component suffered, partly due to the fact that the task manager was an education, not a health, specialist, but also because there were only two formal supervision missions from headquarters for the health component during the first four and a half years of Implementation, neither of which included a health specialist. In the Comoros Health and Population Project, frequent IDA staff changes, and attendant delays in supervision, forced five extensions of the project closing date to enable mission staff to familiarize themselves with the operation. Supervision efforts had to be invested mainly in avoiding further slippages in the project timetable, while other major project shortcomings were left unaddressed. Similarly, in the Mall Health Development Project, Bank reorganization led to a hiatus in supervision efforts, with the result that major contractors were not monitored closely. On the other side of the ledger, a first-rate project Implementari n unit, a good team effort on the part of the Bank (which withstood four changes of task managers over a four-year period), and the Bank's willingness to provide ample supervision resources helped keep ImplementatIon of the Bolivia Second Emergency Social Fund Project on time and with highly successful results. Also, the success of Ghana's structural adjustment effort owed in part to 'hands-on" supervision by Bank staff, which allowed Bank expertise to buttress Ghanalan efforts in critical areas and permitted flexibility in resolving unforeseen problems during implementation. 2.50 An OED report on the Internal project supervision processes of the Bank", was opportunely published during the preparation of the Portfolio Management Task Force (PMTP) Report. Many of the recommendation of the Supervision Study (SS) were incorporated into the latter report. Subsequently, Bank management has responded with 'Next Steps--A Program of Actions' to put into practice the recnnmndtions of the Task Force. 2.51 Both the SS and the Next Steps focus on the lpl ton phase of the project cycle. However, the SS, and to a greater extent the Next Steps, emphasize the critical impact of upstream phases on project outcome. Project design in accordance with economic, sectoral, institutional, and beneficiary circumstances is considered paramount Both documents coincide on the need for increased resources for the supervision/implementation function and the need for incentives to encourage the attention of management and operational staff. Procedures to enhance performance include a more explicit treatment o portfolio management issues in the Country Portfolio Performance Review and in Country Assistance Strategies, greater use of Country Implementation Reviews, measures and studies to improve the efficiency of procurement and financial accounting functions, priority attention to '*Ba i Expmfo l Poject Supevision,- Repom No. 10606, Api 30, 19M2. Openions Poley Depument, 'Potfolio Mmnageou: Nat Step--A Progm of Actions. 51 problem projects, and increased functional training for staff. Thus, the Intent or content of 14 of the 19 recommendations of the Supervision Study have been incorporated In some form in the Next Steps. 2.52 Of the five recommendations not Included In the Next Steps, three may be viewed as implicit (attention to capability of the borrower to monitor Implementation, revision of the supervision handbook, and obligation of the supervision mission leader to nominate skills needed for the subsequent msslonts). A fourth had recommended an automatic mid-term review (MTR) of all projects without an overall highest rating. The Next Steps stipulates that MTRs should not be mandatory Obecause a flexible approach to establishing a framework for mid-course correction seems desirable; It does, however, place great emphasis on country portfolio and project restructuring to address country and project Implementation problems. The fifth had recommended Implementation Plans in Staff Appraisal Reports for all projects. The Next Steps modifies this to reflect differences in the firmness with which projects' schedules could be determined: 'A differential approach to ex ante documentation Is thus warranted to correspond to different project characteristics.' Cancellations 2.53 In the 1992 evaluation cohort, the average cancellation rate remained relatively low, about 9 percent (US$1.6 billion) of the originally-committed funds. This compares with 12 percent in 1991, and 10 percent historically. Design and Implementation problems, reductions in project scope, cost savings due to devaluations, deteriorating country conditions, political changes, and civil disorders were the most common reasons cited for cancellations of operations included in this evaluation cohort. 2.54 When there is agreement with the borrower that an operation cannot be rescued, the Bank is now willing to consider early cancellation. Loan cancellation, as an instrument of portfolio management, has taken on added importance since the PMTF Report drew attention to the growing number of projects encountering serious Implementation difficulties. Taking into account this concern, a study of the reasons cited for canceling loans was completed for this Annual Review. Its principal findings are summarized as follows. 2.55 Of the 3,159 projects in OED's database, three-fourths were either fully disbursed or had negligible cancellations. This included 36 percent which had cancellations of under 10 percent of the originally-committed amount and 40 percent which had no cancellations at all. Loans with large-scale cancellations-50 percent or more of the original loan amount-represented just 6 percent (191 projects) of the total. Roughly a third of these 191 projects underwent substantial changes in loan design and/or other restructuring prior to their cancellation. 2.56 Annex Table 2.10 presents a list of the reasons cited in the PCRs and PARs for these large-scale cancellations. By far the most important was poor implementation, which was cited 62 percent of the timeY Next In importance were an adverse policy environment (27 percent), reductions in project scope (26 percent), shortages of counterpart funding (24 percent), and the lack of government commitment (20 percent). Least significant reasons were noncompliance with covenants, the availability of cheaper non-Bank funds, cost savings, and faulty cost estimates. SGiven that more than one reason was ofen cited in the PCRa/PAfs for the cancellation of a loan, the total number of reasons cited, 391, exceeds the total tumber of canceled projects, 191. 52 2.57 Interestingly, nearly half of these loans were rated "satisfactory," despite the significant cancellation of funds they underwent. They may have been rated satisfactory because, at the time of their cancellation, they continued to offer economically-justifiable development benefits, but became operationally unviable due to changes in the operating environment. For example, in 1985, during implementation of the otherwise satisfactory Korea Small and Medium Industry Project, the Korean economy was severely affected by a slowdown in the growth of world trade and of industrialized country economies. As a result, the demand for long-term loans by Korean industrial enterprises declined sharply. To boost the economy, the Government introduced a subsidized industrial financing program which rendered the terms on Bank loans highly uncompetitive. This triggered the cancellation of the loan at a time when over four-fifths of its funds remained undisbursed. Likewise, the Second Kenya Tea Development Authority Project was rated satisfactory, even though over 94 percent of its originally-committed funds were canceled. This project was intended to help finance a program promoting smallholder tea cultivation under the auspices of the Kenya Tea Development Authority (KTDA). Disbursements were to be used to cover cash deficits incurred as a result of KTDA's field sector operations. However, KTDA's unanticipated financial success eliminated the need to draw on all but a small portion of the credit. 2.58 Of all the reasons cited for loan cancellation, a lessening of the governmental commitment to a project was the one most frequently associated with cancellation 88A an unsatisfactory rating (72 percent), followed by shortages in counterpart funds (70 percent), weak implementation (66 percent), an adverse domestic policy environment (58 percent), and a reduction in project scope (57 percent). Table 2.2 provides further perspective on these patterns. Table 2.2: Relationship Between Ratings and Problems with Highly Canceled Projects Satisfactory Unsatisfactory Rating Rating % Implementation No Problems 49 25.7 23 12.0 Problems 41 21.5 78 40.8 Policy Environment No Problems 69 36.1 71 37.2 Problems 21 11.0 30 15.7 Project Scope Not Reduced 69 36.1 73 38.2 Reduced 21 11.0 28 14.7 Counterpart Funding No Shortages 76 39.8 69 36.1 Shortages 14 7.3 32 16.9 Government Commitment No Problems 79 41.4 73 38.2 Problems 11 5.8 28 14.7 53 töz2.1: ~T1eneroptneufal Costs of MIawed Quality at EnLtry: The Tanennin Mufindl Project textbook exampl' of the costly consequences of iadequate quality at paeot entry lo provided by tho Twananta Moundaulp ~ad Paper Pr jt,whih OED auditod in 1992. Mufdi"was the largest and most complex ausarojetr udetaken i Tanania,with an inivetment cost of around US$300 millico. or nearly 60 percnt of the 1totaedu,stria istme* t in Tm~aola 40ring 1978-82. The itst Bank loan was apprmved In 1979, followed by a second-atae credit in 1983, for a total of US$78 Milioå fom the World Bank Group. Though based on 0sab1lhed uotnogs the mill was suboptimal in s kale from the outset Diflkulties in coordinating ointan Jng, e omarybotges of local counterpaht and external funding, and the latö completion of infrastructural heito triggeted thre.year delay in completion. iThe projoct in oporating at around 40 percent of ts rated epacity, etasson for poor performance to dato incluzde, inter alia, gaws in equipment fabrication, ovemoptimisdec demiand pt~ajeuon art aducti* on tuntorta of imported tiputs, and poort mntenance. About 40 percent of the prglect's. aheoady-meager produQtion is exported below cost. Recaloulated economic and financia rates of return for he rojtaeneatvO.,h i¢ttW operator lha been inaursing hteavy losses due to tow sales leveis, ntegative margins o0.export salesiand high operating costs; is facMg erious finaneial difficuhics, and is nable to service iis debs. To å2s h cn1ifpg the Goernment has njecte new equity funds, conv~rted part of the long4e.n debt int uityrand .duced interest rates on tho remaining oans. Despite this financial support. the company will need further suppöilt fo th ftosable futur if it is to sustaini opeoation. Wha....nt wrong lThe PAR first takes th- Bank to task on the justificatiou of the project at dentfletio:*..h Bank's maanagement rationalized the aupport for tho project by arguing that developmtent of this projectwasjutied tbocauao] the prjetis consistent with the Govemmnt's Basic Industrial Strategy and [because) of th imtdaoridvoedepacity of other priority sectors and tho back of other lidustrial project alteatives? Theo PAR goe on4 jo te that "Th¢ Govemt'st ambition to iplement arr ill-conceived industria strategy% the ono cmmunity's eageress toa at, and institutional pressure to tend within tho Bank coaleaced, and led to tho Bank a Also highlighte is the deain preparation, notably thme design of tho project. The Bank's staff was c alledirpont to "gneaaproject1, and came up with a design refleótig major technical and economnic compromises idenb adv rey8affeted the viablit of the project: "The mil4 ,too large in relation to the size of the domestic rmarket, and, eveanso, its scale is uneconxomic:.Itis notable that annual consumption of paper products at appraisal was malt fractiu of tho proposedcapacity, and tat, eventallowing for tho op stiallygrojected domestic market growths, the þroject would have had to expo.rt patt of is production uritil tho early 1990s, d atj price that hardly .covera variable costs, even under ideal eperaing condhions.' MAso responmibie in past for the design shiortcomings were "tho excessivo reliance on outside specialized finms to carry out market analyses, resulting in Qptimistic demand .. projections; failure to respect the critical importance of scale economies; and the borrower's insufficient budgetary . resoures and detovery capacity. -Thirdly, thes PAR painstakringly documenta how tho appraisal process Jabored under obiously . natennal. conditionas to justify the project, noting that: "In grappling with the issue of planit size, the final costfguration and mcale of the project reflected significant modifications in concept to reach an 'optimubm' between project size, cost, and financial and econoii viability. ..Thmis has led to a mil concept which is expensive in relation to the economy of the country and to the scale of production...The overal å effect has been a sharp loss of scale economies and conomic rate of reur? Byen at project inception, the PAR noes, the risks of proceeding with the projec,t were considered to be significant. Yet, not only dtid tho Bank proceed with the project, but, when it ran into trouble, approved a follow-on .mdit as r operation whih was ationaized an the grounds that: "ome Bank tuppor th pos operation is justified, in part to avoid embarrassment to all concerned if this expensive mnistake were to bo abandoned mid-streamn.,.cven after the proposid operation, the project would remain nprecarious heahh. The PAR's eoveono ussessmn t is that in tho Ta anian ircunsances, and strictly nslcttgios, economi, and financial grounds, the project should have never been undertaken, Undoubtedly, tho Bank exercised poorjudgment in going ahead with this project." 54 C. LINKAGES WITRH REVIEW OF ONGOING PROJECTS 2.59 The *Annal Review of Portfolio Performance for FY92 concluded that the overall portfolio performance was somewhat better than In FY91, but emphasized that the Bank continues to face serious portfolio performance problems. As shown tn Figure 2.4, there has been a widening gap between the ARPP and PCR-PAR ratings reported in the OED cohorts since the early 1980s. However, the projects included in the ARPM and OED cohorts are not strictly comparable: completed projects evaluated by OED were ongoing projects in the "current portfolio for earlier years. 2.60 By matching ARPP and PCR-PAR ratings for completed projects, one can get some idea of how well the ARPP ratings measure the eventual development impact of the Bank's portfolio. The findings of a recent joint Operations Policy Department/OED study on this issue are summarized below. 2.61 The exercise looked at 926 completed investment projects and 104 completed adjustment operations approved since FY79. Comparisons were made between the overall status ratings for the final year of the project and the PCR-PAR ratings at completion.2 For Investment projects, as shown in Table 2.3, 17 percent were rated nonproblem projects in the ARPP but unsatisfactory in the PCR-PAR. Similarly, 4 percent were rated as problem projects but satisfactory in the PCR-PAR. The total "disconnect* between the ratings was therefore 21.5 percent. At a more disaggregated level (Figures 2.5-2.6, and Annex Table 2.11), the following patterns emerge: Table 2.3: Comparison of ARPP and OED Evaluation Ratings PCRIPAR Ratings ARPPR R tiSatisfactoy Unsatisfactory Total No. f Nonproblem 619 160 779 Problem 39 108 147 Total 658 268 926 percentaae of Sample Portfolio Noproblem 67 17 84 Problem 4 12 16 Total 71 29 100 Source: Joint Operations Policy Department/OED Study on ARPPIPCR-PAR Ratings. a Concptay, the development objectives ratings should be closer to the PCR-PAR ratings, than are the overall status ratings. But, in practice, this was not the case (except for later years of adjustment operations). 55 Figure 2.4: Comparison of ARPP and OED Evaluation Ratings, 1975-92 4O 35 30 25 20' 15 10 5 01 197 977 197 1981 1983 1985 1987 1989 1991 1976 1978 1980 1982 1984 1986 1988 1990 1992 Year Source: Annex Table 2.11 0 ARPP % Prob. Proj. - OED % Unuatisfactoy By region, the largest "disconnect was In LAC and Africa.' These regions (along with ECA and South Asia) also had the highest percentage of problem projects. For countries with ten or more projects in the sample, the largest "disconnect- was for Nigeria, Zaire, Brazil, Madagascar, and Bangladesh. By sector, the "disconnect" was largest for technical assistance, agriculture, industry, and water supply and sanitation. For adjustment operations, the overall "disconnect" was 31 percent. Because of the small sample, comparisons at other than the Bankwide level were not meaningful. SIn the Africa Region, the percentage of problem projects jumped fiom 19 pement in FY90 to 30 percent in PY91, soflcting the tigbening of supervision standards and ratings. This should resuk in a lower 'disconucct' la the future. 56 Figure 2.5: PCR/PAR Ratings & OED/ARPP Disconnect, by Region Latia Ameica, Caibbean m 36 South Asia w AVtAca 27 Europ Cental Asia 0 0 5 10 15 20 25 30 35 40 45 OEDIARPP Disconnect (Gross Divorwce) Sowe AnneTabl212 2.62 The major conclusions of the overall analysis are as follows: * There is a wide variation in the "disconnect" between ARPP and PCR-PAR ratings between countries and sectors; there is also a much wider "disconnect for adjustment than for investment operations. * Ratings on development objectives have not received the critical attention that they deserve. The overall status ratings seem to be dominated by short-term concerns about project implementation, while development objectives ratings reflect longer-term optimism that objectives will be achieved once implementation problems are resolved. * Ratings on both overall status and development objectives are excessively optimistic in the early years of implementation. Although this may reflect the lack of information on which to base performance assessments, it also indicates that risks at project entry are not realistically identified and appraised and taken into account in the ratings. 2.63 Some of these issues have been addressed in the Guidelines issued recently for the FY93 ARPP. In particular, attention will be focused on those countries and sectors with relatively 57 high levels of problem projects and *disconnect" with the PCR-PAR ratings. Greater attention will also be given to the realism of the development objectives ratings. Figure 2.6: PCR/PAR Ratings & OED/ARPP Disconnect, by Sector 454 Agicltum a Industy a TdleWl Assiuace a 36 AjuaUntLou* a Water& Sanitation 27 Fina =,%MGInaC v Human Rosaum a Power a Teoommunications a 18 ran a Transport a 0 5 10 15 20 25 30 35 40 45 OEDIARPP Disconnect (Gass Divergence) No: 2ctesTowblmand Proge &Potcy fori mw Sou.: AnnexTaMe2.12 D. CONCLUSIONS 2.64 The principal findings of this chapter are summarized below: Trends in Project Performance Aggregate portfolio trends, based on evaluations by year of approval, exhibit three distinct phases: (a) from the late 1960s until 1973, the percentage of projects that received satisfactory ratings remained significantly above 80 percent, reaching a peak of nearly 89 percent in 1973; (b) from 1974 to 1979, a sharp deterioration in portfolio performance took place, with the proportion of projects rated satisfactory declining to a mid-to-upper 60 percentage range; and (c) for projects approved in the 1980s, the percentage of projects receiving satisfactory ratings has oscillated around 70 percent. For recent years (after 1983), the small number of projects evaluated may not be representative of the total population of approved projects. Therefore, a levelling off or an apparent improvement in portfolio performance during the 1980s needs to be 58 Interpreted with care. Preliminary projections, based on this small sample of projects initiated in the mid-1980s, Indicate a possible deterioration In the overall ratings for at least two or three approval subsequent years before showing signs of a slight Improvement. Adjusting for changes in the country and sectoral composition of the portfolio which have occurred over time does not significantly alter this three-phased pattern- Indicating that differences in overall performance were due, for the most part, to shifts in performance levels within, rather than across, regions and sectors. The following causative factors underlying these trends may serve as hypotheses for further investigation: (a) changing global economic environment (e.g., external shocks) as well as country economic conditions; (b) increasing complexity of objectives (e.g., social issues, and policy reforms) incorporated into Bank projects often requiring Innovative project processing and implementation; and (c) perhaps, expansion in Bank lending levels. Factors Affecting Project Performance * The impact of the external economic environment-especially in terms of external shocks fmanatinr fom the changes in terms of trade-on country portfolio performances has varied across countries. The most critical factor in coping with these shocks has been the form and timeliness of policy responses and structural reforms. Evaluation experience shows that pursuit of appropriate policies differentiated country portfolios that witnessed above-average performance under adverse terms of trade from those that performed worse-than-average under improved/stable terms of trade during the 1980s. * Empirical evidence drawn from OED's evaluations confirms the results of earlier studies that the predominant stimulus for good project performance rests in sound country policies and macroeconomic performance. The performance of investment projects was found to be highly correlated with the performance of adjustment operations-indicating that fiscal stability and steady output growth were vital for maintaining the requisite environment for the successful implementation of investment projects. * The analysis establishes the indispensable contribution of a country's level of human development (as measured by UNDP's index, composed of health, education, and real purchasing power indicators), as well as reiterates the significance of borrower ownership toward explaining successful project outcomes; virtually all highly- successful country portfolios were marked by a very strong government commitment to project objectives including their policy and institutional goals. * As highlighted in the PMTF Report, the importance of the quality of operations when they enter the portfolio (and af sound project processing in general) is reinforced by the evaluation experience of the most recent cohorts. Each stage of project processing 59 provides a value-added to the outcome of the project; specifically, the proportion of satisactory outcomes Increases consistently with the adequacy of identification, preparation, appraisal, and supervision. Equally Important, when the earlier stages of project processing are deficient, monitoring and supervision can play a crucial role In the salvaging of project outcomes. Preliminary analysis conducted for ascertaining any systematic relationship between loan cancellations and performance ratings at completion does not yield conclusive results. However, among the factors associated with loan cancellations, implementation problems were found to be the most significant, followed by adverse policy environment, reductions in project scope, shortages of counterpart funding, and lack of government commitment. Linkages with Review of Ongoing Projects Comparisons of the PCR-PAR and ARPP ratings for completed projects shows a disconnect- of 21 percent for Investment projects and 31 percent for adjustment operations. For investment projects, there is a significant variation in the 'disconnect* betwoen countries and sectors. Ratings on development objectives have not received the critical attention they deserve, while ratings on both overall status and development objectives are excessively optimistic in the early years of implementation. 3. THE AGRICULTURAL PORTYOLO: 19745 A. INTRODUCTION 3.01 This chapter reviews the performance of the agricultural projects approved by the Board from 1970-85. This review is now possible since nearly all of the projects Implemented from this period have been evaluateo. The analysis of the 887 agricultural projects In the set Is of particular Interest because the years co vered were both those of most rapid grwth in agricultural lending and of dramatic deterioration In agriculture's performance ratings. One of the questions addressed is whether there is reason to believe that performance is likely to Improve because of lessons learned during (and after) this sixteen-year period. To set the stage, the chapter begins by recalling the role of agriculture in economic development, foilowed by a broad-brush appreciation of the Bank's evolving approach to agricultural lending. B. THE SEITING Agriculture In Development 3.02 Agriculture remains the main source of income for the majority of people in developing countries. Over 60 percent of the people in the world live in low-income agrarian countries, above all in South Asia and Sub-Saharan Africa. In these countries, agriculture and the activity directly dependent on it (commerce, transport, textiles, food processing, commodity exports) account for more than half of GNP, and the rural population for more than two-thirds of the total (see Table 3.1 below). If agriculture does not grow, the entire economy stagnates. 3.03 Agricultural growth is also key to eliminating poverty. In low-Income countries, 60 to 80 percent of the labor force is employed in agriculture at very low levels of productivity. These countries account for the majority of the world population living in poverty and for most of those whose food-security is precarious. Far more hungry people live in backward villages than in urban ghettos. The situation is exacerbated by rapid population growth. Although the share of the labor force employed in agriculture declines with economic growtN, it does so at a much slower rate than agriculture's share in national income. Moreover, the absolute number of workers in agriculture usually does not peak until a country attains upper-middle income status. During this entire phase of economic development, agriculture faces the double challenge of absorbing additional workers while at the same time trying to increase their average productivity and very low Incomes. Given that few countries have surplus arable land available, these challenges can only be met through significant investments and teclnological change in agriculture.' lla these in paragraphs 3.02 and 3.03 ar also developed in to Bank's rsecnt 'Agrioulral Selor Rview." 62 Table 3.1: Agriculture's Share In National Income, Employment, and Export Earnlop by Country Income Groups, 1989 Country Group Aveage Per Agriculture's Share (%) la Capita lacoee _ _ _ _ __ _ _ _ _ $ GNP Employment Export Earnins Low income Less than $250 per capita 190 49.3 76.5 65.0 $251-500 per capita 380 33.1 68.2 443 Middi Ine $501-1,000 per capita 800 22.8 52.7 40.5 $1,001-2,400 per capita 1,580 12.8 29.8 39.8 Upper-Middle Inoome 2,401-4,000 per capita 3,530 8.7 21.8 16.5 High Incom $6,001-16,000 per capita 11,520 4.6 8.8 18.9 Greater than $16001 20,460 2.8 4.4 10.8 Note: 1990 data. Country income groupingA are taken from the 1991 WDR but with subdivisions within groups that give roughly equal numbers of countries in each. Source: World Development Report 1991, except agriculural employment data which were obtained from the 1991 FAO Production Yearbook. (Reprinted from the -Agrioukurl Sector Review,' 1993.) 3.04 Not only is surplus agricultural land increasingly rare; in many countries, agricultural production per so is threatened by deterioration of land and other natural resources, primarily as Intensive operation of Irrigation commands and a lack of attention to drainage have led to substantial losg of highly productive arable land because of waterlogging and salinization. The growth of population dependent on land-extensive subsistence agriculture is also the most important single cause of destruction of forests and the associated degradation of water courses. Eliminating poverty may have been the great rallying cry of the development profession in the 1970s. In the 1990s, sustainable farm production and resource conservation have taken on an urgency all their own. AchievIng Agricultural Growth 3.05 Agriculture has unique characteristics that profoundly condition its development. In developing countries, it is usually a small-scale, private-sector activity. It is heterogeneous and site- specific, involves a complex sequence of cultural operations during the production process, and is subject to high risks, e.g., in production (disease, pests and weather) and in marketing (price, input supply). In situations where rapid development is possible, e.g., knowledge of a profitable but underexploited technique, management has to be highly adaptive, and the complexity of decisions involved requires the kind of incentive structure that only private ownership can provide. Because of these characteristics, growth usually requires simultaneous satisfaction of many conditions: large numbers of farms with access to markets for inputs and outputs; new or improved technologies available to increase yields; and economic incentives sufficient to encourage farm families to invest, In education and training, land (irrigation and conservation), or directly In increased production. 3.06 Failure to meet one or more of these necessary conditions eplains much of the difficulty eperienced in promoting agricultural development (and therefore in achieving successful outcomes in Bank-financed projects). Two of these are apparently of overriding importance. They are also largely 63 the consequences of public policy. First, development of agriculture is heavily dependent on the "enabling environment" to ensure adequate economic incentives. Agricultural development requires investment, and investment must be profitable if it is to take place. This in turn Is possible only if output/input price relationships are favorable and markets operate with adequate efficiency. Of key importance here are the exchange rate, taxation, the regulation and public management of markets for outputs and inputs (land, labor, credit, and chemicals); plus the impact of the regulatory framework on investment (commercial codes, contract law, bankruptcy legislation, export/import administration). 3.07 Second, recent history shows the fallacy of attempts to pursue what can be called the public production model in agriculture. Centralized management of farms has been unsuccessful be it In China, the former Soviet Union, or Tanzania. Even private ownership with centralized control of cropping decisions has worked poorly, notably when it was attempted in Egypt and Syria. Moreover, public approaches have also had a poor track record in marketing-whether through parastatal input companies, or marketing boards-in stillborn public financial intermediaries, and with few exceptions, in agro-processing units from grain milling to plantation crops. 3.08 In addition to securing adequate policy and organizational conditions, elimination of structural impediments is needed. Technological change-the *Green Revolution* being its most telling illustration-represents a key ingredient for sustained agricultural growth. Research, both basic and adaptive, on tropical rainfed agriculture, particularly in Sub-Saharan Africa, is still at an early stage. Hence, some of the latter's widespread agricultural stagnation can be put down to underdeveloped technology. Moreover, many failures in agricultural projects have been due to promotion of new "technical packages' that were poorly adapted to farmer needs. Similarly, rural infrastructure-both physical and institutional-has to be minimally adequate. Obviously well- functioning agricultural markets require adequate roads and communications. And even the simplest processing of crops (i.e., grain milling, oil extraction, cooling) is greatly facilitated by electricity supply. Less obvious, but of no less importance, is investment in the health and education of rural people. Studies have shown that it is the relatively educated farm families that seize the opportunities offered by new technology in undertaking the risky process of changing long-established farming practices. It is no accident, therefore, that agricultural growth and rural development are seen as mutually interacting and supportive. C. EVOLUTION OF WORLD BANK AGRICULTURAL LENDING 3.09 The Bank's lending in agriculture has been shaped by evolving ideas of both the Bank's role in promoting development and of the strategy for achieving it. The course of agricultural lending is characterized by expansion in the kinds of objectives pursued and of numerous changes in the way they are pursued, most notably in project design. Today, agricultural projects lie at the Iatersection of the special operational emphases of the Bank, i.e., poverty reduction, gender issues, beneficiary participation, environmental protection or private sector development.' 3.10 In the early years, Bank lending for agriculture was almost exclusively production-oriented. Through 1952, mechanization, land-reclamation, and irrigation accounted for over 90 percent of 2 OED, Agricak*ral Markeing: The World Bank's Experience, 197445. a The primary program objectives of agriculhual projects approved in FY93 included environment, natural resourco conservation and forestry (a total of 36.9 percent), poverty aeviation (23.9 percent) and food scourity (13 percent). 64 agricultural lending.' From 1961-65, the single category of Irrigation and fic-1 control accounted for 68 percent of Bank/IDA lending. As late as 1969, mechanization per so was still an important objective, for example in the form of szable DA credts to India and Pakistan to finance the importation of tractors.s Smallholders were still far from the Bank's centerpiece of agricultural development. 3.11 The McNamara Presidency (1968-80) brought far-reaching changes. The mission of the Bank was expanded to include a large role for the social sectors-acknowledgment that investment in human capital is also necessary for development-and by the 1970s a very strong emphasis was placed on development undertakings oriented toward reducing mass poverty.' Bank/IDA lending tripled from 1967-73. Agricultural lending grew even more rapidly, increasing from 11 percent of total Bank/IDA lending from 1961-65 to a total of 20 percent over the seven years 1967-73. After 1972, it again grew more rapidly than total Bank-lending, and reached the all-time peak of 37 percent of Bank-lending in 1978.' (See Figure 3.1 below.) Area development projects with their explicit smallholder focus became a basic element in the Bank's strategy to eliminate rural poverty. They increased from a nonexistent category in 1965, to an average of seven projects annually from FY70-74, to a high of 21 projects per year for FY76-81, accounting for a quarter of total agricultural projects approved during the period 1970-85. Regionally, these projects were most prevalent in Africa and Latin America (see Table 3.2 below). In South Asia, they were rare. Rather, throughout Asia there was a dramatic increase in lending for irrigation as an essential element in the spread of the new high-yielding varieties of wheat and rice. Thus, the Bank played a role in promoting one of the all-time agricultural success stories. Prior to the Green Revolution, most of South Asia (IndIa, Pakistan, Bangladesh) seemed destined not just for poverty, but for Malthusian disaster. 3.12 By the mid-1980s, however, the Bank was reappraising aspects of its lending strategy. OED had delivered some bad news: performance audits were revealing that the majority of area development projects were not working. They were proving too complicated; their technical packages were frequently unworkable (a basic factor in about half the unsatisfactory area development projects). They were bedeviled by a lack of commitment of their would-be beneficiaries and governments, many of which refused to fund their operations. Particularly in Africa, the Bank had overestimated the capacity and commitment of implementing institutions.' 3.13 Several years later, Irrigation projects began manifesting serious difficulties. Through 1987, OED had evaluated over 80 percent of the completed operations in Irrigation and drainage as satisfactory, although with wide variations between regions (38 percent satisfactory in Africa; 89 Mason and Asher, Vae WaWt Bank Since Breon Woods, Brooking Instiution, 1973, p. 178. s Ibid., p. 477. 0We have made a stat at broadening the concept of developmnt beyond the simple limits of economfo growth. The emerging nations need, and are determined to achieve, greater oconomic advance. But...we believe economic progress remains precarious and stOrile without corresponding social improvement. Fuly human development demands atteaion to both. We intend, in the Bank, to give attention to both. Robeut S. MNamara, Address to the Board of Governors, Copenhagen, September 24, 1970. I In 1963, total Bank/IDA staff was 884, about an eighth of the current total (including long*OaM consultants). By 1971, staff had oreased to 2,610. Mason and Asher, p. 67. * See OED's Agriculture Background Paper for the Anual Review ofEvaluebon Resaks, 1990, p. 52. 65 percent satisfactory in Asia). For the 1989 cohort, that is for projects in the majority approved over 1979-81, the satisfactory rating fell to 46 percent, with a notable low rate of 41 percent in Asia.' (Part of this decrease may have been due to the adoption of more realistic Indicators of overall project impact.) Land degradation was equally alarming with widespread waterlogging and salinization. Unresolved institutional problems-but also problems of rural governance--were at the core of these failures: many new Irrigation commands were poorly operated and maintained, because of insufficient fimance and weak management, the latter typically the consequence of a lack of accountability and operational autonomy. Table 3.2: Percentages of Total Agriculture Lending 1970485 by Region and Subsector Irrigationl CredWl Area Perennial Research/ Drainage Floance Development Crops Livetock SECAL. Extension Other Total BAP 8.9 1.7 2.4 3.4 0.3 0.4 1.2 3.5 21.8 LCN 3.4 8.4 5.6 0.4 1.0 0.4 0.8 1.8 21.8 SAS 9.8 5.4 0.8 0.3 0.9 0.3 1.1 2.5 21.1 APR 1.5 0.6 6.3 2.5 1.2 1.3 0.1 2.2 15.8 ECA 3.5 3.5 1.3 0.2 1.7 0.7 0.0 2.0 12.8 MNA 2.3 1.6 0.9 0.1 0.1 0.2 0.0 1.4 6.7 All Regions 29.4 21.2 17.3 6.9 5.1 3.4 3.2 13.3 100.0 Notes: Calculated from total IDA/IBRD commitments for projects evaluated by OED, valued in 1990 US dollars. Secal Londing for agricultural adjustment began in 1979. Source: OED Annual Review Database 3.14 Sustainability was proving weak. Across all sectors, impact evaluations carried out some years after completion showed that the proportion of sustainable projects was much lower than the proportion initially rated as satisfactory (see Annex Table 3.2). 3.15 The enabling environment was also being scrutinized. In the Report on "Accelerated Development in Sub-Saharan Africa (1981). the Bank Insisted that economic policy distortions were an important reason for the region's disastrous economic decline. Worldwide, administered pricing, the regulatory impact of government organizations, and reliance on parastatal organizations for input and other support were coming into question. 3.16 By the mid-1980s, it was clear that agricultural lending was in trouble. OED's annual cohorts of evaluation results showed a marked deterioration in project outcomes: satisfactory ratings had been achieved in over 80 percent of agricultural projects in the 1979-81 cohorts; thereafter, deterioration was rapid, with the satisfactory rating of the cohort falling to 63 percent by 1984. Deterioration in project outcomes was also occurring in other sectors for many of the same reasons, but to a lesser degree." OED, Awal Review 4f BmiWalon Reauks, 1990, Annex 2, Table 2-12, pssit. Annual Reviews for 1988 and 1989, parabs. * Sn Annex Table 3-3 for evaluation ratings for agrioulture and all other projects by cohort year. А1 b4 '� � " Я ед �в�� �,�� ���� � .. � �3���'�8�'° � о``', ео А�,, � � • � •es � о � �, � � � $ � а � '� �����въ.�~� �`� � � � � ��� .�°� '� �� � � � ��� ������ � � � ����������'' �r� � '�+ � �9 о ... � � "� � �,,� . �' м � .ы � р, ������ . .��.� � � �� •� .� � w д .�.. � � � о0 b0 О � � � � � д � � n р � � � ,�. � � � � �� � д � � �� �$�� � � � � а .� 0�0 , ,� �° о '� "� м а � .� � > ,е� � о � s •� � � . � � • � �рΡ� �'�_ � �� С1 � � i� т W • �3 � о " � .S � � � � � ж ..» 3 � ,��а � � °�� � а и'�b�'� '��' ��� $ � � д� � � � ,,,,,, � � . � � w �4 � :� i�i ,о '� �'' � � я � ' � а � -. � •• � � � � �� � .� �-„ а �t � � � м t� � � � � � � .� �д _"' � � ,� бr� �'�' �� �� о � � � � � � � �о���° � �i н ,3 .. � � '� �-,.£1 о � � �� �� � � �• Т � г� � . а � о �т. ,° � а. � .� � � � �, �� � � � � � � � � � � . � � � .. . �_� � � 'l1�SSП ОббТ м � р� �t �, ,� 8 . � � � � � � 67 3.18 Most of these changes can be seen as growing out of the feedback from earlier experience. While the changes were not always timely, their net impact should be a substantial improvement in project outcomes and sustainability. Whether this will be the case remains to be seen. Only seven projects approved dter 1986 have so far been completed and evaluated, of which four were adjustment opefations. 3.19 Another question is whether learning by the Bank can keep pace with new challenges as the hfttation once agam. move& into uncharted waters. Natural resource conservation and management is now the basis of many fivestanding projects and important components In many others. 12 Many of the institutional dilemmas which undernimed rural area development projects (commitment of the authorities, participation of beneficiaries, weak supporting institutions) will have to be tackled afresh in the new coutexL Finally, reforming the agricultural, economies of the former Soviet-Bloc countries is forcing the Bank to face a set of Issues In Institutional and policy reform hitherto unprecedented in its history.0 3.20 This brief "tour d'horizona leads to the conclusion that Bank lending for agriculture has become more complicated and more demanding in response to past experience and the effort to increase the institution's effectiveness in promoting economic development. In irrigation, for example, two decades ago, projects focused most oft= on achieving the successful completion of civil works. The watchword today is not so much bricks and mortar as the sustainable production of entire irrigation systems in which difftcult hiMitional and policy issues loom large.14 In general, there has been an accretion of goals as awareness of the many preconditions necessary for successful agricultural lending has increased. Hence, the -checklist- of items to be covered In the typical agricultural project has bad to grow longer. But much of this accretion is in the nature of safeguards to enhance the probability of project success: due regard for macroeconomic considerations; increased wariness about government commitment; greaW awareness of public sector weaknesses; more realistic approaches to dealing with price and weather risks; search for beneficiary participation; sensitivity to gender issues; and -- P M safeguards. Given the weak track record of agricultural projects, the realignment of objectives and the resulting complexity Is e. Sustainable agricultural development is at the core of the Bank's mandate because of its impact on poverty, on food security (particularly in South Asia and Sub-Saham Africa), and its relevance to environmental management. Nevertheless, switching resources away from agriculwral lending was an appropriate response to the deterioration of the agricultural portfolio. A switch back to increased agriculwal lending, however desirable as a straegic goal, would be warranted only if the lessons of experience are heeded and project performance Improves. U TsA managers decided that 40 percent of the agricultural projects aWroved in PY93 had forestry, mviromneni or almus, resource conservidon as their "primary program objective.* a Fm agnm*ura projects W countries from die foraw Soviet Umon vmm approved Jn FY93- Twelve agrioulturd werabom an WeBeen lbr FY%-95 for a toW of US$13 bMn. M See -AgrWturc and Rural Developa" D%mtmeni, Annual Sector Review,- December M , pp. 37- 42. 71w review offm a good discussion of dw 'new regionalinational approach" as it 4ocuses on institutiond devetopment and minima improvements to physical fi&astrucww...& reaction to strong afticisms in do pad few years from OED reviews of project perftmance in the irrigslift subseftir.* 68 D. PERFORMANCE OF THE AGRICULTURAL PORTFOLIO, 1970-85 General Trends 3.21 This history of rapid growth and rapid deterioration is reflected in the quantitative record. Figure 3.2 below juxtaposes annual performance ratings to the amount of agricultural leading. It suggests a partitioning of the data into two equal time periods: the *Grand Growth" Period of 1970-77; and the "Peak Lending" Period of 1978-85. Over the Grand Growth Period the annual amount of lending more than tripled. Thereafter, agricultural leading leveled off near its peak level so that commitments (and disbursements) over the Peak Lending Period were more than double those of the Grand Growth Period. Performance also changed radically after 1977. Figure 3.2 also shows that in every year during the Peak Leading Period the evaluation ratings were well below those of every year of the Grand Growth Period. Specifically, 70 percent of the projects approved during the Grand Growth Period were rated as satisfactory; the rating dropped to 59 percent during the Peak Lending Period (see Table 3.3 below). Table 3.3: Performance Ratings of Agriculture and Non-Agriculture Projects 1970-77 and 1978-85 Compared (percentage of satisfactory ratings) Agricultural Lending Other Lending Period Mean Range Number Mean Range Number 1970-85 65 53-78 887 79 70-92 1,895 1970-77 70 65-78 429 83 77-92 902 1978-85 59 53-63 458 75 70-81 993 Note: A satisfactory rating requires the achievement of significant benefits relative to project costs and the substantial attainment of the objectives defined at appraisal, taking into account any changes in these goals which may have occurred during implementation. Where economic rates of return are included, a satisfactory rating usually requires an ERR at 10 percent or more. 3.22 In addition to deterioration over time, agricultural projects performed much less adequately than the rest of the portfolio: 65 percent satisfactory compared to 79 percent (see Table 3.3). Agricultural projects also deteriorated relative to others, dropping 11 percentage points in the second period (70 percent to 59 percent), while ratings for non-agricultural projects dropped 8 percentage points, from 83 percent to 75 percent. 3.23 Why should results for agriculture be worse than for the rest of the portfolio? And why should they have been so much worse during the period when lending stabilized than when it was growing rapidly? Both of these questions are addressed below, beginning with the deterioration in ratings over time. Two basic reasons for that outcome are incentive structures and the interaction of Bank and borrower in project preparation and implementation. Borrower Incentives 3.24 In order for farmers to invest, they need favorable and dependable output-input price relationships. Table 3.4 below shows that the prices of agricultural commodities were high with an upward trend in the Grand Growth Period. They peaked in 1977, then plunged in 1978, and continued to decline, reaching levels by 1992 that were less than half those of a decade earlier, levels that had not been not seen since the Great Depression of the 1930s. Moreover, price behavior was similar across 69 the board. As suggested by Table 3.4, for nearly all agricultural commodities, prices dropped radically and continuously after 1977. (The surplus production engendered by the agricultural support programs of the industrialized countries contributed significantly to the price-decline and at the expense of third-world farmers.) How did this affect project outcomes? First, Bank commodity price projections proved to be too optimistic. The estimation error between forecast and actual world prices was erratic but on average increased over the period 1970-83.1 In many developing countries, farm- gate prices closely correlate with world prices. So if project success required increased commodity production induced by farmers seizing opportunities provided through the project (i.e., technical package and credit), inadequate (i.e., subforecast) prices of the commodities to be promoted could be disastrous. This occurred in many such projects following 1978. Figure 3.2: Agriculture Performance and Lending so 7000 75 6000 70 5000 65 4000 60 3000 .55 2000 so - 1000 70 72 74 76 78 80 82 84 86 88 90 Calendar Year of Approval Scu: OEDD m - SAT Rating -a Lending 3.25 The evidence underpinning this conclusion is substantial. First, in the 1990 evaluation cohort OED analyzed a number of unsatisfactory commodity projects (i.e., cotton, palm oil, rice) in which commodity prices fell to levels far below forecast prices. Simply applying the forecast prices to the existing stream of benefits gave seven of the projects satisfactory ratings.' Clearly, this was a lower limit correction since no attempt was made to measure the Impact of subforecast prices in a For an illustration so Annex Figure 3.1. 1a OED Agriculture Background Paper for Annual Review ofEraluakton Resub, 1990, p. 14. 70 discouraging the expected farmer investment Further, the impact of low commodity prices on government revenues was considerable in many agrarian countries. The indirect effect of such reduction in forcing governments to cut recurrent funding across the board, e.g., for area development projects, was probably also significant. Table 3.4: Indices of Agricultural Commodity Prices, 1970-92 (constant US dollars 1990 = 100) Food Weights Total Total Beverages Cereal Fats and Oil Other NonFood (100) (78.6) (32.9) (13.9) (13.7) (18.2) (21.3) 1970 190.0 196.0 226.6 198.4 257.8 132.7 170.5 1971 172.7 175.9 187.2 181.8 244.5 127.5 162.2 1972 170.6 176.0 189.0 175.7 221.3 140.7 153.1 1973 229.6 235.9 207.5 310.0 395.2 146.3 209.1 1974 249.6 267.4 196.2 368.8 366.9 236.2 191.7 1975 185.0 195.8 167.5 259.3 221.6 176.5 150.0 1976 214.8 223.6 316.2 209.1 234.0 129.4 186.2 1977 248.2 271.6 484.5 174.8 264.1 106.5 172.2 1978 200.0 211.4 312.3 182.3 240.0 108.1 162.9 1979 195.5 202.9 285.6 168.0 242.5 116.9 171.4 1980 193.3 197.8 233.6 182.8 204.7 165.4 178.7 1981 168.2 172.4 193.6 194.4 196.7 125.8 154.7 1982 150.8 154.7 200.9 143.8 162.6 108.4 138.4 1983 165.6 167.7 207.8 159.1 196.2 116.6 159.0 1984 175.1 182.4 245.5 155.3 235.8 105.9 151.3 1985 151.3 159.0 224.4 134.7 163.8 101.8 126.1 1986 132.4 143.2 232.8 100.0 107.2 91.1 97.2 1987 109.8 109.8 139.2 85.9 113.6 90.8 109.9 1988 117.0 121.1 144.9 108.1 138.9 94.8 103.9 1989 112.4 114.7 120.4 116.3 126.2 102.3 104.6 1990 100.0 100.0 100.0 100.0 100.0 100.0 100.0 1991 95.7 95.2 92.2 100.0 102.0 92.5 97.0 1992 84.7 85.4 73.6 94.3 104.7 83.5 82.0 Source: World Bank, International Economics Depumeat, Pebuary 5,1993. 3.26 Second, there is corroborating evidence from rate-of-return data. In addition to the estimation error between forecast and actual world prices, there was the increasing bias against realized farm producer prices resulting from increasing currency overvaluation in many countries after 1974. This helps explain that the actual economic rates of return as calculated after project completion were less than the appraisal estimates, with the discrepancy increasing over time. Table 3.5 below shows average re-estimated ERRs that are slightly higher for the Grand Growth Period than the Peak Lending Period. Of greater importance from an incentives perspective is the discrepancy between appraisal and completion ERR. For the Grand Growth Period it is 9.1 percent, but 13.4 percent for the Peak Lending Period, an increase of nearly half. 71 Table 35: Difference Between Appraisal and Completion Rates of Retur breentage Number of outs of Approval Year Poects Appraisal ERR Completon ERR Difference 70 30 23 18 s 30 21 18 3 72 35 23 15 8 43 22 13 9 74 59 28 13 15 49 22 13 9 76 58 27 13 14 60 23 13 10 78 65 23 11 12 53 27 15 12 80 50 25 13 12 40 25 14 11 82 34 25 16 9 22 25 14 11 Note: These are unweighted averages based on 640 projects for which both appraisal and oestimated sates of return ae available. 3.27 Third, the macroeconomic evidence suggests that the Peak Lending Period was less propitious for agricultural investment than the earlier period. There is substantial evidence that overvaluation of exchange rates and other macroeconomic distortions in developing countries greatly increased after the second oil shock in 1979. This occurred, in spite of rising energy prices and tumbling world commodity prices, because many developing countries borrowed heavily abroad. As a consequence, countries pursuing such policies penalized their tradeables sectors, in particular agricultural exports and food production, at a very high rate. In many countries, the terms of trade moved decisively against agriculture in the late 1970s, with pronounced negative impact on investments in the sector."' The Project Factor and the Bank's Role 3.28 The Report of the Portfolio Management Task Force synthesized OED findings about the causal factors that contributed to the secular deterioration in the Bank's project portfolio. Inter alla, the Task Force stressed *project factors," particularly OED's results which showed that the *most satisfactory projects tend to be those in which there has been most Borrower participation during preparation and, as a result, the greatest likelihood of high Borrower commitment.' Based on analysis of supervision reports, the Task Force also stressed institutional constraints, the scarcity of 1 Tds outcome is wel illustrated by the secat history of Ghana, Zamble, and the CPA ftran countries. In spite of deteriorating teams of trade after the first oil shock in 1974, real effective exchange rates somewhat appreciated for developing countries as a group until 1983. For maon-ad4ustment lending countries," they fbsther appreciated beyond 1985. See World Bank, Third Report on Afasoeat Lending, R92-47, March 24, 1992, p. 28. The cwie eresaow: Key to Developraear fqpac, * September 22, 1992, also concluded that the interaction of the changes in the world economy in the 1970s and country responses had a similar impact on prqject outcomes (see pp. 6-7). a Jbid, p. 9. 72 counterpart financing, poor project management, and defective procurement.1 Finally, it dealt at length with the lack of flexibility in project design for dealing with risk and the difficulty of implementing very complex projects. These problems are clearly at the root of many failures in agricultural projects, and as implied by the PMTF Report, they become more serious in the later years of the review period. 3.29 This suggests that the great expansion in Bank agricultural lending during the sixteen-year review period, in financial and new areas and kinds of projects, was not always accompanied by sufficient resources or adequate concern about implementation issues or realistic appreciation of risks. The Task Force concluded that the Bank's role in project development and implementation contributed substantially to the declining portfolio performance," over time. In the language of the Task Force, the Bank's "approval culture,* and the Ooften active Bank role in preparation may connote a promotional-rather than objective approach to appraisal... .the quality of projects at the time of their entry into the portfolio... is not always what it might be....the pervasive emphasis on loan approval is not matched by equal emphasis on implementation planning and identification and assessment of major risks to project performance...The project concepts are not always well calibrated to the Implementation capacity of executing agencies.'0 Thus, the roots of poor performance might be found within the Bank incentive structure which has rewarded new lending, but given far lower priority to good preparation and management of ongoing projects. 3.30 ',eyond these weaknesses, agriculture is burdened by problems peculiar to the sector. In many projects, the technical packages to be adapted by farmers during the implementation period proved Inappropriate. Complexity also proved particularly constraining in the implementation of rural development projects, where multi-sectoral objectives (water supply, access roads, clinics, adaptive research) frequently exceeded the borrower's Institutional capacity. Many agricultural projects failed because of design deficiencies involving poor soils, improper site selection, disregard of ecological conditions in cropping patterns, and insufficient study of climatic conditions. Other failures have resulted from poor understanding of the local social and cultural milieu. In terms of the project cycle, preparation and appraisal were often deficient because basic technical and socio-cultural judgments were taken on insufficient information 3.31 In general, as Implied by the Portfolio Management Task Force, the 'virulence' of all these problems and shortcomings probably increased over time because of the increasing emphasis on meeting lending targets rather than an emphasis on bringing forward only projects where success was highly likely. In agriculture, on average, this might also well have required much more painstaking and more staff-resource consuming preparation work than Bank practice permitted at the time. 3.32 The area development projects suffered in varying degrees from all of the shortcomings mentioned above. Only 49 percent were rated satisfactory for the entire sixteen-year period. For the eight years of the Grand Growth Period, the average evaluation rating for area development projects was 61 percent, compared to the average rating for all agricultural projects of 70 percent. In the * Ibid, p. 7. These weroe de problems most frequently identified in supervision epouts. a Ibid. paragraphs vi and iL a The 'Poouth Annual Review of Project Peforance Audit Resut" discusses projects that miscarried because of insufficient attention to teciial and socio-oultural variables. Report No. 2185, August 24, 1978, pp. 14 ff. 73 succeeding period, the ratings for area development fell by over a third to 40 percent. Since area development projects accounted for a quarter of all projects In agriculture, their deterioration accounts for half of the total decrease in the agricultural overall rating to 59 percent for the Peak Lending Period. Agriculture and Other Leading 3.33 Taking the results for the entire 16 years, the average rating in agriculture was 65 percent satisfactory compared to 79 percent for other leading. Figure 3.3 below gives the comparative performance year by year. It clearly shows that agricultural lending compared to other leading was less successful In the Peak Lending Period than in the earlier one. It also shows that the factors contributing to an overall decrease in the quality of project outcomes were not operating with the same force nor in the same direction on agricultural and other projects. Correlation in performance ratings for the two groups Is negative over 1970-74, but positive for 1977-80, and again negative for 1981-84. 3.34 Several factors explain much of the overall difference (14 percentage points) in evaluation ratings between agriculture and other lending. [They are related to those outlined in the section on Achieving Agricultural Growth (paragraphs 3.05-08).] First, as suggested in the preceding section (paragraph 3.30), because of the importance of agro-technical problems and social and cultural variables, agricultural projects may have been more difficult to "get right than the average non- agriculture Bank project. The requirements for behavioral change by large numbers of rural households In most agricultural projects, and the difficulties in predicting the rate of this change, make heavy demands on project designers. Second, the public production model has proved a resounding failure, given the overall weakening of public administration as well as the relative lack of prestige accorded the sector. But it has been a larger failure in agricultural projects which are usually market- oriented, than in the cases of lending of a public-goods character for certain kinds of Infrastruture (highways, water supply) or the social services. The latter are only indirectly dependent on world prices and commodity demand. Third, because most agricultural projects in the period under review almost inevitably involved market transactions, they were particularly vulnerable to poor macroeconomic management, while again, other sectors were somewhat insulated from the impact of macroeconomic mismanagement and international trade. 3.35 The combined impact of these factors is reflected in the evaluation ratings of lending for Africa. Compared to the other regions, Africa stands out as having the least diversified economies, dependent on a few export crops (and their world prices); the weakest economic policies; the least- developed institutions; possibly the weakest commitment or project "ownership"; and during 1970-85, at least, a pervasive dedication to following the public production model in its modernization efforts." Table 3.6 shows that eliminating Africa from the ratings significantly improves performance ratings of lmir numerical weight in total agricultural lending in tie second period was also higher. Research and extension, which are largely insulated from direct market fories, accounted for 3 percet of agricutural lending from 1970-85. U Most of the modem sectors in African countries were characterized by economically stagnant, Snancially dependent parapublic organizations, in other words by production of too much output at values less than costs. Since they we dependent on government for subsidies (not just fiscal but also overvalued exchange rates, bank credit, seconded civil servants) they consumed financial surplus and checked growth. In some countries, military conflict also was an important factor in explaining unsatisfactory project ratings in Africa. But this would not explain why in Africa agricultural projects fared worse than those of other sectors. 74 both agricultural and non-agricultural projects. But it also considerably increases the ratings of the agricultural portfolio relative to other projects. For the sixteen-year period, on excluding African loding, the difference in performance between agriculture and other lending decreases from 14 percentage points to 10. This suggests that the combined Impact on Africa of the negative factors enumerated above Is considerably greater than for other regions. Figure 3.3: Agriculture and Non-Agriculture Project Performance Ratings 100 90 80 70 60 40 20 10 0- 70 72 74 76 78 80 82 84 Calender Year of Approval -a Non-Agriculture + Agriculture Swo OEDDaabase 3.36 Another explanation for much of the discrepancy between agriculture and other lending targets are the area development projects. Of the 216 area development projects, 98 or 45 percent were implemented outside of Africa with a satisfactory rating of 54 percent. When these 45 percent are eliminated from the portfolio along with Africa, the difference between performance results narrows even further to 6 percentage points (see Table 3.6). 3.37 So most of the difference in performance results between agriculture and other lending can be put down to the impact on the agricultural portfolio of area development projects and to the far greater difficulty the Bank has experienced in lending in Africa for agriculture as opposed to other sectors. But as discussed above, the poor results for the Africa Region and the area development projects are largely explained in turn by causal factors considered earlier: economic incentives, the public production model, and project factors per so. 75 Table 3.6: Performance tathIgs Agriculture and Other Projects, 197085, with and without Afrca and Area Development Ptojects (percentage satisfactory) Worldwide Afica Excluded Afica and A.D.P. Excludid Period Agriculture Other Agriculture Cisr Agriculture Other 1970-85 65 (887) 79(1895) 72(602) 82(1321) 76(504) 82(1321) 1970-77 70(429) 83 (902) 80(281) 86(649) 82(245) 86 (49) 1978-85 59 (458) 75(993) 65(321) 77 (672) 69 (259) 77(672) Note: Number of projects in pameatheses. E. SUBSECTOR LENDING EXPERIENCE 3.38 The agricultural portfolio has ten subsectors. Four of these-irrigation, farm credit, area development, and plantation crops--comprise 75 percent of the portfolio. OED has recently completed a study on each of these. Their results are discussed below.' Irrigation 3.39 Irrigation development has received some 30 percent of the Bank's lending for agriculture, 70 percent of which has been in South and East Asia. Its performance has been slightly better than the other sectors (67 percent compared with 65 percent satisfactory) notwithstanding a significant decline in the eighties. 3.40 Through the sixties and early seventies, irrigation projects were generally large, single purpose, and self-standing. In the late seventies and eighties, they gradually became more diversified and gave additional emphasis to the development of agriculture and the on-farm use of the water supplied. These components proved more difficult than infrastructural investments, and their performance evaluations declined somewhat, although remaining above the average. 3.41 The Bank's activities in irrigation development have been shaped by making loans rather than writing policy. Until recently, there has been no irrigation policy paper and few operational directives (notably, that on cost recovery) concerning activities in the subsector. Changes In the nature and content of projects have been determined by adjustment to the conditions encountered and the perceived priorities of the times. 3.42 Irrigation project evaluations report pervasive design problems. Typically, these projects have sought to develop systems with high delivery capacities, capable of on-off operation that can move water throughout the canals in response to the needs for a variety of crops. In the humid tropics in particular, these highly "reticulated" systems have been plagued by problems of vandalism of structures and "water-hogging" by head-end irrigators. Their actions lead to chaotic operation with serious consequences for those at the tail-end of the water distribution network, and for the irrigation community at large. What to do about this is not completely resolved, but one necessary adjustment OED, *A Review of World Bank Irrigation Experience," Special Study (fotcoming), October 1993; OED, "A Review of Bank Leading for Agriculural Credit and Rural Finance,* Report No. 12143, June 1993; OED, "Aea Development," Lessons and Practices (forthcoming); OED, "Bank lAnding Policy for Plantation Crops," Report No. 12110, June 1993. 76 will be more realistic assumptions regarding performance at the design stage. A second problem Involves the failure to provide or maintain adequate drainage in many irrigation commands. In many areas, the resulting waterlogging and salinization has forced as much as a fifth of arable irrigation land out of production. 3.43 Dealing adequately with these problems, in other words getting the design right, will be even more critical in the future than in the past. Rural populations continue to grow, particularly in South Asia, but it Is no longer possible to continue to rely primarily on development of new Irrigation schemes to meet their needs, as in the seventies. The best sites have already been developed. 3.44 Irrigation evaluations also report pervasive difficulties with operation and maintenance (O&M), with cost recovery, and with water-users' groups. Of the three, O&M is the most Important because it affects benefits directly. Part of the Bank's enthusiasm for more irrigation cost recovery stems from the presumed link between cost recovery and better O&M. But OED evaluations generally show little relationship in that water charges go into the budget as general revenues and that O&M competes with other budget-financed expenditure. Many studies suggest that financial autonomy is the key to major improvements in quality and cost effectiveness. This usually means turning over O&M to irrigators. The Bank already promotes irrigators' groups and turnover of systems, but experiences have often been disappointing. Success requires more than covenants. Findings of empirical studies that isolate the conditions in which irrigators set up groups that work are reviewed in the forthcoming OED irrigation report. 3.45 Nevertheless, the evaluation record shows that Bank-supported irrigation developments have made an impressive contribution to economic growth and equity. They have greatly reduced poverty and malnutrition, and stimulated economic growth. But as indicated, there is strong need for improvement. Since the agdlculturally dependent population in irrigation areas is still growing rapidly, future lending to increase irrigation efficiency is critical. Farm Credit 3.46 Agricultural credit and rural finance has received some 25 percent of Bank lending for agriculture. About half of this has been for Latin America and the Caribbean and South Asia Regions. Credit projects have had very satisfactory ratings over the review period (81 percent satisfactory compared with 65 percent for the sector as a whole). Performance ratings declined slightly between 1970 and 1982, but then increased albeit with minor erratic variations. The average size of Bank loans for agricultural credit is almost double the average for the agriculture sector as a whole (US$97 million compared with US$51 million). 3.47 Lending for farm credit has been subject to specific guidelines, including a policy paper, a handbook and various operational directives.2 It has also been the subject of an extensive literature and vigorous debate. Influenced by this contemporary review, the character of credit projects evolved significantly over the past two decades. 3.48 Notwithstanding this evolution, the central rationale for credit remains that of financing application of new technologies for farmers capable of using them profitably. But projects have moved progressively toward greater emphasis on the viability of the financing intermediary, the generation of M See *Agricuhural Credit,' Sector Policy Paper, May 1975; "Handbook of Agricultural Credit Intitutions," January 1981; and notably Operational Directive 8.30, February 1992. 77 rural savings, and most recently, the promotion of participation by beneficiaries in designing leading programs. 3.49 These new emphases are warranted given the experience of past credit projects. In particular, agricultural credit projects need to support financial integrity and avoid fiscal or other subsidies. The persistence of subsidized interest rates and of negative interest rates, usually due to inflation, has been severely criticized on both equity and efficiency groinds. Similarly, the not infrequent occurrence of poor loan repayment by frmers has been a matter of major concern. 3.50 In response to these vigorously expressed concerns, the number of credit operations began to decline after 1983. Subsequently, Operational Directive 8.30, issued in 1992, codified a set of conditions for agricultural credit that few projects can meet completely. Because of this contraction, some of the progress in strengthening financial intermediaries was lost, since the relationship with the Bank's client institutions became episodic and in some countries simply ceased. 3.51 Nevertheless, agricultural credit projects have supported productive investments of millions of farmers, and have strengthened numerous financial Intermediaries through Bank support for their more effective management. That agricultural credit programs play an important part in financing farm technological change has not been denied. The need for such change in achieving long-run food security is also acknowledged. Consequently, while a return to "business as usual" practices is not warranted given the need to improve the quality of overall financial sector management and agricultural credit practices, increased Bank support for agricultural credit should be encouraged, including appropriate institutional development support. Area Development 3.52 Area development projects constitute 25 percent of agriculture projects and about 15 percent of lending for agriculture. Fifty-five percent of these projects are located in Africa, 18 percent In Latin America and the Caribbean. As noted earlier, their rating is the lowest for all agriculture subsectors, at 49 percent compared with 65 percent overall. 3.53 Earlier studies by OED have found area development projects excessively complex in concept, hastily prepared, and frequently located in relatively fragile agro-ecological areas in an effort to reach poor populations. Most evaluations, including those by other donor agencies, associate the poor performance of these projects with inadequacies in farmer incentives, production technology, institutional capacity, and government commitment. In particular, there was generally found to be inadequate attention to village level institution building and almost no participation of would-be beneficiaries in project formulation. 3.54 Notwithstanding their poor track record, some area development projects have been very successful. Studies by OED have shown that with good local management and appropriate policy frameworks, this type of project can be made to work very effectively.' Recent audits In Africa have shown significant improvement in performance in the simpler second generation of projects initiated in the early 1980s: in the last three cohort years, the proportion rated satisfactory has increased 10 percentage points per year. See OThe Aga Khan Rural Support Program: Second Interim RepoA-Pakistan," Report No. 8448, March 1990; and Dynanics of Rural Developmen in Northeast Brazil New Lessons from Old Projects, Report No. 10183, December 1991. 78 3.55 For Africa, where the need to reach the rural poor Is greatest, the persistent suggestions that all area development projects be dropped from lending programs are untimely. Indeed, with several African countries now adopting more positive economic policies, and many projects underway to strengthen agricultural services (and therefore Implementation capacity), plus the Bank's new approaches emphasizing beneficiary participation and work with NOs, it would seem appropriate to review opportunities to expand leading for area development in Africa and other regions with similar needs.= Plantation Crops 3.56 The subsector comprises 8 percent of completed projects and 6 percent of agricultural lending. Project performance has been rated at 65 percent satisfactory, comparable with the sector as a whole. But the Bank's policy for lending in the sector has been severely restrictive, especially for the beverage crops--coffee, cocoa, tea-and sugar. This has been in spite of the fact that these crops have become Important sources of income to smalholders. 3.57 At its Introduction in the mid-1970s the rationale for this policy was based on the inelastic demand for all four of these commodities. The Bank's fear was that increasing supplies would drive down world (export) prices to such a degree that developing country total export revenues would fall. The Bank presumed that its own lending restriction&-and their example to other lenders or investors -would limit the increase in production of these commodities, thereby firming up prices And preserving developing country export revenues. 3.58 The policy was effective in restricting Bank lending, particularly in the case of tea. There Is, however, little evidence that it was effective in achieving its objective. Tea production grew strongly and tea exports continued to grow steadily after the policy was introduced. Clearly, as the Bank withdrew from lending, other financing interests replaced It. 3.59 But the policy was not just ineffective. It was harmful toward the weakest borrowers. Replacement of Bank lending by others led to increased investment and production in more creditworthy countries, largely at the expense of the poorest of the IDA countries, especially those in Africa. Specifically, for developing countries as a whole, tea export revenues continued to expand at 1.4 percent per year. But for a group of 27 poorest countries (excluding Sri Lanka), revenues fell by 3.5 percent per year. Similar though less dramatic findings apply to the other beverages. 3.60 OED's recent report recommends that the Bank abandon its policy of imposed blanket- lending restrictions of this kind. This does not imply that there will be an explosion of Bank lending for these commodities. Their prices are presently low. International Commodity Agreements may also discourage Bank activity. But such a change could prove helpful to some smaller countries and their smallholder producers, whose production declined in volume and quality, notwithstanding a desperate need for additional foreign exchange. 2 Such an initialve should learn from the many findng of experiace drawn in past years, which have recondy been reviewed and summarized by OED. See OED, "Area Development,' Lessons and Practices. 79 F. CONCLUSIONS Evaluation of the Agricultural Portfolio, 1970-85 3.61 This chapter has analyzed the causes of the deterioration in the ratings for agricultural projects and the poorer performance of agriculture relative to other sectors for the period 1970-85. Several factors stand out in explaining why and how this occurred: Agricultural projects are predominantly market-orlented. Consequently they need favorable price relaticnships to succeed, i.e., to induce farmers to Invest. The dramatic decrease in agricultural commodity prices In the last two decades and, in many countries, increasingly unfavorable macroeconomic frameworks have reduced the profitability of agricultural investment and therefore the prebability of successful projects. * The "public production model" widely accepted In Bank lending proved to be a larger failure for agriculture, which is market-oriented, than for certain kinds of infrastructure (e.g., highways, water supply) and the social services. * Because of the importance of agro-technical problems and social and cultural variables, agricultural projects may have been more difficult to "get right" than the typical non- agricultural project. * Area development projects (integrated rural development) have proven particularly vulnerable to the factors listed above. This helps explain why their track record has been so poor. * The combined impact of these factors is well illustrated by considering the differential impact of lending to Sub-Saharan Africa, where economies are most dependent on a few commodity exports (and on their world prices), have the weakest economic policies, and during 1970-85 exhibited a pervasive dedication to public production approaches. On excluding African projects from the calculation of average performance ratings, the shortfall in agriculture's performance relative to other sectors shrinks from 14 percentage points to 10. Further, eliminating the impact of ratings for area development projects reduces the difference in performance ratings between agriculture and all other sectors to 6 percentage points. Subsector Lending Experience 3.62 The operational findings from this review of recent OED studies of the agricultural subsectors are the following: * The weak operation and maintenance of irrigation commands has caused very large losses in output. Recent analytical work has led to the conclusion that financial and managerial autonomy could greatly improve irrigation O&M. Consequently the Bank now promotes autonomous irrigation commands, and in particular water-user groups and the hand-over of systems to them. Results thus far have been mixed, but can be improved through more careful project design. 80 * In the past, Bank agricultural credit projects did not deal adequately with nonviable intermediaries, acquiesced in government pressures to lend at subsidized and negative Interest rates, and tolerated lax loan repayments by farmers. To deal with these shortcomings, Operational Directive 8.30 imposed restrictive conditions which few agricultural credit operations can fully met. Nevertheless, agricultural credit has succeded in bringing technological change and increased output to millions of farmers. Consequently, while a return to "business as usuala practices is not warranted given the need to Improva the quality of overall financial sector management, increased Bank support for agricultural credit should be encouraged, Including appropriate Institutional development support. * Area development projects had the lowest rating for all agriculture subsectors, at 49 percent compared to 65 percent overall. Today the reasons for failure are clear: inadequate farmer incentives, low beneficiary participation, faulty production technology, weak institutional capacity, and a lack of government commitment. By the same token, the requirements for success are also well-known. Indeed, recent audits in Africa have shown significant improvement in performance in the simpler second gene*ation of projects Initiated in the early 1980s. Taking into consideration the lessons learned so painfully, the Bank should review opportunities to expand such lending, particularly In Africa, where the need to reach the rural poor is greatest. * The inelastic demand for tropical beverages (tea, cocoa, coffee) underlay the Banks restrictions on lending to expand their production. Replacement of Bank lending by others, however, led to increased investment and production in more creditworthy countries, largely at the expense of the poorest countries, especially those in Africa. OED's recent report recommends that the Bank abandon the policy of general lending restrictions. This would make possible assistance to very poor countries and their smallholder producers, whose export production has declined in volume and quality, notwithstanding a desperate need for additional foreign exchange. 4. 1NSTITUTIONAL DEVELOM* A. INTRODUCTION 4.01 This chapter examines the Banks past and current record in Institutional development (ID). Following an overview of concepts and the evaluation context, It reviews the Institutional development performance of the 1992 evaluation cohort against the benchmark of prevailing "best practice. The chapter then explores some key issues concerning technical assistance, the main vehicle for promoting ID in Bank-misted projects. OED's database, which throws light on ID performance, is limited to the 1989-92 period. The concept of ID and the nature of the Bank's Involvement In It are evolving; hence, no trends can yet be spotted from the 1989-92 data and no firm recommendations can be made. Major lessons of experience are, however, drawn with a view to enhancing the Bank's institutional development record. An Annex, 'Institutional Development in the Energy and Infrastructure Sectors,- reviews the Bank's experience in assisting ID in the two sectors, identifies key factors in performance, and, again, draws relevant lessons for future operations. B. CONCEPTS 4.02 Much work has been done in recent years, both outside and in the Bank, to clarify the concept of ID, define its objectives, track its results, and help focus attention of both development assistance providers and beneficiaries on the requirements and means of promoting ID.1 The findings have contributed to a better understanding of why efforts to promote ID have met with mixed results, why a sense of shared objectives between donors and beneficiaries has proved elusive, and what changes may be needed to enhance the prospects of achieving more durable ID. The observations in this section draw heavily from the writings listed in footnote 1. I Primary Bank documentation includes: Artwo Israel, hItudaonalDelop,men.entiv to Performance, 1987; OED, "Pre4tanding Technical Assistance for Institutional Development in Sub-Saharan Africa,' 1990; Samuel Paul, hs*itudonal Deelopnem In World Bank Project: A Crows-ectoral Review, 1990; Gray, Khadiagala and Moore, hasti&alonal Development Work In the World Bank* A Reviw qf 84 Bank ProjectA, 1990; Beatrice Byok, 7he Bank's Use qf Technical Asdrance for Initudonal Developnaat, 1991; Country Economics Department. he Refrm qf PubHe Sector Managemear: Lessaos 4f aperience, 1991; OED, 'Fre-tanding Technical Assistance in Support of Public Sector Management in Sub- Saharan Africa," 199; and Operations Policy Department, Handbook an TechnaE oAws e, World Bank, 1993. Outside the Bank, the modem theory of Institutions, with its emphasis on informational, commitment, and transactions cost considerations in understanding the role of economic and political institutions, was applied to tho analysis of utility regulation by P.L. Joskow and R. Schmaensee lA Markeafor Powr, 1981; to economic institutions by 0.. Wlliamson in as Economfc hs&xWtons of C4pAu, 1985, and Y. Baul in Ecnomf Anakals OfProperty RIghts, 1989; to the firm by RH. Coase in his 'Lectures,' JoawnalofLaw, Eonomics, and Organtrahton, 1988; to the role of public policy in influencing the rules of the game' in Brian Van Arkadia's 'The Role of Institutions in Economic Development," Presontation to the Annual Confrence on Develkpment Economics, World Bank, 1989, and to the entire proess of economic development by D.C. North in istalow, Alssftuional Change,andEconomicPerformance, 1990. InRetdding TehicalCooperadoa*4fornwforC4Pactly-BWdng inAfrica,1992, Elliot Berg oriticizes past methods of providing technical assistance tohat gion. 82 4.03 The term 0Institution' refers to the norms, rules, roles, and structures developed by people to organize and guide their individual or joint activities. Within this broad categorization, institutional economists emphasize the structure of incentives (and deterrents)-"the rules of the game--that govern the production and exchange of goods and services among people. Sociologists Investigate the sanclioned norms of conduct that guide and limit Individual and group behavior, while management spcialists probe the formally organized structure of relationships, rights, and obligations that are valued and accepted within organizations. 4.04 *Institution" and worganization" are often used interchangeably. While their purposes often converge, organizations are distinct from institutions while comprising, of course, a major element of the institutional framework. Organizations (e.g., a governrent department, ministry, parastatal organization, or privately-owned business) perform specific functions. When an organization's utility and/or efficiency is questioned, It may eventually be restructured, replaced, or eliminated. By contrast, Institutions tend to be deep-rooted, and hence more enduring. They are often born out of the free volition of people and, in such cases, represent an acceptance of the need to organize social behavior to enhance both the Individual and common good. 4.05 The norms and rules of behavior embodied in Institutions cannot be sustained without a network of supportive organizations that promote, codify, enforce, and defend them. The attributes of efficient market transactions and rewards for private enterprise and investment cannot take shape in a chaotic, free-for-all environment that is manipulated and exploited for the benefit of a few. To flourish, free enterprise requires a legal system to protect property and enforce contracts, and government agencies to establish standards and restrain monopolies and other restrictive practices, as well as organizations to undertake financial intermediation and enforce regulations. 4.06 According to Berg, "Institutional Development is a much more profound process than organizational development." In order to shape realistic and responsive initiatives, an adequate understanding of Institutions and organizations and their interrelationships is needed to Identify where weaknesses, frictions, and dysbmtions lie. As noted in the Technical Assistance Handbook, 'to equate solely the strengthening of organizations with ID is insufficient as well as misleading. Donors may strengthen the organizational capacity of the treasury to manage the public debt, but unless the values of fiscal responsibility become normative among political and bureaucratic elites, through changes in governance, ID will fail." Likewise, improving an accounting system may be futile, if the accounting data is then not used as a management instrument or as an accountability tool. 4.07 Economic performance is closely correlated with ID. The increased specialization implicit in technological progress enhances productivity, but also requires a greater volume of economic and social transactions. As economies expand and modernize, new markets both within the country and abroad are sought, organizations evolve and equip themselves for the provision of financial and related services, and the legal framework adapts Itself to support contractual obligations. In parallel, the governmental 2 Property ights, an independentjudiciary, and social security legislation are good examples of institutions that have evolved over decades and are therefore enduring, unlike edicts which are sustained only by restriction and force. I aEllot J. Brg, *Rbthinkng Technlast Cooperation, UNDP, 1993. Operations Policy Degartmant, 'Insuiotional Developmaent," Handook on TechdooAssitance, 1993. 83 machinery on all levels develops to: enforce standards and create a policy environment that supports growth and modernization; promote transparency and accountability; protect the environment; and assist the most vulnerable groups in society. Thus, ID is designed to assist this evolution and make it a self- sustaining process. While temporary disruptions may occur, Institutions that are stable bring forth corrective measures-difficult though they may be-nstead of allowiwg every problem to become a crisis. Conversely, where ID Is lagging, fragile, built on weak foundations, or thrust upon a reluctant country or agency, not only Is economic development hindered, but long-term dependence on infusion of funds, ideas, and techniques from the outside is induced. Bot 4.1: Insituiona Deveopment Institutional development is the creation or reinforcement of an organization's capacity to generate, allocate, and .usq human and fnaucial resources effectively to attain development objectives, public or private. It includes not only the building and strengthening of institutions, but also their retrenchment or liquidation in tho purouit of institutional, sectoral, or government-wide rationalization o. expenditure. 'D is typically aimed at iMproving and strengthening: * Internal organizational structures; systems, including monitoring ad evaluation; . .*flnancial man et (budgeting, accountng, auditig pancedures) n pling systems; .*paras=W =wVgm=nt,'staff&dlpment and training;. ** institutional structures of subsectors or setors; * legal frameork and * government regulations and procedures. Bigatico Buy*, "The Bank's Use of Technical Assistance for Instiutlonal Development,* WPS 578. World Bank, *Deaumber 1989. 4.08 ID takes many forms, including components that support a specific investment project, policy reform program, or the creation of broad-based institutional capacity, inter alia, for analysis of policy alternatives and improvements of administration and civil service management. In undertaking such activities, ID may involve the creation of new organizations, or the strengthening, restructuring, or dismantling of existing organizations (see Box 4.1). It may require new regulations (e.g., the protection of the environment or denationalization of public sector enterprises) or streamlining and abolition/modification of existing regulations (e.g., outdated controls, licensing and permits that restrict private enterprise, or trade liberalization). Whether at government or enterprise level, ID connotes better planning, more efficient and equitable resource allocation, enhanced beneficiary participation, better accountability, and transparent evaluation. The success of these measures depends largely on societal acceptance and active support of the norms and values that underpin ID. 4.09 This is why, beyond its positive contributionto successful development outcomes, institutional effectiveness is critical to sustainability. Whereas many physical investments can be designed and executed by relying on imported skills, there are others (e.g., rural development programs, 84 population/health and education) for which 'domestic" skills are important both for design and execution. When institutional development lags, investment benefits are eventually reduced below the anticipated level. For example, inadequate maintenance of a road or Irrigation project due to insufficient funds may result from the lack of a policy (resulting from socio-political considerations) to levy and collect appropriate user charges. Effective long-term economic management can only be assured with corresponding progress in the country's institutional development. C. THE BANK'S INSTITUTIONAL DEVELOPMENT ROLE 4.10 Since its inception, the Bank has supported Institutional development in its investment operations. Initially, this support was limited to such project-oriented concerns as training. Later, in the context of its sector loans, the Bank also assisted systemic institutional improvements such as sectorwide policy formulation and planning. The Bank's involvement in broad-based ID with countrywide implications, particularly for policy development and capacity building, Is of relatively recent origin, stemming from growing concern over the macroeconomic crises in which many member countries found themselves in the 1980s. Adjustment lending was a means of addressing some of the deep-rooted structural problems, but it soon became apparent that, however well-conceived tb3 policy reforms and remedies, a number of countries were experiencing difficulties in implementing them. Not only were there political constraints, but policy analysis skills were insufficient and institutional abilities required upgrading. Because the longer timeframe needed for ID reform was not well-suited for these fast disbursing operations, parallel TA loans were instead made to enhance borrower capacity. 4.11 In the meantime, there was a growing realization that the results of training programs supported by investment projects were in some regards less than had been anticipated, and that in many countries (particularly, but not exclusively, African countries), analytical and implementation capacities were lagging behind the demands of development. The Bank's response to continued weaknesses in pub.ic sector organizations and the unrealized potential of the private sector was a new series of free- standing loans aimed at building capacity, restructuring and divesting public enterprise, and facilitating a larger role for the private sector. Today, the major thrust of D lending Is the improvement of public sector management in its broadest form. 4.12 Within this context, the Bank's involvement in the governance aspects of ID is limited to "the manner in which power is exercised in the management of a country's economic and social resources for development." Efficient and equitable development occurs when there are well-established (and understood) rules and procedures, and an institutional foundation that ensures both accountability of those in power and transparent and consistent enforcement of rules. When the overall environment leaves much to be desired, organizational improvement supported by the Bank cannot achieve much or for long. Therefore, it becomes incumbent on the Bank to engage the borrower on broader ID issues. Box 4.2 discusses the manner in which the Bank may exercise its mandate as laid down in its Articles of Agreement. 4.13 The Bank's support of ID is in transition. The recent decline in adjustment lending comes at a time when the Bank has broadened its sectoral agenda, placing emphasis on poverty reduction, environmental management, and an enhanced role for the private sector-all of which require organizational and policy development. The amount of free-standing TA directed toward ID has increased from a FY88-91 average of US$163 million to US$204 million in FY92. Already represented in the 1992 cohort are operations approved during FY81-87 that aim at assisting public sector 85 management (Chile), accounting and audit training 'Madagascar), economic recovery (Philippines), and state economic enterprises (Turkey). In recent years, the range of activities supported by free-standing TA has broadened further as evidenced by some of the FY91 loans for economic management capacity building (Angola), economic management support (Ghana), public and private provision of Infrastructure (Indonesia), economic reform (Bulgaria), and financial and program management improvement (Jamaica) Some of the FY92 loans aimed at promoting ID are for privatization (Cape Verde, COte d'Ivoire, Egypt Guinea, and Zambia), tax administration (Argentina), institutional capacity for macroeconomic management (Mongolia), treasury data systems (Turkey), central p:ning (Yemen and China), and sectorwide Improvements such as power (EI Salvador), telecommunications (Mexico), and environmental impact management (China). Given the growing interest in ID-oriented TA, not only among smaller countries, but also larger countries endowed with a broad range of sophisticated skills, and the desire of some newer member countries to adopt market-based economies, it appears likely that the Bank's TA support for ID will be sustained and perhaps even expanded. Hence, the need to strengthen the Bank's ability to provide quality ID assistance to its member countries is particularly urgent. Box 4!2 World Bank's Mandate 'Nhe World sankls mandate, as laid down in its Articles of Agreement and as applied in pactice, is to promote e economic and social development. Its concern for govenance must be driven b that mandate. TLhe Baple's General Counsel has identified five aspects of governance that mr beyond the Bank's *nat:the Bank cannot be tafluenced by the political character of a member; it cannot interfere in the 'piritsn politics of die mem.berit inst not act an behalf of cerin member coutries to influence a bonouing a*bbr's political.oricaatioe or behaviors it cantbe intueced i.its decisions.bypotcal factors that dd Iat liave a proonderant economic effect; anid its staff must not build theijdgmenits on the ptesible~ recdn of.priclrBnmme ormebesaH hasigested that.gvera5l nc sy bowror, be televan.t to the Bank's work if it is addressed in terms of having good order and discipliniein the alangemnt fQacuntry.sxrsoures. Thus, there could well be a aeced for the bank to encourago, for example, ivsevico reform, legal reform, accountability for public funds and budget discipline. ..................... iV - -:w b W .... 4.14 The changing mix of Institutional development objectives and the interplay between macro and micro concerns make it difficult to interpret aggregate institutional development ratings. Unlike Investment-related TA where there are measurable performance indicators, much of the ID supported through free-standing operations Is qualitative. New, imaginative quantitative measures are, however, being introduced in a few cases, e.g., the Argentina Second Tax Administration Project, which uses yardsticks such as the amount of taxes collected or expediting the resolution of tax-related cases (tax auditing), and the Venezuela Judicial Infrastructure Project, which aims at improving courtroom productivity and efficiency (e.g., number of cases handled, time lag from filing to judgment), and reducing public and private costs of litigation. 4.15 The growing complexity of institutional objectives has contributed to the unpredictability of their eventual outcomes. For example, a study conducted by the Bank in 1990 reviewed Bank ys Samuel Paul, "Institutional Reforms in Sector Adjustment Operations, Disusion Paper No. 92, World Bank, June 1990. 86 experiences with institutional reforms in 55 adjustment operations (SECALs) in the agriculture, trade, and industry sectors. The SECALs supported three types of ID: (a) organizational restructuring and strengthening; (b) regulatory and procedural reform; and (C) sector policy and planning capacity building. Under the first category, restructuring covered wide-ranging reforms including the reorganization of existing agencies/activities, and divestiture of public agencies or some of their functions. Organizational changes fostered better personnel policies to strengthen leadership and improve training. Regulatory and procedural reform included deregulation, changes in laws to facilitate the implementation of new policies, and streamlining of day-to-day procedures to achieve better results. Sector policy and planning capacity building entailed the broadening and deepening of skills to: (a) develop sector strategies and monitor sector performance; and (b) undertake policy analysis regarding prices, Investment priorities, and trade. The trade SECALs had the broadest macroeconomic content and Intersectoral or interinstitutional impact, while industry and agriculture SECALs focused on the particular sectors concerned. In terms of regions, the scope of SECALs in Africa was more limited than SECALs in Asia and MENA, but wider than SECALs in LAC, which reflected a judgment by Bank staff on the relative capacities of the regions. Technical assistance was provided in most SECALs. The distribution of sectoral institutional reforms is shown in Table 4.1. Table 4.1: Distribution of Sectoral Institutional Reforms In SECAls Policy Planning Oaanizational Restructuring Resulatory and Capacity Building and Strengthening Procedural TOTAL Number % Number % Number % Number Industry 13 25 21 40 19 36 53 Agriculture 38 32 52 44 29 24 119 Trade 13 24 27 50 14 26 54 TOTAL 64 28 100 44 62 27 226 6 The total number of reforms is larger than the number of SECALs under review because each reform type is further divided into sub-categories. 4.16 The report concluded that SECALs do not seem to have been an effective vehicle for implementing complex institutional reform, especially in countries with limited institutional capacities.6 Serious problems were encountered in implementation. Politically sensitive reforms were particularly difficult to achieve. While some of the conditionality, e.g., creating a new agency, was met, this did not necessarily ensure that the agency was able to deliver as planned. The more complex the reforms, the less the success achieved. The strengthening of organizations proved, on balance, more achievable than restructuring or divestiture, given the pull of vested interests. The report offered various possible reasons for the limited success of SECALs in promoting ID which, inter ala, are discussed in Section E. Ibid. 87 So .3 ercaa sisac DeliveryMehd. .... Technical assistance is normally provided through: (a) lang-term resident advisers, normally expatriates, assigned individually or as a group; (b) abort-term consultants, usually with more specific expertise and taks; (c) consltants-Intemational, local, or joint venture-assigned to con4uct studies or piovide advice, oftimi .tiough.a combination of (a) and (b), above; (d) twAning, a profesianal xilationship between an operating mntityin a developing couniy and a iiitar it more experienced organization in another country.. (e) Bank staff, or Bank-managed consultants, for very specific sharortam tasks (or an aongoIng bas. s as part of otbr Bank programs); (t) acadeate degree weik, study tours in-contry or inaioala iner and ta be off ron the job (with apeciallad trainers, such as equipment suppliers); and' . (g) provision of kupporting equipment, software, technology, and programs inextricably linked with . - a technical task (such as building capacity to monitor social indicators).. 4.17 Complexity In ID intervention has two other dimensions. The first is the multiplicity of instruments employed-both lending and non-lending. The second is the large number of bilateral and multilateral donors involved in assisting ID. Box 4.3 shows the wide variety of delivery methods used in TA operations to assist borrowers. Some of the methods are investment-specific while others contribute to the broader objective of ID. While It may be possible to evaluate the end results of some types of TA with reasonable precision and objectivity, others yield only qualitative and subjective judgments, thus complicating assessment of U) initiatives. Moreover, while the task of interpreting the results of TA, the major vehicle used in loans and credits, may be difficult in itself, there is also the complication that the Bank's ID function has increasingly been waged through nonlending instruments, e.g., aid coordination, cofinancing and advisory services funded out of the administrative budget, grant funds, and a variety of collaborative programs and trust fund activities. 4.18 The presence of numerous donors in assisting ID enhances resources for evaluating results, but when there is not a clear and agreed agenda, and when there is a lack of consistency In procedures, interpreting ID results becomes complicated. Aid from multiple sources also imposes a heavy burden on borrowers, many of whom lack the capacity to absorb and manage TA. Table 4.2 shows the funding sources for the 1992 project cohort. There is, however, no discernible correlation between performance and the variety and number of funding resources. For example, some projects with multiple donors (e.g., the Guinea-Bissau Structural Adjustment Project and the Tanzania Mufindi Pulp and Paper Project) performed poorly, whereas others (e.g., Madagascar Industry and Trade Policy Adjustment Project and Jamaica Agricultural Adjustment Project) showed no evidence that funding complexities affected either ID or overall performance. There is, however, some evidence that cofinancing channelled through the Bank reduces procedural complexity and transaction costs. 4.19 The Bank's role in providing technical assistance in areas of policy reform, ID, and capacity building "upstream" of normal lending operations was reviewed by the Technical Assistance Review Task Force (TARTP). In its November 1991 re.ort, TARTF recommended that the Bank strengthen the internal management of TA activities, improve coordination with other agencies, and intensify cooperation 88 with the United Nations Development Program.' Among the recommendations (since implemented) to improve the Internal management of TA are: (a) the Issuance of a revised operational directive on Techncal Assistance" (OD 8.40); (b) the formulation of action plans to Improve TA management in all regional vice presidencies; and (c) the preparation of a Handbook on Technical Assistance to provide guidance to staff on all aspects of TA, and to serve as a benchmark against which to measure practices. Table 4.2: Funding Sources for the 1993 Project Cohort Investment Techinal Program Projects (226) Assistance (17) & Policy (31) Funding Local Sources of Funding 15 1 0 Bank Financed 202 17 34 Cofinanced GS 0 14 Other Sources 41 2 6 4.20 Also addressing the problems stemming from multiple sources of financing was a Development Assistance Committee report Issued in December 1991: *Principles of New Orientations in Technical Cooperation (see Box 4.4). However, the implementation of the "Principles" remains a distant goal and, consequently, effective coordination between the Bank, UNDP, and other development agencies remains a continuing challenge for institutional development.' 4.21 From an evaluation perspective, it is best to assess the institutional Impact of development assistance against the backdrop of individual country programs, taking into account each country's history, political economy, and stage of development. This requires close monitoring over several years of the results of free-standing lending operations and individual TA components in projects which support ID in the countries concerned. It Is also necessary to evaluate the results of other initiatives, given the similarity of objectives being pursued by various agencies. Many governments are seriously handicapped 7 TARTF also recommended the establishment of a new grant facility-the lastitutional Development Fund. The facility would finance small action-oriented schemes identified during, and closely linked to, the Bank's policy dialogue and economic and sector work in countries where ID is a significant country assistance objective. The approved FY93 budget included a provision of USS25 million for IDP, which began pilot operations in FY93. The IDP is managed as a trust fund and is designed to inance technical assistance for institutional development work associated with reform; country management of technical assistance; and special operational mphasis initiatives, particularly for poverty reduction, public sector management, private sector development, and environmental management. In its first year of operation, the Committee received 95 requests for grants and approved 57 requests. Total expenditure amounted to US$20.7 million including administrative costs. * A seminar on Technical Cooperation for Development Agencies, sponsored by the Development Assistance Committee, the UNDP, and the Bank is planned for January 1994 to discuss possible ways of improving the development offectiveness of technical cooperation. 89 in undertaking such assessments because of limited evaluation capabilities and a multiplicity of donor Initiatives, each with its own peculiar features and evaluation requirements. Accordingly, the National 4 4 ilm a lesfNe Odentloo in Technical Cooperation ~ Set as a tratege objev of Technical Cooperation, the longernm capacity building in dvlp ountries rather than immediate short-term performunce improvement; P lace great emphasis on the central role of developing countries in the planning, design and mpagement of Techniical Cooperation; Stit) ees .senatiknortance of imprved planning in die context of coordinated support for see. toa bobie n policies 4a4 in particular, use of a p1rogram rather than a prot-by-prqect Encuaiprosesip," i.e., responsibiity and control of Tecnia C oronograms and rejets altage by thie ntendedi beeficiaries through participatory approaches, including local N.~.~GO prticpon p ft thcky hoporancfor sustainable developmt and self-rlianice of long-term mlidigrg qspe~ialy~ in the aeas of policy alysis an eeomn nemeont: *Take fuh O cooent the new tocognition of private sector seoe for TechnlicalCooperation: *Sadourage ter use of local expertie and exitinzg struture; chootetiein~ trms of ouoms to tie achieved rather than inputs to be~ provided, * tress th n to pay greater attenton to the ot. n ototsieesfehia Cooperation activities. 1OECD Paris 1991 Techmical Cooperation Assessment and Programmes (NATCAP) was introduced by UNDP in 1986 to enhance the role of borrowers in the management of TA, including those TA programs and components which promote ID. NATCAP is government-led. An important product of NATCAP is a Technical Cooperation Policy Framework Paper which sets out the governments priorities and position on key Issues. The paper is 'intended not only to draw attention to neglected policy issues, but also to be a consensus-building exercise." NATCAP Is a recent innovation, and it may be many years before its impact is fully known. While some doubts exist about the extent to which the NATCAP process has been Internalized, it is nevertheless considered to have acquired more local ownership than any other reform efforts. Berg notes that "although the diagnostic-analytical aspects have moved ahead quickly and often effectively, operational follow-up has been slow and uneven." * BHiot J. Berg, *Rethinking Technical Cooperation." 90 4.22 While this review focuses on ID as a means of strengthening governments and improving the performance of the public sector organizations, it also recognizes that many governments are now actively pursuing, with the Bank's assistance, the privatization of publicly-owned assets and enterprises. The implications of a changin (shrinking) role of the state, and possibly the growing needs of an enlarged private sector may have to be reviewed in the context of channelling the Bank's future support of ID. An Important question in this regard Is, whether given its strong support for privatization, the Bank can provide more direct assistance for ID in the private sector. D. INwITUTIONAL DEVELOPMENT PERFORMANCE 4.23 In 1989, OED began to systematically assemble data relating to the achievement of Institutional development objectives In Bank operations. From 1989-92, OED evaluated 1,170 projects approved during 1971-90.0 Among the projects, 343 (29 percent) had "substantial" ID achievements, 559 (48 percent) had "partial," and 268 (23 percent) had "negligible" ID achievements." Among the projects deemed to have "substantial" ID achievements, 94 percent were considered "satisfactory," while 66 percent of the projects with "partial" and 21 percent with "negligible" ID achievement were also rated "satisfactory." This shows a high correlation between ID achievements and ultimate project outcomes. A similar correlation between ID and project sustainability was also noted among the projects evaluated in 1989-92 (Annex Table 1.7). Of the sectors with 100 or more projects evaluated during this period-which together accounted for about 65 percent of the total number of projects-25 percent of the 379 agricultural projects had "substantial" ID ratings, as did 31 percent of the 107 finance projects, 28 percent of the 105 human resources sector projects, and 24 percent of the 115 transport sector projects. The energy sector, with 81 projects, performed best with regard to ID, with 53 percent recording "substantial" ID achievements. Among the regions, there was again a high correlation between "substantial" ID achievements and "satisfactory" overall ratings ranging from 87 percent to 100 percent. The achievements of ID objectives in the 1989-92 cohort by sector and region are shown in Tables 4.3 and 4.4 respectively. 4.24 There was no significant trend in ID achievement during 1989-92. Table 4.4 shows that there was a considerable decrease in the proportion of evaluated projects with "substantial" ID achievement between 1989 and 1990, but that small increases were registered in the two subsequent years. However, in none of these years have "substantial" achievements in ID been recorded by more than a third of the evaluated projects. About one-half of the projects show only a "partial" achievement. The main explanation for the rather low proportion of projects with "substantial" ID achievements is a notable rise in the share of projects which are designed to achieve capacity building objectives going well beyond organizational improvements at project level. The number of projects in the former category, i.e., capacity building focus, accounted for 7.4 percent (63 out of 845) of the total number of evaluated to The l989 cohortalso includes a project approved in 1964 which, becauseof completion delays and hence a delayed completion report, was not evaluated by OED until 15 years later. I The criteria for measuring ID perfrmance are understandably less precise than those for investment operations and, as noted in paragraph 17, the judgments are generally qualitative and subjective. However, some imaginative steps have been taken to link ID performance to the achievements of tangible (and even quantifiable benefits - paragraph 4.48), and there is merit in a broader adoption of such innovative procedures in the Bank. There must be increasing recognition that ID is not an end, but a means to achieve greater efficiency in the production, distribution, and utilization of goods and services in order to raise levels of living. 91 projects in 1989-91, but rose to 12.4 percent (33 out of 266) in 1992. Given the inherent "demandingness" of projects with a dominant ID thrust, and the learning process the Bank is currently going through in this relatively new area of Involvement, it is not surprising to witness a rise In the risk of Bank intervention. The learning process encompasses all phases of processing ranging from the identification of the need for an operation focused on ID, agreement with the borrower on its priority, its subsequent preparation, appraisal, and eventual implementation. Some of the lessons learned from completed projects are discussed in Section E. Table 4.3: Achievement of ID Objectives, 1989-92 Cohort Rating (% of Evaluation Cohort) by Sector 1989 Coboat 1990 Cobot 1991 CoboAt 1992 Cobolt Total 1989-92 Sub&- Par. NeiU- Sub- Par- NegUi- Sub. Par- Ngll Subs- Par- NetU- Sutbs- lb. Neg8* Ron buial tial gible teal tial gible tntial tial gible hanl tid gbe tndal tial aibl Agicuture. 28 53 19 21 57 22 20 S2 29 32 48 20 25 53 23 Energy 43 43 14 56 31 is 54 38 8 64 36 0 S3 38 9 Finance 41 52 7 30 35 35 33 38 29 13 47 40 SI 42 27 Human Resource 42 37 21 24 54 22 26 48 26 25 46 29 28 48 24 Industry 42 so 8 29 36 36 33 42 25 25 69 6 31 so 19 Pollution Control 100 0 0 0 0 0 0 0 0 0 0 36 100 0 0 Power 60 20 20 34 52 14 15 54 31 32 53 16 36 46 18 Program and Policy 60 20 20 25 44 St 0 82 18 12 60 28 21 53 26 Technical Assistance 25 50 25 18 36 45 27 55 18 25 25 50 24 41 3S Telecom- municatioes 43 57 0 33 50 17 14 29 $7 67 33 0 35 43 22 TorAim 0 50 50 33 33 3 67 33 0 0 0 0 38 38 25 Tranport 15 45 39 25 50 2S 18 55 27 39 46 14 24 49 27 Urban 3S 33 33 22 44 33 33 22 44 46 31 23 33 35 33 Water & Sanitation 33 67 0 38 35 13 57 14 29 st 44 25 40 42 18 TOTAL ss 45 20 27 4 24 27 46 26 31 48 21 SO 47 23 4.25 The absolute level of ratings offers no ground for complacency. The more demanding development environment and the growing focus on policy and institutional reforms has led to rising complexity in relation to both the absorptive capacity of the borrowers and the conceptualiza- tion/processing/management capacity of the Bank. The findings of evaluation suggest that a greater degree of realism is needed in fixing the parameters of the projects, paying particular attention to project design and borrower capacity. 92 Table 4.4:.Adblevement of ID Objectives, 1989-93 Cohorts Rating (% by Evaluation Cohort) by Region 1989 Coboat 1990 CobtM 1991 CobMt 1992 Cat" Toa 198992 3ub.w fkr N4*h Sub& Par- No*. Suws PW, NegU Subsw h&v NsgU- Sdb& har. ftil. Romi.~ *1a AWbE naw dda gI" Us"l twa giAD %MWs tWa Albt uaw tAd gEA ANls 27 52 22 20 47 3s 21 4S S1 37 44 29 2S 47 50 &sA As sad oaft 45 38 19 41 47 12 48 SS 20 56 35 9 46 $9 15 SoWa*19 5 2 2 21 57 is 22 56 2 so aS S s2 s4 ts CeNalAs 50 44 4 27 S3 20 3 52 14 14 71 14 27 45 28 N04AMo so 45 5 36 48 26 25 SO 25 44 56 0 34 49 16 Zabk Amgakca Crtbbef 34 41 25 25 45 30 26 43 30 21 52 27 24 5 19 TOTAL SS 45 2 7 27 4U 24 27 46 26 t 48 21 30 47 25 4.26 On the other hand, It may not be appropriate to conclude that the hitherto limited success of the Bank's ID initiatives calls for a scaling down in either the number of ID operations or their scope. Instead, the Bank should adapt its operating practices to help its member countries deal with institutional reform in a way which is more effective, based on much closer borrower and beneficiary involvement, and directed more clearly towards local capacity building to improve the prospects of sustained develop- ment. ID initiatives need to: (a) take into account the capacity constraints of borrowers; (b) adopt a more gradual phasing to reflect at each stage the highest priorities (rather than a "catch all" approach); and (c) show flexibility and resort to corrective action with regard to objectives and implementation methods as circumstances change. The role of borrowers and beneficiaries, both in identifying problems and proposing corrective measures in light of changing needs and priorities, is critical to the success of ID initiatives (see Box 4.5). 93 .:�. •.л ¢ .:..:�. л :чч.::{i•:•:г.'ц. .i.�.}Ч:..... .. ::>� r�:::.::.т:.:'.:' :, <:}..}, ,�iг' • • : :•.: v -.и:. л ...: .. :: . ...: , т :?г }: м л .. ...: 'и, v.. i:: г..,:.:,::. : ..}. .W-i:\3:vti:!ц.�' :�:лС-:::�.,б:3: ••' .•а:и .г:ti�:::' г}йC::iF{-.:н.:.Н:i:л:и: , чi::'. г. г. ........ ... ...л:..�.л.,•г,:...:....:::л.. . ь......::: -.. ... i::.:.::,v: .:: ::.:. _' ..4;:, ,... и:'гi:.: ,. .л. .^ .\: . : ::С":v::....... ...:..:. у:.. }. . :• н::....:.:::лл..... . . ......... ... : ...г,':::iг,:?.:л•.ry::w.. . . ... •k .. ..... ь . ... ....... ... . .... . ... ... . .:...:: .: .. . .г :.: .: г: ., ..-..л.,и....: :..vi:0.:гi.Ji:i.:г.4<:гт:т.г:г:iwi:тгv. - У..ь:. и:у.:.......н..........д..: ..ч..л.,...ч .. .. ....чл.н ...... .. ... . ... . ... ....:.. ... . .:..м::.::.. :г. г...л.-:.:: лv.::::. лv.:w....... . ..::::г::л:: '6f:8Y,•"r.�' ... .}... г л . ... г.. л .. ..л.:не. . . :.:?,:. •: м.vг : . .... .:.: ............... м...: ;... .... ........ ....... . .: . . , л. . .. . . .... .... . .. .. ... .. . ....:ь �....,:...г.., г. .: : . ... v: л: . .. ..С .., : . .. : .. ...:: У:"{:.:::..-ч . ,У.. - :-:. м: �У л.: : : : ;: : : •;' :. . .,.,:�::�.:л, .:: � ::•�т� ,:.т,. � _i.. ..- . .п.:. ... .... iл..... л. •л.... .. . г::. . .а::: -: -:: - , ., .:::.:. . . . . :. ........:::.:;:<. .. .... ".:,::�:,..и:: . . м. � �,. :.:, .Т .....:: ::.:. .. т: s••,: . : �... . .. ,:.м: ::. : .. ..::.. .. . . :т::.;.::. . # -- , :. .т. ... .... . �:.::>:� '?�°,• : '� EFw �л':: i' rar.�.:1�>`� ::'#i►���Ta .. �.%.'�.�11.1� ....... "'..:. � . :.�;:г:.:::..:::.и.:<:;г::ба:�:::':.i.?.:::::_:.:,>�?:г д'��>z:г::�::��':'�;iг:z:.:`?: ..у,...х...м :.......т ... ..::::.. .... �? �и. ..::..:.гч,.�,::.:.�.,...,и .С: м, м:, г~.;:.,:гo-:::.: т:.: .....: .:: л.:; л... , ..л 4 4 ......... :: : ..,. :.. ..:: .. ц:. ; ..: г : ...-::.: : . .F .:г.:.• :::, :-. . ,...:ви....;...,., а}..........., ...... ..:...: .....:: r..v. И . . • 4 л}4:•: . •.: w: :..: .. .... . ...т . . .:., .. . .: лл.г... .:':'.:; •.: ' ..бл . . , ..л.:: м.::.г-:i:::.::г:: :•::.• . . � .,. •:::n..i.4,::й:.гt:::: v.:. - ........'г.....: .. :' . .: ..:г., ...:.л...:. �...л.-: ,.....: ...: "гг:..v.n. . . ...:. ..;': .,:.., .. .. ,..:f4.т '4:� •::.:ыг:.:. v>::"�::�.:: ::.:'.'� 4 : ,: :.: . .. . �: : . м:. ; .. л; :.; .: .,у ... . :. . .. .: ...� .. .};.:�: ', •г; - . . .: . .:. : -,::...-'. : - 'г::':::.:.... . . ц. ; ::.<г.�. �: `i:;i:.>� �'.-.� 3l7}7/: :. tl: _ :: ... : ; ' :.. • . .. . . :": . �. _ ." :.,''j•ri (��'уΡ, I�� уΡ�.�{tl�q СΡj �}� `��. � ��уΡ��уΡ,�.,��` ::Клi �уΡ�� j]q��� , ... ,� �`f(!.`.�..l.fMRM"��'S.�вв�в ,. .:..11F�1�V � т'�•:::�КЯ"Ai'iVliVwG�{`j{W���:г:.<�:С •;: .. м.. .,.а�иду .. : .... '.:. < ' :...M±m�r►.1�3�. .: , :...т�::. : , р�"4tiкг8iiдб'д>'aiыt.`:л.:.: ...., . -. :�т:• ..:м;>.;:-::т; :' iv�dUa[ :::::.: ,., .. ......:.: : . .. . . . :. .: .. .�:.,..:••>:•;: <.:;;:::.: б:,,>•.� ц..уΡуΡ : : . . , . • 7,�.� ..�>:Е �I1tЯ:•t�1.�SC@гСН�Я:� :pit!�!""'"' п .�:;:.-::: : .. � : ... v �t1uA .. . . Oi1ta_4lгеsти'811xcQBatiil:ŭt:�Pata• � �п��ttiat:iq:0_tfteci.��:;'1�ieF �п1!bYaпiati�>�{'� +'.:'ч lC'';�.. ... ,и:;':.:.:... , .: ..и ::::...б. . ...: ... .:: .. . . :: : ,,. :,: ;:.:.;..-:�� ',.i'.9л` .:..: .. .. ::.-: и;.,:_.. ::::: . >,т;;.:,, :' :.,.г;:м::::,F.:: :::>:::....:::,�...л. . . .: � •:• - л ...с'�-#�д`..д�еа�t0.'tь::.. .. '.. Criiв.a3K�яatv'дя0 lifx '�'fd�l"iцо�д:-:и'.,: •:<:::>.:;;.,ь;.::.:�;<.>::;•.. •.,;:... , ...,... ..•<:: .:. . .. �"''' � ......:'.`..... � :�!?::..: ... : .: . :!�l!. ... �У�..::. : :..:.:....: �г�.����fv�: ;� ;,.. : ': `:for:?�I{..feгnз�ors� : :� : ���.�::.:ч>::.:- :.:,::.и.,'�:.�::1:.., . .. . �,. . ...... . . ..:...... ... . . _.... . . . .... ..,: :...у, ' ::< л::. . ..`.:.. :.: . . :::, .:,-��. л. а�tlяя: ... ; artid я� ягц�а Q.Q� ёi�iг.tт6пrпёaiiгliak �eaaiiz�j��рΡ� ♦� :.,- �Fp � ..,,- - .. . .. r�tv�.�d и:�а�:я�`� ��аа�3': ;.у.ч . ......:�,. ....... ,.. ..,-.. : ...:::.::.... . .:.. .:... . ..:..:.: .. ;.,. �i""... , �..ь::.х . �:!?д., . ., r, с.:..::. - • .. п:••г.•. .:г:,'Г. у. :>:гi.in?t::isi::.,:.::.:: • .. . ..': .'а,а., .•ч... , . ... ....: . . .:. .ц, ,.�,.�. .-:,г-:А :.,:.и:.. . :, �g��� :,,\.. .,:�'. ":Ч1.V� �- '•-:1' :. ;-;:'iF:::: n:л ::.н . � - ."#!�!�Я....•...�А�..:: ...R..;.... . .. :..:А�8й��f�'Я�.�QI1iQA�: , :�;�R:�7-Q ..'. ..: :���`.�.::,:,.:.:.: , . Jq�:1� ' -:'•;� :::•::,..и.:. .::...;..::..:..... :•: :.<..:.:.,..:,.:....:: ... �� .. . :.: ::: . ::: л, . . . �...::::.:'.:.: :�� ��:�:� . .�;:�>,: � А ч{ . !;�..q *. :�.� . �..:;�е, �...,,и�,t ���� '.::,( .. ч . ♦�рΡ,,�МΡУΡ�иwxw� �},,����` ,�дt,� .L,в�, �ri,.,,,� гп:�� :�lHRdSKiii!�+':;1Fi4R,�Я4MMy`��l.,,ë1�,e.��f,MlV.t•".1Р!'�WMN�dOi�� . ... ...рΡ..�:�.t1.� ц� -WPM.MЧiVSF.!3.• " .:.. .^'7^р;. ,.::••:.•:-:.F°^^`" . ;�}!�!!F0.. :`т.~чг::•:::•J5*.•:;:�:Jл••г::::г:..г::-: а;: -::::.. . ...:. . f�i9�F . .. .. л л. ....:.,`1 ... ... . . .::л-чo-;::,>o-г:•г:� .:.:.. ... ...:>г.::..'..'- :::.:f'.'...;.. :r: ,т.:::': а,;. - г,.-.,; т . . . �.�} . . V• :• . . . .. .. . . . .. . .. . . . ... . .... .... . . . �д '�'WVa:рΡ�ц� ..:vn::C?: гл:: : . ..:.: . : � . ,.,.. {:: ..... .в.г...в .. .Р:, � . .:.и. :..,}.v.. Q� nгj{уΡ�рΡ� :у`i.:л �г:�'б4':лvгь :'�� �' : л . •. . :•г::и:гг:'•:.i.::.• �.�1.� г� ггтггi:'гт::.:: .г..::. ,-. ::}б.: .ггv,ч.:., .. .. , .. . �/�-��j i::{л �Т. .. . � . ...тS. • , г: . �:<,.:.п,.v}нмг: � �. ... .... ... .:лм: '.:г.,:: ::.и.м�•.У::••а .}, �: �•: •dd.v...v: ..и.з.:..,.лн ' г �PF•�•:•..... : �\.:,. 'i}; 1 . ..�.. ... . . .... . .. . ..... ::..,:ц:.-:. .. :.v.s .., s :+и. . ....... ... .. ::: ::::.:: . ..,+.су^ } �' г;,:tti>- +,г�}:it . . . . � :: . . : . . ..6': `" ' " :.:. . . . ... . ... л . ..... л . . ...... ....ч, . .... ....: .... .., .:С•К . . . . . . . ,. . . . .. .. . .. .... ... ... .:. ..ч :. .: . .л н. ,,:, . .:vл. .. ..... . ..." :. .. .. .. .._..... - :?:... : i\:. ti•:h:;г: л�'::%.i{.:: . . .. .: . . . .. ... :. .. ... . . м.. :.. , л: ::...-..... . ... .:. ..,.. . . ..... .,. ... ....: v:}:г :ч\: . 4�.{?�:�г .. .. � . . . .. . .. . .. ......... м. .... .. ... :.... .. . , . г . <+ .:: . .г..:: n.:::м.: ..\...•} вч'ЧЧ.г. ~:}':.{г: т..}:'г . i г: ..�.. ..✓...л. ...ч.....г.,.... г.,..:�_ \........г.г ...::. ......... ........,.... . ...г........::::.л...4 ..{л ::�.. :. < .лм.:л...,.......л.л.. .:.v.........v .............v:'-:г.:v.=::глил..,м:•г}:лммг-.и:м.:: лvн:.У.иг.лжл,, ч: -:' .�' :.:.r::::.......; .г..м :..:..::...: . ..:,. .... ........,.: .: . .. .. ..... .. ... ..... .. .. ... ... ... м.г . :,:�:i ..\.. „У+, .ti:� . .: . .::.. .... ..... ...... ...г....., ...... .. ... ... ..,.... .. ..... .. ....v......л•:::.::г•.:.г::..:.:., ....... ..:..... .......: :.., мг,:вгл{{��{tij�i г�Кv }sN.:..,yv,.:.,.. м.. . :.. ...:л..........: ...... :......\.......ч.........гг. .. ....... ......... .........:..н..........✓:w::::лмнvл ••`л..�i,•: .. т ...... ............ ... . ..л . .:,.. .. ... . . ,лУ . . .. ...... . .. . . .. .л.. , .... . . .. ...... ... . . J+:ц.v.:::.ц:л-м:..м :::: .г:'н , . , ,. . . ., . .. . . , .....,.... . ::.�л. м . .... ...:^.,. . ...: .г.....г. ......::.К^:•:Л'.i`4_:�?:.: г}'',:тм:sС�.:г:'.г�S::.'л .. ...... .. .. ... . . . ... . .... .. .. ... ....,.... . ...... .. . ..... , л. . . . ..- .и�г•:.,„....:.,w,.н::.,..::.,.:..-..::...,,л::..:..:::.:.::.•.:.....::ч:....•..и.:.. ..:.....::....ч .::.:............. ....:.и...т:.• ,:ь:o- .: чл. ......... .,. ......... . ..... ........ . •. . . ,.'....:... .. . .: , л.. ,.. г:. •.: . :..: ...; � ,:.:л:.;.... ... �:......г.л . .. ... . . ... . ,.... ...... . \ . .. , ...... . . . . .. . . v .,. . .. . ... .. ... ..... .::.w:..г.:. ..:.:v :.., ::•,F,_i . . ....., .. . ,м . .. . . ::.:..:•.м:: ..... . .. v л� r •:,•�iiг /���, {,�� УΡ.�`'' ::4.\`:г :>ч4'�4I -':�i•� .::i: дiтРттл} . и:.лхм....:<.::::::•., . . . . . . ... ... . ..,. :,.: ��уΡ .: ч:< к. гi:т� т• . S.ч�i'`. ,.ь�-:�< J"в'� :.;ч.г.:.:. :}.v:v:v.vr:: ,т:i?i:. v-л;бгг :.{��F: :{{�QL -:� � �l:.:::� ::чУ:: ::<'< .}-5 } ...л. :г. :-:::....... ......л.: i,:L:}i : ..ц ..�.. ........ л... ..; . .. .. . . . .чwл..... .:. \v::i:.,:::л г ..мт.. ., л.... . , ч .......... .. ...... ... л. .. ...,., . ..и . .Г"�` .л �...,Г..:.. . ,м,,.. . . ... м .. . , . . ,.... м- 1ч , ... . . . .. . ... . ... ... .. л ... .. _ . . . . .... .л'.: л ..., , ;..;. - л _ ...,.. .. . ...... : .. ... ... . . .. . ....::..::pi:.:S:{:;,:лh:•г>�vлv,Sv.тл,:.{:vv::r{:гьi •л'•k.г. ..г,ч::.г...г:?,:,^..:•:.:•..:: .. '....:.: .. .:.v.., .� ... :. .... . :... .. .'Cti:`:,чu.,iv�:'{:�г.ti:::i:v:::::л` .::::4i:: ` .:- .: : . �•.,\,,:::: sw ... ::....:.. :: ... ::...: ,.::: ::'.: г,r.:' ... .::. ,.. _ ... . _. . ... .: : . ..,- ., . . . :::лt:;:гг:.т,:г::: ?:.::::.;:д.,:.:. . �:..iл:тцiг.i . ,: ..т,.,.г.....,._.,...........:..... ,:.:....:..: . ,,. .... . --р• .r.... :. л ... .. . .......:... .. .. ... . .'>...: :..: :..:.:.: . . . ..:•. h.. h.,, . .,г.....л,и,. ....ь ..ь . . ... .. . . ......, ... . ..::� г i:S': ,...... ..л :..... .... . ........ 'г.:::.л::..- п т,гг^:т .:. ..s :;г?, . ..... .:л.л:.:.....:.:л .:...........n. , гл .ч.. .. .... . .......и.у. .: -:..=г:.:.. Liгiгi'ггi'.ë::::r::,vin�: . ...... . . .. . . . .. . . . . ,.. ... .. . . . ..:.. ... . .. . .. . . .....й, . .. ...:ц.�,.:.; :�',•; .:.�лгл:'м :,:. ч ..::'.•г а5:б?:�5ti...• ;.: ,<а. .йг,-•, _ .. . .... . :л. .::: _. ......."..•.. :: ..... .......:. . ,. . .. ... . ..... . .. .•.t�.. :хё� :З:i.:: j;;t;::т: л.. . t,,-.. .::.. ... ......: :л.� ..... ........ ... . . ..... ... ..,.,... �+v'^.'.л. л..т:::.;�:г: м. л. Ч v. .� . . . .k . .:.ы.. нт.: .... .. , .. ...:.: . .. . : ..: ::: ..: ... . л.... ... - . . .. . . л. . . . .:л.гг:'• ::г:: ^:::,v::":':;'г: :м-: г:'i;:.; г. ,n; г.:..; . . и,:т.л:г }.ут .:тг. • . . .. . . . . .-' .: :. ..;.: .... . ... . ,.. ... и; _ � р i >::::::>°:: ,•.: ,, : . :�<. , 2�' , г:.-: : iну.:.�:п ::r ..г. v-: :ь:л .tг.ч.. .di :}. . .•,л]�,':.'v•� дг'. . al1 А . `'.:т .. л .х'?z<а`vч: �' !<'�..,.�;��ncT�f�it.. .�r�:ra� .tr�E.�4w�:1��qaal,cqаад' +ьf� roгсt�То��..... :л>::,�;: ; . г ... ..,, _ .. :...: : л.. .,.: •;б,:........:. '<: .:. - --- . .. :,,» �: ;,-, :;.::. -н., л,;:-:..,-,�-: -. (6j рго]еа taaka wвre partlcnlaгly aiafpla, or nвw ageaoiea соиlд ata�t otY wlth eaaier tirat taakr ot tAe pro]есс'а � dea3ga аrав aimplf6ad; {rj iLad alior,atioaa from кhв goveaamaat аад beaaficiary coaцibutioaв wero oбtaiaod promptj , �(д) а$еасlеа were pz�apred Ьу beaвftciariea. acata ageaciea, даvвlортвак baakв, N(ЭОа. екс., to adharo to the ' ' • 1mplamo�tatioп асЬедиlев; (о) therв was camplemeacarity Ьвtа+ваа aaioa taken Ьу atatв аад lоса! govвrдmeat, гЬе lаtквr Ьвlрiаgдо reduoe costa •, аад доlау, апд alicйing greatar usa of lоса! maceriala аад 1аЬог. Wh�e ilае Haok аад other доаога customarUy адорс а"auppfyaido' арргаас6. bp bцildiag ар tLгe cagaciqr of ageaeloa, tlie • Northвafit Btazil рго]ect ед,ратiедее Ьав вЬоип tf�at детаад-дг3vед ртгааогоlтаа� bеаадlс3ат}вs sad ot1a� соаs�дод pattiв� саа ' •. Ье 6� oi�Ca1 faawr iя кЬо pro]aCta' виссеаа. 1�►moog цiв apxtfic тгд�ив адОрсод w�o; btealc+a6 ир Piaa�iцg ер¢ aaocutroa �югjддs . •; . iдW Егоqиааоiов e1�at eaiaaided wit4- dGпion ayote� w аагаок еlесtвд laadera, w6a would t�aa аеlеск &от а'manu'� of pDaa�lе . • acxiviU�3 апд aub]ectiag вдеоицаg agвoGioa w`дотаад а5одса' Ьу aaking that pmt о! tdeir Я�лдIаg DO сЬалтеJl4д ihYOUgh Rьо ааега о` their aarvicaa»ао1 jaat beae�c':агу groupa but ather paбic a,geadea аvеЪ аа devatopmeat Ъааkа апд atiuдIcipal • goverameata. Fundiag ageдcies also aaslaced Ьу rвqoiriag aarmwty вресМвд taalca, meaBUraЫe акаадвгдв foa� рег{`ormaa� адд оlваг реааlцев for aot perFormiag. •'I3в аисевавЯ�1 ооатроаата id�citiad Iu tЬо ewdy йад. aaioag оtЬвеа. good atato-lovet pplhlcal auppo�t, иroц-Qаfтед аад • :high-prloriry моеда, empowв�rmeAt оР beaвflcfarlea, аад асо�в to addltional reaow�oera.. ТLв atudy вuggoeta that wЬце диrв сап' ,�:'be an giiaraatoo that висh аиссеааеа can bo гореаtед. wheкver р�аавlЫе. poafdve predlspдяia8 �Оога ейоиlд ba 3деасКlод snd '. , iacбrpocatad iato рго,�всt design. F.двтрlев аге: � ,�.' (в) :nakiag s aia$�a рго]всt иац rospoдaiЫa for дю pmjec� аа s w4ola, hvt poгmйtipg ic't4ь t�го �,j г�дi0riв6iрв: ' . �����•: . . .. . . . ..; ` :••(Ь) : iavoiviдg beдeficiary or$aдixadaтu;thraug6 tце рмjан аоц �а кЬе � iadasдit �ot tЬв йяaigo of iave�ata ао t1�k . л tha deвigA гаау Ьо iтргоиед: 8оод ре�fотттке atimWated адд аддlкiояа�l к+ивоиrеоs •moЫlйerl;. • . (eJ diвtiaguie#�iдgbeкwaeд tboge iaeerveatioдs wЪiоЪ аае suicaЫв for deeepce�li2od inaaagameat'аАд qтhe� toctirrit� •.' . � а}:аге 4coaomiea vf асаlе di�eo praviвioa Оа а regioaal ar вtara-wide beaia; :.. {д) г�вiа8 AkY�oa1 Вад aocial %fraa{ruCWro tD дгдsиет beaofita to th� и�г pobt аяд iвадlеяS vvl�о mf,gdt ot�'ю{аэD: •�} • Ьа вхсlидед fmm а pt�odt:ctiod ot�ted program: ' . . � (в) �у appraisiag аад moahoria$ exiatiag orgaaiтatloaa сЬаr maq ао1 tuvsriad azperlг,nce ofpravldia$ вervloes { со tha nual роог адд wbkh а�ау Ье иааЫа to адарц • .. (Q tceeping crвdit compoa�ta атац аад eaperiдuental iu а LigWy inlteцoaary е®vlroameat: аад • (� daaigningpюjoaawdlchasвabietoreapoadtothвpolfцcalprloritiea--aotbqfiaaaciagaaygoverameat�dor8� асЬетв, бut Ьу baing aOJo со awhch юаоикхь whea а goverpmeac'а priorftiea promiso rвaJ апд sustaiaaЫo . fгpprovomeata for rural familiaa. •�',Dудатfсв оВ Rura1 Developa�ot й Nautboa� �га�i1: Naw I,�гrna �ат С!д P�rojecta.• . •. � bч 3оо Аrйка lвкаеl,'[аацt�нiодв! Dwotopmoat--iacapcivas ко Рег!'олоаиое,' Wodd Ваах.199А. �аа1 eit'gaa9 that ооацпцед .. рсее®иге оы govarnmoat agыtcieв Угот iptaaded 6anaflciarias аад othara ооасвгаед witb aervico dWrvery can be ва вРРесцvо •, р�радн of gettiag atŭoient аад timely actioд. Ra doвcriboa tЬia аь а'compWtioa аап+о$аtе," вiаев ie ia c�Amparabk �о тсроаиге •'. to wmpetitivo рreааигв ia cria privato aoctar. 94 iSSUES 4.27 A review of the IM cohort shows that the issues and constraints heed by the Bank In promoting M are generally the same across seat= and among regions. ft also shows that the manage- ment of teclinical assistance In support of M b lea sadsthaory than the outcome of Bank-assisted projects and programs. However, the review also shows that there is a growing awareness among staff of the Lmes relating to M. There is a noticeable improvement In the Identification of potential ID issues In some of the recent hWW Executive Project Summaries, and a more refined presentation of sucb Issues and possible means of addressing them during appraisal In the Final Executive Project Summaries. 4.28 Studies have established that there are continuing problems associated with ID projects, some of which center on the Bank, and others on the borrowen. Among the lew satisfactory aspects within the Bank are. (a) the uneven quality of the Institutional andysis (limited staff skilb, little or no borrower Involvement); (b) the lack of a strategic context for ID, leading to an uneven treatment of ID Issues and failure to recognize the key links between strong Institutions and overall development;12 (c) deficiencies in design (e.g., Inadequate recognition of borrowers' limited implementation capacity, excessive components often requiring an Inordinate number of expatriate personnel, and a rigid approach to 113); (d) lack of indicators (quantitative or qualitative) by which to measure progress; and (e) shortcomings in supervision (e.g., lack of appropriate skills to address complex and difficult ID issues promptly, and delays In . project components that are doing poorly). 4.29 11e more common problems borrowers have are: (a) an inadequate appreciation of the importance of ID for sustained development; (b) lack of a sense of ownership of 11) operations; (c) reluc- tance to accept a situation In which an external agency is perceived to be taking the lead in . - and shaping the domestic policy and institutional agenda; and (d) the perception that Bank staff are not sufficiently sensitive to (or knowledgeable about) local cultural fimrs, socio-economic system, and modes for consensus building activities. These aspects are discussed below on the basis of OED findings, with particular reference to the 1992 cohort. Sector Work 4.30 There is mounting evidence, stemming from reviews of the Bank's ID record, that operations in support of ED are rarely built on quality sector work and lack a strategic context. The Bank undertakes comprehensive economic and sector work ni many fields. In the IM cohort, less that 10 percent of the projects were not preceded by sector work (most of these projects, eg., disaster relief, did not require sector work). A source of continuing concern is the uneven quality of the institutional analysis (and the limited attention given to M issues) and the resulting mismatch In requirements of the rapidly widening scope of ED in Bank operations. The concern about the lack of adequate Institutional analysis is attributable, In some measure, to the OED finding that only SS percent of the projects evaluated during 19W92 (about one thousand projects) were sustainable in the long run, and the recognition that the institutional. ftnilation of development is a key strategic element in safeguarding the long-term viability of development projects and programs. 4.31 Despite the recognition that institutional analysis is a key element, the resources deployed to support and expand it operationally and to place M in a strategic context have been modest. Concerns sw TedmW Asdsb= Task Pone, *Maw&gTechnWAsWsMn*ainft IM,R Rwit, World Bank MI. 95 relate to both the number of staff called upon to deal with ID Issues and their lack of a strong ID back- ground. However, three regions have stepped up their efforts to address ID issues in the context of both ID components in investment projects and free-standing TA projects. The Africa Region's organizational approach and analytical work has been instrumental in initiating ID-related work in several countries In the region. The Public Sector Management (PSM) Division in the Latin America and Caribbean Region has a staff of 11, and its work program includes several new ID initiatives. Because of the need to transform command economies Into market-oriented economies In several of its countries, the Europe and Central Asia Region is supporting concurrent efforts in some of its countries to facilitate the transition by creating or adapting appropriate institutional mechanism. 4.32 Institutional sector work is demanding in terms of the skills and resources needed, and has a much longer time horizon. A good example of promoting ID within a well-defined and borrower- supported strategic framework is provided by the Bank's assistance to Indonesia's power sector (Box 4.6). A strong commitment both by the Bank and borrowers to improve the performance of the sector, a good understanding by Bank staff of the key ID issues, the phased introduction of policy and Institutional improvements in a flexible manner, and a consistently productive dialogue characterize effective ID assistance in these cases. When the availability of staff with ID skills is limited, and cannot be readily obtained from the outside either, it may be preferable to deploy available resources more selectively so as to maximize the impact. Specifically, a determination needs to be made as to which borrowers are most ready, committed, and able to benefit from the Bank's support, which is then directed in large measure towards those borrowers. It also avoids the possibility that the Bank's ID assistance may be spread too thin and have a much lesser impact on a much larger number of countries, many of which are ill-equipped to benefit from It. A more selective effort in initiating ID may also facilitate better supervision of ongoing projects, particularly those involving a *process' approach which requires the Bank to be very alert and responsive (see paragraphs 4.33-4.41). If a more selective involvement by the Bank enhances the success of ID operations, it will also have a better demonstration effect for other countries, besides providing a valuable learning opportunity for the Bank. 4.33 Specific guidance to staff undertaking institutional sector work is provided in the Handbook on Technical Assistance, which recommends that a strategic framework for ID be created through sector work involving four stages: (a) mapping relevant institutions; (b) conducting a qualitative assessment of institutional arrangements; (c) promoting an understanding among donors (in countries where multiple donors are involved) about the proposed key strategic initiatives; and (d) executing wide- ranging discussions with the government and leaders of public opinion to ascertain their will and capacity to support reform. Design 4.34 The rising volume of Bank support for ID has resulted in projects (and TA components) becoming more complex since they deal with many organizations and address interinstitutional relation- ships. A growing share of projects escape the purview of a single (or even one dominant) organization, do not pursue finite and measurable end results, and are therefore more difficult to design, implement, monitor, and evaluate. Table 4.5 lists the ID objectives and Table 4.6 the specific components for the 1992 cohort. SSo "nstitutionAal Devlopmnit Scotor Work and tratgy,- Handbook on Tecnudcal Assiawe, World Bank, 1993. 96 4.35 Project design involves three distinct elements that have a bearing on the ultimate outcome. The first concerns the number and complexity of the components, their sequence and phasing, and the borrowers' absorptive capacity in relation to available skills. It is axiomatic that the full range of project components should be acceptable to borrowers without whose support success would be limited and questionable. Equally, project design should be consistent with other efforts already under way, as well as initiatives being supported (or contemplated) by other development agencies. Finally, appropriate specificity of objectives, end products, inputs, and sequencing is imperative. 4.36 Given that countries with the most need for ID to improve their development management are often the least capable of absorbing assistance, the question of what constitutes a minimum package of components assumes considerable significance. If components are too few, they may have little or no impact; conversely, too many components would strain administrative capacities. Among the evaluated projects in the 1989-92 cohort, overly complex designs arose in the Burundi Power and Distribution Project, the Rwanda Power Project, and the Zaire Shaba Power System and Second Power projects. All these projects included demanding U) measures-reorganization, improvement of planning, operation and maintenance, strengthening financial management-without adequately recognizing that the agencies concerned were also concurrently carrying onerous responsibility for implementation. A more manageable sequence might have been a first phase with limited objectives, followed by a review of lessons learned, and subsequent phasing based on realistic assessments of feasibility. Cumbersome institutional arrangements led to serious implementation problems in Ecuador's Guayaquil Urban Development Project, where a multitude of executing agencies and several levels of government were involved. Overestimation of the country's absorptive capacity resulted, for example, in Tanzania's Petroleum Project training component not being accomplished for want of staff to train, and in Guatemala's Power Sector Project ultimately failing because of the agency's inability to handle a technically complex and risky project, which the Bank did not fully recognize. On the other hand, the choice of a simple organizational set-up for Ecuador's National Low Income Housing Project resulted in successful implementation. 4.37 A lively debate continues on the issue of specificity versus flexibility in project design. The blueprint approach, which connotes specificity, has a long and proud tradition and is deeply rooted in "hard" investment projects such as in infrastructure and industry. This approach has played a major role in building up domestic institutional capacity in developing countries. In a wide range of situations, it has led to successful technical assistance funded by the Bank, partly because of the accountability and transparency associated with this kind of intervention. The blueprint approach involves a clear definition of objectives, specific components that serve as the means for achieving the objectives, firm cost estimates (including contingencies), and a predetermined timeframe. It also involves precise terms of reference for the key actors, including consultants. It is, therefore, particularly appropriate for projects involving physical investments, especially those that have been preceded by feasibility studies, detailed design, and engineering. 4.38 However, the blueprint approach is not equally suitable for projects focused primarily on ID, because of political factors, less than full commitment to agreed policy and institutional reforms, instability of institutions, and arbitrary and unpredictable personnel changes. In such situations, a *process" approach is more appropriate because of the flexibility It imparts to objectives, means, and priorities. The process approach is deliberately designed to build and maintain ownership among participants (who play a key role in initiating changes and certainly in reviewing and endorsing them before they are adopted), to emphasize learning and capacity building, and to cope with changes in the operating environment. � <•wwx•г•С':: •ц'л.'t?�l�сг.:. г•у;•..:, . ч't�`-г.#,i''чtл4ч,•:j•г.� чп ;ч;••v,;;;.'�г.; • '::�r::... �}:г?чг? ; ч;чч - . �..ч. •:<:. � ; :� :; � . �'t':, ч л-•. э. м�а л.t. ..с: ,}. /:?4г,t' ;•н.4�`::;ж:>•.',<й%tC't;.. ?.;л':;м�•.4г�н•г:"'}::$: ч;�п::;= ,9vh..i.$?, ...'�ч.•}гt±4...г. гi3.:iл::i�:г:г::'г'.:ii:Ч:v:i�:%:лi:.н.v:?yS;-T�:. . л'ч•чг . J...�. . �}:.:'+.ч: � {{+.• . �{ .{� . ;Ow ч9.....•AS:•: :•У.ф:г .'г •i.::i?:.?'::::::iг:i.v::г:i?:.?::ч.}i:;уы.г:'i+:.гг+ч?чt:iYt :гi.Mi�Y.';;чв[vivvК{в•}чv� :?.ч:•г.ч: ч{И'лчv: ..:':F9 'Лч.:. :it���... ..Ку : :. ;jг.:: •:�"•ч .{}} .�fi.:г.чиг.нч��лчч:+jv�v+M •. ...:: :...... .. .:::ч. .:: �•: : , .н .-ч�;�гг?;•.,'.;ижи'.. .....�-3+"' .,. . гУ.}. '.�.`�с:•ч :,•.;.,,,ч:;;:.ч.; ; •;,чч}.ч 5• о-•'�t • ;ч,;гг, • `+'j,+ ......... . . ............;...:.,,cv..cv,-•s�}�гп.:;#;:.л, эС.. .. . .�о.�•л. 'ч •н. : ч, , •} • . . f. •'•• л • . ;т.. 1гу}�г;г.г�,. �,' t. \'чд'�.,";гчг•;'лс;}л.: r.w�t .;:л., ; �..,. ч....i:;•ч...v,г-!!-:?:"v:� :-....; :г . ..:2�}:;;. `;?:S;?';М.;;с.�.4*:м2�а.. .:�к�мw:•.�:. �.,У г.г.. , , .3ч:: ��ч'- }::$:ч ;а k't ......:. _ ...:. .. :ц:. i:Y.v. :гЧг: •г,Сч;;}г} .:.{:г}..:4. лиW;iч+,С •г�:н' �и. :•i} ..д;'•ч'гУ. :.гб7}•- +•г; ^г: �. .л ^ .гж,•:гt: •г•::;;.: '�:i}i:::.г?.nг,}�: �::iл::. v.i::�: ..... л; .. г{;::Тг•ч, . ч.. ...ч�...л::чп.,.ч �кч:::: л :�4 уΡуΡ�� �ч�: :`-i�; ,:с.. .t•п:: ,�-�ц ��уΡ�;���'уΡ� I1{/�г :1is�:i г•:Jгг:• ;:�:: -:F.. q Х' ...:./f�N.%•-4 Л.. :.цу .. :: . ... У�д.г ч.�':...... . .... �:.•г.ч• � ��.}. »•..•�� '{ :'�s:%'; к'.с�?чаок . �. - чг .ч.'�> ...�:�дМ! 'и:�г'SR:},'!?7г�. л:�•:::•}•�•::: . :.: .:,?:�v. ... .: i>t i: н<:;:. У.�. .� нt':yhd. .vh•: й ': • .;;... �':; v.,L..�'{•.•:?�' ?.•:4.^г : л•.14'•� .,�г�,�•л ..S . vvW'.. . ;{;:К•. ':v:.: п: ...::::�:.:... .. 2;. ..; :\:. ., .у.. .:.'^». . , :ч • ;';г,?;?: .: ;гл:?t•:. гi. . i4 : ;•::-t`�vЛ:�• К{ �Ri4-:t :•г?чч'i :'члч .....;г � г :н'+ч�г: : 'г. , _" �;: ' • г•? .::::...: ..: :^?.�д:'9г.i?:}5:• ?г.: .... ?':у?:.:` _. . . , . ;....... . • . : ...1,. : . ...:: -..._ tr •�; •} }�7�рΡ 1 /�/� � {�QСΡ $ .��'� .•... .���•��.....'^ . 3:.7�:. :..�:�::�".°�•.�:. � ..х•. s,'ч ,,•�;: ;:г.., ;;.;�, ч.�..;.. ..ю� <:.;г, t'• . г�. � '?�. . .�•'ч'•:•? •: ' . .У.мl.> .�' . ч•ч'. :•.{: ; ::••.••�.•вч.. .v . .•.�. `.�ли г.г'��д { г. � ^: �w, л. . } :,ч•. :+.{�л.'.;!�:' �:л ..<.ч :•:{tч.ч ..п•. [тТ��[?:•;• :смчг:��:; ;;£:м:;�:t`? ,•;,м�,н:.�...}г,,...ч;о:;;.:ч�:.,,ч..,.•,:.: , ..,... �.. . .':.......: .�. . . _ , :. . .. � . •....: :• �::. .,; ...ч�: ч ... ,.... . .:л�:;.. ��.и., r`.: г?.г:::. � .ч:..:.::.....н: ;.>?•:.ч.. . - � ;ч.. :... ... .♦�� iiw ? � ,. .,�j}��}� .+� :.ч, ,� �+.: �ц�� �/ .��:� �s�t�::'°:� � � {.-. :. � ?��:�?�??�Iй!3�� .. :: ;�.:��� .ч�г�. .м.. .СΡ.i�?i•.ч �:~,г; : iF :•.•.••.�l.•.•f ��;: ��. ....•�г'���,ч. . гпчд� ?}ч. . ..л :;;ю.. ;%Т.�::С:.�"�,%'_i°.�iт ::;t. .t.v ;•п::3�:'•7�"•'л�'; �;""[�.., ::4чv...,;.ц<г.;.ц;.}. :.+.,.: г:?3:::г.^•..... ч:5s�г:г;:л :ч�..м...r.. ;iд}�:..;��,�+;•��,�'.'��'.�'^.•'�'.'�.��'��.ч��''��'�'.'h.vv.A;...'+•v}�.�h{•}чйй}i�,v.w.vA.v? .r.vi;�Yt"Q�,�ц�� � 1вУ,M1Л + � +...��.., у�.4•�Ч�ч: :.с�(�� ч`'�1��9gil�йRftv»��''�#l����Т��� ��''$�:i�"'7�f1�"'�: � �"��Ч.�R' ��г�. г. � аго��sод �вг�nвац в�t 8ap�q�8aar�s � Лi7д Р��вв ss4.� �`е��� ��[ �° uo�Msaa'(R Уа � �nmпQa�uaa pns s�fo�d.;a so�,toe.8tтo� зв�tр т р��.wдлооnг. озиs�а �вnnд�рвп! �. г�nps�. .. •рацгтn. аiам �тsо�гиоак аор�арапаb рае �rtt�гd . �УО:� ��� ��YvpL� �w;и"" �• ��i"Sr"•" 7'i""' �R �� р�.�"�"" :�м "°��"►I ��'�'v • .�• ;�:л � '��г� � �! Р� i�II� • � �: ПОреаtгО,р� i�AB�аер�. ;.`�s""а� •'1��2ци�;• AR� '�.., '� i���� F� Ра�l�С� �•� � �• ��••oIPPP� �• � �] �:•. ариаrд о; риа •.`вдад�. $с�,rагр�} ,�м;аи. дп �s..a� 'в�дв �ыiвnt� �� 9°� �� `���:: Ynamdoiaлqp :�лиодtс�и в1ри�. э�р �8х�tю .р� s�тг8юат , а4� �дааао�.•�а�аее� �; OZ�&А '�l.•; в F�t� Ч�3i8�!+.У. �LL 'lт�s:атрв.е�вдаоv я7�sод+ � в��вр1��3 !рп� �tг:.К�кn°�"�t.$'°1murJ "qo['�at��. -ао PaPI�� �'�.� � Р� 'а�� �� �[ 3n �� � �.� �� I�pT!+ozd в�v�атд q�m.N. P�i` ЧаЯжЕ3 $Ч.I. `еввsюл° lfa� �� о�м OOZ �l�ut '8тв!в� Iiaг+p� ia�P�"� (ЮО`� �гаоs b�d� :. to3 `b8-$36L arI `�а4'г�гах► Э3е�в т цсgа" g19+r,ROB!�too р� о� ь�пи!�т� `� �8 ��+ `рав gopxu�Ic� . aoiцW WsrP Рие саnтs�шва�ат� 'аоРв.�аЕС т эвр.t�д� а�lвп�в Р��. �Ч 1�Кд '�08GT �I� �� �� � •saвol �q�m� 3о ютт�в 8и�оЕ ®� РА�ю � �I '� � � � �l�лер � �F �i ���Р� 1�Я�4Р�.�Я� � . Р� 80L6i �А т �^�+о18 s�w?sеат'д� �._� � � г� Р�Ч �о�д. �1%,�и �лв.�,Х:. 7� • эlрр�ш рое �asrлsadn�в w3 �П� � `s�8c►�i..8� 3о �д �R !� � ��.. � 3о g�g !� �:� -- ��. Ре. ps�s�sss �"поц�ррs т`mq сiв�д lюлоаак .p�iu�n� в tю pas�o,� . омi 8итмоiТо3 �4,L `�rQ 3о ��в яоРпЧ.�lF �'i�I �� F� �+� �й Эи �пвтаnо�дап � рвв aa�т�rqeq� Qo ро�да�� вд�вfозд о� �sтц aq,L 'Р��бF �� QT �'� Р°g . ��F �i1�� '�� ��4N�IP р� �[д �амод и�,iйт зо Qop�гuqsao� sq1 пг �в Sa!PN►ad о� �и!PPg �i '�R� 1�!�► 3о ano атл� ��оз�в пв 8в�доreлвр �в раnпив '�tвро3 в�nпгlиоз Р� �896 t a�g[ �� т ugRвq q�rqм 'хо��ав iвn�od e+q� ц! �ивт�ло� nиisвunpгrl � Р� �S 9Ч� �� �'��1�[Io'J ,•впорв2вда avanbas4� ��1�! ��R64� �� P��n д�влl��� �Ч ЛiУд '3�$� 1�n�j вlPрст e.N"Id 3о �4��9 � ��1 '� � ��I 3о �9� � Р��д sвц . Ртпг '�nstd sвм,ад в8� �о аоратадо . ров аот�пдв�о� Рае � вепо�'�Е 's�nц�'�гц � �3?IF� �Е. �!1�1� �i'Idi в.� �[tгоУл� �S � � • �� #� • �+►вр аамаТаге�сt �to3 9о�гвт�s'ва le�p�1 }��$Р�р ti�M. й� �'iloJ• � �ii!�►� �[�� tQN[ ��`!►Z Р� Сц�[ф� �lZiцwb[ZZ ��i3 �.•. т в��вtыд �arood R��� 1�'�й�4L. �� �Ф � t�Id ���. tгo:qspi^э о'q.t. x�S �+од S.�oPal И °0р��� �qS Р� �NpM•��S �'b �Н Lб 98 Table 4.5: M %j!!qve4 IM Coked hVesus" Tech" Program and ED objectiva ftjwb M4 Assionce (17) Policy (34) TOW PfflMM adeahu I I I ROODWOO &U=v=W - 1 6 28- W %" ftUMMU aeam= 1 o 3 4 BwArouwsaVNaWdRw*vs*W InfiastructutWUsban 0 2 10 12 Prkvsw SectorMweial Intetmadiation 2 0 is 17 Establah a Now Otgemization 2S 1 1 27 Efirnhwft an Bxkft Chganhadon 1 0 3 4 Reghvdwa and StrenOwn = Orsanizabon 198 13 is 226 -Now Logitwon 3 1 7 it D=Vgadan 0 0 4 4 Decooftinflan 4 0 0 4 Table 4A Dbhtmdn of ID Cmponmts and Frequenq, JM Cobort hriatowd Tedn" PbUoy and Ploieft MO Assidazw(17) Pfogmms (34) Total QMBWM Planning and Pblwy A4811YOSS 26 7 5 38 ResouToo MwAgealft 55 6 6 67 SkiUs Uppmfigg 164 8 2 174 MW09MON"Minift Ill 7 2 120 QUAgbBWWB L%d Pmnwmtk to 1 9 20 Repb" Chansea 20 2 is 37 CWH Suvift Rdogm 1 2 6 9 bW4=*Ud= Taking 4 1 0 5 Planning, Budpft, Auditing and Evabwfion 12 4 9 25 4.39 However, dw blueprint and process approaches (specificity versus flexibility) to project design do not have to be viewed as mutWly exclusive. Each presents advantages and disadvantages. Specificity promotes clarity of objecdves and Introduces greater discipline with regard to how and when the objectives are to be achieved. But objectives, mean , and timefiames; should not remain rigidly fixed, even in a changing a:vh=mW *.S., when the original paramean are no longer fkdly relevant. As Boyck notes, -Fleftlitty In design allows all pardes Involved to be particularly responsive to changing requirements in a choWn environment, as well as to redesign. non-performing components during 99 Implementation without amendments to the legal agreements and covenants."" Buyck also notes that *the disadvantage of this approach, however, lies in the heavy demands it tends to place on the Bank during implementation. Without doubt, flexibility for the purpose of masking imprecision and uncertainty as to what is to be done, or as a result of a hurried design, will inevitably lead to poor performance.* 4.40 Clearly, stringing together loose and poorly-defined components or accepting open-ended timeframes can be recipes for failure. Having well4efined objectives at the outset, embodied in explicit terms of reference, are seldom a disadvantage. But when the objectives are Immutable, rigidly pursued without regard to their continued relevance, they become obstacles to meaningful change. On the other hand, the constructive change permitted by flexibility will not automatically occur unless the new ends, means, and timeframes: (a) do not undermine the core developmental objectives of the operation; (b) are consciously adopted; (c) lead to clear mandates for all major project actors; and (d) trigger suitably revised terms of reference to TA personnel to reflect the agreed revisions. Thus, the pattern of the administrative costs is closely linked to the approach adopted. In particular, the process approach requires heavier staff inputs during Implementation. 4.41 The choice of approach to ID should be guided by the requirements of the particular operation, the involvement and receptivity of beneficiaries, any expectations about changes in key personalities, and relevant experiences from similar earlier initiatives. A number of examples testify to the effectiveness and the limitations of the blueprint approach. Outstanding among these is the remarkable rise of the National Thermal Power Corporation (NTPC) in India, with considerable financial support from the Bank. The principal objectives of the Bank's support of NTPC were to: (a) create a central organization that could efficiently and speedily construct and operate large thermal plants and high voltage transmission lines; (b) introduce long-term sector planning; and (c) serve as a model for state utilities. The projects that supported NTPC included substantial physical components as well as considerable ID. The blueprint approach was used in both. The highly successful outcome of these projects is attributable to: (a) the strong commitment of the Government of India (under whose auspices and guidance NTPC operated), and its willingness to provide NTPC with sufficient autonomy in managerial and financial aspects and in personnel policies; and (b) a clear understanding of the future role of NTPC in the power sector (including a performance contract which clearly indicated what NTPC was Axpected to achieve, at what cost and by when). 4.42 On the other hand, the guarded expectation that NTPC would also serve as a model to the power systems owned and operated by the states has not materialized, largely because of (a) political interference in day-to-day management of some State Electricity Boards; (b) lack of commercial orienta- tion; (c) the launching of uneconomic activities such as rural electrification, without a specific provision to cover costs; and (d) an excessive number of employees and bloated payrolls. 4.43 A combination of the two approaches has been useful in the several agricultural research and extension projects in the 1992 cohort. These projects offer a wide range of options with respect to the choice of technological messages, the density of extension coverae, or the sequence of implementation. But while they are process oriented (continuous feedback from bieficiaries, demand-driven advice), they also embody fairly rigid operating procedures characteristic of the blueprint approach (frequency of 14 Beatrice Buyok, The Bank Use 4f Technical Assistane for sstudonal Development, WPS 578, World Bank, 1991. 100 contact between farmers and extension personnel, and between extension and scientific personnel, need for farmers to conform to advice regarding inter-row planting, and input usage). 'ox4.'Iile- rnzt, Dwsign Chaqgm through the. P$WMee Iprac o 1"hdutondl The Chilean Public Sector Management Project offers important lessons for ID component design and implementie. The adical restructudn of Chile's public sector in the late 1970s n4 the recessionof the early 1980s left the government with little capacity to reassume ,t $ole as economic ,polleymaker and anager. Badk asitancewas sought to strengthen public sector decidenam=lvg, d an a US$20.7 milliii project with tI$1 illion Bn Dank fmancing was designed to: improve public seotor Inves t platming, the single largest component; enhance public enterprise management standast incteage the quality of government statistics; ard promote exports. Tw years later, less than 5 percent had bean disbinsed, and changei miicroeconomic 1ithusaceq *ad goverment policies required major revision iin projeit desipn4 Accorditigly, had at the *initIitiv of the ovrnmient, project revisions resulted in the change of' project priorities, with support to pilie sector investment planning and economic statistics taking precedence over st nthening public anteije maneent4 The initial alow start.to project imnplemrentation was compounded by a lack of goverament commitment, bureaucratic decision-aaking, and frequent changes in goverinmnt-appointed prqJect coordinators. H*owever, with the revisioni in place, government comnmitment improved, and speclfic -proposals consistent with project objectives and government strategy were prepared, disassaed with thellank, and quickly finatlized; The decision-making framewvorkc was simplifinexperienced toordinator appointed' and given managerial autonomy, and close and constant comunuication with the Blank was estab,lished. The result was a relatively smooth and effective prg ect implementation, ith $ yercei2t of the 1aexpended funs befng disbursed over the seit two anid a half years. Despite a neqjor aliange to govenlament prior to close, atmost all project resilts are being utilised and can be viewed as sirtaabbe. In many cases, the government is financing continued development and implementation of pojeof coinponents. This high degree of sustainability demonstrates the 'value of fmnancing components, whert possible, wvhich have been put forward and are desited by the borowr. ....*... 4.44 On balance, the need for greater Bank flexibility to adjust or redesign ID) components after project approval has proved an important lesson of experience. In six out of ten Bank technical assistance projects rated outstanding in the 1992 cohort, Bank flexibility in redesigning the project was cited as contributing to the success of the operations. For example, the Tunisia Mining Technical Assistance Project, unrealistic expectations about the absorptive capacity of the implementing agency and the timetable led to a pruning of some ID components, after which implementation proceeded without further significant problems. In the Chile Public Sector Management Technical Assistance Project, the initial ID objectives were too open-ended, but the projects built-in flexibility permitted a substantial redesign, which led to a sound and successful project after the first two years (Box 4.7). The evidence suggests that in TA projects, the quality of the redesign is at least as important as that of the original design. 4.45 OED's findings confirm that the blueprint and process approaches have the potential of handling different kinds of development problems, and that the final choice should be influenced by the kind of activity and the borrower's capacity in terms of skills and organizations. There may even be a transition from one approach to the other during implementation. The implication of this finding is that 101 a more conscious institutional design strategy must be validated during the early stages of project preparation and firmed up in the context of appraisal and negotiations. Indicators 4.46 The design of ID projects or components has rarely provided for the systematic monitoring of progress against explicit performance criteria or concrete or dated targets. It is assumed during design that the benefits of ID are largely, if not entirely, qualitative, and that any stipulation of time-bound results might be inconsistent with the slow and somewhat unpredictable evolution of ID and the flexibility needed In approach. The lack of criteria by which to measure ID performance is comparable to the lack of a required calculation of ERR for social sector projects, which often end up without any benchmarks for measuring performance. 4.47 The lack of indicators creates two types of problems. The first is in judging whether an ID operation achieved its objectives. The second, particularly in operations where multiple components are involved, is in judging which among the components fared well or poorly, so that the poor performers can be weeded out and the scope of the promising ones increased. ID components continue to be supported on faith rather than on the specific promise they hold, and ID is increasingly perceived as an end rather than a means to an end. The experience of Paraguay's Highway Sector is instructive in this regard: in spite of the Bank's involvement in seven projects over 30 years, many of the ID issues faced by the sector, and repeatedly identified in successive SARs, remain essentially unsolved. The Bank's efforts at institution building are replete with such examples (agricultural credit in Pakistan, the power sector in Turkey) where the description of the ID component remains identical through a succession of projects implemented over two or three decades. This suggests that a series of repeater projects under the auspices of the same entity does not necessarily guarantee progress in ID. A far more critical element is the institutional analysis through good quality sector work, as well as a productive dialogue based on mutual trust and confidence (see Box 4.6 on Indonesia's Power Sector). 4.48 New ground is being broken. The Latin America and Caribbean Region has taken the lead in providing performance indicators in some of its projects which only a few years ago might have been regarded as the least likely candidates for such action. The Tax Administration Project in Argentina stipulates improved tax collection/arrears reduction, the Venezuela Judicial System Reform Project requires the speeding up of case disposal and reduced costs of litigation, and the Chile Public Sector Project calls for faster processing of bills introduced in the legislature, from tabling through discussion to final disposal. Staff need to be encouraged (and selectively required ) to come up with similar imaginative indicators-quantitative or qualitative as circumstances warrant-to measure progress in achieving ID objectives, while also recognizing that the indicators may lack the vigor and precision of the measures employed in investment projects. The use of indicators will take nothing away from the "process,* but will probably impart far greater credibility to the ID components from the borrowers' perspective. 4.49 The Portfolio Management Task Force called for a strong effort to introduce performance indicators. As stated in the PMTF Report, "But without clearly defined success criteria, it is impossible to recognize-and in turn to eliminate-components that are unlikely to succeed. Without performance 102 standards, it is impossible to Identify shortfalls during supervision and to set in train corrective "IS measures. 4.50 OED is currently undertaking a Study of Monitoring and Evaluation in the Bank that includes the development and application of "key indicators," and a review of the Interdependence between indicators and the monitoring and evaluation system. The Overview Report is expected before the end of FY94. Supervision 4.51 The supervision of projects or components aimed at ID requires staff skills and numbers which, because of the rapid increase in the number of such projects under Implementation, are in short supply within the Bank. While in-house staff resources can always be supplemented by consultants, the overall scarcity of resources to conceptualize, prepare, appraise, and supervise projects is a cause for concern. The TARTF noted that "expertise in institutional development, without substantive experience and knowledge of its application in a development setting, is not sufficient." The situation is particularly worrisome for ID projects based on a process approach. ID projects require close monitoring to facilitate early recognition when a component is faltering or when an objective is becoming unrealistic in content or timing. In either event, appropriate, achievable alternatives need to be developed in close consultation with the borrower, and consultant expertise and equipment needs to be realigned and redeployed in accordance with the revised design, all within the framework of the original legal agreement and financial provision. These tasks require staff with participation skills and experience. 4.52 The overwhelming evidence from the 1992 cohort suggests that cases where ID strategies and monitoring plans are consciously and explicitly drawn up as part of project preparation and appraisal are few and far between. One undesirable consequence is that the pattern of staff usage over time is not systematically adapted to institutional development requirements. In particular, there is evidence that standard appraisal and supervision coefficients are better adapted to blueprint than to process designs. In the case of large physical investment projects, in which technical, managerial, and financial matters tend to receive most of the attention, ID-related aspects consequently receive less. Since some of the ID components may not have received adequate attention during preparation/appraisal (either being afterthoughts or gap fillers at a late stage of appraisal), cursory supervision may fail to identify problems promptly and to propose appropriate solutions. The limited ID expertise of Bank staff and resulting negative impact on the quality of ID work and performance of ID components have been highlighted in numerous Bank reports. The dearth of appropriate Bank skills is also mentioned in a number of PARs and OED studies concerning the energy and infrastructure sectors, although not analyzed in detail. Empirical evidence suggests that several factors may explain the Bank's relatively poor performance in helping to design and follow up on ID activities: (a) the small proportion among Bank sector staff of former managers with hands-on experience (and "feel*) for ID problem-solving at the corporate level; (b) the lack of familiarity of the Bank's technical specialists andlor inexperienced generalists with the broader set of ID issues (at the sector level), which the Bank is now tackling; and (c) until recently, a reluctance on the part of Bank managers to commit substantial administrative resources to the analysis of ID issues, presumably because of the inherent difficulty of dealing with the subject, and the high cost s ffective Implemdentaion: Key to Development Impact" 103 of retaining consultant expertise, also In short supply. An OED report on supervislon noted that -in the softer areas of policy and institutional reform, vital design components were often left to be decided during Implementation, either by default, or explicitly as employing a process approach...Whereas a degree of the process approach is legitimate in pilot or experimental or innovative projects, the Bank by skipping this pilot phase for many of its new style projects enormously increased (a) the risk of failure, and (b) the difficulties of project supervision.' 4.53 The review of 11 TA operations inthe 992 cohort that were directed prWacipaly towards ID shows that the time devoted to their supervision, the skills deployed, and the frequency of supervision varied considerably. For example, the staffweeks (sw) ranged from a low of about 12 (Paraguay: Preinvestment Studies - 12 sw, and Philippines: Economic Recovery Assistance - 12.6 aw) to a high of up to 100 (Madagascar: Accounting and Auditing Training - 62 sw, Chile: Public Sector Management - 74 sw, and Jamaica: Technical Assistance - 100 sw)17 The limited amount of Information on skills deployed or needed makes it difficult to conclude that the missions lacked skills that were sufficiently present, particularly skills to review progress in ID-related matters. However, of the 87 supervision missions for the 11 operations, 40 (just over 45 percent) had only one person on the mission which, given the multiple components and ID focus, may not have been adequate, particularly since several of the operations were first generation. The number of supervisions during the life of the project ranged from three to 11, with most projects being supervised between seven to 11 weeks. In complex and multifaceted projects (especially those based on the process approach) where the quality of redesign may be as important as that of the original project, the quality and quantity of resources atvoted to supervision are critically important. The Bank is continuing its efforts to increase the number of staff with experience In ID matters, and the regions have in-house specialists to guide task managers on various aspects of project processing. Borrower Commitment 4.54 Lack of borrower ownership is cited among the principal reasons for the limited success of ID operations. Three factors have in the past influenced borrowers' attitudes towards ID projects and components. The first is an Inadequate appreciation by borrowers of the extent to which lack of appropri- ate policies and Institutional capacity constrain development efforts. The second is a perception that funding agencies do not try hard enough, or devote adequate time, to understand the cultural setting, socio-political motivations, and consensus-building methods before proposing project design features or pressing for specific conditionalities, in part because of their own priorities and timetables. The experiences of two cohort projects partially validate this view-Brazil's National Land Administration Project in which a controversial land distribution component raised major institutional problems, triggered by affected vested interests to whose concerns insufficient attention had been paid, and the Thailand Northern Agriculture Dc'*-opment Project In which the sensitivities and cultural blases of hill tribes were underestimated. The third factor Influencing borrowers Is a reluctance to accept a situation in which an external agency is, for all intents and purposes, conceptualizing and shaping the domestic N OED, -Bak Experienc in Project Supervision,' Repon No. 10606, April 1992. 11 Data relating to supervision are extrcted from PCRs, and may not always reflet the fl resources devoted to the supervision of the projects concerned. Por exauple, in the case of Paraguay, progress was closely folowed by UNDP staff in a parallel UNDP TA program. In the Philippines. the tise devoted to supervision may have been included in another concurrent operation. 104 policy/institutional agenda.1 The end result is a lack of shared objectives, i.e., borrowers never fully identify with ID initiatives stemming from the outside. They may go along with the hope that ID components will accomplish some good and the belief that their own responses and actions will ultimately be the principal determinants of the eventual outcome. In the meantime, the Bank (and other funding agencies) proceed on the (often justifiable) assumption that ID has been recognized by the borrower as a critical element in development, since progress without it will be limited, halting, and unpredictable. Thus, many ID initiatives proceed in an environment where misconceptions prevail, exchanges are not free or frank, and actual collaboration between borrowers and lenders is limited, unduly formalistic, and probably conducive to the achievement of only minimal results rather than the broadening of collaborative horizons."' 4.55 The commitment of governments to Institutional development tends to be strongly influenced not only by their cultural and value systems, but also by the political stability of the country, internal and external threats to security, and any unresolved disputes concerning territorial boundaries, some or all of which have characterized many countries (particularly in Africa) since their independence. As long as governments are preoccupied with political stability/security problems, it may be difficult to engage them in a meaningful exchange on broader ID issues, and possibly even on the narrower subject of enhancing the efficiency of public sector management as a whole or individual entities within It. Therefore, the Ba,''s ID initiatives should be tailored to the prevailing circumstances of the country, and be based on a thorough evaluation not only of the country's willingness to promote ID but its ability to implement and sustain major changes, taking into account the forces that resist or obstruct such changes. Specifically, the Bank should refrain from promoting ID in situations where governments are unstable, lack a popular mandate, face serious internal dissention, or lack the capacity to deliver on assurances or conditions. In these circumstances, the Bank's assistance may be directed more beneficially towards specific (and limited) organizational improvements, which tend to be generally acceptable and may have a quick pay-off. 4.56 Thus, among the key requirements for the success of Bank-assisted ID is the borrower's unequivocal recognition that- (a) ID is needed to meet an existing deficiency (e.g., in policy formulation and/or implementation); (b) ID programs serve objectives that have been assigned high priority; and (c) local capabilities are inadequate to promote and sustain the proposed ID, and hence need to be supplemented with outside expertise. The borrower should also explicitly recognize that poor (and in real terms, declining) remuneration, political interference in day-to-day administration, and uncertain career prospects for civil servants seriously constrain ID, and that when commitment to ID is not accompanied by a willingness to address these matters, any lasting gains in ID are unlikely. A borrower's endorsement of the above criteria offerb the best hope for successful ID assistance. The initiative for seeking the Bank's assistance would clearly rest clearly with the borrower, and the Bank's role would be to respond a Another factor is the lack of involvemnt of the intended beneficiaries in discussions pertaining to the design of the ID component. The resulting lack of interest on the part of beneficiaries may severely constrain the impact of an ID initiative, even if national officials (e.g. treasury, planning) are highly supportive of it. 19 While the ID record to date shows that these factors have limited achievements to well below their potential, there are exceptions that demonstrate the Bank's sensitivity to borrowers' mores and attitudes, a successful display of good faith and intent, and exceptional care in providing for the borrowers' close and continuous involvement. Indonesia's power sector, India's NTPC, and Nigeria's agriculturalservices demonstrate the successfulachievementof ID. Some of the Bank's earlier work, e.g., Israel and Heaver, Cowntry Cofnanent to Dewopment Projeas, and Gray et al, 'Factors Affecting the Reality of ID Work in the Bank,* also confirm this finding. 105 to a felt need of the borrower. In this situation, TA would be *demand driven,* unlike many past Bank- assisted operations conceived and designed by the Bank (Box 4.8). h bteh a osan nte seaned forTA, a osdrbedegre f m al rofessonaltrust distiestbbltd a al lves ssesig teelieath k- to TKis~ not elaply a umtter of Wikn a pinistersO acptance of the need foritane Consuet denotes a borrower's wllingness. Aleisthe~ or atd laws i,s esand procedures; commit staff, resources, equipmeat, and buildings; abide by agreed conditions and undertandings in the Loan Agreement; anLd take whatever action is deemead alcoeary for the successful.Implementation of the project or its TA components. Many agreements reached at the top levels of government are often highly tenuous, and TA is often requested not because it is needed, bnt bepause it might pave the way for finandial assistance. In such circumstances, the TA may be resented bth sational.ataf hence, chances of its success would be eriously impaired. Borrower commitment is likely to change over time during pcqject implementation because of social, a*tiC1 and ecootcors TheOrefore, task managers have to endeavor to keep it at a reasonably high lelb(a) wtshluldwl canorate targets and monitorable performance indicators in project design; and (b) coutleiall sseusi peiformance and attaining reconSrmation of a borrower's commitment in light of ay ane t project environment 4.57 When severe capacity constraints limit a governments involvement in ID and require the use of a large number of overseas personnel, both the government and the Bank should plan ahead for the reduced need for and eventually disappearance of such personnel. This will increase receptiveness to overseas personnel who will then work in greater harmony with local staff. The productivity and contribution of overseas personnel will be signiflcantly enhanced, as opposed to a situation where they are at best tolerated, and at worst resented a ,ietracked. 4.58 OED evaluation has developed empirical criteria for assessing borrower ownership," which include: (a) locus of initiative; the degree to which the program was authored by the government or was government-centered; (b) the level of intellectual conviction among key policymakers; extent to which policymakers agreed about the nature of the crisis and the necessary remedial actions; (c) the expression of political will by top leadership; actions and statements by the government and key political figures indicating support for the program; and a ~See JII. Johnson and S. Wasty, "Borrower Ownership of Adjustment Programs and the Political Economy of Reform,' Dircussion Paper No. 199, World Bank, 1993. 106 (d) efforts towards consensus-building among various constituencies; efforts by policymakers to evoke cooperation from, among others, beneficiaries, local authorities, and implementing agencies. 4.59 Using the above criteria, the 1992 cohort of adjustment and TA loans shows that the ownership Index was very high or high for 17 projects, of which 13 (76 percent) were rated satisfactory. The Index was low or very low for 18 projects, of which 13 (72 percent) were unsatisfactory. These results are shown In Table 4.7. Table 4.7: Borrower Ownership Index Satisfactory Unsatisfactory Ownership Index Overall Rating Overall Rating Very High 3(75.0%) 1 (25.0%) High 10(76.9%) 3(23.1%) Low 5 (27.8%) 13 (72.2%) Very Low 0(0.0%) 0(0.0%) Note: The first figure indicates project numbers; percentages are in parentheses. New versus Existing Organizations 4.60 There are two contexts in which new organizations are created in Bank-assisted projects. The first is when launching new objectives and related functions previously not the responsibility of any existing agency. While the functions can be grafted onto an existing line agency, the resulting load may overwhelm the agency. The second is when the line agency under whose jurisdiction the functions would normally lie has limited potential fbr expansion or reform. Particularly, but not exclusively in agriculture and in Africa, the response in the latter case has been to create a project management unit to undertake the day-to-day implementation of the project. The presumption in this case is that the line agency, such as a ministry of agriculture, has a scarcity of staff with the necessary experience and skills, and is so encumbered with performing day-to-day routine functions that it may not be able to adapt its working methods and procedures to the requirements of a new and innovative project. 4.61 While PMU's have been a feature of Bank-assisted projects for many years, and a valuable vehicle for implementing certain types of projects, they have also been the objects of considerable criticism, because they: a The subject has been addressed in several reports prepared by OED, e.g., *World Bank Experience with Rural Development: 1965-86; Performance Audit Report on six agricultural development projects in Nigeria, and Report No. 12016, June, 1993; Impact Evaluation Report on three agricultural development projects in Malawi, Report No. 4850, December 1983. 107 (a) have proved to be a means of circumventing lne agencies, which has resulted In a limited Interest and commitment on the part of the latter; (b) have operated as enclaves with little or no allegiance to the parent ministry, and have made no positive impact on the latter; (c) tend to preempt the best staff and resources by virtue of being funded out of the resources specific to a project, and providing staff (including locals) with higher salaries, and better living conditions; (d) provide an unsound base for continuing project activities after the reduction in donor support, thus placing long-term projects in jeopardy; (e) are often dominated by expatriate advisors who are recruited at the urging of the Bank, and who leave little or nothing behind when they leave; and (t) preclude the expected demonstration effect because of their isolation from the normal governmental machinery and its rules and procedures. 4.62 Because of the mixed experiences with PMUs, the increasing resentment within line agencies toward the privileged position of PMUs and, most importantly, the continued weakness of line agencies, the Bank has for many years been questioning the desirability of establishing PMU's outside the parent ministries (Box 4.9). PMUs that are not closely linked with line agencies or that require a considerable number of expatriate staff have also been discouraged. Increasingly, the objective Is the long-term strengthening of a line agency rather than the successful implementation of a particular project.- However, there may be exceptional situations, for instance, when the governmental machinery is too weak to undertake the implementation of a project which is urgently needed, and hence an ad hoc PMU may be unavoidable. Such exceptions can only be justified if concurrent steps are taken to enhance the capacity of the parent ministry or agency, promote its involvement in project activities, and devise a time table to transfer the activities to the Ministry after a suitable transitional period. In this situation, a Project Management Unit is seen as an essential step in a longer term ID effort. 4.63 In the broader context of ID, the Bank has assisted in the creation of new organizations whose mandate is much wider than those of PMUs in investment projects. Some of these have been successful while others have not. Successes include NTPC in India's power sector, ISKI in Turkey's (Istanbul) water supply/sewerage sector, and the Lesotho Highlands Water Authority. Failures include ISA in Colombia's power sector and the Philippines Rural Road Maintenance Project. While the decision to create a new organization (rather than use an existing one) was not in itself a cause of success or failure, it is clear that the organization's success depended on (a) the government's willingness to provide it with a clear mandate, as well as financial and managerial autonomy, and (b) a clear understanding of its future role. 2 In a recent address to the African-American Institute Conference, the Vice President for the Africa Region announced that there will be no undercutting of government responsibilities in project work by imposed project implementation units (which are heavy users of expatriate staft or resident expatriate help without the approval of the Regional Loan Committee. 108 4.64 Creating new institutions specifically to manage and/or implement project components can sometimes disrupt orderly institutional develo,ment. An example of this is seen in the CESAG Project, a newly-created management institute established in Senegal, but sponsored by the West African Economic Community to serve the training requirements of the surrounding African region. As an institutional concept, it originally had to compete with an equally plausible (and, in retrospect, more viable) alternative: expanding the existing Senegalese management training institute, which would have been enlarged to include other students in the region. As a regional institute, CESAG was plagued from the outset by divergent opinions within the host country about the merit of the concept. Both accountability and budgetary security were dimivished by the decision to create a new and competing organization not strongly tied into the host country structures. 4.65 On the other hand, the Indonesia Second Polytechnic Project's basic agreement about continuing the institutional framework for post-secondary technician training established in the predecessor project was never in doubt. Although the project was adversely affected by a change in management and a relocation of the project office, it speaks for the robustness of the institutional structure that the project withstood this setback without damage to its overall performance. Instruments 4.66 For many years, the Bank's principal Instrument for accomplishing ID objectives within an agency, and occasionally a sector, was the standard investment loan or credit. The ID objectives were limited (e.g., training, expert services, simpler data systems, computerization). While these components served a useful purpose within the context of the particular projects, there was a growing perception that the scope of ID should be broadened to address longstanding systemic problems and deficiencies (e.g., by introducing comprehensive and sophisticated management information systems to serve as an effective management tool), and that this would require a long-term association with borrowers rather than a one- time operation. The perception was reinforced by OED studies which showed a strong correlation between ID, sustainability, and overall performance. 4.67 The choice between a free-standing ID project and ID components in an investment project is clearly linked to the range of objectives to be achieved and to the number of institutions involved. In the latter case, a free-standing umbrella type operation is more suitable because of the different jurisdic- tions that need to be accommodated. Investment projects with well-defined physical and financial objectives are generally within the jurisdiction of an existing line agency that is accountable for the implementation and eventual outcome of the investment. However, broad-based ID operations that involve many agencies with divergent responsibilities need implementation arrangements that take into account the complexity of coordinating and orchestrating the variety of objectives and roles. 4.68 Conditionality may be employed in both adjustment lending (SALs, SECALs) and investment leading. However, in the former case, there may be no specific provisions for enhancing the borrower's capacity to fulfill the conditions as in investment projects. For the instrument to be fruitful, the conditions must be clearly formulated, genuinely accepted by the borrower, closely monitored, have a realistic timeframe, and carry an unmistakable understanding that noncompliance could lead to the use of remedies. Evidence from many adjustment loans suggests that conditionalities were vaguely defined or worded, and the related covenants were usually limited to actions such as "carry out studies, discuss recommendations with the Bank or implement these satisfactorily." Because of both the lack of precision in conditionality and the borrower urgency for loan withdrawal, the ID outcome was often unsatisfactory. 109 4.69 Free-standing ID loans have longer implementation periods and are thus more realistic in terms of accomplishing results. When it is determined in consultation with the borrower that ID requires a TA input, the precise methods of TA to be used should be decided in advance through a consideration of alternatives (see Box 4.3). The breakdown of TA methods used in the 1992 project cohort Is shown In Table 4.8. This table demonstrates that expatriate consultancies were about three times more numerous than local consultancies, and that long-term consultancies were more than double the short-term consultancies. The incidence of expatriate and long-term consultancies was the highest in TA (ID) operations in relation to the number of such operations. Training components (in-service, domestic, and overseas) were an intensively used instrument of ID projects in the 1992 cohort. Though Table 4.8 captures the frequency of utilization, the numbers of individuals trained were also very considerable. The investment, TA, and program and policy projects reviewed all included a substantial number of studies, but there is inadequate information on the utilization of the studies. Twinning, which is increasingly being Identified as an efficient, economical, and far more acceptable form of TA, occurred in only two of the 1992 cohort's projects. Similarly, the participation of NGOs was limited.0 Given the many advantages of twinning and of NOOs as a low-cost means of service delivery and enhancing beneficiary participation, particularly in small and widely dispersed rural development projects, their limited use within the 1992 cohort merits attention. Table 4.8: ID Methods Used in 1992 Cohort ID Methods Investment Projects Technical Assistance Program and Policy (226) (17) (34) Local Consultants 52 2 2 Expatriate Consultants 150 16 6 Long Term Consultants 140 13 6 Short Term Consultants 64 5 3 In-Service Consultants 80 5 1 Domestic Training 74 8 1 In-Service Training 80 5 1 Outside Training 82 2 0 Studies 132 13 9 Twinning 2 0 0 NCO Participation 6 0 0 4.70 Significant conclusions cannot be drawn from a review of TA methods used in the 1992 cohort, particularly in the absence of comparative figures for 1989-91. But the continued use of a large number of resident expatriates is a source of increasing concern within the Bank for several reasons. In the first place, this indicates that only a limited amount of progress has been achieved in increasing the number of local consultants or enhancing the capacity of government agencies to become more self- contained with regard to skills. There is also a growing dissatisfaction with the scale of emoluments and a Only six cohort projects were identified as having direct, significant NOO participation. Minor or peripheralINGO involvement in suboomponents, the proffering of advice by international NGOs, beneficiary groups, and others such as churches providing de!!very services, are not included. 110 other perquisites that expatriates enjoy, and with the tendency to extend expatriate contracts repeatedly. Information In PCRs regarding the design of training programs, the selection of training facilities within the countries or abroad, choice of trainers, selection criteria for trainees, and the deployment of trainees on completion of training are too limited to warrant firm conclusions about the effectiveness of training components. Box 4.10, based on a review of the Bank's role in human resource development in Sub- Saharan Africa, presents a number of proposals for enhancing the effectiveness of TA for ID. The proposals have significant relevance for other regions in the Bank. .o ,90 Prac Mnaann and startse Mdany ptqjects tha aceMly achieve their phyuial targets have been eriticized in evaluation port f. . hi.g .lh i.pact .n instituton building .4 N tesiltioaldevelopment h~a s affered most when autonomous or semi-autonomious proJect babemet Uiti&hae substituted for line agencies instead of strengthening themu. Becauue of the pressure geeatdb ihipeetto calenidars, these units often had a hot-er otlo and di no conriut la th r4uerets ifefrt of izntutional development. Thleir advantage is that linstitutionially weak eanlopdentsedhe c~an prvide substantive implementation capaility, especi.ally where there is a substantial 1gam R itagrctpecnistruction, is hzas often been the case. However, autonomoua project Maisgemafiifitkhaeyoprovn vry ffective in interagency coordinationi%eino cotibuting line agencies ftby* hi w fprogrOa.saimplotunt Also, inproso ugtr rsi,atnmu nt ui ee h irt uffer cuts. Yet for ack of inistituiona deabliy,patiulrli Affica, enclave-type bit dy liite ben jdged mor cessful than othrai terms of Bankc iteria, weighted as they are in ofhottemproduction incessand pia completion; Pat ilit latnin.ami it .hs profect units (a anlzdi ueosBn eot)i evd sa*suind a for continuing prjctatvities afe the 1tedutin a dr suppoxt. This tig cptject benefits in jeopardy. A partia satto is 1avidspcil xe.tngunt by deutt pjtsopnntin difemont ministrie thghtewresaatpoec,bu ~coorhidednt, located either ins asectoral meinistry or, somietie better, in 4 nonsector mnsr 10 PhionitalcGohedent1 Of5ce This apec,howevot4 also hans flaws, The oordnaiting nk.smy gi. tproi mowpiorty individual sector ministries may have a greater interest.i syet b teyti uly espnsble ad ooriatio my otl eeffecie inclnung tha among donors d1'iffol s comxponents~ Thai teins inaJthe dod duif6ecause ofu Th ct an rdac of fant aupported.I la th An>of 4dequzatn loa stffteouce ad it poorly performng training comnponzents, 196an. ofthapcteade to dier rahe thani create adhditional human resurce. During project ins metaio. pcia plaii f tafg of Bank projets a have substituted for addressing more fully *h haaetlcue of hua eorecntsns Tackling the human resource constraint problem moirithivot, atistdy n Afica reminsa challenge. Thelitii aii5rpetive vU thin is provided by policy-based leading could be beneficial in .sa *tebmrource.esof Bank paojects in the wider context of conty constraints and priorities. O4D, oild Banik P,apoileuce with 14tral Dl oment: 1954, Av' irl 88. . ~ . ..:........:.:..:.:.:.:..:.:...::<:.::. .... ... . ..... ..... ..... Ill x 4.10: Improving the Effectiveness oflDTA T¶A for trainfog and lastitution bouiding has absorbed large am ouofforg esistance, espoirally la Sub- Sabaren Afdca~, without a comasrtehprovement in domiestic aaiy h rpoast eeyti situation faltinto arve eategories: * Improve the processes and procedures by which TA is delivered and menaged. The approach has been to devote more effort to the design of training programs, specifyiug objectives in terms of improved sustainability of local capacity, involving the borrower in design, intensifying supervision, and carefully monitoring and evaluating results. * Use different approaches to transfer knowledge and build capacity, maidag greater use of short-term advisors, local consultants and researchers, and twinning anaugements. * Improve working environments. The short-run response is to shore up implementing agencies or to establish new agencies provided with expatriate staff; salary supplements, and euqipment. The longer run response is to promote civil service and anistetive reform. Remove price distortions in the markets for higher-level skills, which encourage expatriates to be hired in place of local personnel, who in turn are induced to seek enployment.abroad. Remedial suggestions propose the revision of contracting guidelines to incorporate vout . considerations, making the full cost implications transparent, and requiring recipient agencies to pay for consulting services (to the government or to a fund) even when they are received on a grant basis. * imit the need for TA by reducing or consolidating government operations, simplifying projects, and using less skill-intensive technologies. Ther is some evidence tha the poposals in the.first and second :ategor"s at bding implemened but tlw. gap between tae and practice remains wide. Evaluatoan of the third category responses has hot toposted anc success to date, but perseverance is needed. The problems salsed by de footti category have been largely neglected, both In analysis and practice. Efforts to improve procedires and delivery modes will have warginal impact until the market incentives are reversed. More attention needs to be paid to the molution of thk constraint. The fal category concerns influencing the factors determining the demand for TA; such as growlk in the sagnitd and complexity of government functions, and increpsed foreign assistene. Theie iecomidation have not"been fully explored, but a recent OED studyt sugges that the proposals- haf substantial potential for enhancing the effectiveness of ID. : 'Th World Danks Role in Human Resource Development in SubSaharan Afhica," Report No. 12144, June 30, 1993. 4.71 The 1992 cohort does not provide much information on the role of management development and training systems in re-orienting, reforming, and upgrading the civil service systems. Unlike the training components in investment components, which typically address training needs of a specific nature within a single entity, the training of civil service personnel, particularly at the senior levels, is critical for enabling governments to play a major leadership and strategic management role. Whereas integrity, dedication, competence, and incorruptibility are among the main virtues identified in with a well- performing civil service, inadequate attention is generally paid to the management effectiveness or the 112 Innovative capacity of the civil service. Hence, management effectiveness, which is regarded as the hallmark of private enterprise, Is not regarded as a crucial ingredient In day-to-day civil service administration. As long as this aspect Is not accorded the importance it deserves, the civil service will be identified with routine, stereotyped, and humdrum activities like tax collection, law and order, and routine maintenance services rather than a dynamic, leadership role (e.g., enlarging the scope and coverage of education and health services on a cost-sharing basis with the beneficiaries, and promoting greater community participation, through a bottoms up approach, in the design and Implementation of projects). This aspect deserves to be closely integrated into PSM projects. F. CONCLUSIONS 4.72 While commendable progress has been made in operations directed toward Improving PSM, the Bank has had less success in addressing some of the culturally Important determinants of borrowing governments' performance, including social policy assessments, attitudes to change, and methods of consensus-building. Though the Bank's economic work is of high quality, its institutional impact has been constrained by a weak "capacity building* focus and by an excessive emphasis on reports instead of field work. Reluctance to tackle institutional development is caused, in part, by a lack of access into areas of the borrowers' domain that are considered privileged and that raise governance issues over which the Bank has limited or no jurisdiction. Conversely, there is evidence that the intellectual underpinning of ID work has been neglected, that ID skills have not been urtured, and that the Bank has often failed to involve borrowers in identifying domestic capacity needs and institutional development priorities at an early stage of project conceptualization. 4.73 There is an acceptance that ID is a slow process and that it is incumbent on the Bank to become involved with borrowers on the broader ID issues. However, valuable lessons from the experi- ences of completed and ongoing operations can be used to reduce uncertainties and improve performance. Among these are: * paying more attention to ID issues and institutional analysis in economic and sector work, and recognizing that the strengthening of the institutional foundation of development is a key strategic element In the long-term viability of projects and programs; * ensuring the fullest possible borrower involvement at all stages of processing and, in particular, promoting an active role for the borrower in upstream diagnostic work; allowing the borrower's consensus building mechanisms to produce understanding and popular endorsement rather than pressing ahead prematurely to suit the Bank's schedules; * eschewing the overdesigning of projects, recognizing the borrower's capacity limitations, and appreciating socio-political constraints, however obscure, that may be present; * recognizing that the borrower's unequivocal commitment is the key to sustainability and substantial ID improvement, and ensuring that the borrower's role, responsibility, and accountability for the final outcome is an integral part of project design; * appreciating that flexibility to adjust or redesign ID components after project approval and during project implementation can be an important ingredient of success, and that the participation of borrowers and especially beneficiaries in identifying problems and proposing corrective measures is critical for the attainment of ID objectives; * introducing performance indicators in ID projects and for ID components in investment projects; 113 * refralaing from unnecessary interference with knplmnaIn and from creating any perception in the mind of the borrower that the Bank is Ia any way *tunnlng* the project; being highly selective in advocating the use of long-term expatriates and active In promoting the involvement and skills upgrading of local consultants; * Including appropriate skills on supervision missions for project ID components, especially in complex and multifaceted projects; * devoting particular attention to consultation with beneficiaries and involving local people In the design of training programs to the extent possible; 115 LIS OFANEES Chapter One Anne Figure 1,1 Interrelationship of Project Sustainability, Institutional Development and Performance Ratings, 1989-92 Evaluation Cohorts: Number of Projects Figure 1.2 Interrelationship of Project Sustainability, Institutional Development and Performance Ratings, 1989-92 Evaluation Cohorts: Satisfactory Ratings Table 1.1 Average Re-Estimated ERRs of Evaluated Operations, by Sector and Region Table 1.2 Ratings Disconnects on Project Yield and Outcome, 1992 and 1974-91 Cohorts Table 1.3 1992, 1991, 1990 and 1974-92 Cohorts: Commitments by Sector and Region Table 1.4 1992 Cohort: Distribution of Evaluated Operations by Years of Approval and Closing Table 1.5 Performance of World Bank Commitments, by Sector and Region Table 1.6 1992 Cohort: Project Performance Ratings, by Lending Commitment Lev.'s Table 1.7 Outstanding Projects from the 1992 Evaluation Cohort Table 1.8 1992 and Combined 1989-91 Cohorts: Achievement of Institutional Objectives in Evaluated Operations, by Sector and Region Table 1.9 1992 and Combined 1989-91 Cohorts: Sustainability Ratings in Evaluated Operations, by Sector and Region Table 1.10 Interrelationship of Sustainability and Iastitutional Development Ratings, 1989-92 Evaluation Cohorts Table 1.11 Interrelationship of Sustainability and Institutional Development Ratings, 1989-92 Evaluation Cohorts Table 1.12 The Likelihood of Satisfactory Project Outcome, Given Favorable or Unfavorable Ratings on Project Sustainability and Achievement of Institutional Objectives, 1989-92 Evaluation Cohorts Table 1.13 1992 Cohort: Achievement of Physical Investment Objectives in Evaluated Operations, by Sector and Region Table 1.14 1992, 1989-91, and 1974-91 Cohorts: Cost and Implementation Time Variations in Evaluated Operations, by Sector and Region Chapter Two Annex Table 2.1 Performance of Operations Evaluated, by Approval Year (1968-84) Table 2.2 Performance of Operations Evaluated, by Approval Year, Weighted by Commitment Amount (1968-84) Table 2.3 Appraised and Re-Estimated Economic Rates of Return on Operations, Evaluated by Year of Approval (1968-84) Table 2.4 Sector and Region Performance by Time Periods Table 2.5 Country Performance by Time Periods Table 2.6 Country Portfolio Ratings, Overall Project Performance Investment & Adjustment Operations, Approval Years 1990-88 Table 2.7 Portfolio Performance and Country Factors, 1980s Table 2.8 Project Outcome and Project Processing Table 2.9 Prqject Outcome and Incremental Contributions of Project Processing Table 2.10 Principal Reasons for Major Cancellations of Bank Loans Table 2.11 Comparison of ARPP and OED Evaluation Ratings 1975-92 Table 2.12 A Comparison of OED and ARPP Overall Ratings in their Final Year by Region and Sector 116 Chapter Three Annex Figure 3.1 Intermational Rice Prices, Actuals and Bank Forecasts Figure 3.2 Project Processing and Evaluation Ratings Table 3.1 BankIDA LAnding by Total, Agriculture, Structural Adjustment, 19-92 Table 3.2 Comparison of Agricultural Performance and Sustainability Ratings by Region, I970-90 Table 3.3 Satisfactory Performance Ratings by Annual Review Cohort Table 3.4 Percent of Agricultural Loans and Credits Evaluated by OED by Calendar Year of Approval Table 3.5 Ratings by Approval Year and Region Table 3.6 Ratings by Approval Year and Subsector Table 3.1 Agricultural Lending Performance by Regions and Subsectors, 1970-85 Table 3.8 Agricultural Projects, Percent Satisfactory and Number Rated, by Country and Time Period Table 3.9 Project Evaluation Ratings at PCR Reviews Compared with Evaluation Ratings after Audit, by Sector Table 3.10 Agricultural Projects, 1970-88 Evaluation Ratings by Lag Between Completion of PCR and Closing Date Chapter Four Annex Institutional Development in the Energy and Infrastructure Sectors Annex Five: Statistical Annex Table 5.1 Current Regional Classifications of Countries Represented in 1992 Cobort Table 5.2 1992, 1991, 1990 and 1974-92 Cohorts, Operations Evaluated, by Sector and Region Table 5.3 Performance of All Evaluated Operations by Year of Approval, Sector and Region Table 5.4 Distribution of All Evaluated Operations, by Years of Approval and Evaluation, 1974-92 Table 5.5 Distribution of Operations Evaluated, by Sector, Region (1988-92) Table 5.6 1992 Cohort: Performance of Operations, by Sector and Regions Table 5.7 1974-92 Cohort: Performance of Operations, by Sector and Region Table 5.8 Appraised and Re-Estimated Economic Rates of Return on Operations Evaluated, by Year of Evaluation Table 5.9 Gaps in the Economic Rate of Return, 1989-92 Cohorts, by Magnitude Table 5.10 1992 Cohort: Implementation-Time Variations in Evaluated Operations, by Sector and Region Table 5.11 1992 Cohort: Implementation-Cost Variations in Evaluated Operations, by Sector and Region Table 5.12 Average Project Indicators by Year of Approval Table 5.13 Average Project Indicators by Year of Evaluation Table 5.14 Average Project Indicators by Country, 1974-92 Cohorts Anne Siz: 1992 Cohort Table 6.1 Operaticl Evaluated in 1992, by Sector and Country 117 Annex Figure 1.1: Interrelationship of Project Sastainability, Institutional Development and Performance Ratings, 1989-92 Evaluation Cohorts %Substantial 30 /1/ 20 IG Partial Institutional Development Unaaiegligible Sustainability Unlikely 118 Annex Figure 1.2: Interrelationship of Project Sustainability, Institutional Development and Performance Ratings, 1989-92 Evaluation Cohorts Negligible Unliely Uncertain Partial Likely ~Subt ata Percent Satisfactory 0 90 100 nstitional Development M SustaiVAbility 119 Anne Table 1.1: Average R-estmated ERR ofEvaluated Operatons, by Sector and Region Operatlos Evaliated la Re-est Uated ER 1992 1989-91 1974-1 1992 1989-91 197441 Aricultm, 40 15 198 21 575 21 13 11 13 Pow 12 36 48 21 179 21 9 12 11 Thapot 13 7 79 16 393 26 22 19 Indusry 8 15 27 19 84 16 10 11 F'mance -- -- •• • • Programa&Poseoy - -. • TechnicalAsismance - - -l -l - -l - - Urbmn 4 22 26 4 53 2 17 16 18 Teleconumications 2 2 20 6 63 5 25 16 17 Tomism - - 5 1 17 1 - 10 12 Energy 4 3 16 11 31 6 38 12 26 Water&Sanitation 4 1 23 2 74 2 7 9 7 Pollution Contr& Sol W. - - - -1 0 .- - -5 Africa 30 9 122 8 449 9 12 13 14 East AsiaandPacific 21 54 93 24 272 18 16 15 17 SouthAsia 10 13 71 18 168 15 2 15 19 EuropemndCentralAsia 7 8 30 13 129 19 I5 13 13 MiddleEastandNormAftica 9 3 52 10 153 9 22 17 14 LatialAmericaandCaribbean 10 14 74 27 299 29 4 10 13 Total 87 100 442 100 1470 100 13 13 15 /a EuMusmaionserwMeh dataueaaaNe andOPSrJU truMhmIuiappredau. Sar anele,utInanyIipipIidatrflObU .rgmat.io emebylmaaumasseam ma~1.- uI-aau.amd bypassmandaliyhadns 12© � •а а►$; мо� � •�м0@р�..��оО�.�л+�` vмt� .�.�.�iiZ�i.N�+.. о � � � � U о '�' � �� . , х�� � • �м����м � ������� о -� r" �� �� � � а�ооо••о.rооинг уми;� � aNм.•�.rи;g � � � � � � Q� � � �$i� � � ��O�v�i�� � $����1 � � '�..оацо$ooe�..r►•»�м � �R:::.►й: � .. � � ii+ ОΡ G t� •�r � вГ�0о0оо.и..ооN0��?�'�р � •1�iNr" ���СО •Sбгi�1 vY � � � О л �� � . � . � . . � ' �О�ь� � ����ь�й � � � . R� � а.-�•уоооо.•оооо�-ооа � оо•»мооv Я .. .✓ е ` й� t . � . � • • � • ���� ��������3о .� W � аноооооиооооооме• � �нн•-•�*•-ro� .. � � � � н �• � � � � � • � �� �са� � ��� � нFнн�3�» ао������- � � 121 Annex Table 1-3: 19929 1991s 1990 and 1974-1991 Cohorts: Commitments by Sector and Region conunitakents M 1"1 1990 1974-1M t WMIIIJ ty (USIMIly Ow a="!u tv dMI/L tw) Sector Agriculture 4,437 25 4,236 so 5,674 27 28X3 27 POW 1,947 11 1.298 9 3,092 15 13,168 12 Thnsport 1,718 10 1,882 13 1,411 7 15.990 Is Hunan Resource 706 4 907 6 944 5 5,537 5 IU&I*Y 1,410 8 784 6 957 4 6,300 6 Finaum 1,709 10 1,365 to 2,413 12 91659 9 pfo9mm & Policy 2,855 16 1,055 8 3,409 16 12,658 12 TednicalAssistance 85 0 78 1 121 1 388 0 Ud= 1,299 7 680 5 725 3 3,020 3 Telcommunicatio-M 99 1 145 1 96 0 2,495 2 Tourism - 60 0 147 1 5" 1 EaerV 727 4 1,106 8 917 4 4,053 4 Water & Sanitation 645 4 420 3 1,004 5 4,259 4 PbIlution Contrl & Sol W. 0 - 131 0 Region Aftica 4,178 24 2,335 17 3,202 15 14,69S 14 East Asia and Pacific 4,610 26 2,632 19 4.415 21 23,569 22 SouthAsia 2,004 11 1,999 14 3,843 18 19,157 18 EmWe and Cc" Asia 11095 6 2,016 14 1,193 6 13,134 12 Ididdle, East and Nordt Afi= 894 5 1,182 8 1,449 7 9,558 9 IjAm America and CariMbean 4,843 27 3,851 27 6,707 32 26,462 25 Total 17,624 100 14,014 100 20,808 100 106,57S 100 Notr Sub&otds=yaA=wwtobhdwt*mwft=w. 122 Annex Table 1.4: 1992 Cohort: Distribution of Evakuas4 Operations by Years of Approval and Closing UvulastIon NuSMer Nuaberla Year Approved Closed 1975 3 1976 2 1977 2 1978 8 1979 10 1980 19 1981 20 1982 47 2 1983 49 3 1984 32 3 1985 35 7 1986 19 11 1987 17 18 1988 8 30 1989 5 46 1990 1 7S 1991 - 75 1992 7 Total 277 277 * Nopmdomf I.1thansoindassagmstrec~u.lyaerssaldishissacrtMomea~t. 123 Annex Table 1.5: Performance of World Bank Co=mmitments, by Sector and Region Operamlns Judged to bo Saisfactory 1992 1991 1990 1974.91 Comitments Commåtmento Commitmentu Commitments 9~ml (M m$dt % (USmL) (M (UStaD) (M Sector Agricultue 3,127.5 72 2,025.2 49 2,369.4 42 16,836.1 60 Powr 1,353.8 70 807.0 62 2,152.1 70 10,715.0 82 Transport 1,515.3 88 1,275.9 68 1,174.5 83 12,990.1 81 IupanResoe S8.1 83 588.7 65 703.8 75 4,185.6 77 Industry 1,197.5 85 537.9 69 262.3 31 4,539.8 73 Finance 835.8 49 1,203.9 88 1,491.6 62 7,657.7 79 Program &Policy 2,216.4 78 879.4 83 2,026.4 59 9,756.2 77 Te~mialMi~ama 49.6 61 51.2 66 68.5 56 258.0 67 Urban 1,250.6 97 506.0 74 681.1 94 2,495.3 83 Telecommuoicatio 98.0 100 38.5 26 88.2 92 2,314.7 93 Touri m 0.0 0 59.6 100 132.5 90 413.7 76 Energy 546.2 75 933.1 84 472.0 51 3,393.5 84 Water& Sanitation 320.5 50 264.5 63 402.5 40 2,877.6 68 Pollution Cantrl& Sol W. 0.0 0 0.0 0 0.0 0 58.0 79 Regin Affica 2,490.9- 60 1,221.8 52 1,325.6 41 8,948.7 61 East Aia andPacific 4,340.4 94 2,162.8 82 3,524.3 80 19,913.7 84 SouthAsia 1,178.3 62 1,688.1 87 2,296.8 60 15,165.4 79 Europe and CentraAsia 903.4 83 1,185.9 59 527.3 44 10,318.9 79 ~idd eEast andNorthAfica 893.9 100 651.7 55 1,228.5 85 7,495.2 80 Latin America and Caribbean 3,291.4 68 2,260.6 59 3,122.4 47 16,649.3 63 Total 13,098.3 75% 9,170.9 66% 12,024.9 58% 78,491.1 74% Sowee: OED Databas. 124 Annex Table 1.6: 1992 Cohort: Project Performance Ratings, by Lending Commitment Levels Total Loan/Credit Number of Percent Rated Commitment Projects Satisfactory (USS Mllions) (No.) (%) Sto<10 44 61 10 to 25 72 65 25 to: 56.05 Sm of 200.15 Erob > F: 0.00 Regression Variance: 2529.73 R2 - 0.79 Corr(Y,Y): 0.80 WH1TE-Corrected Stndard Errors Independent Parameter Standard Error of t-Statisti Prob > 1t Variable Estimate Estimate L: 0-0 W 54.96 4.26 12.89 0.00 ODIP2 3.19 0.72 4.42 0.00 INF2 -0.11 0.02 -5.48 0.00 HD12 19.32 8.48 2.28 0.03 143 Annex Table 2.8: Project Outcome and Project Processing 1991 and 1992 Evaluation Cohorts Naber of PCRPARRLatis Percent Identification Preparation Appraisat Supervston Prolects Satisfactory Unsatisfactory Satisfactor Adequate Adequate Adequate Adequate 277 225 50 1.8 Adequate Adequate Adequate Deficieut 49 36 13 73.5 Adequate Adequate Deticient Adequate 25 17 8 68.0 Adequate Adequate Deficient Deficient 12 2 9 18.2 Adequate Deficient Adequate Adequate 11 10 1 90.9 Adequate Deficient Adequate Deficient 11 7 4 63.6 Adequate Deficient Deficient Adequate 57 25 32 43.9 Adequate Deficient Deficient Deficient 34 9 25 26.5 Deficient Adequate Adequate Adequate 1 0 1 0.0 Deficient Adequate Adequate Deficient 0 0 0 Deficient Adequate Deficient Adequate 1 0 1 0.0 Deficient Adequate Deficient Deficient 0 0 0 Deficient Deficient Adequate Adequate 0 0 0 Deficient Deficient Adequate Deficient 0 0 0 * Deficient Deficient Deficient Adequate 36 14 22 38.9 Deficient Deficient Deficient Deficient 34 7 27 20.6 Deficient Not Available Not Available Not Available I 1 0 100.0 Not Available Adequate Deficient Deficient 1 0 1 0.0 Not Available Adequate Not Available Not Available 1 1 0 100.0 Not Available Not Available Not Available Not Available 4 1 1 50.0 55 SS 195 64.5 larMspojectedid #A MoielFWPARtRatiV 144 Annex Table 2.9: Project Outcome and Incremental Contributions of Project Processing Project Cycle Process Percentage Qf lbae oits the Percentage Rated Of thm sica Incremental Rated &.0g$Ae: Deficient (%) trting are Contribution to the Adequate Sat. (%) Unsat. (%) Sat. (%) Unst. (%) Average Proje's (%) Probability of Being Satisfactory Identification 86.8 70.0 30.0 13.2 29.2 70.8 12.42 Preparation 66.4 77.3 22.7 33.6 39.3 60.7 6.49 Appraisal 63.7 80.1 19.9 36.3 37.4 62.6 38.27 Supervision 74.5 71.7 28.3 25.S 43.9 56.1 20.60 Probit Model of Bank Processer PROBIT Confidence T- statistic Incremental Coefficient Level Contribution to the (2.Tail)b Average Project's Probability of Being Satisfactory Intercept 0.83 100.0 4.71 Identification 0.28 84.7 1.43 12.42 Preparation 0.15 55.9 0.77 6.49 Appraisal 0.88 100.0 4.75 38.27 Supervision 0.47 99.9 3.45 20.60 The preceding analysis is based on 54S OED pwojects which have PCRIPAR tatings for all four project processes. SAT/UNSAT (overall rating) and ADEQUATE/DEFICIENT (ratings of project processes) have been converted to 110 dummy variables to carry out this estimation. The PROBIT method of atimation was used beausethe dependentvatiable (overall rating) takes on of two discree values, 1 (SAT) or 0 (UNSAT). The marginal contribution of the Jth variable to the prqoject being satisfactory is: L *(X'if) - Oy OE%AdequateA (summing across all variables, including the constant) where the probability distribution of the model is 8x; represented by P - F(a + A ) where P is a cumulative probability density function, and * e xnr(-S%Adeavate-SP/21 (2r) * Where Pi is the probability being satisfactory, a is the constant, and A is the coefficient of the ith process variable. 6 A project process with a confidence level of 95% (or greater) is signifioant in the determination of the overall rating. 145 Annex Table 2,10: Principal Reasons for Major Cancellations of Bank Loans- All Loan with Large Can~ ana 80-100% 70-80% 60-70% 5060% Total With SAT Wh UNSAT ~Ratig Rating hmpløentation problem 45 11 24 39 119 41 78 % - 62.5 52.4 64.9 63.9 62.3 34.5 65.5 Advero Poliy Environmnt 14 4 16 17 I 21 30 ________________________ 19.4 19.0 43.2 27.9 26.7 41.1 53.8 Reductions in pro~t scope 12 5 13 19 49 21 28 ____________________ 16.7 23.8 35.1 31.1 25.6 42.9 57.1 Shortage/day of coun~rpat Ad 12 4 13 17 46 14 32 % 16.7 19.0 35.1 27.9 24.1 30.4 69.6 Lak offChangcd Government Commitment 21 5 4 9 39 11 28 % 29.2 23.8 10.8 14.8 20.4 28.2 71.8 Faulty Cost Etimat s 4 0 3 13 20 11 9 % 5.6 0 8.1 21.3 10.4 55.0 45.0 Financing plan ~hanged due to avaiability of cheaper 10 2 3 3 18 15 3 funds % 13.9 9.5 8.1 4.9 9.4 83.3 16.7 Non-compliane with covenant; Suspension of 7 1 5 4 17 4 13 disbursnments % 9.7 4.8 13.5 6.6 8.9 23.5 76.5 Extealshoks 2 1 4 5 12 6 6 % 2.8 4.8 10.8 8.2 6.3 50.0 50.0 aProjeat refinaced for administative reas~ns 8 2 0 0 10 9 1 % 11.1 9.5 0 0 5.2 90.0 10.0 Cost Savingt i 1 1 3 6 5 1 % 1.4 4.8 2.7- 49 31 83.3 16.7 Cosa overruns 2 0 0 2 4 3 1 % 2.8 0 0 1.6 2.1 75.0 25.0 Tota exced the nubr of caoefm d projects due to the fact that more than one reason was cited for cancellation in a ~age nuber of cases. 146 Annex Table 2.11: Comparison of ARPP and OED Evaluation Rat~ngs, 1975-1992 AmP OED Tear % Prob. Prols. % Unsat. 197 17 12 19m 11 14 197 8 10 am 9 1979 10 12 1980 9 11 1981 12 15 1982 12 20 198 13 15 1984 11 26 1985 11 30 1986 15 18 1987 14 28 1988 12 26 1989 13 30 1990 17 36 1991 20 37 1992 18 33 147 Annex Table 2.12: Å Comparison of OED and ARPP Overall Ratings in their Final Year Perent for Regon Naber OE FMa~ PPOE D~vr- Regon Us. Poect gap gEncM Afica 291 33.3 19.2 14.1 25.1 EastAsiaPaciac 187 18.2 8.6 9.6 15.0 Europe, Centra1Asia 67 26.9 19.4 7.5 25.4 LatinAmerica,Caribbean 164 41.5 18.9 22.6 27.4 Middle Eat, North Aica 94 17.0 8.5 8.5 12.8 South Asia 123 28.5 18.7 9.8 19.5 Total 926 28.9 15.9 13.1 21.5 Pre~t for Sector Nu e rD Nobe PPO D~wr. Sectr PM~lUn. P s Gap ---Jr-~ Agriculture 288 40.6 24.0 16.7 28.5 Eneg 82 13.4 2.4 11.0 13.4 Finance 78 26.9 21.8 5.1 17.9 HumanResoumce 98 24.5 8.2 16.3 20.4 Tuanetry 44 38.6 22.7 15.9 25.0 Pollution Cntrl& sol W. 0 - - Power 63 22.2 12.7 9.5 15.9 Program &Policy 2 50.0 0.0 50.0 50.0 Technical Assistan e 36 38.9 11.1 27.8 38.9 Tel~nmmniention 21 19.0 9.5 9.5 9.5 To~uism 1 0.0 0.0 0.0 0.0 Tuansprt 114 17.5 12.3 5.3 17.5 Urban 46 17.4 17.4 0.0 4.3 Water&Sunitation 53 32.1 9.4 22.6 22.6 Total 926 28.9 15.9 13.1 21.5 148 Annex Figure 3.1: International Rice Prices, Actuals and Bank Forecasts 350 300 -Actals 250 Forecasts 150- 50 -_- 0 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 2000 2002 2004 Year Source: OED, draft, A Review of World Bank Irrigaton, 1993. Annex Figure 3.2: Project Processing and Evaluation Ratings 31% Sat Deficient 13 37%Sat Adequate 219 toaNo. of Projet- IDENTIFICATION PREPARATION APPRAISAL SUPERVISION 177 28% Sat Deficient A uAdequa DeficientD 48 100% sat DcnDeficient Adequat 2 6 75% Sat Adequate SAdequate 4 145 0% Sat Deficient Deficient2 11 67% at Adequate Adequate 9autnu 64% Sat Adequate 14 so 78% Sat Source: OED Annual Review Database, PIP Ratings from 1991-92 Agricultural Evaluation Cohorts Ad ut 150 Annex Table 3.1: Bank/IDA Lending by Total, Agriculture, Structural Adjustment, 1967-92 Year of Adjustn.t Agricultural Total Agriculture Shar Approval LeU ndin Undfngt Undlna of Total I*nding (Ussai. (Ussa.) (Usmai.) 1967-73 1,349 9,135 47,250 19.3 1974-76 1,260 9,389 34,327 27.4 1977 265 4,335 11,669 37.1 1978 235 5,768 15,791 36.5 1979 152 4,477 15,860 28.2 1980 875 5,197 19,369 26.8 1981 969 5,052 19,079 26.5 1982 982 4,296 18,240 23.6 1983 3,406 6,111 20,937 29.2 1984 1,284 3,625 16,769 21.6 1985 2,116 5,129 20,828 24.6 1986 4,458 6,036 19,887 30.4 1987 4,560 3.619 21,505 16.8 1988 4.937 4,113 20,576 20.0 1989 5,861 3,562 23,203 15.4 1990 4,597 3.722 21,676 17.2 1991 6,971 4,326 23.330 18.5 1992 4.071 2,841 20,182 14.1 Total 48,345 90.734 390,478 23.2 Notes Agiouturat tending icudus agioukum adjutaent hending. A ~jua t~ending dom not knlude dobt rmductio Inding. c.h.duYDarofAprval, 1990 USS. Somae: Wld BankFianialnatabuun,Doo~tb.31. 1992. Annex Table 3.2: Comparison of Agricultural Performance and Sustninabiity Ratings by Regon (CY of Approval, 1970-90) Nnbe of P«Cen Perfonnce Rang R =sT Performce Rag UN p^# R~ &~o P u~M~O R~ oe mnb. ef ct Maber offö $"~ SusataaMbwy Sotaab UnsuLaable ToMl UT SutaIn~bI Un~utnMe Total UNSAT APR 120 38 39 18 57 6 57 63 PAP 84 52 41 12 53 3 28 31 ECA 28 61 14 6 20 3 5 8 LCN 49 42 13 3 16 7 26 33 MNA 28 68 ta 2 20 1 7 8 SAS 65 51 33 13 46 0 19 19 ALL 374 48 158(15%) 54<05%) 212 100%) 2012%) 142<88%) 162 «00%) Sor: 0ED Dat.~um 152 Amex Table 3.3: Satisfactory Performance Ratings by Anmual Review Cohort Year of Agiculturc Non-Agrculture AB Seton Evaluatio ffo.) (% Sat) o.) %§a (No.) (%Sag 1974 6 83 44 86 50 86 1975 8 63 49 92 57 88 1976 21 67 49 94 70 86 1977 17 76 92 92 109 90 1978 27 78 70 96 97 91 1979 49 80 81 93 130 88 1980 32 84 55 91 87 89 1981 40 80 68 88 108 85 1982 54 72 73 86 127 80 1983 43 70 135 90 178 85 1984 56 63 114 80 170 74 1985 55 67 133 71 188 70 1986 54 76 193 83 247 82 1987 66 61 119 79 185 72 1988 62 60 105 82 167 74 1989 82 56 176 77 258 70 1990 114 52 245 69 359 64 1991 94 53 182 69 276 63 1992 86 66 188 67 274 67 Overall 966 65 2171 80 3137 75 S~mo: ~ED Annua Rrvi~w Databas . 1 i 1S3 �i о в ,� � �� � - � � g � ��j � �`��������.���.����� к� � „°1�"� � .. �i � гГ вл en �► и � и ♦ м �в ..� « н � S� � Q а � � �ч�г�дан�� �� и � � д ��� ��. � � � �� � .. .• � м м м� w� �ri й' .г � м й� a�i �� м+F � So й � � 'RKK�l�`НН '4 �'tAa�RH`�i�':Q�" ~И � � � � й � � й � 8ь � � s�i � � а� � м iS i3 °� :З р � � ; � в � � � ������� ���.����а�����й �� � ������� �� � � .� �Q�K��4a "'� AStS:�a8 ",°; a�s � � � � � � � •+�. �. � � � °� А! � � а й' v й м; � � � �, '8. 8 � �Q � � О о в � � � �i6й�йй���Siй�йSогО���й�е3.4 �� � о� � � � � � � '� iA д! .� И F�б l� 00 О� F А��О И� и w е! 'r О•r С? О �� � �в =й у►� � А ,� � � .Gt 3 � � � �s�.��,. в• � �'�е'��ФR%ГoE►SB'S'ь�аr't��СiЕЗ�й�'r'i`й�S�iй �..: м, � � � ем � � � �о� �' � � �о Е+ R����1e��f�f���3��3����3��Жa��� �' • А о � � � � � � . � iii+++ � т� � �� �' ������ �л �� � � � �� � � �� �� � $�� � � �� � $ р$ у � �° � �е � � �� у� ��z��я�����$ �� � ? � � �w 8�и�► a�eo��r 3 � � � � о� '�' 8 _ �, °3 � 1е Х'� Ни $и °•iи �tл �!л EaS � °.3.'"о $и 8и '3.t °.iw 8и ё:й � ag � b 3�йi $а $`л So 8w 8aw $iй � а �q � + °.i $ .ai •е � о. �8 ае � и а й �°w !� Т к $Ъ'д й.� йа аёи 8.► йа Eil� � � � й.'�. ei о � ее й W °.i w Е4 а $:'.!' � � �й �� 8w So $8е� $а l�Qi � "' Li'о йН $и �'W $i..1 йq i�i� в7 О� �D А $ Ч О w й.1 О А 8 в � йi �1 � J а3 V/ Yi СО вrw.i г.� О�в �• � й о й.� $ А о О i.�i А .1 Ч •� � Eie ао�О +а)А i�fe Ои йЧ РМ � iJ и V ьf О и iT �О О и О'о Ч ii � •в W О О О О i� ♦ О е t� А О и � .� � $ r $ r � М л и $ А �ii .i $ b О�+ � и О r Ф л �• $ 1в � Ф W -е О О+ О О е л $ r О вв � , �+ ngr b�,а 8 и о е о о а а � � л р i{ $.. е е е а 8.. е о е л е р � • • й � � b V �О й � в� 1� �i. $ $ � � ��8 р� н8 �� �s �8 �� �ех я� �ra в� �g �r . � о е .. � е е а ra о е д а о о е е о а е е о о о s �. � .. 44 оо ev� ео оо ое ео ео ео �� ое оо ое м4 ' � аго ..g ое :.� ое о о �.8 ее ео оо оо ое ag � � оа т8 оо ну� оо ае ао ее оо и8 «�о ое ах � .•� •� оо �е� ое •.8 .•о ое •г8 ме •.о оо �� � � ео и8 .ro м� ..$ ..� ..8 ое -•8 оа ее оо ;�х �" "'SZ `"7$ •°•St L'�Е't "8 `''8 '"$ ^е оо м8 м7$ '"5� "aLt � "'й °° =S'г '�5� "'S� '°`w�' и8 '8 п8 '�1�" ^t`о °° 9'i}�д � �еик ..е й}� rog ..о еая �t'г и$ ~b� н5t ai� ий Rx � '"w "'8 N r �я NS� и�О Nй ~8 �О а$ м$ м0 �в�0 ... �' их '"8 �й '°f'д '^$ °'8 зй '°ro "8 `°й о8 '° �",8 1р g$ ее �ir ий ••о +yZ A.r,o '.,'8 °° �8 °°g "�$ $Рл" 1ьr £ ' � � и8 оо ;;й 00gg '''8 '"!`о й� "8 "'8 '"8 •E8 �° 1!,'t� .� � �е г.� ео eei5i а$g .вм ••8 �й 'Ol�о "'й '"� 'v8 '8 �й � "� � � ое оо �н,� �n$ ао �'+Д �д8 0°м "8 00й '''8 "'8 °r4�$' � � �" �е •� оо ;о�. °М '''8 '''S� �&8 .^.`$ оо �•� оо м�е �Sй � О � �n м^ е е е уг �. 8 е�+ 8 о е 8$ а� `'' 8 д'�' °° "" 8 LZ iE G b� 4 д,'is у�ч м 8 о е м 8 и 8 о о - е м^ .�о,� и е� м� о о •• 8 �� b� � ,� � �n$ ао еее� и8 е�я оо м8 нв ле их ее w$ хХ � � � g оо ое л� oogg .-о .•е и8 и$ ое м8 оо •�$ �ч�s м iл � ы ^ е"�а ���� � � �� �� ���� �� � ���� � � н ��� �� � � � � �� � � �� а � � � � � � ,�� � ,�� � �� � ��� ������� � � � � д ���'� �а �� �°� �а.уΡ�.�а� �о� еа���'' �г� � �а �о� �а� '�д �'г G�i � 'ii.'it�i�i ZвZ $i oZвi�1.'ё SaLii�Zв��7Lв� �{Zi 'l..S� �Х.� � {1г� p�i ��, �о'Б�8!^o�88.o�o5�iwR° • V vNa'✓оо�г.-•иио�еиN �� а�Sй�о�iй$ 1 $ 1,r, 1 �и„ �� ;к�$�и�•-.rо•�•оиол � �� °v4 о°м'ййRQlC"���i$й � �� ;�°��илинvоооо`�" � � ar �л $� e�%�$o��00� t 1 в�. +� CCY.i � � . Гii N�+►�••Р.�'•'��иоо^'� �Ci � � . � � ��, °��3о�'�35t�rS.лo�$i�" � � ,,,Q а 04 .�,. ��,. �г � и и ... О� .,. о �` � � ��� � � _ � О ��' пйv�иеь�мп�iе8йа�v '� L � а '� 'С � � . � �v$�:°.rаоое��еоом`� � ' � � ��, i оа�'Ь��S�R'v�°��о'8$ � � � � '� ii!!ss � •�+ � еИнй�N+ли,лмNи0 �.. р О � 3� $� � � � ti �� м g�� � н ; �� � � � �� � � н � � � � � � г��,�з� �. �,�а � � 157 Annex Table 3.8. Agricultural Projeetss Percent Satisfactory and Number Rated, by Country and Timne Period C~try Pormt SZ*kckry ti=berRet" IM" Ign-O 1~ 1978M Igms 1~5 C~= ~on LCN Iw iw Iw l 1 2 om= Iw Iw Iw 4 3 7 ChirIft EAP Iw Iw 100 3 18 21 B&~ LCN Iw 100 100 1 2 3 SAS Iw Iw Iw 2 1 3 Korea, ~ blic EAP 89 97 93 35 39 74 Nimum LCN 89 Iw 93 9 5 14 LCN 91 91 91 1 l i 1 22 M«~ 88 93 90 24 28 52 &i~ SAS Iw 86 90 10 21 31 Svd~ AFR 83 Iw 90 6 4 10 MYRn~ EAP 90 90 90 10 10 20 ~ flus MR 78 Iw 89 9 10 19 Burundi AFR 83 92 89 6 13 19 Pm~ LCN Iw 79 89 10 9 19 Botr~ AFR 90 Iw 89 10 8 18 Ttalland EAP 92 87 89 24 39 63 CostaRica LCN 91 86 89 11 7 la cma~ UY Iw 67 89 6 3 9 om= ECA 92 67 a 13 3 16 j~ 89 86 97 9 14 23 Chile LCN 75 iw 97 9 7 15 b14=må EAP 90 83 97 62 65 127 Y~ PDR 86 87 86 7 15 22 T~ 92 80 86 25 25 50 EAP 75 Iw 86 4 3 7 New 0~ EAP 91 80 86 11 10 21 P~ SAS 82 88 85 28 32 60 YU~ ECA 91 76 84 32 29 61 Bok AFR 71 90 82 7 10 17 Malawi AFR 95 80 82 13 20 33 C01~ LCN 93 70 81 29 30 59 Målayså .AP 89 71 81 27 21 48 T~ ECA 75 84 80 28 38 66 El Sal~ LCN iw 33 80 7 3 10 buffa MS 86 72 79 84 67 151 ESM 79 79 79 24 29 53 P~ ECA Iw 75 79 3 16 19 TM AFR 80 78 79 5 9 14 AFR 60 88 77 5 a 13 AM 80 67 77 20 6 26 cypgus ECA 88 67 76 8 9 17 Y== Arab ~ Ik 85 69 76 13 16 29 Romazda ECA 86 68 76 14 19 33 Cote dlvoire AFR 74 75 74 23 16 39 Philip~ EAP 83 67 74 35 46 81 Rww& AFR 63 82 74 8 l 1 19 Ban~ SAS 98 64 73 32 50 82 Ghana AFR 71 75 73 14 12 26 Ecuador LCN 80 60 72 15 10 25 158 Annex Table 3.8: Agricultural Projects, Percent Satisfactory and Number Rated, by Country and Time Period Country Regon Percent Satisfactory Number Rated 1970-77 197845 1970-85 197077 197845 1970-95 BurkinaPaso AFR 69 70 70 13 10 23 Mauritania AFR 63 75 69 8 8 16 Somalia AFR 73 64 68 11 11 22 Mexico LCN 84 56 68 25 34 59 Cameroon AFR 75 59 68 20 17 37 Gambia AFR 50 80 67 4 5 9 Guinea AFR 67 67 67 3 12 15 Dominican Republic LCN 86 50 67 7 8 15 SierraLeone APR 100 25 67 5 4 9 Brazil LCN 74 58 66 47 52 99 Niger AFR 86 so 65 7 10 17 Nepal SAS 69 61 65 13 18 31 Uruguay LCN 57 71 64 7 7 14 Mall APR 64 62 63 14 13 27 Senegal AFR 63 62 62 24 21 45 Argentina LCN 80 50 62 5 8 13 Bolivia LCN 73 29 59 15 7 22 Zambia AFR 83 41 59 12 17 29 Agia dNA 62 50 58 13 6 19 nW r APR 44 63 S 9 19 28 Central Aftican Republic APR so 57 56 2 7 9 Panama LCN 80 36 54 10 14 24 Nigeria AFR 58 so 54 19 22 41 Sudan AFR 5 50 54 12 16 28 Libria AFR 71 35 52 14 17 31 Kenya APR 59 44 51 29 32 61 Western Africa Region AFR 0 67 50 1 3 4 Uganda APR 50 50 50 2 8 10 Haiti LCN 67 42 50 6 12 is Congo APR 57 40 50 7 5 12 Syria MdNA 60 25 so 10 4 14 Zaire AFR 71 27 48 14 IS 29 Tanzania AFR 45 48 47 31 27 58 Jamaica LCN 42 50 46 12 16 28 Guyana LCN 50 40 45 6 5 11 Pera LCN 60 26 38 10 19 29 Western Samo EAP 100 0 33 I 2 3 Las AP 0 25 20 1 4 5 Chad APR 17 0 14 6 1 7 Annex Table 3.9: Project Evaluation Ratings at PCR Reviews Compared with Evaluation Ratings after Audit, by Sector Subsedor Number PCR Audit Dowagraded PCR Audit Upgraded Number eremt Audited Sat. Unsat. % mber Unsat. Sat. (% number Changed Changed Auie Audited) Agrictume 1os 61 is 14 44 s 5 20 19 Transport 37 30 5 14 7 0 0 5 14 Ham Resomces 31 26 1 3 5 3 10 4 13 5org 14 12 2 14 2 1 7 3 21 Water/Sanitation 10 7 2 20 3 0 0 2 20 Finance 16 8 1 6 8 0 0 1 6 All Subsectors 251 176 26 10 75 9 4 35 14 All Subseectors Minus Agriculture 146 115 I1 8 31 4 3 15 10 The number of projects evaluated as unatisato at audit fram the set of prjects rated as satisfactory at the PCR review. Note: Database defined as those audited projects for which an evaluating rating was also made at OED's PCR review. Soure: ODD Annual Review Database. 160 Annex Table 3.10: Agricultural Projects, 1970-88 Evaluation Ratings by Lag Between Completion of PCR and Closing Date 100 so 60 40 20 0 l=sstha I I to 2 2W to3ve 3 Years Lag (in years) Less than I 1 to 2 2 to 3 More than 3 Total Percentage Satisfactory 65 73 70 64 49 Number of Projects 903 120 326 282 175 Source: OED Annual Review Database 161 C PTE FOUR ANNE: INSTTTONAL DEVELOPMENT IN THE ENERGY AND INFRASTRUCTURE SECTORS A. INTRODUCTION 1. This Annex alms at (a) providing a general assessment of the Bank's approach to institutional development (ID) in the energy and infrastructure sectors; (b) identifying key factors of performance; and (c) drawing lessons relevant to future operations. It is grounded primarily on an empirical review of recent OED work (studies and audits) and did not involve the collection and analysis of new statistical data (although it does refer occasionally to data compiled for, and Included in, selected sector studies, e.g., on gas, water supply, and telecommunications leading). It should be emphasized that because this review is based primarily on evaluated projects (most of which were completed before 1990), it does not purport to assess the Bank's =grant practices and approach, which have evolved in some significant ways (particularly regarding the framework for private sector participation), but whose success cannot yet be measured., B. BACKGROUND ON SECTOR CHARACTERSCS AND INST!TTONS 2. The energy and infrastructure sectors covered in this review include electric power, oil and gas, telecommunications, water supply and sewerage, highways, ports, railways, and urban development.2 These sectors account for 39 percent of the total number of projects evaluated by OED between 1974 and 1992 (40 percent in terms of dollar commitments). 3. Overall, these sectors share common characteristics Identified in previous Bank reports:' (a) predominance of public ownership; (b) lumpiness in investments; (c) monopoly status; (d) high level of externalities; (e) intermediate input characteristics; (f) Important network effects; and (g) difficulties in recovering costs. The oil but not the gas) sector stands somewhat apart as it is more akin to industrial activity (even though the high economic rent involved often leads to public ownership in oil-producing countries). 4. It must be recognized, however, that the institutional setting for each of these sectors can vary significantly due to sector specificity. The intitutions encountered in electric power, oil and gas, See in particular A. Galenson, -Bank Landing for Infrastruoure,- Draft Repoit, 1993. 2 This Is alightly different from the definition used in meont Bank papers, particularly the Infrastrucne Sector Policy Review Paper (draft, Dec. 1992), which includes Irrigation but excludes oil and gas. See A. samel, Issues for Infrastructure Management in the 1990s,- Dscwsioa Paper N. 171, 1992. 162 railways, and to a lesser xtent telecomnunnkations, water supply, and ports, are to various degrees (and often only on paper) mostly comnerially-orlanted, semi-autonomous, and quite often monopolistic public enterprises. On the other hand, Institutions involved In the highways and urban sectors, and also in selected lsac power and gas distribution, are mostly government departments and/or municipalities. 5. the lega.adU 1 gr.framework and the degree of government intervention also tend to vary in relation to each sector. The sectors with the highest potential for monopolistic conditions due to investment lumpiness, economies of scale (power generation and transmission, tel communications, water supply), or which involve a large economic rent (oil exploration and production), are mostly characterized by large, central monopolistic public enterprises and a restrictive legal and regulatory framework. In contrast, oil production and distribution, and transport and urban development, which lend themselves more easily to operation in a competitive environment, already involve (or have the potential to involve) a larger number of Institutions, public or private, and operate (or could operate) in a less regulated environment. The d2 g MMm= JMMI= anuMadon also tends to be higher in sectors with the greatest monopolistic features (I.e, power generation and transmission, telecommunications), high rents (oil), or significant externalities (highways), compared to other sectors (e.g., road transport) which can function with minimum regulation. 6. Finally, the Mialfo pai N of prate sor operators also varies significantly among sectors. This Is due not only to the monopolistic versus competitive nature of each sector, but also to other factors such as: (a) local availability (or lack thereot) of advanced technology; (b) degree of risks involved (e.g., oil exploration and production); (c) potential for hard currency earnings (oil as compared to gas); (d) capital intensity and ease of entry (rail versus road transport); and (e) political sensitivity of output pricing (energy as compared to telecommunications). Patterns of institutional are also affected by factors other than sector specificity. One such factor is clearly a country's politicalleconomic system, .e., large, centralized public enterprises andlor government departments predominant in command and control economies. Another important factor is history, with many African countries, for example, having adopted the general Institutional models of their respective former colol powers. C. OBJECIVS, SCOPE, AND DELIVERY OF ID ACTIVITIES Objectives 7. The review revealed that the objectives of Bank-financed ID often were not made explicit in Bank reports when the borrower/etecuting agency was an slting Institution; rather, vague statements were made regarding the latter's strengthening. This was particularly prevalent in the case of public utilities (power, water supply, telecommunications) where projects focused primarily on the construction aud/or rehabilitation of physical facilities and where I) components (mostly technical studies, orgnization and MIS studies, and training) appear to have been included more as an afterthought than as a reflection of a primary project objective. In some case, this lack of explicit objectives was rationalie by the need for lxibiity darig ID Implementation (wrongly so, as provisions can always be made to #djust original objectives periodically in the light of experience). Only in a few cases were more specific objectives explicitly set: technology transfer was a main objective of the Hungary Petroleum Project and was successfully achieved. Elimination of labor redundancy was stated as a key objective of the Pakistan Tenth and Eleventh Railways Projects (but was never seriously pursued by the government nor by de Bank). However, It must be pointed out that progress on that count appears to have been made in more recent yem, wtness a number of(recently completed but not yet evaluated) power projects (Ghana Guinea,rTsy) that were almost exclusively dedicated to ID in the face of institutional collapse. 163 8. The strengthening of an institution cannot be undertaken without a clear understanding of that institution's future role In the sector. The absence of explicit objectives is not sumprising considering both the Bank's general lack of attention to the sector's overall institutional and organizational framework, in many of the cases reviewed, and its tendency to approach ID at the level of institution(s) in charge of the project (or even only at the level of a project implementation unit), rather than at the sectoral level. 9. Only in a few instances were ID components integrated into a long-term sectoral ID strategy: in Indonesia's power sector, the Eighth, Ninth and Tenth Power Loans fully complemented each other and were part of the long-term sector strategy successfully pursued by the government and the Bank. Indeed, in this case, the Bank also devoted a major portion of sector work to ID issues. However, the review found no instance where ID activities were explicitly integrated into an overall countrywide ID strategy, in spite of the recognized importance of generic country factors in ID performance (see Section D below).$ 10. ID objectives were more explicit whenever the projects were aimed at creating, rather than strengthening existing sector institutions. Through a series of 13 power loans made to India from 1977- 87, the Bank supported the initial creation and subsequent building up of National Thermal Power Corporation (NTPC) in order to: (a) build a central institution that could efficiently and speedily construct and operate large thermal plants and high voltage transmission lines (given the state utilities' poor record in this respect); (b) introduce long-term sector planning; and (c) serve as a model for state utilities. The Bank's substantial financial support to the creation of Interconexion S.A. (ISA) in Colombia's power sector followed essentially the same rationale, in the context of a highly regionalistic political system. And similarly, albeit on a smaller scale, the Bank promoted the creation of Istanbul Water Supply and Sewerage General Directorate (ISKI) in Turkey through the Istanbul Sewerage Project. The whole series of Bank TA loans made for petroleum exploration is also a case in point-their very purpose was to build up the governments' institutional capacity, and revise the existing legal/contractual framework in order to promote exploration to the oil companies and monitor their activities. Scope and Nature 11. Nonexistent, narrow, or vaguely defined objectives most often led to either too limited a scope for ID activities/components or, conversely, a *christmas-tree" approach where a shopping list of activities was attached to a major investment project without any serious needs analysis having been performed. The recent review of World Bank lending for natural gas points out in particular that "aining programs would often consist mostly of short lists of training areas without consideration of the availability of staff, course contents, training arrangements." In addition to training, MAeM Information System (MIS) studies and/or assistance to the implementation of new systems were a very common feature of ID components included in the projects reviewed, their inclusion sometimes based on an initial diagnosis carried out by a management consulting firm. Also quite common were various types of tecmical udiesassistamnc related to system planning and/or operations and maintenance, the degree of complexity and specialization of which depended mostly on the sophistication and expertise of local staff (e.g., specialized enhanced oil recovery studies in Hungary as compared to basic long-term expatriate assistance to O&M in Botswana's water supply sector). Hardware and/or facilities associated with ID were occasionally included in projects, consisting mostly of computers and training materials S"Tower Sector Instituional Development Review-Indonesia," 1989. S It must be soted, however, *ta mome recent (but not yat completed) seor andlor public enterprise adjustment operations (e.g., in Argentina and Morocco) appear to have Alled that gap, at teast partialy. 164 (although a number of exploration promotion projects, particularly in Africa, did include such items as laboratory equipment, vehicles, and new buildings). 12. Some projects also included studies related to sector planning andlor oranization, to be implemented by the government rather than the utility: examples include the Egypt and Bangladesh gas projects (studies of gas development and utilization) and the Brazil Power Sector Project ["Piano 2010" for L-T system planning and a study for sector organization REVISE]. Indonesia's power sector again stands out, as the Bank appears to have invested an unusually high level of efforts and resources in addressing sector institutional issues, not only in the context of individual power projects, but also by sponsoring a high-level conference of experts on the subject and by carrying out substantial corresponding ESW (para. 9). Also, most petroleum exploration promotion projects and a number of oil and gas investment projects included legal assistance to governments for revision of the sector's legal and contractual framework for foreign investment. Delivery 13. The overwhelming majority of ID activities identified in this review were channelled as components (proportionately small ones) of traditional investment projects. Few cases of stand-alone TA projects were encountered, in contrast to the number of engineering and TA loans made in these sectors in the 1970s. One already-mentioned significant exception, however, was the series of petroleum exploration promotion projects, a large number of which were in Africa. The review identified only two cases of significant ID sponsored under sector adjustment lending: the 1986 Brazil Power Sector Loan, which included REVISE and a long-term power master plan (Plano 2010), and the Bangladesh Energy Sector Adjustment Loan, which involved specific ID actions in the power, oil, and gas subsectors. This contrasts a significant number of such projects more recently approved, but not yet evaluated (e.g., in Mexico and Argentina). The review found very little reference to ESMAP-funded ID activities in OED materials, which could mean that such activities were either very limited or that, if indeed some were carried out, they were probably not fully integrated into the Bank's operations. Nor did the review encounter any significant mention of NGO-funded or sponsored ID activities. This may be due to either insufficient attention paid to these aspects in OED reports or, more likely, the Bank's historical lack of preoccupation with the micro ID aspects related to, among others, demand management, quality of service, and community involvement in these "hard" sectors. 14. The majority of ID activities encountered were carried out by foreign consulting firms (particularly major technical and MIS/management/organization studies) and occasionally by individual consultants (i.e., expert panels for hydroelectric projects in Guatemala and Mexico, or high-level seminars on major power sector issues sponsored by the Bank in Colombia, Indonesia, and Brazil). Long-term resident expatriates were used in several instances, as in Oman's telecomanications agency and Botswana's water supply company. Quasi-twinning arranaements were carried out in a number of witer projects, as well as in several gas projects (Egypt, Indonesia, Pakistan and Tunisia), due to the sector's prevalence of joint-venture arrangements. Lastly, training was most commonly provided on-the-job (in connection with consultants' studies) and through short-term seminars/courses in foreign countries. There were some instances of the Bank funding long-term attendance of students to foreign universities (as in the Ethiopia Petroleum Exploration Promotion Project), and one case (Botswana power) of a successful training program implemented by equipment suppliers. Energy Sctor Management Assistance Program, a tchnicalmassistanceprogramjointly haded with UNDP, for which the DBank is executing agency. 165 Targeting is. ID was predominately targeted at the project's executing agency, with some occasional instances of ID targeted at the central government for sector planning (see above) and/or policymaking and improvements in the legallregulatory framework. In addition to the sponsoring of high level conferences on the power sectors of Colombia, Brazil, and Indonesia, the Bank also helped with strengthening Brazil's environmental agencies, in connection with substantial lending to the power sector. D. TRENDS AND PATTERNS IN COVERAGE AND PERFORMANCE Coverage 16. Although no specific statistical analysis was carried out for this review, there is clear evidence that at least some sort of ID activity was included in the vast majority of recently evaluated projects in the energy and infrastructure sectors. Data compiled for the review of natural gas lending showed that about two-thirds of Bank-financed gas projects included technical assistance for project implementation, operations, improvement of accounting and financial systems, and planning. Sixty percent included training components. A recent review of Bank lending for telecommunications indicated that projects involved institutional improvements related to organizational structure in 72 percent of cases, technical staff training in 69 percent, improvements in accounting and MIS systems in 76 percent, and personnel policies in 21 percent. A cross-sectoral review of ID in Bank projects prepared by the Country Economic Policy Department (CECPS) in 19907 showed that 86 percent of public utilities projects approved between 1985-87 included ID components (versus 79 percent in 1978-81), as did 82 percent of the projects approved in the same period in the transport, urban, and tourism sectors (versus 77 percent in 1978-81). Coverage of ID, at least in quantitative terms, increased substantially during the 1980s in the energy and infrastructure sectors, as in most other sectors. 17. Coverage of broader sector institutional issues, although still scarce overall, was mostly found in the energy, telecommunications, and water sectors. A significant number of oil and gas projects (and most of the exploration promotion projects), in particular, addressed the need for changes in the legal/contractual framework for foreign investment. The telecommunications lending study indicated that revisions of the institutional framework were pursued in 58 percent of the projects; indeed, proposed changes followed the same pattern, starting with the organizational and financial separation of postal and telecommunication activities, through the (more or less effective) commercialization of the telecoms entity. The survey also revealed that project- and agency-based ID, still the most prevalent form, had itself broadened in scope, emphasizing the building and strengthening of whole institutions rather than project implementation units, as was often the case in the previous decade (and still found in some road projects, for example). Performance 18. Previous Bank reports have documented a general detericration in the performance of ID activities financed or sponsored by the Bank, from which the energy and infrastructure sectors are not exempt. In a 1989 review of 1,699 Bank projects implemented between 1978-88, CECPS estimated that 7 See S. Paul, AD in World Bank Projects, 1990. SIbid. 166 the share of projects with negligible I) achievement had steadily increased from 15 percent to 25 percent, and that the share of projects with substantial ID achievement had declined from 42 percent to 35 percent during the same period. In terms of sectors, transport/urban and public utilities also showed a similar (worsening) trend, though the latter appeared to be stabilizing towards the end of the period. Compared to other sectors, the same data showed that the telecommunication subsectors had a better ID success rate, that public utilities (power, water) and urban showed a mixed performance, and that railways' ID ratings were significantly poorer. 19. These findings and patterns are essentially confirmed by the empirical evidence gathered for this review, by the statistical data compiled for the recent reviews of Bank lending in selected sectors (gas, water and sanitation and telecommunications), and by recent OED institutional ratings,9 shown in Table 1 below. Table 1: Institutional Ratings (1989-92 cohorts) (Percent of No. of Projects) No. of Negli- Subs- Not Not Total Sector Projects gible Partial tantial Applicable Available Energy 77 8 40 52 0 0 100 Power 82 17 43 32 2 6 100 TelecoMs 25 20 40 32 8 0 100 Transport 115 26 48 23 2 1 100 Usban 53 30 36 32 2 0 100 Water Supply 57 18 40 39 4 0 100 s/tot. 418 20 42 34 2 1 100 All projects 109 22% 45% 27% 4% 2% 100% 20. Not surprisingly, empirical evidence and ratings confirmed a strong correlation between ID performance and overall project performance, as shown in Table 2. 21. The number of overall successful projects exceeds the number of successful ID projects. This appears to confirm the review's findings that quite a few projects that had essentially achieved all their technicallphysical objectives were rated satisfactory overall, in spite of the failure of their (often limited) ID objectives. As somewhat of an exception, Indonesia's East Java Water Supply Project achieved its limited institutional objectives but mostly failed on the technical side. 22. Some of these statistical findings need to be qualified in light of conclusions reached by recent reviews of the selected sectors. The study of gas ledn did conclude that over half of completed gas projects made substantiIl progress on ID compared to other sectors. It also pointed out, however, that although success was significant in strengthening the technical capabilities of most beneficiaries, results were mixed in the case of long-established national oil companies or natural gas monopolies (e.g., in India, Pakistan, Bangladesh, and Nigeria), and that the Bank's impact was indeed minimal in restructuring the overall sectoral institutional framework and improving in regulation. The water MW Snstitutional ratings have only been available in OED's database sioe 1989. 167 gud. concluded that the Bank had rarely suceeded in bulding up instuions tchnicauy capable of impiienting regional or natonal programs (thus contradcdg the impresson givm by thi above ratings), In spite of some notworthy suces stories (e.g., in Botswaa and Tnala). MIs my be du to the fact that ID objecves may have been too modet to start with, or becaus ID achievemmnts were not sustained in the long term. Ie telemnicaionsdx con&rmed dt many (but by no Means all) projects in that sector did achieve their limited ID objecdves in terms of strf-gtIening the te~nical capabilities of FIT agmncies and bringing about the Inial steps of sector reform (e.g., separation of postal and telecommulcations). However, the study a showed that th~s projects fell short of promoting more radical sector changes (e.g., introduclng some lmmens of com~petiton and improving the regulatory framework) that rapid tchnologl~al changes madi possible (if not outght necessary). Table 2: ID and Overal Project Perbrmnce for Energy and Infrastructure P*rojects (1989-92 cohorts) Overail Project Ratin ___ Satlafactory Unsaduauteory Total Negligible 22 27 61 73 83 100 Pardial 142 81 34 19 176 100 substantial 140.- 98 3 2 143 100 Total 304 76 98 24 402 100 23. T1he sample of projects reviewed (or included Inthie sector studies) æay not bo geographically representativem and deginit conclusions about geographical patterns of ID performance cannot bo drawn. Thei review did show, however, a coentradlon at ID success storie in last Asia (e.g., Thiailand'spower and telecommunications sector, Slngapore'swaterandsewerage agency, andJndaneia's power and water utilities), compared to a hlgh lncidence at failures In LAC (whness the power sectors of Colombia, Guatenala, and Panama, and Ecuador's Guayaqugi Urban Project). E. MAIN FACTORS OF PERFORMANCE 24. Although not based on detailed statistical data, the review provided a suftHlently wide range of ID experience in the energy and infrastructure sectorn to illustrati main factors of success/fagur. flese can bo grouped in three categorles: (a) exogmnous (mostly *country") factors; (b) project-related factors (i.e., mostly related to project design aud Implementation); aud (c) process-related factors (i.e., "Bank-controlled" factors). Country Factors 25. Political and Social Context. Although clearly pervasive, the intluence at political, social, religious, aud cultural factors on ID is probably the mot diffEcult to analyze givmn the complexity aud sensitivity of the issues involved. To cite only a few examplus, there ar. obvious links between factors such as: (a) religious/cultural values and staffing iusues (e.g., role af women in the workforce); (b) a In perdtiuar, projuca reviewed lnetuded a tstlwly smali proportion of pstjsos in Akles. 168 political system and scope for local participation; (c) extent of political patronage and productivity; and (d) regionalism and model for institutional framework. Such issues do not permit easy generalization on either geographical or sectoral grounds. The review carried out in the energy and infrastructure sectors did illustrate, however, the critical importance of such factors in the performance of ID activities, as discussed below. 26. The Issueof fdlm lIonalisBproved particularly daunting in sectors like power, which require at least some degree of central planning and coordination to function efficiently. In Brazil and Colombia, these issues proved insuperable, whereas in India, they were addressed with some (limited) success. The retrospective review of power lending to Colombia showed how the Bank ultimately failed in its attempts to restructure the sector, particularly in the creation of a strong central utility (ISA), because it overlooked, and failed to take advantage of, Colombia's strong regionalism. The Bank's more timid efforts at promoting sector restructuring in Brazil, via the REVISE study (para. 12), were also thwarted by its inability to recognize, and adjust to, the new political tendencies towards increasing regional autonomy and decentralization, highlighted in recent changes to the Brazilian constitution. The Bank was more successful in building up a strong and efficient central institution in India (NTPC), in spite of major changes it involved in the relations between the central government and the states, and in the federal set-up; however, this same set-up has so far proved a major obstacle to further institutional change at the level of the State Electricity Boards. Legal, as well as political, constraints have also occasionally been overlooked or underestimated by the Bank, as in the case of Indonesia's water sector, where the Bank required sector organizational changes that went against the laws of the country. 27. ID efforts have also been thwarted in a number of Instances by high man tunoe resulting from political instability or the country's political culture. Mexico's power utility (CFE) saw a succession of five general managers between 1970 and 1982, which led to a significant decrease in efficiency. The water supply sectors of Colombia, Mexico, and Kenya also suffered from frequent wholesale staffing changes throughout the sector, as a result of political appointments down to the second level of management. 28. Overstaffing due to oltc ptna is also a common occurrence in public utilities, and particularly glaring examples were encountered in Pakistan's railways, Bangladesh's power company, as well as in Nepal and Ghana's water utilities. In Mexico's power sector, the political Influence of CLFC's (Mexico City's distribution company) trade union has, for the last 30 years, thwarted the governments attempts at improving sector integration and productivity. As somewhat of an extreme case, overstaffing was actually a legal requirement for Kenya s water utility. And the Bank's Inadequate initial understanding of the local political circumstances in Ecuador's Guayaquil Urban Project (particularly the municipality's attitude toward social development) contributed to the failure of most of the project's components under local government responsibility. 29. More generally, the review confirmed one of the findings of previous Bank reports:" the Bank's analysis of ID issues (however superficial) tends to be inward-looking. focusing primarily on the implementation and management capacity of project entities, and largely overlooking external environments and linkages. Only rarely did the Bank attempt to identify the socio-cultural conditions, the main stakeholders, Interest groups, and *winners and losers,* in order to assess related risks and long- term sustainability. It See D. Brinkerhoff, befafoal AnalysIs ad &ssinaloal Developimea A Mowv 4f World Bank Prja Eqpedence, 1989. 169 30. Human Caital onti. A general dearth of skilled human capital has been a major constraint to the Bank's ID efforts, particularly in the poorest countries. The energy and infrastructure sectors were not exempt from these constraints. In some technology-intensive sectors (e.g., oil and gas), engineering skills were especially hard to find or retain, particularly due to the brain drain phenomenon (e.g., petroleum specialists' exodus to the Middle East). This problem was found to affect even good performers such as Thailand's Posts, Telegraph, and Telecommunications (PTr) agency. Projects in some countries (e.g., many LAC countries and Turkey) were found to suffer from an acute lack of floanial expertise, often due to competition from a thriving private sector or a basic lack of appropriate training in the financial and economic fields (the latter particularly in socialist or ex-socialist countries). Paraguay's highway sector, as another example, suffered a severe exodus of qualified staff as the result of undertaking two giant bi-national hydropower projects with Brazil and Argentina. 31. The review found noteworthy examples of successful approaches to the skills shortage issues. Long-term expatriates were used successfully, for example, either In executive positions (in Oman's telecom agency and Botswana's water utility) or in advisory roles (e.g., in Ecuador's National Low- Income Housing Project). This success is attributed to the implementation of a long-term, realistic plan for the gradual transfer of responsibility to designated local counterparts. Twinning arrangements were also used effectively in several oil and gas projects, and showed the most promise in the energy and telecommunications sectors where large utilities from developed countries have the skills and interest in entering into such arrangements. The review also showed, however, that even successful efforts at the sectoral level were no substitute for a long-term countrywide approach, which would essentially entail the strengthening and/or restructuring of the whole educational system. 32. Macroeconomic Environment. Restrictive fiscal policies were foremost among key macroeconomic variables affecting ID performance in the energy and infrastructure sectors, as they directly affected the ability of public utilities and/or government departments to attract or retain qualified personnel through appropriate wages. A case in point, among many others, is the Philippines Rural Road Project which failed to create a strong provincial road maintenance capability due to a lack of attractive employment conditions. Conversely, the water supply study points to the institutional success of Tunisia's water supply utility due to, inter alia, a high staff morale and low turnover brought about by an adequate salary scale. Wage restrictions were a particularly limiting factor in the Bank's efforts to strengthen central government sector planning and regulation, as government departments were usually subject to even tighter budgetary constraints than state-owned enterprises. 33. The review also found that if excessively low wages consistently resulted in poor ID performance, more limited wage differentials could be offset by nonfinancial factors. India's NTPC, for instance, was able to attract first-caliber staff by promising, and delivering, faster career advancement in a rapidly growing company. Extensive staff training (US$550 per employee per year) is a key element in Singapore's water department's ability to recruit and retain skilled staff. For many countries, however, no significant improvement in sectoral wage policies (particularly for government employees) will be possible without an overall civil service reform. The review found no example where sectoral staffing policies improved significantly through wage increases alone. 34. The direct impact of other macroeconomic policies on ID performance was less obvious. The review found, for example, that very few ID actions in the sectors suffered significantly from a lack of counterpart funding and/or access to foreign exchange during implementation. On the other hand, the incentive framework for forein investment was clearly a determinant in permitting a major institutional restructuring of the oil and gas sectors in which the Bank was involved. 170 35. Administrative and Legal Environment. Administrative and bureaucratic bottenegks were shown to have often impeded the effective implementation of ID activities. An extreme case is that of Bangladesh's power utility where 29 separate procurement steps were required before foreign consultants could be hired. Bureaucratic obstacles relating to tax treatment and provision/import of vehicles and computers, among others, were also found to be common in many projects, Irrespective of the sector, although they were more prevalent in government departments than in public enterprises, which sometimes benefit from special exemptions. In a few countries (e.g., Uganda, Somalia, Ethiopia), security risks also made it difficult to hire and/or retain qualified consultants. 36. The existence of an established general framework for &overnment-public enterprisg relations was also helpful in facilitating institutional reform and effective ID implementation. The concept of performance contract ("contrat-programme"), for example, which had been primarily used in French- speaking African countries and North Africa, has recently gained ground in other regions as well. A similar contract signed between the Mexican government and CFE, the power utility, has provided a framework for future productivity improvements in the sector. In India's power sector, NTPC's relations with the government have been governed by annual Memoranda of Understanding that commit the company to specific performance budgets. Project-related Factors 37. Autonomy of Executing Agency. Managerial and financial autonomy (with accountability) of the public utility or project implementing agency was a key factor of ID success, as the lack of autonomy invariably resulted in overstaffing, low productivity, high management turnover, low staff morale, lack of appropriate MIS systems, and lack of long-term planning. The water supply study attributes the institutional success and indeed the overall success of Bank-financed projects in Tunisia (until the mid-1980s) and Botswana to the high degree of autonomy enjoyed by their water utilities, with regulatory oversight provided by parent Ministries. The same study also points to Singapore's Public Utilities Board (PUB) as an example of outstanding success for an autonomous public sector agency, which interestingly enough manages to combine the provision of gas, power, and water supply services under a single umbrella in a very efficient fashion. The successful evolution of India's National Thermal Power Corporation (NTPC) into a strong and efficient institution was attributed in large part to its operating autonomy, a reflection of the government's natural desire to foster the growth of its protege, its confidence in NTPC's management, and its early recognition that its role should be essentially supportive rather than supervisory. Other success stories (Turkey's ISKI in the water supply sector and Thailand's telecommunications agency, for example) are similarly characterized by the large degree of autonomy enjoyed by the corresponding institution. 38. Conversely, the ID objectives pursued in the Pakistan Railways Projects (labor redundancy), the Philippines Water Supply and Sewerage Projects (improvements in maintenance practices), and the Bangladesh Telecommunications Project (improvements in project planning and implementation capabilities) were all ultimately thwarted by their governments' unwillingness to provide the implementing institutions with sufficient managerial and financial autonomy. (Bangladesh has clearly backtracked on this count in xacent years.) 39. Project Design and Readiness. The review confirmed that design factors had a maor influence on the effectiveness of ID work. Excessive complexity of institutional arrangements, for one thing, often led to implementation problems, as in Ecuador's Guayaquil Urban Development Projec where a multitude of executing agencies and two or more different levels of government were involved. On the contrary, the more simple institutional set-up chosen for Ecuador's other urban project (the National Low Income Housing Project) proved successful. But more often than not, urban projects, in 171 particular, were characterized by excessively complex institutional arrangements: a recent review of 28 LAC urban projects revealed that each project involved an average of six different institutions and 23 separate components. The "shopping list approach" used in many training components of gas projects covered by the gas lending study also led to inadequate implementation and follow-up. The overestimation of the country's absontive caacity was not uncommon: in Tanzania'a petroleum projects, training could not be implemented because of lack of staff to train. In Guatemala's power sector, the Bank overlooked the power utility's weakness and lack of experience, and entrusted it with an especially technically complex and risky hydroelectric project that ultimately failed. 40. Most Importantly, a clUa commitmn on. and ownershW-Qf. the objectives of the proposed ID activity by the government and/or institution in charge was often lacking at the outset. Previous OED workn identified four main criteria for assessing the extent of such ownership: (a) locus of initiative; (b) -evel of intellectual conviction among key policymakers; (c) expression of political will by top leadership; and (d) efforts toward consensus building among various constituencies. In the case of China's Three Ports Project, lack of initial government commitment led to two important studies (on long-term port strategy and economic impact) not being carried out and the project's ID objectives not being achieved. On the other hand, NTPC's institutional success story in India's power sector is due in large part to the central government's strong backing (including financial support). Generally, it appears that commitment on institutional objectives and targets was secured in a more low-key manner than that for other pr ject objectives (physical, financial). This may explain why ID objectives were not always taken seriously by both governments and borrowers. 41. The design of institutional development components rarely included provisions for systematic monitoring of progress against explicit success criteria or concrete and dated targets. Needless to say, monitoring targets never extended, in any case, beyond the expected project implementation period. Most often, Staff Appraisal Reports (SARs) did not include a timeframe or specific modalities for phasing out ID assistance. This may have reflected an underlying assumption that such assistance (and the related Bank funding) would continue forever (or at least the foreseeable future), and may also explain why repeater loans sometimes appeared to tackle the same ID issues over and over again. The experience of Paraguay's Highway Sector is instructive in this regard: in spite of the Bank's involvement over 30 years through seven projects, many of the ID issues faced by the sector and repeatedly identified in successive Bank SARs remain essentially unsolved (uncompetitive employment conditions in the responsible government departments, ineffective axle load control capacity, unclear definition of responsibilities among various sector institutions). 42. The degree of preparation of ID components was also found wanting in some instances. The gas lending study found that most ID components of gas projects had not been designed with a clear strategy, explicit action programs, and details for phasing out various institutional actions. In some cases, terms of reference for key studies were not ready, or were too vaguely formulated, by the time of loan approval, and the studies were never carried out for lack of appropriate Bank follow-up. 43. Quality of Leadership/Management. The quality and motivation of implementing agencies' managers were also a key factor of ID performance. India's NTPC again stands out: much of its success has been attributed to the drive and leadership qualities of NTPC's first chairman, who recruited a small group of top managers from other sectors, who he felt had the potential to become starters and builders rather than bureaucrats. Among his top ID priorities were the establishment of a high profile corporate 12 See J.H. Johnson and S.W. Wasty, Borrower Ownership of Adjustment Programs and the Political Economy of Reform,' Draft, 1993. 172 planning unit, recruitment of skilled and motivated staff, and training and set-up of adequate management systems and procedures, all of which were successfully carried out. The importance of management was also found to be critical by the water supply lending study, which pointed out that breakthroughs at the political level require initiative and determination on the part of public utility managers. The study also indicated, however, that making recommendations on management frequently poses a dilemma for Bank staff, as the Bank has an unwritten policy to encourage staff to develop specifications for a management position, but not participate in the selection process. 44. Type and Quality of Consulting Services. Poor performance by consultants was W1 found to be a recurrent cause of ID failure in the projects reviewed. Instances of failure were moOdly attributable to inadequate terms of reference, inappropriate supervision by the hiring agency, and unclear definition of responsibilities. It must be pointed out, however, that Africa was underrepresented in the material reviewed. Ongoing OED work indicates that poor consultant performance may indeed be a serious problem in that particular region, as international consulting firms may have a tendency to assign their less experienced staff to Africa. 45. The review did not identify the form of delivery (e.g., short-term versus long-term consultants, consulting firms versus individual experts) or nationality as being consistent determinants of performance. For example, the use of long-term expatriates was successful in Botswana (water supply), Oman (telecommunications) and Ecuador (housing), but failed in many road maintenance projects. Although not very well documented in the material reviewed, the use of local consultants does not appear to have been widespread, particularly in the energy sector, due to lack of local technical expertise, with the exception of a few countries (e.g., Mexico's power sector). Indeed, India's NTPC is reported to regard the opportunity to use consultants from a variety of countries-a means of gaining access to the latest technology and most advanced utility practices-as one of the major benefits of its relationship with the Bank. 46. New versus Existing Institutions. The review encountered striking examples of both successes and failures in the Bank's support to newly-created institutions. Successes include NTPC in India's power sector, ISKI in Turkey's water supply sector, and the Lesotho Highlands Water Authority; failures include ISA in Colombia's power sector and Philippines' rural road maintenance project. It, therefore, does not appear that the choice of a new versus existing institution to implement a project was necessarily a factor of ID performance. 47. These examples do indicate, however, two things First, the successful build-up of a new sector institution was largely dependent on (a) the government's backing and willingness to provide it with sufficient managerial and financial autonomy, and (b) a clear understanding of that institution's future role in the sector. Second, the successful build-up of a strong and efficient sector institution was not in itself sufficient to resolve deficiencies in the overall sector framework, as shown by India's power sector where major problems persist at the level of the state electricity boards, in spite of NTPC's success as an institution. 48. Local Participation. Very few cases of community/user participation were encountered in the review. In Liberia, an attempt to organize local communities for routine rural road mair tenance was unsuccessful due to shortages of manpower and government funding, lack of coordination between government agencies, low motivation, and opposition by segments of the rural population. The Bank was more successful in promoting effective local participation in the design and implementation of substantial resettlement programs in power lending to Brazil and in Mexico's Hydroelectric Development Project. Experience with the latter, however, where the Bank's activism may have exacerbated the resettlement program, illuminates the pitfalls of Bank involvement in these issues. 173 Process-related Factors 49. Choice of Bank Instrument. A large majority of ID activities in the energy and infrastructure sectors were channelled through standard investment loans (main exceptions were a few telecommunications TA loans, the Brazil Power Sector Loan, and the series of petroleum exploration promotion loans). Possible reasons for this prevalence are: (a) the "hard" nature of these sectors, where borrowers usually view financing of physical investments as the most pressing need; (b) an implicit bias on the part of Bank staff towards large and more "visible" projects; and (c) the fact that many projects reviewed were approved in the early 1980s, at a time when sector adjustment and lending instruments such as Public Enterprise Rehabilitation Loans (PERL) were not yet widespread. 50. The experience of a number of projects that failed precisely because the institutions in charge were too weak to implement them (e.g., the Guatemala power projects and many water supply projects) tends to indicate that the Bank would have been better off strengthening the implementing agency through a stand-alone TA project or other means (e.g., sector work or PPF-financed ID) hefor agreeing to finance a large investment project. The (generally successful) experience of petroleum exploration promotion projects in the oil and gas sector indicates that stand-alone TA projects in other "hard" sectors should have been given more consideration. Alternatively, the Bank could have held off on further investment lending until required institutional was implemented; indeed, Turkey's Istanbul Sewerage Project audit concluded that the Bank was well justified in waiting for the water utility to be granted the necessary autonomy before proceeding with a project. 51. The review included only two cases of ID supported by adjustment lending: the Bangladesh Energy Sector Adjustment Project and the Brazil Power Sector Project.' Both projects failed to achieve their ID objectives, due to: (a) lack of government commitment; (b) inappropriate understanding by the Bank of political constraints; and (c) insufficient and weak enforcement of conditionality. The poor results of these projects is an indication that the quick-disbursing character of sector adjustment loans may be a handicap to tackling institutional changes that require a long gestation period. Recent progress reportedly achieved on the ID front in the telecommunications, energy, and transport sectors in PERL- type operations (e.g., in Morocco, Argentina, and Mexico) highlights the potential usefulness of this type of lending when major leverage is requlied. However, this remains to be corroborated by forthcoming OED work. Further, if the inclusion if ID in large investment projects did increase Bank leverage in promoting institutional change at the level of the direct beneficiary (i.e., usually the public utility), the review holds no such evidence that it exerted the same leverage on governments to implement broader sectoral changes. The review did indicate that most success stories (e.g., in Turkey and Jordan's water supply sector and Indonesia's power sector) were associated with a long-term Bank involvement through Epeater rects. This tends to confirm the premise that ID Is a long-term proposition and is unlikely to be achieved as the result of the Bank's one-time involvement (usually limited to a three or four-year period). 52. Conditionality. Although ID-related conditionality was common in the loans under review (the gas lending study, for example, mentions that about 60 percent of Bank-financed gas projects had covenants related to nonfinancial institutional issues), more often than not these conditionalities were vaguely defined or worded. Related covenants were usually limited to commitments to (in essence) "carry out studies (mostly on organization and/or MIS improvements), to discuss recommendations with the Bank, and to implement these satisfactorily." This is not surprising given the already-mentioned U .'though not officially classified as a sector adjustment loan, this loan had all the characteristics of such an operation, wvh the exception of a formal letter of development policy. 174 absence In most projects of any in-depth institutional needs analysis that could have served as the basis for a well-defined ID action plan, to which legal documents would then have referred. This lack of specific ID action plan was common, even in repeater operations that should have been abie to build on ID diagnostics and actions of previous projects. This was probably due to the lack of effective and systematic ID monitoring of most projects (see case of Parag-.ay's highway sector, para. 41). One clear exception Is the Bangladesh Energy Sector Adjustment Loan (ESAL), which included very specific ID conditionality; IID objectives were not achieved, however, due to the Bank's misreading of the governments commitment and the country's socio-political constraints, and lax enforcement of conditionality. 53. Ouality and Intensity of Bank SuDervision. The projects reviewed were not exempt from the main weaknesses in Bank supervision, which have been well-documented (most recently by the PMTF Report) and identified as one of the main causes of poor ID (and overall project) implementation. In particular, the prevalence of large investment projects in the energy and infrastructure sectors meant that technical and (sometimes) financial issues were largely given priority during supervision missions, to the detriment of institutional issues. This in turn was compounded by a general lack of Bank staff ex-ertise on ID issues (see below). The water supply lending study found that supervision time as a whoie for water supply projects was reduced by 48 percent between the mid-1970s and the mid-1980s, and pointed to the high turnover of responsible project staff or the hiring of staff inexperienced in ID as major issues. The audit of the Pakistan Railways Project concluded that the Bank's supervision erred in failing to follow through on specific government ID commitments; indeed two years elapsed at some point between supervision missions. Similarly, the failure of the China Three Port Projects to achieve its ID objectives was attributed in part to low supervision frequency and staff continuity, as well as inadequate staffing of missions. Furthermore, a lack of systematic country-based portfolio management and a related lack of focus on generic countrywide ID issues (also documented by the PMTF Report), most noticeable before the 1987 reorganization, contributed to the lack of priority given to these issues during supervision missions. 54. Bank Staffing and Oranization. A general lack of Bank staff expertise on ID and the resulting negative impact on the quality of IID work and the performance of ID components have been highlighted in numerous Bank reports." The lack of appropriate Bank skills is also mentioned in a number of PARs and OED studies concerning the energy and infrastructure sectors, though not analyzed in detail. Empirical evidence suggests that several factors may explain the Bank's relatively poor performance in helping to design and follow up on ID activities: (a) the low number of former utility managers on staff with hands-on experience (and *feel") for ID problem-solving at the corporate level; (b) the lack of familiarity of the Bank's technical specialists and/or inexperienced young generalists with the broader set of ID issues (at the sector level), which the Bank is now tackling; and (c) until recently, a reluctance on the part of Bank managers to commit substantial financial resources (e.g., for consultants) to analyze ID issues. 55. Indeed, one of the few outstanding examples of ID success covered by the review is Indonesia's power sector. Its success has been attributed, inter alia, to the combination of several factors: (a) the sensitivity of relevant Bank staff to ID issues; and (b) Bank management's willingness to carry out *expensive," dedicated ESW work (including hiring world-renowned personalities and organizing a high-level conference on these issues). Budgetary constraints may be one factor explaining the Bank's common practice of financing institutional needs analyses (e.g., diagnostic studies by management consultants) UWr its loans, rather than carrying them out g1k to loan approval, which would ensure See Gray at al, hutf afot*al Devlopme Work in the Bank A Review of 84 Bank Projecs, 1990. 175 that the findings are taken into account in project design. Project Preparation Facility (PPF) advances, which can theoretically be used for this purpose, are sometimes resisted by governments, and their availability is in any case subject to overall regional ceilings. Further, the Bank's recently-established Institutional Development Fund'- is specifically excluded from funding the preparation of ID components. 56. A realization that the Bank was not adequately staffed or properly organized to tackle ID triggered the creation of central pools of ID experts at the level of Technical Departments during the 1987 reorganization (in Public Sector Management, Public Enterprise, or ID and Management Divisions/Units). There was no mention of this change in the material reviewed for this study; hence, its impact cannot be determined. The limited results that are available are inconclusive. In the Jordan Energy Project, for instance, the regrouping of all energy subsectors in the same division did allow the Bank to carry a broader and more coherent dialogue on ID issues for th. energy sector as a whole. However, in the Bangladesh Petroleum Exploration Promotion Project, the breakup of the central Energy Department had adverse impacts. 57. Restrictions on Bank Funding. The experience of several projects tends to indicate that the Bank's inability to finance certain types of expenditures has sometimes hampered its effectiveness in dealing with ID issues. The water supply lending study, in particular, highlights the severe gpgations and maintenance problems faced by the sector, which it attributes in large part to lack of funding. It is possible that selective funding by the Bank of critical O&M expenditures would have helped to alleviate these problems. O&M funding would, however, have to be combined with clear-cut measures to secure financial soundness, given the Bank's past reluctance to invoke remedies in the face of noncompliance with covenants. There are also Indications that the Bank's ability to address severe labor redundancy problems in a number of projects (e.g., Pakistan and Mexico railways, Mexico power, Philippines rural roads, Nepal and Ghana water supply) might have been enhanced if it had been allowed to finance at least part of the cost of severance payments or similar schemes. 58. Coordination with Other Donors/Financiers. With the exception of water projects and two power projects in Africa (see below), the material reviewed did not contain much information on ID components financed by other donors, nor did it discuss the way in which the Bank coordinated its own ID efforts with these donors. Since there is prima facie evidence that a substantial amount of bilateral assistance has been and is provided for ID in the energy and infrastructure sectors, as in all other sectors, this dearth of information is in itself probably an indication that coordination with other donors is insufficient and that important ID may remain unfunded (or that in some instances duplication of efforts may be rampant). The water supply lending study does recognize the important role played by the World Health Organization and the Pan American Health Organization in helping member countries develop policies and programs through sector studies and surveys. A similar role appears to have been played by the International Telecommunications Union in the telecommunications sector, but is less well documented. On the other hand, the sheer complexity of the cofinancing arrangements provided in the Botswana and Swaziland Power Projects (ten and five cofinanciers respectively, each with its own procedures and requirements) strained the utilities' limited absorptive capacity and hampered their intended institutional strengthening. 59. Also, scarce reference, if any, is made to the possible role played by the joint Bank-UNDP Energy Sector Management Assistance Program in the energy sector, or by the numerous trust funds used by the Bank for project preparation and supervision. This apparent lack of coordination is all the more Regarding rationale for establishment of IDP, ace *Management of Technical Assistance in the 1990s.* 176 worrisome since the Bank's comparative advantage in financing ID activities is not always clear to borrowers, who are likely to be reluctant to borrow for activities that are potentially eligible for bilateral grants. E. CONCLUSIONS AND MAIN LESSONS LEARNED General Findings 60. This review suggests that issues and constraints faced by the Bank in its efforts to promote ID in the energy and Infrastructure sectors are much the same as those identified in a number of recent Bank reports for all other sectors. As in other sectors, the somewhat broader approach to ID followed by the Bank in the early and mid-1980s (characterized by attempts to tackle ID at the institutionallagency level, in most cases, and the sectoral level in some cases), in contrast to the prevalent project-based approach of previous decades, did not result in marked improvements in ID performance. The same factors, which have already been shown to be at the root of such poor performance in other sectors, were present in the projects covered by the review: (a) lac': of genuine commitmeit on the part of the government and/or Implementing agencies; (b) inadequate institutional analysis at the outset; (c) absence of countrywide ID strategy; and (d) insufficient Bank ID expertise and lack of priority given to ID issues during supervision missions. 61. The review could not identify clear patterns of performance, whether at the subsector or regional level. This tends to reinforce the general message of previous Bank reports on the subject, that in the energy and infrastructure sectors, as in all other sectors: (a) ID issues should be systematically addressed (through institutional analysis) early in the project cycle, and on a case by case basis, particularly taking into account the country's political and cultural specificity, and absorptive capacity; (b) long-term ID objectives should be clearly defind at the outset (albeit with provisions fr adjusting them periodically in light of experience) and the borrower's clear commitment on these should be firmly secured; and (c) the Bank should devote better skills and more financial resources to support the design and follow-up of ID activities in closer coordination with other donors, while remaining flexible as to the means and instruments it uses in doing so. 62. A further general finding of the review is that the coverage of ID in PCRs as well as in PARs was often insufficient, particularly regarding (a) ID activities financed by other donors and related coordination, and (b) possible recommendations for future ID actions. Main Lessons Learned 63. Beyond these general findings, this review of OED work in the energy and infrastructure sectors illustrated a number of more specific lessons: * Government Commitment/Ownership: A broad vision shared by all parties concerned of the objectives of ID (as well as for the project as a whole) is a prerequisite fox success. This lesson was highlighted in most of the reports reviewed, and evidenced in particular by successes achieved in the case of India's NTPC and Turkey's ISKI, and the failures of Colombia's ISA, Ecuador's Guayaquil Urban Project, and China's Three Ports Project (para. 40). (See para. 41 regarding OED criteria used in assessment of ownership.) 177 * Medlon-term lD Sraegy: This broad vision must be translated into a medium-term strategy which the Bank should be ready to support through several (repeater) projects, while remaining flexible in conception and implementation of individual measures. This lesson is best illustrated by the success of Indonesia's power sector, where the Bank financed ID components in a long series of projects (as well as a major piece of ESW) that were complementary rather than self-contained, while all fitting into an agreed M-T strategy (para. 9). Other examples of successful medium-term ID strategies pursued through repeater projects can be found in Turkey and Jordan's water supply sectors (para. 10). Condidonality: ID-specific conditionality, if made sufficiently explicit, can be beneficial in helping the Bank and the borrower monitor the achievement of ID targets, but will most likely be worthless If It does not reflect a clear borrower commitment to ID objectives, as well as a realistic assessment of the borrower's ability to comply; the formulation of conditionality (and its relation to tranching) is all the more important in adjustment operations, as shown by the failure of the Brazil Power Sector Loan due to the lack of a requirement for key upfront actions on sector reorganization (itself a reflection of insufficient government commitment). * SockalandPolkcal Constraints: In designing and implementing ID activities, more attention needs to be paid to possible social, cultural andlor political constraints (e.g., impact of regionalism, ethnic and religious rivalries, influence of trade unions), and the Bank's skill mix needs to be adjusted accordingly (para. 54). The problems faced by the Ban'. in the power sectors of Colombia (regionalism), Mexico (political strength of trade unins), and Brazil (recent constitutional changes) are particularly illustrative of this need. * The quality of management and leadership of local Instituions and the attention paid to personnel issues are key factors of ID success/failure, and the Bank should be more forceful in its efforts to ensure that qualified managers are hired or retained for the agencies in charge of implementing projects. The success achieved by India's NTPC under a strong, stable, and competent management stands in contrast to the numerous examples of ID efforts thwarted by weak or incompetent managers and/or excessive management turnover. It is also apparent that the perennial and pervasive issue of uncompetitive employment conditions is not likely to be resolved at the project or even sectoral level; instead the Bank should try to find ways to promote civil service reform at the country level (para. 31). * Managerial and Fbnncial Autonomy: The best utility managers can be powerless in promoting institutional change if the government is unwilling to provide an appropriate degree of managerial and financial autonomy. Autonomy (with accountability) was empirically shown to be closely correlated with ID performance, both at the level of the Institution (in terms of achieving productivity improvements, effecting organizational changes) and at the sectoral level (in terms, for example, of opening up the sector to private investment and/or some form of competition), as evidenced by the greater degree of ID success achieved in the oil and gas and telecommunications sectors (which usually benefitted from greater autonomy) as compared to the water supply sector. Helping in the elaboration of a general framework for government-Public Enterprise (PE) relations (e.g., performance contracts) can be an effective way for the Bank to promote greater autonomy for public utilities. 178 * New Iinalmron Where favorable conditions exist, the creation of new institutions free of established sector constraints can be a highly effective means of achieving selected objectives or discreet needs (e.g., meeting a growing demand). Howe"er, even when successful (e.g., India's NTPC), such creation is no substitute for addressing deeper and more pervasive sectorwide Issues. In any case, a lack of consensus at the outset on the future role of a newly-created Institution Is a sure recipe for failure, as shown by the example of Colombia's ISA (para. 26). * Compkxiy: Complex design of ID components and/or institutional arrangements for project Implementation is more likely to result in later problems and ultimate project failure, as shown in particular by the examples of Ecuador's Guayaquil Urban Project and a number of gas projects (para. 42). This Is all the more true in poor, unstable countries with limited absorptive capacity. Conversely, straight-forward design and dependence on fewer institutions appears to greatly enhance the probability of ID success, as Illustrated by Ecuador's housing project. Also, the Bank would be well advised to give more consideration to the processing of stand-alone ID projects designed to strengthen weak institutions (or at the very least, carry out detailed institutional analyses) ar&at entrusting them with the implementation of large Investment projects. Sector-specific "institutional safeguards" (e.g., Board of Experts for large hydroelectric dams) should also be considered. * Cjnaindg: Although the net impact of cofinancing is in most cases beneficial, ther. are instances where excessively complex arrangements can be detrimental to the achievements of ID objectives. This is particularly true in small, poor countries where such arrangements may overwhelm the absorptive capacity of the borrowing Institutions, as they did for instance in the Botswana and Swaziland power projects (para. 58). In such cases, the benefits of incremental external fhnding must be weighed carefully against related institutional costs. At the very least, close coordination among donors should aim at minimizing the administrative burden involved (e.g., through joint supervision missions, common progress reports). * Bank Researas and Comminen The Bank's success in promoting ID in the energy and Infrastructure sectors has been shown to be highly dependent on management is commitment (and asinent of adequate resources, e.g., for specialized ID consultants and/or dedicated ESW); appropriate stafag of, and guidance to, Bank missions; and staff sensitivity to ID Issues. Conversely, the Bank's effectiveness has been hampered by its inability to concretely address the labor redundancy and O&M issues, and consideration should be given to designing new Bank policles with appropriate safeguards to replace current bans on certain kinds of finance (e.g., severance payments, selected recurrent expenditures). 179 Annex Table 5.1: Current Regional Classifications of Countries Represented in 1992 Cohort Afdca Bonia Madagascar Botawana Malawi Burkina Paso Mali Burundi Mauritics Cameroon Mozambique Central African Republic Niger Comom Nigeria Congo Rwanda Cote d'ivore Saaega Djlbouw Seychelles Eastern Africa Region Somalia Equatorial Guinea Suda2 EilopIa Swaziland Gambia Tanzania Ghana Uganda Guinea Westera Africa Region Guinea-Bisaan Zaire Kenya Zambia Lsotho Zimbabwe Liberia East Asia. Pacific China Myanmar Indonesia Papua New Guinea Korea, Republic Philippines Malaysia Solomon Islands Maldives Thailand Europe. Central Asia Cyprus Romania Hungary Turkey Potugd Yugoslavia Latle Amrca. Car%bean Argentina Grenada Barbados Haiti Bolivia Jamaica Brazil Mexico Chile Panama Colombia Paraguay Dominican Republic Peru Ecuador Uruguay Middle East. North Africa egypt Republic of Yemen Jordan Tunisia Morocco Yemen Arab Republic Oman Souh-Asia Bangladesh Nepal Bhutan Pakistan India Sri Lanka 180 Annex Table 52:1992,1991,1990 and 1974-92 Cohorts, Operations Evaluated, by Setor and egion Operations Evaluated 1992 1991 1990 1974-92 Agdoultur 87 31 96 35 114 32 973 31 Pow 21 8 14 5 31 9 262 8 Tä~upot 29 10 24 9 28 8 506 16 25 9 24 9 37 10 308 10 dusky 17 6 12 4 15 4 135 4 lianes 16 6 24 9 40 11 286 9 P'e å& li 25 9 13 5 23 6 140 4 T n~ Affl~ 11 4 11 4 12 3 70 2 Uiban 13 5 9 3 18 5 96 3 3 1 7 3 6 2 83 3 Touri~ni - - 3 1 3 1 25 1 Bnrgy 14 5 26 9 16 4 103 3 War&sniean 16 6 IS 5 16 4 167 5 PollutinCat&SolidWai- - - - - 5 0 Regon Afua 126 45 105 38 113 31 987 31 EsAiauandacHo 43 16 40 14 68 19 537 17 SouthAsia 22 8 34 12 55 15 400 13 EurdpenCenbaAsia 14 5 21 8 21 6 258 8 M ~iddleEuasdNortAfioa 16 6 29 10 31 9 325 10 LatinAmmricandCribbmn 56 20 49 18 71 20 652 21 Total 277 100 278 100 359 100 3,159 100 Not.: 8.labfsayau tatalat,dåegslingar. � � � ����� ���у��Н � � . � . � $. is Rt ii► � � �� � +� . S� � � ��'�.�� b у � т �, � � . � � - � � � � � . � � � � � � �' b - � ..a�e.ее .�.�л.. и w ... .и.. в� . � � wAOиb+P • ииАVиА+D�оАв��+«:w�p � � � А О. О � � ffi � 8�8�W� . �8�8�388�ffiЙ8g� �� . о !� � � � «. оо , � «. 00%`13�w г,r".��о: Z3й s'Ь'8$$°У'л~� � � � . А � _ � � rt �ffiй8::`.3 8йR4�''w88 ,g�►�'я'3�8:й3.'Р�°�3 � 17. � � � т8°1�'w°:�' .'k'i•+й`�д':1►i►'$ййй��i � ' к • � � � ей�i�Р.iй � й88�i���i►РйоiРiйй � !д � �► fJ4/A$i ♦ А �j � Ч ы+�0Йwr Wi1в�.+АЙ�8��м�в�Й�в~1 � ��а�х� ��88ga8��8�a8� �� ������z$�к����s����� � � � , М � " �п � 7ь � а N �i � � .. w • :S й � w � и • .в п � � � � а $ п е� � � '� в s� R � s3 � � и•огtа � � � � � м.. и Е 7 7i R s' � � � � � и и w О А$ 0 С � � �i д Ж �'илпдкRяа � м � � � :+ � � и и ♦ а'3 п П а 1." l1 Т� � �` � W s� � � и и.е « А й R 8 ii. R А � � �ia С� � _- и в« п 0 я я А а п• � � � at 3 � � м w н е л g �$ я$« w w и �.: � g п д� б n+ и �s л g, Х: g${ а v е и и N � � � � 4 � и • л'� .и. R Х li 3i 3 w r _ и о� � � i�r � � � и_ л м S'. s я и R lQ п• - и � � � � � .. и л i1 3 А S`. Я:6 $ в_ и Р+' � ёа • �� �+ _'. $ ц! R"J R$'$ w и w м S�i � � � м � � b & и и л= Z д S�j w St и и .. .�'. � .. "' .,� � � � � w л j3 (t я F: « л и• � ® � �в .•. 3 я«$ w w и и r �` я � w - � � • « 4 $ • и и и _ _ .. � � � � � _ л Н �О _ w М вв � � $ W М� '� л л• в и_ .� ffi i�r и и и w r � � - s � � О ii 3 е _.. и и _ ., М � � о й .. � Q _ _ _ „ , . � � _ .. , . А < � ,. _ .. , . � � v's � � _ � � � . . � � at н � а �г &��������$��������� � я � � .в м .в в+ .+ w м .в w. вв в � � G+ � . � � 183 Annex Table 5.5: Distribution of Operations ~»ted, by Sector and ~ont 19~2 im 7 7 3 AVI~ SECAL 2 1 3 3 S Am Devd~ 14 11 36 19 25 c~h~ 9 9 13 11 a F~ 0 3 0 4 4 4 9 l 8 8 D~ 18 24 27 17 17 Livedock i a 4 6 0 OaffAvå~ 0 1 4 5 4 Pa~ CMP 4 9 8 9 6 R#~ -k Ex~ 5 6 9 3 6 sow~ 0 3 0 2 1 p~ 6 16 31 15 21 2 0 8 3 1 POwer 0 3 5 6 7 Oampower 0 0 0 0 1 p~SEM 0 0 a 0 0 R=~ *. 2 1 1 1 l Th~ PO~ 1 7 12 1 4 Tnasmisdu 1 3 3 4 6 Tr~ rt » 34 28 24 29 A~ 0 0 0 1 0 a~ 16 25 19 16 20 pab 3 2 5 3 7 ~»P 1 6 4 4 l Tra~ Fm~ Rdm~ 0 0 0 0 1 Tt~ SECAL 0 1 0 0 0 Hk~ RNOUM 21 19 37 24 25 7 7 a 9 7 ~«Ed~ 3 1 9 2 3 Hm- Row~ SEXAL 0 0 0 1 0 ~Ek~Pn~ 0 0 1 0 1 PO~~ax~ 4 3 a 1 5 ?~~~ 1 3 6 4 a sm~Få~ 1 2 1 1 2 TU~ -minink 0 0 0 2 0 VET 5 3 4 3 5 12 13 15 12 11 11 7 11 5 10 kdu~ S8M 0 1 1 2 5 1 3 3 3 1 Ouffh~ 0 2 0 2 1 9 27 49 24 16 DFC a 11 30 17 11 rm~ SIWAL 0 0 0 0 2 Odufi~ 0 0 1 3 1 184 Amex Table 5.5: Distribution of Operations Evaluated, by Sector and Region, 1988-92 Sector 1988 1989 1990 1991 1992 Sa h~eMediuindsties 1 6 9 4 2 Program4?eney 3 n 23 12 25 EmercyRcovay 0 1 1 1 0 P4grmLOan 0 1 0 1 0 SAL 3 2 17 5 14 SBCALA 0 7 5 5 11 TechcalAscstace 5 4 12 11 11 Urba 5 13 18 9 13 W.UrbanDevelopment 0 3 3 1 2 MunicipalDevelopment 2 0 1 0 2 OthrUrban 0 0 3 2 0 Regional Urban Development 1 2 0 1 1 Shelter 2 7 9 4 4 Urban Emergency Rehabilitation 0 0 1 1 3 U RbanSECAL 0 0 0 0 1 Urbn Tranport 0 1 1 0 0 Teleenmnntenanls 4 9 6 7 3 Tourism 0 2 3 3 0 Eurgy 6 22 16 26 14 Coal 1 2 4 1 1 EnerySECAL 0 0 0 0 1 ExplorationPromotion 1 8 5 4 5 NaturaGa 1 1 1 1 1 0il 0 2 0 2 1 OtherEnergy 0 0 0 1 0 PetroleumJa0atmucture 2 8 4 14 4 Renewabe&thr 1 1 2 3 1 Water& Sataton 16 10 16 15 16 RumWater 0 1 0 0 3 Sewerage 2 1 2 3 0 Water Supply 7 3 8 7 11 Water SupplySanitation 7 5 6 5 2 egion 1988 1989 1990 1991 1992 Africa 44 62 113 105 126 East Asa aud Pac 24 65 68 40 43 Suth Asia 20 44 55 34 22 Eurp and Ceutrd1Ada 12 20 21 21 14 Mdde astaud North Afica 27 24 31 29 16 Laie America and Caribbean 42 48 71 49 56 Total 169 263 359 278 277 g�у рΡ g� s � � � ��q • t� � F �i Я � $ /f ������ � ���� ���� ����� ���� ����� ������ � � � � у �� � ���� � � �� � �� � � � � ��� � �. � 4 � � � � � � а � .. � . .. .. .. .. . �► А w s �" . .. .. w 00 . r и и А ь. . ... • . .. и о .. � • о р и «. . в • � �р • а • и ♦ • r p+n, м « � � . .. r . r . 40 W и и р . r . .. ы . .. и . 0 ► . . и . . w у1 r • �в а � r .. . . . .. р . . • и и .. .. • .. . �" � Q� + О °��°�°f'�i/fi���' �°�°�iA°��°.��oogooll���.%А�1���ооо��оо�91l���и°1Q � bрΡ • .. . .. . . •. w рв r . „r . . . .. рв .. . . и W «. и . . . .. . р . . . • р и и . . .. . q .. w . • .. � .. м .г � 'L° � � Q .- . .. • в м W . r в 1м . • в r rr .. в • .. Ч «. и • . в «. в р в • . • р и и • . .. • р ♦ и . .в .. .. . дв .. .. r � �. �. . ��. �. �. . . �g�. . и°i��. . . �. $. . . �8��. . �• g• �• ���в ��гв� �° � � О� м�рΡ . . r в .. . W . . . р . . . . р . . . и Ч . . r . . .. . Ч . в . . р . . r . s.. . Ч . . • w .. . r . w м т V! . . в+ . r . Ч , . . р . . . . р . . . в. fr . . .. . . w. . � . . . . р . • . . � . 1в . . . • r . . . , .. и .� � � • ♦ $ • � в р . . • . в . в • . • • в � � в . � • в `з . р в . • в . . в • . '$ . � . . в � � . • • � � � �уΡ. С С � . . в . . . р Мв . . р • . . . М . . «. и W . . . . . . . р . . • и W . . . . .. . ►r . . . .- .. . r и . л tq �� � . . . . . . р . . . р « . . . ,r . . .. и W . . . . . . . р . • .. и W . . . . . . р . . . .. . . � и . . р � � . . . . . . , , . . . � . . . $. . � � $. . . . . . . . . . � � $. . . . . , о. . . � . . � � . . � . $ � ....в.р,..rЧ....р.....рв.....в.Q.в.....и......р...и«.иии..0 �� � . . . . . . О . .. .+ Ч . . . . р . . . .+ 1.� . . . . . . . р . . ., ь. м . . . . . . р . . . и � и ьв и . . � �• � . . . . . . , , 88$. . . . . . . . �$. . . . . . . . . . $$8. . . . . . . . . . fiв�в�а�. . $ iffC �i . и .. • . .. А А w ы О► ... . .� • О► . . «. .. р,1 .r . в .. .. . . и . . .. .в А в ,.. . .. . .. у� . � . и . . .. ♦ .. • �О � � . и .. . . . W У w .. р .» . .. и р . . r . ьв и . . . «. . . W . . «. и Ы . .. . r . . Ч . . . .. . . .. . .. . Ч У � . � � . . . йй� и °.i� . � �g °�. . � . j�►9 . . . � . . �. . � а �. � . ~g . . �. . . �q . . я . � . � � �® • • . • . . �� • В . �; � ..т, . . . ." . « . '�r � ..� . . . ..... .р . «« й !� • . . . . . . . � . . . 8i � � .. р . . . . . . . р . . . в� ��ii •р . . . . . . . р . . . � Ф � • . . . . . . . • . . . Р . � � . р . . . . . . . р . , . ."�i VVW�!! �� • О . . . . . . . р . . . .дв � �$•��..в..... � А. � � и � . .. и . . .. . 0 . . . � � р. - � r А . .. .. . . .. . р . . . � - � .8�..�•�•���• х � • �1у и • • .r • и • Н .в • • 17 �МММ � � . ,(� .. . . .. . .. . Н .. .. . � а � • }� е�ае$ее��$о�л �е! � � • � • • • в • • r � • ♦ • h • � s . � ь • 1 • . .. r а « А • f� � � � ов � � � � ' � � К н � а 4� ��� � ����аl�+���� . . � �' � � - � � � � ��� � � � � � ��'����� ���� �������� �������д ����� �������� � � у � � � � � � М М � � ���� ��� � ��� � ��� �� � � � � � М � � . � А � � � � � � �1 М.� и и�о и»��и а а ц��о и� а w. . и��.+.. � tj а и и�а g м. .� а�и м ffi л У О1 Li �" t� �'p "� #А b и . � w .. . а � и в. и и �" а ► й .� • . . и а � .. .. � �+ � .. w $ м g • . . L', а � и и YS в3 » м у', и а $; w и �А" � � �°�ll4e� ,�+�а1О3�°•i�4iSi1й•а92! ое�8 � . ��й�овваs�о�� ��а�в���а�:аа� и .. и+е . . ц �м и . �, vа м й • � . . G p �. . G g "J . .. �r � • . . t'+ в► в�+ti й 1� и r 8 м и � � и 3iij � � � и .. ри . . ь� �►в .. • � Аа r м ♦ • . . � r �, . r й 1! • ' �jj•� g • в . а r йr о g и ♦ О �r • g д r � �� � � о "в��4а. . в8���• ����`в�в�. . зй�• � з��• • �й��• • �вg9����г�sзха�аw � � рΡ~рΡ � и � и . . � � .. . . � а и . . и а . . g .. а . . �s w ц . . � w а и .. . и и � и �i • ♦ w � .а ♦ С G в }i $ +•� о g и �w . . ц �• . . yr v.. . . ы и . . и .. �+. . а и g . . �а �в м .. . и и �. � • и в � л и � М 31б•ii� � �ё�е. . ��ё. . ���. . �g. . ��Е8• • as�• • �а��а�• ��88• R�a�soгs�a���� .. .. W и и . � � .. . .. а � е . .. . . . . и в G . и $ w g . . w в н . . . • r � • .. .. .. й �„' ♦ . i„ вr •• • `�i � .. «. 00 и и . � � .. . • � '�" в . .. . . . . и ы о . и .� и g . . � а и . . . а и '�" . . .. .. g •� и . : а . и � � � �'4 �йffiй�• �г88�в• �г��#. $. . . . ��р�. �а�� • • �а��. . . а��� • ййа:г�• а��г• •� � • � � . . р • .. . х: � . и •- ",i �' .о . • и �в . ь. • а 'А .. . а �о а . . � о й и . . и и � . . .. • , и !! .. w pi �i w и р � � � . . А w .. . � � . ы . g '�и д . в. «. и . . и ц ��1 » . и �о !3 • • В w $ в • . и и а . . . . .. & .. w i'n ч и w � А' . . $й� � вffi• аг • �s�� • ��а • • иа�� • ��� • � �йаs • . ���. . . . ���цsк�х� �� � о д аи �: м..а�. ,...«��� . g наи . ..ii�r• • i� 1�.+w$i"wв • ..'•11:в�w♦♦и�tl'ииУК• а� � миg.еа..��.....С,'�'�•�а•и..=�..аS�им"�Gii....к��ии..иL'iC,....1Lб"'м� � ай.°i��в�ай• а��8дп • ��г�� � • ��• • е�а�g��аег • • gвх��ага$�гте:гххка��g � с � � � �� у �у � � ������° �$� $� � ������ ���.�� " й �� � � �� �� �. �� � � �а� � � � °��� � � � � � � � � $ �' � � � � � � � � � � � � v $ и и �! и ,+; . .. . ' . и � А �j . • и iл, .. .. .. и �' ю •р ,i3 •• � М 1 и$ и . � и •Мм . н . �' . и�' W р в • и '�' • .w п и Q а L i�p ,r � N А �р � � !.° 9 ° й $ 8 ° � ° $ ° $ t�ii �i � ° ° 8 $ ° � � 8 .�i +� = м � � о � - yt 1r .• �•.. 0.. и... и и. а V Ч v и .� . G и и.+� д и а• �{, � " М 'у}•в ♦+ • у • ... � .. и . r и м . л � � V w .• . � и и . • # ЕА и • . fA � � v+ о '�° $ • $ 8 8 �� � ' � � $ • g SS' $� 8 ' 8 8 � ' 8 $w8 9 ' � - �С ~ 1� � � • • • � в .е . w. М м •+ М � рв v Ч • • • . . r М 1iQ �Q .в • • О � 'VV14/ � /�'��! f+ • а и . . � . ао . .. и и .. .. � ►+ „~j .+ ь . .. . . .. • е1 д0 . w . � в� / г�� � $и• • �• $. �"$"в'в"яй�g$. . 8. . $$��. �. А � {� ►•► и а �+ • � У а в .• N .• • • � W в+ . • в r • .. • в и М • J • Ф О�, Ч и и и . � .. а . . и в+ . . р У� М+ . . . .. . . . . м в . а . � � ,i'i"� о � '�"� � $g g $ . r$ � . . 8 � . . ��$. . . 8 . . . . � i . ц . . � иу � � •f ,л а .. �в .. и » . а м . . �' А в0 и . . и . . . и СО W и� .. . ё �у � � � Ч g�" и r Q• N вв . а М 1 • Ч А�С1 r♦ в и.•. и в1 IJ У w. • iы� �$�иt Qв N� � $ 2R Et $ м� � $ ' 8 8 . . ж�$g . . $ . . . 8 ëi.�n� _ • о. f$ {� �'$ й" и r�.' в М . М в и • М� в.� А• в М t'j М и и•��� j. � %A � ii �в .' " � • ♦ в r .. м • в+ о .�i � • • и +О в м М в и' �О �О 1в в � �•�вш��� s• ья$• ��'"�"$• ��гн$а• ойаес• ОУ� •. � 189 Annex Table 5.8: Appraised and Re-Estimated Economic Rates of Return on Operations, Evaluated by Year of Evaluation (1974-92) Number of Average ERR /a year Operations Appraisal Completion Gap (NO) N) NO M) 1974 31 16 17 .1 1975 22 Is 17 1 1976 41 is 16 -1 1977 70 is Is 0 1979 55 20 19 1 1979 94 20 Is 2 IM 52 21 16 5 1981 69 19 14 5 1982 77 20 16 4 1983 97 22 14 8 1994 107 21 12 10 1995 111 25 17 8 1996 131 21 16 6 1987 98 29 16 13 1988 95 26 16 9 1989 117 21 14 7 1990 127 22 12 10 1991 118 25 17 9 1992 98 25 15 10 Total 1580 22% 150/0 7 /a tkwWghud xvmga bwd an pudeeft for W i& both the affnisd wd w-adimded ato of - - , m swMI& ExcludW p4coft htaht& DFCa, SATA tedz&d asddnn pop"= b=M wd w&dt^ Wuutia% pwsmm IoWA Sa &VwAtwg UWIL � ��.��� �' � ��Q� � . а � � � � � � е� �� �� о � о !�I � � �н� �� �� � �� 8Ж ,� � �. ... � � гл р�„ •°* й l����3� �� А ё � � ё Г/� Й ЙJa0o1�i ��� � � ё де ,�Ь�b+С►� ��� �л � 191 Amex Table &10: 1992 Cobort: mentadon-Täme Variadons in Evätuated Operadom. by 5~ amd ~on ov~la 09~1b Undw~ N«b« of Av~ No~d N~ of Ave~ oD-era_uon_a Over~ -92~ . 0~ om Ujiderrm (NO M (MO (XL) N Avi~ 80 60 6 24 pma 19 65 2 Ta~ 26 66 2 1 22 23 66 1 11 al 5 1 9 7~ 11 176 5 P~ AP~ 14 81 10 51 Tk~Affi~ 8 52 3 Udm 10 49 2 18 i dm 3 69 T~ z~ 12 90 2 32 W~&soa~ 14 55 1 5 Po~ com a ed W. 1~. Affim 107 73 17 2 14 EndAdaaud~ 34 53 8 1 a ~Ada 20 66 2 36 B~ #ad cc~ Asia 10 63 3 1 22 Uddia Bed ud Nw* ~ 15 50 1 9 45 86 10 1 st 231 7091* 38 8 24% 192 Annex Table 5.11: 1992 Cohort: Implementation-Cost Variations In Evaluated Operadons, by Sector and Region Overruns On Target Underruns Number o Average Nuber of Number of Average Operaons Overrun Opao Operatons Underrun Sector Agricultur 21 30 3 57 33 Power 8 35 12 20 Transport 10 23 2 14 18 Humn Resourcm 10 44 1 10 30 Industy 4 44 2 8 25 Finance 7 105 5 2 15 Proga 8 10 7 4 26 Te~ Am- 2 7 39 Urban 6 33 1 5 40 Tteeonmnniention 2 79 - 1 26 Tourism - - - - - Energy 4 11 1 8 23 Water&Santaton 4 29 - 11 29 Pollution Contr Sol W. - - Region Aûica 43 29 8 51 23 EastAsiaandPacific 12 31 1 27 29 SouthAuia 9 39 1 12 33 Eup and Centrl ia 3 21 1 9 33 M~ddlEästandNrthAfica 3 24 4 6 47 LatinAmericaandCaribbean 10 56 9 34 32 Total 80 33% 24 139 29% - No0pendin Ngte Exlds 34 qwa-.uforuMoh oou data wue not availabo. Annex Table 5.12: Average Project Indicators by Year of Approval Averages by Year of Approvul Year of Nuber of PCR/PAR COammet Cancelation Project Cost Completion Dates Clouing Evahatl Approval Projects Rating ERR RER Amount Amount Estimated Actua Estimated Actual Date Year 1958 1 100% na na 29.00 0.00 47.00 46.70 na a Dec45 74 1961 7 86% 15 15 23.04 1.06 49.66 53.69 Sep.65 Jun-68 Nov8 75 1962 1 100% 17 17 10.80 0.00 13.40 22.80 na a Dec-72 74 1963 3 100% 16 22 14.67 0.00 24.37 36.53 De-66 1 70 6un-72 75 1964 14 79% 12 13 24.17 2.69 63.63 93.51 May.9 Feb75 Sep-74 77 1%5 14 86% 15 15 24.86 1.49 51.09 64.15 Oct69 May-73 May-74 76 16 26 96% 16 17 16.83 0.73 38.49 43.56 May-70 Jn-73 May-73 76 1%7 35 83% 16 12 13.58 0.92 34.% 40.05 Sep-71 Nov-73 Feb.74 76 1968 69 84% 17 16 14.94 1.03 47.67 71.32 Aug-72 Jul-74 Jan-75 77 1%9 97 89% 18 17 16.14 0.69 34.86 40.04 Aug-73 1y75 Dec-75 77 1970 116 82% 21 21 19.65 0.49 44.89 65.91 May-74 Aug-76 Ot-76 78 1971 117 84% 19 16 21.11 0.53 53.63 69.35 Apr-75 Der.77 Feb.78 80 1972 142 85% 19 16 20.86 1.02 68.85 95.85 Jun-76 De-78 Dec-78 80 1973 170 85% 20 13 24.21 1.29 62.87 84.31 Jun-77 Dec.79 Dec-79 82 1974 180 73% 23 13 25.14 1.05 72.83 103.08 Feb-79 May-81 May-B1 83 1975 184 77% 22 15 32.94 2.09 114.36 141.54 Sep-79 Feb~2 Feb-2 84 1976 211 76% 23 13 29.90 2.17 85.65 92.19 Oct-80 Apr-83 Dec-82 85 1977 215 75% 22 15 31.01 2.31 86.72 91.52 Oct-81 Jan-84 Nov43 86 1978 242 64% 24 14 39.76 4.23 110.15 115.46 Nov-82 Mar-5 Dec44 87 1979 226 68% 24 15 40.99 3.41 124.37 125.03 Dec-83 Apr-6 Jan-86 88 1980 250 72% 23 15 48.71 5.22 155.77 139.17 Nov44 Feb47 Nov-86 89 1981 211 71% 23 14 50.61 6.74 156.05 125.88 Aug-5 Aug47 Jul-87 90 1982 216 67% 27 16 50.99 9.20 157.28 129.98 Ju~46 Aug-88 May-SS 90 1983 173 71% 24 18 59.81 5.93 141.65 127.12 Jun7 May49 Apr89 91 1984 77 73% 32 22 64.68 4.69 174.12 177.56 Mar-8S Ju49 Sep89 91 1985 74 71% 20 15 60.48 5.11 114.48 140.39 Ju149 Apr-90 May-90 91 1986 41 54% 18 24 128.59 2.84 617.17 713.34 Jun-89 Mar-90 Nov.89 91 1987 29 83% 60 54 125.76 2.07 140.65 137.20 Oct49 Mar-90 Mor-90 91 1988 12 83% na ra 148.14 0.09 253.97 256.88 Apr-90 Dec.90 Ot90 92 1989 5 80% na na 217.52 0.12 270.43 271.18 Sep.91 Jul-91 Sep-91 92 1990 1 100% na ng 25.00 0.00 25.00 25.00 Da Da Mar-91 92 Overan 3159 75% 22 15 39.32 3.39 107.78 11102 Mar41 Jun43 Sep43 86 Annex Table 5.13: Average Project Indicators by Year of Evaluation AveraRes by Year et Evaluation Eval.ISUn NUmberet PCRIPAR Conuient caouai~ Projec~Costs Appr~val comp~etio Dates Casig Y nar Pr~cas Rating ERR RERR Amount Amomnt Esimated Actua Date Z~mAtual Dt 1974 50 86% 17 17 18.35 1.01 5218 67.01 Jul46 Nov49 Mar-71 Ap-72 1975 57 88% 18 17 21.05 0.54 49.66 63.38 Ju148 Jan-72 Apr-73 Atg-73 19 10 86% 16 17 18.95 1.49 42.49 49.09 Mr49 Jun-72 Nov-73 Ang.74 197 109 90% 18 18 20.19 1.39 73.42 77.51 De~.9 Feb.73 Nov-74 Jul-75 19 97 91% 20 19 1631 0.73 29.82 38.38 Jm-1 Aug-74 Mr-76 OcV76 1979 130 88% 21 20 21.61 0.60 60.61 80.09 Sep.71 Ja-75 ~m.77 Derm77 198 87 89% 22 17 18.38 1.01 414 57.06 Sep-72 FeM6 May-78 Sep-78 1981 108 85% 20 15 29.83 1.05 73.24 100.21 Jan-73 Dec-76 May-79 Jul-79 1982 127 79% 20 16 27.26 1.47 51.40 62.40 Apr74 Feb.78 May30 Sep80 1983 178 84% 22 14 20.19 1.23 .4S 6246 Aug74 Jul-18 JaneS Jul41 1984 173 73% 22 12 27.29 1.82 68.20 85.98 No75 Oct-79 Mar42 AX-f2 198 192 70% 25 16 39.73 2.27 100.23 109.20 Sep.76 Mr48 J43 Feb43 196 248 82% 22 16 38.13 3.07 127.66 154.10 May- 7 Ju1 De~-3 Ot3 198 187 72% 28 15 40.08 3.50 116.02 120.10 Ju-78 Nov42 Feb85 De44 198 169 74% 26 17 40.23 4.82 126.28 118.01 Sep-79 Nov43 Oct45 Nov.5 1989 263 70% 22 15 52.26 5.44 143.30 122.38 Apr-0 Ang54 D~o-6 Sep36 1990 359 62% 22 12 57.96 5.4 177.10 161.01 Jul4S O0b-5 De047 0ct47 1991 278 62% 25 17 50.41 6.26 133.05 128.29 Jun-2 Oct86 Mar49 Deo-8 1992 277 67% 25 15 63.62 5.42 173.90 183.45 Ju143 Se87 Dec49 Dec89 Overal 3159 75% 22 16 39.32 3.39 1778 113,8 May.77 Ma~41 Juues Sep43 195 Annex Table 5.14: Average Project Indicators by Country, 1974-92 Cohort Country Averages Number of PCRIPAR Commitment Cancellation Project Co its Country Projects Rating ERR REER Amount Amount Estimated Actval Afghanistan 3 100% 7 -5 11.00 5.64 9.10 10,55 Algeria 21 58% 25 14 52.40 21.43 131.12 1 J,7g Argentina 18 56% 23 12 86.66 13.89 270.27 3ul.I6 Babamas 3 100% 16 11 7.60 0.25 26.41 2i,0 Bangladesh 87 72% 30 17 35.68 4.05 51,94 1?.1 Barbados 6 100% 18 9 8.20 0.73 11.75 18.03 Belize 1 100% 17 27 S.30 0.00 7.10 8.60 Benin 19 79% 27 39 11.21 0.35 15.56 17.42 Bhutan 2 100% 21 12 6.00 1.98 13.47 11.06 Bolivia 28 68% 20 15 15.22 1.05 25.55 2748 Botswana 20 85% 15 11 12.00 2.17 27.83 29.18 Brazil 113 66% 21 14 82.78 8.02 399.02 445.47 BurkinaFaso 24 71% 26 17 7.76 1.38 12.7 11.06 Burundi 23 83% 20 - is 13.53 0.41 12.99 13.61 Cameroan 41 73% 20 17 18.57 1.11 38.26 41) ' Cape Verde 1 100%'. 17 14 7.20 2.16 31.70 25.08 Caribbean Region 2 100% ns 0 25.00 2.66 0.00 000 CentralAfricanRepubl 13 54% 21 10 13.25 0.02 19.27 18 61 Chad 14 30% 13 2 6.68 1.41 10.96 1 3.34 Chile 19 89% 24 23 68.55 2.36 110.03 117179 China 21 100% 32 32 107.07 5.34 322.34 295.83 Colombia 74 82% 24 is 47.32 2.82 106.61 12214 Comoros 7 57% 20 4 4.66 0.32 6.71 6.65 Congo 14 43% 17 5 16.41 0.19 43.97 50.16 CostaRica 24 88% is 17 16.73 0.12 32.79 38.04 cotedivoire 44 73% 21 18 37.43 2.68 46.60 48.38 Cyprus 20 79% 17 20 9.71 2.26 24.83 29.22 DJibouti 3 100% na 0 5.80 0.06 10.66 13.07 Dominica 1 100% 22 17 5.00 0.14 7.20 7,40 Dominican Republic 16 69% 22 12 17.26 1.65 40.93 4.50 EastAfricanCammunit 8 63% 22 24 22.85 0.15 42.80 52.48 EasternAfica Region 1 0% 10 3 45.00 0.32 84.90 79.90 Ecuador 32 74% 19 13 26.49 4.09 38.05 37.85 Egypt 53 79% 27 24 48.68 3.32 167.01 184.77 El Salvador 12 83% 17 17 15.90 0.72 28.02 36.39 Equatorial Guinea 3 33% 44 -1 7.23 0.00 9.63 9.14 Ethiopia 33 79% 18 10 16.89 1.11 35.52 32.87 Fiji 7 86% 17 14 11.96 0.04 28.09 42.40 Finland S 80% 25 14 19.00 2.86 80.50 109.00 Gabon 4 75% 19 19 5.58 0.57 9.73 10.83 Gambia 10 70% 31 16 5.19 0.01 762 R,62 Ghana 33 79% 27 14 28.59 0.11 46.05 54.12 Greece 16 88% 20 19 29.89 7.35 79.41 93.81 Grenada 1 100% na 0 5.00 0.03 7.91 9.55 Guatemala 12 92% 27 25 26.87 0.94 59.77 105.72 Guinea 18 72% 36 33 16.27 1.01 27.22 34.29 Guines-Bissan 8 63% 30 24 10.36 0.06 12.69 13.55 Guyana 14 54% 18 9 8.86 1.04 15.67 18.26 Haiti 19 53% 19 12 13.21 1.13 17.14 18.15 Honduras 25 92% 16 14 21.34 0.65 49.07 53.41 Hungary 5 100% 29 31 128.84 0.00 30.3). 2 7. 196 Amnex Table 5.14t Average Project Indicators by Country, 1974-92 Cohorts Country Averag Number of PCRPAR Cnalumet Canc~fiatlo Project Costå Country Projects Rotmn ERR RERR Amount Amount Estimated Actual lceand 3 100% 19 14 7.03 0.00 27.13 36.90 Jndia 163 81% 25 17 87.31 2.49 236.54 241.20 indnein 138 84% 21 16 66.60 5.11 149.51 130.21 bm 17 71% 17 12 32.00 5.69 99.24 125.32 raq 4 100% 15 14 25.85 10.79 37.87 45.20 arelmnd 8 100% 21 18 19.06 0.27 58.90 73.97 ael 7 100% 22 24 27.14 0.09 72.12 84.42 Jamaica 36 54% 22 13 20.02 1.19 22.56 24.23 Jordan 25 88% 21 16 15.94 0.75 69.18 68.04 Knya 72 56% 22 10 29.75 4.38 43.50 38.98 Korea, Republic 81 91% 23 19 73.87 4.97 239.96 297.10 Laos 5 20% 31 8 10.64 0.03 17.76 17.89 Lehanan 2 100% na 0 19.80 0.97 64.20 65.50 Lasotho 13 77% 22 14 7.02 0.51 12.06 11.49 Lib~ia 32 53% 23 13 8.32 1.10 19.14 18.32 mad r 33 61% 20 12 17.71 0.75 26.65 31.57 Malawi 37 78% 17 12 15.89 0.54 21.31 21.95 Malaya 55 84% 18 14 27.25 4.39 80.50 77.74 Maldives 2 100% 64 25 4.10 0.39 8.24 8.98 Mali 28 64% 31 17 10.73 0.05 18.22 18.88 Manitanin 18 72% 16 6 11.00 0.05 49.76 43.55 Mauritius 20 90% 18 14 11.35 1.64 21.52 19.78 Mexico 67 69% 21 15 124.34 10.13 312.95 312.32 Morocco 58 88% 17 12 48.33 5.91 144.33 145.38 Mozambiqne 1 100% na 0 45.00 0.34 45.00 44.70 Myanmar 20 90% 34 24 26.85 1.27 51.20 55.02 Nepal 33 67% 20 11 12.20 0.75 14.92 16.71 New'ann 1 100% 18 18 16.00 10.51 88.00 71.90 Nicaraga 15 93% 15 11 13.48 0.10 19.79 28.11 Niger 21 62% 25 13 15.48 1.67 19.15 16.81 Nignda 45 58% 19 13 48.86 3.40 112.11 104.49 Oman 7 100% 35 40 12.14 0.08 86.77 87.84 Pakist 75 85% 27 18 35.54 3.86 108.16 125.47 Panama 25 52% 17 10 25.07 5.62 47.57 55.72 Papu~NewGuinea 23 83% 19 17 13.06 0.36 22.13 24.34 Paaguay 22 90% 28 28 15.78 1.57 26.05 26.55 Para 34 45% 25 16 35.36 7.92 77.57 66.36 Philippines 88 73% 23 15 49.78 8.09 77.73 71.65 Portugal 20 79% 10 10 49.32 11.35 369.90 336.95 Republic of Yem 2 100% 16 12 6.50 0.00 18.71 25.92 Ramania 33 73% 16 10 66.19 0.06 391.67 395.73 W~anda 20 70% 24 24 9.99 0.37 14.66 17.53 Senal 50 66% 23 18 12.35 0.84 14.68 14.75 Seychuus 1 100% 18 31 6.20 0.00 16.80 16.20 SierLeono 11 73% 19 16 7.20 0.63 11.90 13.06 Singapore 11 100% 12 17 15.41 0.08 30.35 36.76 Solmon~Islands 3 67% 16 14 2.83 0.28 8.25 8.03 Somala 24 67% 17 12 11.20 0.16 17.88 18.21 Spga 11 100% 15 13 45.06 5.59 210.48 274.34 SiLanka 37 86% 21 16 19.27 4.07 34.61 32.37 Sudan 33 45% 26 11 23.98 3.20 54.48 60.44 197 Annex Table 5.14: Average Project Indicators by Country, 1974-92 Cohorts Countrz Averae Numberof PCRWPAR Commitment Cancellation Project Costs Coastry Prolects Rating ERR RERR Amount Amount Estimated Actual Swaziland 10 90% 19 10 7.38 0.44 16.38 17.44 Syria 16 53% 12 3 30.94 5.09 78.47 97.03 Taiwan, China 8 100% 17 23 29.96 0.90 $4.37 71.25 Tanzania 64 48% 22 4 18.26 0.95 35.87 35.69 Thailand 69 90% 23 19 49.02 6.06 141.95 130.87 Togo Is 67% 33 Is 11.46 0.22 16.21 16.27 Trinidad and Tobago 12 73% 18 7 10.23 2.62 23.34 41.53 Tunisia 61 87% 21 20 27.06 2.20 60.18 56.69 Turkey 71 80% 29 18 73.39 3.93 185.97 221.41 Uganda 17 63% 16 9 27.64 0.11 15.55 15.91 Uruguay 14 64% 21 10 24.48 5.01 62.24 59.29 Vanuatu 1 0% Ma 0 2.00 0.79 3.88 2.23 Venezuela 7 100% 14 16 25.86 5.10 115.08 142.72 ietnam 1 100% 18 8 60.00 0.30 110.00 124.10 Western Africa Region 4 50% na 0 24.53 1.15 109.93 143.40 Western Samoa 3 33% 14 7 4.80 0.80 12.50 11.03 Yemen Arab Republic 31 77% 19 is 10.02 0.65 28.20 34.98 Yemen, PDR 21 86% 20 13 6.57 0.12 15.56 18.36 Yugoslavia 66 82% 20 is 57.08 5.24 208.81 212.52 Zaire 33 47% 29 10 19.46 1.03 54.35 SS.74 Zambia 35 66% 19 8 23.43 2.60 50.48 50.41 Zimbabwe 9 100% 15 19 43.10 3.46 88.36 79.82 Total 3159 75% 22 15 39.40 3.38 107.78 113.02 ia 7 � � s з й s У � s $ ° � ь�К� � й i 3 й � .S s ; й r&g о ь.i� � й � з � � � �' ы'� '� ы ы ы ы�ы �q� ы ывг�.i ы Z.i й'� я.�i '•i ыI� w ы.i ы ы ы��'� �� S � � �+� в� Ч в•в М~ в� М в вв ы У М� У У � ы У { М вi в•i У У Ч.Z ы� ы� Ч �М в ��Ys а "s�a� п��::��i���а"s��гг:��i"sii� "s$: � � ��� � �й�� ������������������������� ��� �p(рΡ SaS,� о с�.�`'�+о Soa'wi"Ь��н���"д-�.'Ч83�о о8Ч'Soë�o Ri.� У �о а �R Ф �1 �V . М в1 вв .� в� вV в� М вМ вМ Ч в1 в� А в.� вв .+ А 7� О О дtl А рФ .в Н О р О О аФ� р? О А д•� бь Q вь ОA О � ? �Т Р �g О j•��1 g1� р �g р Р ��g 1 1� � g♦� О� О Р,�� � +z9s � �3 7 ~77 �i'�R7a����J���ы���i �ы����иi �� `�'�Т� в ы �i .S � м а и�.n .� й л."i .g i� ."'i n R м .� n:. й n.л л w в� :� .�i л w.oi n.°� п -� а У м .в n р нм о о ��� s��и^р. ��� V � � т� �г �к�� �т ��ёт�$�$уΡ�"� �уΡ � x ��� � 7��7 '`���?�7'���`t��7��� �9'����'ц� �°iR � � w �. м r. .д • и &' w й � » °° ^ й � Si .. й и й й °.i ^. .7. rni ° ^ � й '^ н 'ы .^,. а � � `�� �х3 ��i� :���«р ��«R:�°����������a5i 8�г � М А � �� М у в� b О О О.r О и� О N�в Ч н М р О� S О♦ н М р 1r 1д h О в1 р.+ О О о . �/,А� � +++888jjj, О�► Мв•` � �� � в•i w Si 'ы �.•: � Y.i sei si i.°�. у w и а w е:+I �ri й а й�� л:r Si й.д. i: i л '^ ^� .-� � �r ".� " �.i Si н ig й �� д у,'i• °�• м g i� Q '• е�В а ё�.°i. w � Li й $ .°pn й й � w ��"-i М н о� н .ч у',У S� 3"�i� и §'�S � а и �е а +� w � .i а с"`i v и'.i' '�i v v'.� °J й i� и v З v v.`� � З�.о 3�'.3 `� "�'.i Ps 8� v и',3'�' . "О . д�+ � . в� м ,1g'W7�1 У н � О � ы' й � � � tT �` • i � ы � �а�g � �3 °. 3 3 ° ~ р о д � ы ° й ы � � :� � � р ы * ч �3 � � � ы .. ,. � �• � ��зв �у�.,��>� �� ,���, �ы��:�����, „ �' ,�� � у � а"s а.��а�� �� р з�> ��;��$ыы���ыy � ё � -°о т° о е°о о � ва°а( гйуΡ о � й^ii °4 �°q у° г� е°е �,МΡй � ва� йдрΡ�в.�уΡ+ �� ю й�рΡ� й� S ы° л.у V рΡ О J О М д �Ф Н>,!д �� F@ ч}� IqO Dh tl 4qрΡ F М Ф Н 7 �i ы О ыь �О tl1 4 АуΡ С д � k G.i ~ УΡ V й 9 Аa д А 9 oi Ч q рΡ�! � у�q А СΡ4рΡ vvi J О А уΡ� рΡ i11 ы'д Я { � р � 1 W ш4 ы w � �вуΡа р �Ч tl О i•i ыЧ У Л ОФ � д {�уΡ М< С У 0'! >в ' 1 м � .i а (j у � ° ° � q � '� � ы ы ,�0 �°а � � т �уΡ о w° ��° � ы 7 � о w � � � � � 4 � � � ы Мн � .н7i � а " о° � ао гi а� '�I °ё `и1' ��� е'� и°+ й е� � О й�й т д' н b'' о.i и° � ы ы 4цs ta so � :Q .� � ь..а � Rg+ r� / � � а д °v � � �, " `" � � '� о �} У �рΡ '''��� � � � �s, Ф Ав�qq{ Ч � д �� чq1 � ДЧ � д^t 4рΡ�уΡ ОрΡ �д тi �q А 4 Й у� � � о � ,�уΡуΡ О � А �, Т С.-ФqГ � � О 4 М д • � б Ч У О V MwA1 дq.�+ФрΡf рΡ4 4Ч � �! ��рΡq�уΡ р г�� ц О О �1 V l '4 k% .Гь S4 'ДДG�� 'U йf G "! С. N Ф Rt 4д V V ь9 S.r 7{ 7С 05 iY а tд Fь Ч N М С� Ч У'. 1111111111 fil 11111111 lit l til 11,111 l lý l 11.-1 11 l lir rb - x Cs ������������� ���� �������� �. � � �� . � � � �� � � � � � ������ � �� � ����� � � � �� �� � � � � � ��� ��в � � ��q � � ��� �� � � � �j � 4 ' `�рΡ � п{в � 4 У � � � � � � FFF � `�КΡ Q � ОО � Ч � r . Ч М IBRY' � iS7� МУ М rЗW�M � Q � �� " � - � � � � � �. . � �' ,� а � � ллrr�л л л �л � л �rлл ��о� й �i �'в в°и' м�й о Z'в� � ii �r�очй «fr�.✓i�� л � . 3й i V+Г��О Y4i MEi • iL�вГвы4.Свv °�вiУ$•✓p � � �� О О � � � • i� �о 4 ооиоwо е �io Е�' ✓мС.".и« о%oooir �� v% й ✓ь й Si � й�« S н� i �`м й !r iв ii е.°i М � ii ✓ ppw �i � М� вi р � � � Рi ✓ ^уΡ � N � � • О � � L � � � i � � � L/рΡ� � � �I � 5 � Й а � � й �в � � й 'вi йдS в4в ••� � �i� � М � �в iв � � �' а � д н О О 4 м r в w вV. r r Ч i4 q м r М М w � � ������ � �� � ������ ������ � : L �о i� � L . S L � е М� � и М М вi �С 4 n ✓ • ✓ ✓ V в� ✓ ✓ ✓ ✓ М I+ ✓ ✓ М ✓ ✓ ✓ М ✓ ✓ М �в � ОW • рΡ рΡ ✓ 1� О ✓L✓�уΡ ✓ ✓ уΡ ✓✓ М аи �q+ вв � О • О О�✓М О д Ч Р Р � V ri1 w р Й У �Ч А М�� О � � ������ t� �� � й����� ���s��� � � ё ����s� � � � ��в��к к����� � � а � ь к��к� �s � 3��3 �ss�� � ; � ��� 3 � � �� � �� �� � � �� ���� � ����� ��� ��й� ���� ������ �� � � � � i . � �� � . � � � � � �r� ���а� � ��������� ������� ������ ����� � � i М М� С и У+ й М М М М Р М♦ � ���� � � � �� � � � М � � � � � М и � � � � Д о � ! � ��вr ~ и� r � . k� � � . а � � аС Сл лрл�+лл а�w+{['�j! '�'{�+j �}�! аrлqr а л д ��+в С �+ � [$� �j'и ' ♦� р� Ч�� Р Й������• У V W И 9 ва. ��♦�� М У О А� М и �/ L w • �+ л N u йwiiL'vv ёиё'�ii' SL'iCëCBййioŭ:a r�"о о'�i".o�Si'i $�о� �� � � '� б � � О н О♦ О О� О О О О q О О О О О О р о О О О О rв О М О в• р Z! О О У Й �� рΡв� вв ввМΡввт�вriй�вii1в�S!ввwiв1i►11ii� � р�i ♦ У у М о ч М С• М w и w �Т Р М М Й�+ М д М о Р в~.. О О � в♦+ w fO М�ii � �i ,в V О V V 0� Ч О V � 1в О А н~ ви+ М М♦ V�иi1 О V w�! � ������� �������������������� ������ ���� � � � � я � g������ ��L����� �� �С���С����}. �� � � ��� � � � � � о � � « о 8 � v .4�°- � � «'�'. ч � � � «. v .. � � :: � ,8 � � � а � й � ,�i .• °м �iйоw�й Si�вg►�«8я�:й�Y��vvSг�af �.' й���«'8L° iPйii � � � � � � М А � � � � � � 3� R � М � � � � � � й � � � � � �.� � � � � Я � � � т�п aass т пtп s� sп �т �� �� �� �� �� � � � �" � ��� � ���� � � �� ������ � �� т �� � �3��� ���� � � � �� �� r � ��� � �� � � �.���� � � �� � ������� � � � . � . � � � � � т л � � А� �� �� ����� � � � � ������ � � � � � � � х М � � � � � V М ♦ М ♦ � . �j �уΡg ~ � � � � � � М G � М � � i � М � . r � �fVf » � �. �� � � � . � � � � ��� лл Мллrл, л л л лeле�лы�� � �� � �` . М О1 М �� �♦ ��� р� � О�� �С Р� М V У М • л Ом. О О r• М р V у И М r V О� V�1 �О у1 q О О М А � + О � � • � � • � и� � � е w v о.. й.� е о о о о о о м в м м о е.. е и о мч �� ^и� �+ � и p МрΡ �.в уΡ и уΡ уΡ уΡ мz М 01 F О� � М д М М О М О Q •♦ Омi М Р� R •� � М й +� и У S� N q �� А�i i'+ й� й 7�� й О м.У.• ч м +� � 1а� ��ь �� �� й���� � ��� ��т���� � � � � р О О О)� у 1. � Оf� 1У.. М М ' Ь!~в МрΡ1 1Мв 1У. У 1у+ О b М У � � � � Р V � У � О � � � � S 1в � � � � � � � � `1�а � � � О w О r� r О r � О g �1 ь�о д n� м е4' • м О� м м м л м �Qв м м м М м Ув м 1в М♦ М М М М М М hв М м$ муΡ ! � b� V М� И V М��� У М О� М � р М М Ч� У М Ч С ��� �� �� ����� � ��� ������� � � � ��� � � � �� ������� � � � х �� � � � ������ � � � � � � � � ���� � � �'����� � � �'���� ��- . . � � , � � � � ��� � � � � � � Ч. ����� � ��� �' ����r��� °� � � • � й � � р � � � М �� .,��� ��� ы �� � � '� � �� ьо � � ���� а � � � � � М Ч 7 � � у � � » � � � � о � � � � � и �i р ы � vуΡ1 • 6У Ч ;±� � �Ф�в1 � ыёй�'°!!п+!l}8+! ьiLСΡZв�!gц+tSS!! �"► ьл$. РL.�i.лрΡ.!ы�' �"ыыа".oCv'CS'�'ыь �'ь'и" n � � . � в Р а w i i v v i Ч о А У й S� i~� � й о i� а R: ь'�. а w�.i � 8� " �О N М О н }� УΡ �рΡв А �в O вв Ф АΡвbви в`аС� �в � � � О в•• н•� М О� О О вн. О N О М У О О L О Р О �R Й v О �О У �. Ч � F • ✓ W у � � Ч н О 1 н Ч r У М О О М М н rв О О Q Й Ч О О» О н i. ♦ Р р у ih� � и M М �в уΡрΡ� рΡ уΡрΡ рΡ уΡ уΡ уΡ QхjQ� Р О О ��� i V О О О О У М М» v О� О Ч » М Ч�� Л � М V Ч'�О У М ь М М V У и». V V 1нО 1� � О I� М а М У. Р ММ � ����������� � � �а��� ��t������� �� � � � УЧ+ LО Ч У✓� М» Чв+уΡ М О V О р W М М о►»в О О А О � � �gq`����L ��s`рΡт�уΡ � � �� �ls��j ���е����УΡ �� � � � Р� н н �1 ���• О � н М� н О� � в r S� � Ч О� н Ci ы О М О М��� М О у Й Р � � � М О М� А О� ва+ ?� 4� р Р У Р � � й����йя���� � � ����� ���������� �� � � � � п�� � �� ��� ������ а� � � � 3�� �� ��� �� .�3 � � �� ��� �� �� � 〕〕〕·〕！〕〕〕〕’牌！&&&!,;：〕，- � � � � ���� � � � �� �� � ���-i ���' �i�� `' � ' � а � � � � � � � т � � ���� �� �� ���� � � � � � � � � R �. � � �� � � � � � � и� ы � ы � и � � ы�ы/ы�уΡуΡуΡ���jjj � М � � � � у � . � � � о � *' � � � . � iiii � • � � � '"� � ,� а�уΡ p лN V М МΡ н i+ � ���!j+ вА}.�. W }ьq� Ч л л в'�{в ОуΡ� в�. врΡi в�i ��/�j {" '1 / . М м V � О� О О • А О О � Р О Z i♦ V� М р 1ив � � А � #в � М� �� О V р ! М �О 4 • М М О О/г М М А♦ r fyA/' � f . � О � � д� � ц ооо ооое ч йо or иео`оее.-еа..М $ �� � v � уΡ и Qи � r S V 1'Zi �r" I� й ~w V в,'. V�' М О О А М� V и• ��в � N � М 4 М д w V А в+ б1 � Ч V Ч V О вМ. �ii М У 1�i i F� � � � л ��� ���� � �� �� ����������� � � � M5)�� УΡP)� L а "G �p� ����дΡУΡв � �� �� ��7НΡУΡ���9RY��`УΡSi� � � q О� ��в 1rв А � �+ в+ в+ в+ 4 О 1У �О А Ч д 10 � 4� в. м в+ rуΡ r r в. в+ в. вQ. вi+ в$. вQ+ нS ы а�уΡ. ыi р �t � М�� W 4 ви+ У • У М �� N У fв • Ч 1в w М v М� ' ��� �r��й й �й �� ����������й � � � � �� � ����s���� � � � � � �������� � r � � � ��r � � ' � An6eg ТаЫе G.1: Operations Evalnated in 199Z, �.�у Sector аад Covnhy . lоаоf0евдls IIsta срвs 2ц tко7веs Овtвв 2вротs tвтlоsв�тв се�sи с�ц. пав.к с�wсл s,м«: ни. rrc в..os к.аео a.s аито..к в>,�.1ев м. �rм о..т.кк 2кsяв х�а�ов.s,. аг�т в.�глг 1� wви ат к к:аг.rsт 2к.r.,г.я1 кож ха ие:вк.ае�т Твlесоттттlсаtiоаа а.n2к.а..Ъ �.eiomr s скик и в 1n 2s.тnиы 2о.лт-я1 tosи тв иц.аnео�д о.т тw�.и.rmsacseo. s lasu 2а о кв9 кк.а�.и ао.л�wsк кота2 see иек,кааsо� аи� � 1 ът2s а кв 2и т.s�г.вк 2о.sт�.и sasx wa ..:s.ки:о� Еnет�г С.од lhвциri 21вs 1io 1,i2otЧ 2 L2N7 !9 Ч тб 2Чips.tt lО•�рк.9к 1070i т'Ct ввеSвlmtm� �� 1во2кадий 2�sp fвssos �ыиаt С199! 1тт 0 1т! кl.вуиN l1..Ои.як 10т00 !0[ иtlKвatos, �� Iauвtoefвl0utop Msaiom lваtег tвoLdeal Мв3вsаоов с120в t 0 2 !0./ог.i2 ц.Dво.и 10т2! тСZ ввtlцаеtеtт �tоуlв r.eralиs �lersetao кrахlао сии i о 1о tв.Лт.вs 21-иат-sк 1кИв vв иеlвtикакт О�иwlвw тrееоlиа 2�рlоквtlоа 2 С1ив ц 0 22 ц.цвк.в.1 и.�.я0 10т2т FCt ицвtааtее, >,.л.р,а.s я.ssок.�о 1op�soeasoa тв�о.ео�tа■ сlокв кs а к2 в.w,ло 2к.п.а.вs 11ви tгs искалаип х� т+кеек.и � я�+� ск2ео я а • 1о.а�и и.о.е.яо копl rae иа.е.�, �tNnntf�a f�1ps �!я Увекокю L22S2 101 ♦ в37 29.цвr.А! fк.4ва-90 цвк7 т+Сt иtiвfaC.� � � aspaetпs Оц вад Qи c:вбSt t�21 100 22 Iи т.Пь1-/l 21.Dи.и 20707 2�CR тиtfвlaatael РвцОl�еаая I01kOStraelае�о � 2� 2� .яа т.tss. а.ц.�. ск2» и кв 2о 3о�г.7.ы i1.о.а.вя 10то2 ке ив2.каоее� Сlfeus Rвкрв� lвtкокв� к�2f2б 1и кк !б3 2яitq-M 21�1far-91 11121 YCt ввtlвlsаиs, lрдцииs 8�af 0!l i�2osвttm С129в к2 0 1к кб.:оr.в2 31�Dи.b кквS2 тQ иtiвlваГоq рКп ?вtкекв� Рtодиае3оа 19оАwомваt к,2193 вк 22 1Ч 1Q.1ис.в! M.JYm.p кц3т 1Ct тввs3дваsат� ЛwwнlЪitdcQl�r iмqa ОвоtЪкввl �еад�оа С1Чi и 0 b-:в��вв 21�Dм.Ч кlвi! !Ct иWtваее� �� � ����� ����� � ��� � � . � . � � � � у � �� ������������ ��� � � � ����� ������ ��� � . � � � У � � ����� ��� � � tl � � ' дд.. � � � � � � g п � . . � вд � С а � � � �уΡуΡ У �i л�$! !! а SC йи �`уΡ S. i+ � � А " aii � О• ТМ• � f6 Sfкв У� r11 О М М V�М • �� �1уΡI �• ] 5 вУв •• У V V О М У r О# О О �О � � � r' V � � ь � . ! � QQВΡB Гw О О О О О q О М О р♦ О а М r О � 1{ М+ a�а� �+'•riс QхвiGУΡJ � •i 1в J i� Ч Р р $ ё� Р•� ai М О М r� � � # Г+ � а �� ����~ ������ ��� � � � �� ..:�..� ������ �.� g � р � � ��g� ������ �..� � � �� v��S� е� �о ё�.� ..�,'�' � в+ врΡ� h� I� вв М М �в М hgв М�Q в+ М М � � � Ое�'! О ОуΡ М А�уΡв�� �уΡуΡ���уΡ Уtl~уΡ• УвуΡ Ч�уΡ� � w � �в � i� Л П�в w �в 1в � П Р� � п п r ��� �� � � � � � � ��3 �� � ��� � �