Report No. 6436 Annual Report On Operations Evaluation (FY86) October 10, 1986 Office of the Director-General, Operations Evaluation FOR OFFICIAL USE ONLY Document of the World Bank This document 6as a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not therwise be disclosed without World Bank authorization. ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) Abbreviations Used DGO - Director-General Operations Evaluation EDI - Economic Development Institute IFC - International Finance Corporation JAC - Joint Audit Committee OED - Operations Evaluation Department OPS - OperaLions Policy Staff PCR - Project Completion Report PPAR - Project Performance Audit Report FOR OFFICIAL USE ONLY THE WORLD BANK Washington. D.C. 20433 U.S.A. Ooice of Directocrwal Opuratin Ivalutun October 10, 1986 NENDRANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Annual Report on Operations Evaluation Attached, for information, is this year's "Annual Report on Operations Evaluation." The report provides an overview of the operations evaluation function in the World Bank Group, feedback and dissemination mechanisms and the major evaluation activities during FY86. It also discusses issues relevant to future programs of operations evaluation. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank Authorization. FOR OFFICIAL USE ONLY ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) TABLE OF CONTENTS Page No. I. THE BANK'S OPERATIONS EVALUATION SYSTEM .................. I The Director-Ganeral ................................. 1 The Operations Evaluation Department ................. 2 IFC Operations Evaluation Unit ....................... 2 Joint Audit Committee 2............................... 2 Project Evaluations .............................. 3 Special Studies ..................................... 4 Other OED Activities ................................. 4 IFC Operations Evaluation Activities ................. 4 II. FY86 RESULTS ........... 5 A. Project Evaluations .................................. 5 B. Special Studies ...................................... 7 C. International Finance Corporation .................... 8 D. Other Evaluation Activities .......................... 8 Research ........................................... 9 Economic Development Institute ..................... 9 III. OPERATIONS EVALUATION DEPARTMENT ......................... 11 IV. LESSON LEARNING *.*.*......................... 12 A. In the Bank ........................................ 12 Monitoring and Evaluation .......................... 13 B. Gvurdination with Other Agencies ..................... 13 V. ISSUES ................................................... 14 Project Evaluations .................................. 15 Special Studies ...................................... 16 Feedback and Dissemination ........................... 17 Annex A: OED Reports Issued During FY86 .......................... 19 Annex B: OED Policy Reviews and Evaluation Studies ............... 32 Annex C: Monitoring and Evaluation in Projects ................... 69 Annex D: Projected C1*ntribution of Borrowerp to PCR Preparation through CY87 ............................. 81 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. - 1 - ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) 1. This report is the eleventh in an annual series begun in 1976. It reviews the Bank's operations evaluation system as it stands today (Chapter I), the major evaluation activities during FY86 (Chapter II) as well as management and staffing issues of the Operations Evaluation Department (Chapter III). The last two chapters focus on the effectiveness of feedback and dissemination mechanisms (Chapter IV) and highlight possible new airec- tions for future evaluation work (Chapter V). I. THE BANK'S OPERATIONS EVALUATION SYSTEM 2. Operations Evaluation in the World Bankl/ aims to provide a syste- matic, comprehensive and independent review of the Bank's development experience. Its principal purpose is to help improve the design and conduct of the Bank's development assistance. It also enables the Bank to account to its member governments for the results of advisory support and investments in human or physical capital, and for its efforts to help improve development policies, institutions, and approaches to development problems. The operations evaluation function in the World Bank is organized as follows: The Director-General 3. The Director-General, Operations Evaluation (DGO) has overall responsibility for the evaluation function. As described in Organizational Manual Statement No. 1.70 this includes: o assessing whether the Bank's programs and activities are producing the expected results; o incorporating OED's assessments and findings into recommenda- tions which will help improve the efficiency and effectiveness of the Bank's programs and activities, and their responsive- ness to member countries' needs and concerns; and o appraising the Bank's operations evaluation system and reporting on its adequacy for use within the Bank and by member governments. 4. The DGO reports to the Board and has an administrative link to the President. He is appointed for a five-year term and can be removed only by the Board. To further ensure his independence, he may not join the Bank staff upon termination. The incumbent took office March 3, 1986. I1 The term World Bank includes the International Bank for Reconstruction and Development, the International Development Association and where appropriate, the International Finance Corporation. - 2 - The Operations Evaluation Department 5. The Operations Evaluation Department (OED) is the staff arm of the Director-General. It is headed by a director appointed by the DGO. The incumbent took office August 15, 1986. The principal functions of OED are to: o assist the Director-General in making periodic assessments of the adequacy and effectiveness of the operations evaluation system in the light of the objectives and programs of the World Bank; carry out performance audits on selected completed projects and to conduct evaluation studies and operational policy reviews; * help the Bank encourage and assist member countries to develop their own operations evaluation system; assess actions taken by the Bank on the findings of OED studies and report thereon to the Executive Directors and President; and o help disseminate evaluation findings regarding Bank opera- tions both within the institution and to the wider development community. 6. OED has a staff complement of 60, including 39 higher level staff, currently distributed into a front office and three divisions: Agriculture and Human Resources; Industry and Policy Review; and Energy, Infrastructure and Urban Development. IYC Operations Evaluation Unit 7. IFC's Development Department is re3ponsible for the operativns evaluation function within IFC. It carries out this function through an Operations Evaluatiou Unit (OEU) consisting of two higher level staff and two research assistants, under the general oversight of the DGO. OEU maintains close relations with OED, consulting with OED staff at various stages of the evaluation process and observing OED's criteria for independent evaluation. Joint Audit Committee 8. The Joint Audit Committee (JAC) is a standing committee of the Board of Executive Directors. Its responsibilities include, inter alia, reviewing the adequacy and efficiency of the Bank's operations evaluation procedures and reviewing selected OED reports to identify problems or policy issues for consideration by the Executive Directors. 9. In addition, since 1975, the JAC has established a subcomrittee to review a number of project evaluation reports and to assess the adequacy of the project evaluation system in the Bank (see paras. 25-26). -3- Project Evaluations 10. The Bank's evaluation of project performance is a two-tier process. The first assessment of a project experience and its results is normally made by the operational unit concerned so as to assure a close and continuous link with ongoing and future operations. Operational staff are required to record their assessments of all completed projects, without exception. 11. The outcome of this process is a Project Completion Report (PCR). Guidelines for the preparation of PCRa are contained in OMS 3.58 of 1977 which, since 1982, has included guidelines for preparing PCRs on structural adjustment loans. A public version of these guidelines has been available since September 1983. A general revision of OMS 3.58 is currently planned by the Operations Policy Staff, working in cooperation with OED. 12. All PCRs are then reviewed independently by the Operations Evaluation staff and either "passed through" and forwarded by the DGO to the Executive Directors and the President (after soliciting and incorporating any comments received from the b-rrowers or co-lenders), or used as part of an independent evaluation of the project by OED. The department's evaluation includes an examination of the files, the Board minutes, interviews with operational staff and, in most cases, visits with the borrowers and beneficiaries at project sites. The department's findings are then written up in a Project Performance Audit Memorandum (PPAM), which is circulated (together with the PCR) to the Executive Directors and the President as a Project Performance Audit Report (PPAR), after recording any comments received from borrowers or co-lenders. 13. Since 1977, borrowers have been required in loan agreements to prepare their own PCRs to foster feedback and lesson learning through self-evaluation. Co-lenders are also invited to participate in the evaluation process. 14. Until 1982, OED made an independent evaluation of all PCRs; since then, under the pass-through system, about half of each year's PCRs are selected by OED for independent performance audit, in accordance with criteria laid down by the Board. The share of PCRs audited varies by types of operation; for example, SALs and sector adjustment loans are all audited. The implementation of these criteria and the suggestion by several Executive Directors that consideration be given to reducing the proportion of projects independently audited by OED is discussed in paras. 56-57 below. 15. Every year OED prepares the Annual Review of Project Performance Results, which provides an overview of all PPARs and PCRs issued during the preceding year. Findings are analyzed along with patterns of experience and their implications for future operations. The Annual Review is also released to the public (para. 27). -4- Special Studies 16. Special studies focus on areas that have been identified as of special interest for the Bank's lending policies and operations. One group of studies reviews the economic and s*ocial impact of projects several years after completion; to date, OED has istued 27 impact evaluations covering a total of 40 projects. Closely related are studies focussing on the sustainability of project benefits, two of which have been issued (with another three at various stages of preparation). A second group of studies deals with the experience of Bank operations in specific srbsectors (e.g., smallholder livestock; project related training) while a third group reviews Bank operational policies (e.g., institutional development in Africa; compli- ance with loan covenants). 17. The fourth group are country program and policy reviews which not only deal extensively with policy issues but also evaluate the entirety of Bank-borrower interaction in a given country over an extended period of time. Two country studies have been issued to the Board; another four are expected over the next three years. More recently, OED started a new series of studies which address policy issues at the sector and country level, such as the first review of experience with structural adjustment lending and a study on program lending in Bangladesh. Other OED Activities 18. Other OED activities, reviewed in more detail in the following chapters, include participation in the evaluation of research projects and EDI programs, dissemination and feedback within the Bank, and sharing of evaluation experience with borrowers and other donors. IFC Operations Evaluation Activities 19. IFC prepares PCRs on completed projects and, after review by IFC's Portfolio Committee, sends them to the DGO, who has them available for reference by interested Directors. IFC's operations evaluation activities focus on reviewing groups of projects to identify patterns of experience and lessons that may be of broader application. These studies are intended to assist IFC in the selection, appraisal, structuring and supervision of its investments (paras. 31-33). - 5 - II. FY86 RESULTS A. Project Evaluations 20. As Table 1 below shows, the number of projects for which PCRs were received by OED has increased gradually from 159 in FY82 to 197 in FY86.2/ Together with the carry-over from FY85 (178 projects) the total number f projects under review during FY86 amounted to 375. For 218 of these projects, evaluation reports were issued to the Board.3/ The remaining 157 projects were at various stages in the evaluation process and were carried over to FY87. Table 1: STATUS OF PROJECT EVALUATIONS FY82 to FY86 FY82 FY83 FY84 FY85 FY86 PCRs brought forward /a 136 173 184 167 178 PCRs received 159 180 178 190 197 Total (A) 295 353 362 357 375 Projects evaluated: PPARs 122/b 91 111 86 124 Pass-through PCRs - 78 84 93 94 Total (B) 122 169 195 179 218 PPARs as % of total projects evaluated 100 54 57 48 57 PCRs carried forward: (A-B) 173 184 167 178 157 /a At various bLages in the evaluation process. 7W Last year when OED audited al. projects. 2/ Over the next few years, this number is expected to increase further to a level of approximately 250 p.a. See Operations Evaluation Department, Status of Current Work and Proposed FY87 Work Program and Staff Budget, R86-81, April 9, 1986. 3/ All evaluation reports issued by OED in FY86 are listed in Annex A. .. 6 - 21. The number of projects carried forward has been significantly reduced in the past year. This reflects a conscious effort by OED manage- ment--in response to a recommendation by the JAC4/-to avoid unnecessary delays in the processing of PCRs. However, the flow of PCRs to OED remains highly uneven, with close to half being submitted during the last quarter (see Table 2). This bunching inevitably inflates the number of projects carried over at the end of each fiscal year. Table 2: BUNCHING OF PCRs Fiscal PCRs Percentage Received Year Received Last Quarter June only FY86 197 47% 26% FY85 190 45% 26% FY84 178 51% 31% 22. PCRs themselves continue to be delayed in many cases, primarily because of other pressing activities in the operating departments. About 60% of PCR's are received in OED more than a year after final disbursements have been made, about half of this total being received two years later. However, the requirement that PCRs be prepared within six months of final disbursement has been relaxed. Compared to original plans, half the number of PCRs are delivered to OED on schedule, but about 15% are a year or more late. In scheduling PCRs regional staff have tended to plan 20% of PCRs for delivery 1-6 months after final disbursement, 22% for delivery 1-2 years after final disbursement and 16% 2 years later. Such delays have been aggravated since borrowers started preparing completion reports (see para. 13) so that the process no longer remains under the full control of the Bank. OED is reviewing these issues with operational staff (see paras 58-61). 23. Of the 218 projects for which evaluation reports were issued in FY86, 124 weri audited, while for the remaining 94 projects the PCRs were issued after having been read by OED staff. The audit ratio of 57 percent reflects the relatively high number of agricultural projects in AfrIca, as well as six SALs, which were all audited. In other areas, especially public utilities, transport, industry and DFCs, the audit ratio was lower: Table 3: EVALUATION REPORTS 16SUED IN FY86 PPARs Sector PPARs PCRs Total as % of Total - number of projects - Agriculture 50 20 70 71 Human Resources 10 7 17 59 SALe 6 - 6 100 Technical Assistance 3 1 4 75 DFC/Industry 13 14 27 48 Transp. & Urban Dev. 24 24 48 50 Energy & Publ Util. 18 28 46 39 Total 124 94 218 57 4/MO Ro ot= te =vb 4/ Report of the Joint Audit Committee, R85-321, November 6, 1985. - 7 - 24. The cumulative number of project evaluations reached 1545 at the end of FY86. Access to this large and unique source of information is assured through a computerized retrieval system which continues to be expanded. Enlarged storage capacity has provided access to much more extensive textual and numeric information than that contained in the initial classification. As a result, OED no longer issues the "Conordance to Project Performance Audit Reports" which in any case, was more limited in scope and content. The concordance will be replaced by a directory of all OED reports which will be widely distributed in the Bank and to bilateral and multilateral agencies, while access to OED's data base will be facilitated by the Bank-wide spread of new office technology. 25. In FY86 the unit cost to operating departments of preparing PCRs averaged just over eleven staffweeks, while OED spent on average about six staffweeks for an audit and 0.6 staff weeks for "pass through" PCRs. This represented little change over FY85. With a 57 percent audit ratio, the overall cost to OED per project evaluated in FY86 was 3.7 staffweeks. This figure includes evaluations of SALs which cost about 10 staffweeks each. As the number of policy-based operations increases, the average unit cost of OED evaluations is likely to grow, as ling as the present 50 percent audit ratio is maintained. The cost effectiveness and efficiency of the selective auditing system was reviewed in 1984 and 1985 by Subcommittees of the Joint Audit Committee who expressed their satisfaction with the manner in which the system is functioning. 26. In discharge of its oversight of the operations evaluation func- tion, the Joint Audit Committee of the Board appointed another Subcommittee in FY86 to review a number of PPARs and PCRs to assess the quality of project evaluations. The Subcommittee focussed on the experience with SALs and sectoral import loans. PPARs covering seven such projects were selected for detailed review and discussions were held with staff from OED and the operational complex. The Subcomittee's report is expected to be issued shortly. 27. The findings of both PPARs and PCRs are synthesized in OED's Annual Reviews of Project Performance Results. The twelfth such Review was recently released to the Executive Directors. It covers 192 projects that were evaluated in CY85. As in previous Reviews, the 1985 projects have been analyzed in the context of prior results, so that conclusions can be drawn from a longer time series. B. Special Studies 28. Nine evaluation studies have been issued by OED in FY86 (see Annex A). They include two country policy and program reviews (Pakistan and Sri Lanka), a study on program lending in Bangladesh, a review of the sustain- ability of fertilizer projects, one study each on cre experience with smallholder livestock development and with conditionality related to irrigation water charges, as well as three impact evaluation studies on agricultural projects in Korea and Malaysia, and on lending for education in -- 8 - Korea. Work also began on a first review of experience with structural adjustment lending, which was completed and issued in September 1986, and on a study on rural development, scheduled for distribution later in the year. 29. These studies were important elements of OED's work program for FY86 which was reviewed by the JAC in February 1985 and approved by the full Board in May 1985. As the studies are issued, they are reviewed by the JAC. During the JAC meetings Bank staff are usually requested to comment on the issues raised by the OED reports. A summary of OED policy reviews and evaluation studies is given in Annex B. 30. Most of the special studies are resource intensive, claiming a large share of OED's budget. In FY86 some 15 staff years were spent on studies, and during FY87 about 14 staff years have been allocated. This represents about one third of OED's total manpower budget. The cost of individual studies ranges from less than one staff year (impact evaluations) to almost three staff ye-rs (country program and policy reviews). C. International Finance Corporation 31. Prior to FY86 IFC had completed two evaluation studies, the first dealing with IFC's experience with hotel p:ojects and the second with the cement industry. 32. IFC's third evaluation study, dealing with IFC's experience in the non-fuel minerals sector, was substantially completed in FY86 and is to be circulated to the Board in the Fall. Work has also been initiated on two further studies--one on the Corporation's experience in the textiles industry and the other on its experience with agricultural processing, both to be completed in FY87, The latter will be the first of a series of studies on IFC's experience in the agribusiness sector. IFC is focusing its attention on textiles and agribusiness because of the poor performance of its investments in these sectors, the importance of its financial exposure and the expectation that IFC will continue to make substantial investments in these sectors. 33. Eleven PCRs were submitted in FY86, about the same number as in the previous two years, but the gap between the number of project completions and those reported upon continues to increase. IFC's operations Evaluation Unit has brought this problem to the attention of IFC management. D. Other Evaluation Activities 34. While OED's focus has largely been on the evaluation of lending, attention has also been given to the evaluation of the Bank's research complex and of the Economic Development Institute. Reflecting Board expecta- tions, resource constraints and its own comparative advantage, OED has been relatively more involved with the latter than with the former. -9- Research 35. Evaluation procedures were established by the Research Policy Coti.cil to clear the large backlog of completed but unevaluated research projects as of July 31, 1984. This was basically done by limiting evaluation to research projects with over $100,000 in funding from the Research Support Budget. Thus, 38 out of 93 research projects were evaluated by six sectoral panels, each under the chairmanship of a member of the Research Projects Approval Committee (REPAC). 36. For research projects completed after July 1984, regular pro- cedures are being applied, i.e., they will all be evaluated regardless of their size. Nineteen such research projects, for which completion reports were available at the end of September 1985, were evaluated by REPAC in FY86. 37. The results of past evaluations are summarized in the Report on the World Bank Research Program.5/ OED does not have the staff to evaluate independently the Bank's research output. The practice that has been estab- lished, therefore, is for OED to participate selectively by applying its contribution to research projects most clearly relevant to operations, and in which OED has the necessary expertise. Evaluation reports for all individual research projects are on file in the office of the DGO for reference by interested Executive Directors. Economic Development Institute 38. EDI has continued to evaluate individual courses/seminars through questionnaires distributed to participants at the end of the training program. These and other evaluations are covered in EDI's regular annual reports. EDI's evaluations have considerably deepened, partly as a result of the creation of an EDI Evaluation Committee in Jvly 1985, and partly by integrating evaluations more clearly into the mid-term evaluation of EDI's five-year plan, as explained below. 39. In May 1983 the Executive Directors approved a new mandate for EDI to strengthen its effectiveness as a development instrument. A five-year plan was subsequently approved. EDI's FY85-89 Plan gives priority to activi- ties intended to help training institutions in developing countries expand and improve their training capacity. An integral part of the five-year development plan is a mid-term evaluation in FY87. The purpose, scope and methodology of this evaluation, to be carried out jointly by OED and EDI staff, were reviewed recently by the JAC.6/ 5/ The last one was issued in December 1985 (No. 5961), and was discussed by the Board in January 16, 1986. 6/ Approach Paper, Mid-Term Evaluation of EDI Five-Year Development Plan, FY85-89, JAC 86-10 dated June 3, 1986. - 10 - 40. While each evaluation activity is intended to meet particular needs, eight specific evaluation activities will be undertaken that will provide the basis for the preparation of the mid-term evaluation: a evaluation of EDI's China Program; o evaluation of EDI's Activities in the Transport sector; o agricultural management training for Africa; o evaiuation of assistance to institutional partners; 0 country impact evaluation, o results of action planning in EDI-sponsored training; 0 evaluation of seminars for trainers; and a evaluation of training material. Some of these evaluations involve OED while others are carried out by EDI alone or by consultants. The evaluation by OED staff of EDI's work with the Pakistan Administrative Staff College is under way. Another type of evalua- tion that was recently completed in Zambia focusses on the effectiveness of EDI's policy-oriented seminars; a similar evaluation has been undertaken in Mali. OED staff participated in both evaluations. 41. The evaluation of EDI's China program--the Institute's largest country program--got underway during the Spring of 1986 when a questionnaire was mailed to some 1,500 Chinese participants who attended EDI courses and seminars during 1983-85. An EDI/OED mission then visited China for follow-up interviews; the mission's report, which will provide the basis for strength- ening EDI's China program, is being prepared. 42. The results of the eight special evaluation studies, as well as of the review of recent course and seminar evaluation reports, will be incorpo- rated into a comprehensive evaluation report to the Executive Directors. It will draw appropriate lessons from experience, and present any proposed changes in direction for the remaining years of the Plan. The report will be supplemented by a separate OED assessment of EDI's self-evaluation. The report is expected to be submitted to the Board early in the second quarter of 1987. - 11 - III. OPERATIONS EVALUATION DEPARTMENT 43. In FY86, OED filled all vacancies outstanding at the end of FY85. However new vacancies occurred again in FY86 which had to be filled. Table 4 below provides OED staffing details over the last three years. Table 4: OPERATIONS EVALUATION STAFF FY84 FY85 FY86 Higher Level Staff /a Authorized positions 37 39 39 Staff beginning FY 36 34 34 Recruitment 5 9 8 Departures /b 5 9 5 Staff end FY 36 34 37 Vacancies end FY 1 5 2 Support Staff Authorized positions 20 21 21 Staff end FY 20 21 20 Vacancies end FY - - 1 /a Includes assistant level staff. Tb Reassignment, retirement and resignation. 44. Staff turnover has been high, ranging from five to nine higher level staff per year, equivalent to 14-23% of authorized positions. Staff turnover continues to be high in FY87 and staffing profiles are being reviewed to ensure that new appointments reflect the country and sector policy priorities of OED's programs as well as the need to strengthen OED's administration, budgeting and data management. Continued support by Bank management, which guarantees re-entry of staff released from the Operational Complex, as well as efforts being made to increase the visibility of OED's staff a-ad output should ensure continued recruitment of high quality staff. PMD has also supported OED's request to be included systematically in the YP program, starting with a first YP who joined OED last July. 45. For the first time in the history of OED, a departmental retreat was organized in Annapolis in May 1986, which included participants from outside the Bank, external relations, operations policy staff and the regions. This provided an opportunity for all staff to reflect on the external and internal environment in which OED operates. Topics discussed included the effectiveness of aid; environmental issues; the future product of OED; the evaluation process; support staff concerns; and internal issues of personnel management, organization and procedures. This has generated a - 12 - number of initiatives, currently being followed up by actions and task forces, where further study is needed. These processes are helping to further enhance the high motivation of OED staff. 46. Given the volume of OED output and the growing stock of data to be managed, accessed and disseminated, an Office Technology Committee has been appointed to advise on the acquisition and maintenance of adequate office technology skills and equipment, liaise with the Information Technologies and Facilities Department and advise on data management. 47. To improve the efficiency of distribution of OED reports, the report distribution function was transferred from OED to the Internal Documents Unit, and OED reports are now available from the "Reports Desk." IV. LESSON LEARNING A. In the Bank 48. There are numerous instances of lesson learning in the Bank. In Africa, Zollow-up actions have been taken, and others are on the way, partly as a result of OED's review of institutional development there. Special units have been created in the African Regional Offices to provide expert help in designing institutional development projects. In Western Africa, a systematic country-by-country analysis of public sector enterprises has been undertaken to identify needs for technical assistance. The design of a new lending strategy for agriculture in Sub-Saharan Africa is another example of learning from lessons of experience. The new approach is more responsive to local conditions, deemphasizes substantial institutional requirements and places strong emphasis on agricultural research and extension. 49. There are a number of mechanisms in the Bank to disseminate the lessons derived from OED's work. For instance, there are memoranda prepared by the Operational Policy Staff (OPS) which summarize lessons on a sector basis; these are distributed to operational staff at headquarters and in resident missions. The content and frequency of these dissemination papers differ from sector to sector, depending on the number and diversity of projects evaluated, and the number of Bank staff involved. OPS has begun to prepare annual analyses of lending operations by sector, which draw on the lessons derived from OED's work. 50. Another effective tool of dissemination is the discussioa of OED reports. All audits and special studies are submitted in draft to Bank staff and borrowers for their comments. In the case af the Annual Review, special meetings are arranged with operational staff for each sector chapter. Through such interaction, the lessons of experience and the necessary follow-up in terms of operational policies and procedures are clearly identified. In addition, both OED and OPS are increasingly active in orga- nizing meetings and seminars, for both Bank staff and borrowers, to disseminate OED findings. - 13 - 51. At the request of the JAC, dissemination mechanisms and related issues were covered in two papers prepared in 1985: o "Mechanism for Integrating OED Findings into the Policy Formulation Process of the Bank" (JAC/M/85-10); and 0 "Dissemination and Integration of OED Findings in Bank Opera- tions" (JAC/M/85-20). There has been active discussion of these two papers and of related issues that have arisen when other reports have come up for consideration by the JAC. Also, OPS is now requesting OED comments on draft policy papers and guidelines, and uses OED studies as a source for developing Bank policy and position papers. A recent example is OED's study on smallholder livestock development which provided a substantial input into the Bank's policy paper on that subject. Furthermore, after review by the JAC of the Sri Lanka and Pakistan country studies, seminars were organized by the Region to discuss the follow-up to OED's recommendations. In Nigeria, two PCRs provided the basis for a weeklong seminar, attended by senior civil servants, project and Bank staff to discuss the future strategy for agricultural development in that country. Also, as a result of the Malaysia Jengka Triangle Impact Evaluation, Malaysian Government officials expressed keen interest in reviewing experiences in other countries that the study had analyzed for comparative purposes. Arrangements for such visits have been made. Monitoring and Evaluation 52. Lesson learning is not linted to ex-post evaluation. In fact, continuous supervision ensures that feedback takes place throughout the project cycle. Experiences are aggregated through project implementation reviews and reported to the Board in the annual report on project implementa- tion and supervision.7/ At the project level, this has required more empha- sis on monitoring and evaluation. Monitoring and evaluation of ongoing projects has been a concern in the past OED Annual Reports. It has been receiving increasing attention not only in agriculture and rural development but also in other sectors. This is reflected in Annex C, which this year covers not only agriculture but also water supply an- urban development, education, energy and, quite extensively, transport. B. Coordination with Other Agencies 53. In recent years, there has been a growing interest in evaluation among bilateral and multilateral aid agencies. The Bank, and OED in particu- lar, has remained in close touch with these developments so that experience may be shared and cooperative efforts enhanced. An example of this collabo- ration is OED's participation during FY86 in work undertaken by the OECD/DAC 7/ The Eleventh Annual Report on Project Implementation and Supervision (R86-15 dated January 13, 1986) was discussed in the Board in March 1986. - 14 - Expert Group on Aid EvaluatLon and by the Inter-Agency Evaluation Group of the Joint Inspection Unit of the United Nations. The subjects reviewed by these groups included evaluation of non-project lending, environmental protection, and the role of women in development. Another important topic discussed at these meetings was how to improve exchange of evaluation findings through better information access. 54. During the year, the DGO and/or the Director also visited OECD, CIDA and evaluation agencies in China. Several other staff of OED called on bilateral agencies to compare methods of operations evaluation. Staff of the Asian Development Bank, African Development Bank, the Kuwait Fund, Islamic Development Bank, the Commonwealth Secretariat, the UK ODA, the Economic Commission for Africa, UNESCO, and the EEC visited OED to exchange information on the work of their agencies and on the Bank's operations evaluation system. 55. In addition, OED evaluations give increasing coverage to the subject of donor coordination, where despite significant progress much remains to be done. Apart from sharing experience, a coordinated approach to borrowers should also help reduce the demands placed on them for purposes of evaluation. Cases in point are the two country reviews of Sri Lanka and Pakistan which paid substantial attention to the interaction between donors and the impact they had on the countries' development. Another example is the Bangladesh population program audit As many as six cofinanciers participated in the funding of the project and all were extensively consulted during the audit process. V. ISSUES 56. The scope and direction of OED's work program8/ was discussed by the JAC in February and approved by the Board in June. The FY88 program, due in February 1987, will be developed with due regard to the set of issues outlined in this chapter. As a result of the growth in numbers of PCRs and PPARs and the increasing complexity and cost of PPARs involving projects with a poverty or policy focus, project evaluations claim an increasing share of OED resources, which have remained generally constant in the three-year period through FY87. The proportion of OED resources devoted to the review of individual operations and the preparation of the Annual Review and Report has risen since FY85 from about 50% to an expected 65% in FY87, under current assumptions of PCR growth and an audit ratio maintained at 50% (compared with 57% in FY86). 57. One consequence of this shift is that resources available for special studies, which have become a particularly valuable instrument for lesson learning and feedback, may fall to 14 staffyears in 1987, i.e. five 8/ Operations Evaluation Department, Status of Current Work and Proposed FY87 Word Program and Staff Budget, April 9, 1986, R86-81. - 15 - staff years less than in FY85. The suggestion made by several Executive Directors, in connection with the FY87 Work Program and Budget, that consid- eration be given to dropping the audit ratio below 50% is thus welcome. However, dropping the audit ratio without sacrificing accountability and the quality of lesson learning requires that project completion reports be timely and of consistently good quality. This, in turn, will require that OED and Bank staff allocate more time to monitoring and evaluating PCRa. More time also needs to be allocated to interaction with Bank staff and borrowers. The issues involved in shaping future programs are outlined below. Project Evaluations 58. Self-evaluation through PCRs, which cover all completed projects, remains the foundation of the Bank's evaluation system. While there has been some improvement in the quality of Bank-prepared PCRs since selective auditing was introduced in July 1982, there are still wide differences in coverage and analytical depth from one PCR to another. This is partly due to the growing share of borrower prepared PCRs, which has risen to one-third of the total in FY86 and is expected to exceed one-half in the current fiscal year (Annex D). 59. Borrowers' PCRs are 1 neven in quality. While in some countries and sectors borrowers' PCRs are fully satisfactory, in many cases they are not given the priority assigned to new and ongoing operations by senior staff. The greatest deficiencies are in the more complex sectors where institutions are weakest. To upgrade the quality of borrowers' PCR9, Bank staff would have to continue to guide and help borrowers in the actual preparation of such documents. In addition, medium to long-term technical assistance programs will need to be developed to strengthen borrowers' ex-post evaluation capabilities. Such efforts would be staff intensive, exceeding present budgetary resources available for such activities both in the operational complex and OED. The issue will be further pursued in OED's next work program and budget proposal after more detailed analysis and discussion with Bank management. 60. Timeliness of project evaluation is another issue that requires attention. The preparation of PCRs is often seriously delayed (see para. 22). The delays occur not only for traditional investment projects but also for quick disbursing policy-based operations such as SALs, sector adjustment loans or export development projects. The Board has become increasingly concerned about such delays, and has specifically requested that evaluations of SALs and other policy-based lending be completed on time. This will need continued monitoring. 61. To address the issues of quality, scope and timeliness of project evaluation, the following steps have been initiated or are recommended: 0 OED needs to spend more time and effort to monitor and assess the quality of PCRs, to identify areas of improvement, and to systematically feed back these findings to operational staff and borrowers; - 16 - o to enhance the quality of borrowers' PCRs, substantial technical assistance would be required in many countries to help borrowers in the actual preparation of such documents and to strengthen their ex-post evaluation capabilities; a some regions are in the process of introducing more effective medium-term planning for the preparation of their PCRs. These efforts are actively supported by OED and should, in due course, reduce delays in project evaluations; 0 audit memoranda need not necessarily be very elaborate; where issues are adequately covered in the PCR a cross-reference may suffice. There is need to strengthen evaluation summaries, highlighting all important issues raised in the PPAR and bringing out clearly the lessons of experience; a operational guidelines for the preparation of PCRs (OMS 3.58) will be updated by OPS in close cooperation with OED. Issues to be discussed are the format and size of PCRs, packaging of several projects, and subject matter to be covered; and o such improvements are staff intensive and will take effect over a period of time. Whether there is scope to reduce the audit ratio and if so how soon, will be discussed in the context of next year's work program. Special Studies 62. Special studies have become a particularly valuable tool for drawing and feeding back lessons, not only in specific sectors and subsectors but also with regard to the balance of lending programs, the quality of country economic and sector work, and social impact. By synthesizing the experience of past projects in specific sectors or subsectors and by focussin,g on particular issues, these studies provide a deeper insig)- into the factors that determine the outcome of projects such as design, borrower involvement, Bank support, internal institutional and policy environment, as well as external forces over which neither the borrower nor the Bank has control. 63. The nature and focus of recent studies undertaken by OED has been highlighted earlier (paras. 16-17 and 28). Over the next two to three years, the Department intends to carry on with country program and policy reviews, a second SAL overview, impact and sustainability studies, and reviews of Bank policies and procedures. Emphasis in these ex-post evaluations will be given to issues of present concern to the Bank, including experience with export development loans, protection of the environment, the role of women in development, social dimensions of Bank supported projects and technology transfer. - 17 - 64. Demand for such studies has been growing both within the Bank ana among borrowers. This reflects a strong desire for direct feedback - most recently expressed during the Annual Meeting of the World Bank - which can only be met by an increasingly diversified product. At the same time, the principle of accountability through individual project evaluations needs to be upheld, .1iis raises issues of budgetary resources and the proper balance in OED's work program. 65. Consideration is also being given to publishing some of these studies. This would make them accessible to the general public and further contribute to the dissemination of lessons from past experience. Consider- able interest already exists in the publication of the two country reviews on Pakistan and Sri Lanka, and seminars are planned in both countries to intro- duce these reports to a wider audience. Other studies will have to be trans- lated to disseminate their findings more widely in francophone countries. Feedback and Dissemination 66. The ultimate rationale for operations evaluations is to be found in their impact on the quality of Bank operations. In a large and multi-faceted organization like the World Bank, with its global outreach, feedback takes place at many levels and in many different contexts, some of them quite informal. Project design and project implementation are two basic areas where the record suggests that there has been a positive Bank response to the lessons of past experience. However, more needs to be done, particularly in such e ?as as social impact, institutional development, role of women and environmental issues. In shaping evaluation programs and priorities there must be more interaction with Bank staff and borrowers so as to maximize the relevance and impact of the evaluation process. 67. Clearly, in the coming years, OED will have to focus more attention on both internal and external dissemination. Areas where continuing progress is needed to build on past efforts include the following: o closer interaction between OED and the Bank's operations staff. This will not only reduce overlap between the learning facet of OED's activities, and the research and policy function of the Bank's operational complex, but also enhance institutional knowledge through frequent exchanges on project and country experience. Discussions are now taking place to increase interaction with operational staff while maintaining the independence of OED; o wider dissemination of OED findings by selected publication of OED reports, and by more frequent conference participation, contributions to journals and the like; o increased interaction with borrowers so that OED's findings translate into improved evaluation procedures in the field and more focussed use of the lessons learnt through past OED work; and - 18 - more emphasis on the presentation and quality of OED output. More concise reports on individual projects or review of comparative experience will find a wider readership if they are written and presented with specific objectives and audiences in mind. This is basic to improving dissemination of findings among Bank staff, member governments and, where appropriate, the development community at large. 68. All these initatives will be staff-intensive, requiring an effec- tive marshalling of limited resources into priority areas without compromis- ing the independence of OED and the quality and intensity of its evaluation of individual projects, which must remain the cornerstone of the operations evaluation system. - 19 - ANNK A ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) OED REPORTS ISSUED DURING FY86 A. PROJECT PERFORMANCE AUDIT REPORTS Report Report Covering Memo Agriculture No. Date to Board 1. Algeria Technical Assistance-Rural Development 5825 08/22/85 SecM85-985 (Loan 1159-AL) 2. Swailand Rural Development (Loan 1375-SW) 5829 08/23/85 SecM85-1001 3. India Third Agriculture Refinance and Development 5857 09/18/85 SecM85-1111 Corporation Credit (Credit 947-IN) 4. Indonesia Seventh Irrigation (loan 1268-IND) 5881 10/04/85 SecM85-1192 5. Yugoslavia Macedonia-Strezevo Irrigation 5917 10/30/85 SecM85-1317 (Loan 1616-YU) 6. India First Maharashtra Composite Irrigation 5940 11/21/85 SecM86-94 (Credit 736-IN) 7. Philippines Tarlac Irrigation Systems Improvement and 5969 12/12/85 SecH85-1322 Jalaur Irrigation (Loans 1080/1367-PH) 8. Nigeria Lafia Ayangba and Agricultural Development 5972 12/16/85 SecM86-7 (Loans 1454/1455-UNI) 9. Philippines Mindoro Rural Development 5978 L2'18/85 SecM86-95 (Loan 1102-PH) 10. Yugoslavia Second Agricultural Credit 5981 12/19/85 SecM8S5-1345 (Loan 1477-YU) 11. Yugoslavia Agriculture and Agricultural Industries 5999 12/26/85 SecM86-42 (Montenegro) (Loan 1370-YU) 12. Somalia Drought Rehabilitation (Credit 623-SO) 6011 12/26/85 SecM86-149 13. Thailand Northeast Rural Development (loan 1198-TH) 5990 12/27/85 SecM86-118 14. Cyprus Pitsilia Integrated Rural Development 6012 12/30/85 SecM86-150 (loan 1483-CY) - 20 - ANNEX A Page 2 Report Report Covering Memo culture (Cont.) No. Date to Board 15. Ethiopia Drought Areas Rehabilitation (Credit 485-ET) 6018 12/31/85 SecM86-43 16. Cameroon Coastal Treecrop Plantations: Second CAMDEV, 6019 12/31/85 SecM86-91 Second SOCAPAIM and Second HEVECAM (loans 1508/1391T/1392/1791/Credit 975-CM) 17. Malaysia Western Johore Agricultural Development; 6023 12/31/85 SecM86-90 North Kelantan Rural Development; and Coconut Smallholders Development (loans 973/1294/1618-MA) 18. India Uttar Pradesh Public Tibewells 6038 SecM86-185 (Credit 1004-IN) 19. Paraguay Livestock and Rural Development 6126 04/02/86 SecM86-463 (loan 1674-PA) Livestock Component and Sixth Livestock Development (Loan 1979-PA) 20. Syria First Livestock Development 6142 04/14/86 SecM86-474 (Loans 1311/1312T-SYR) 21. Ecuaaor Seeds (loan 1229-EC) 6147 04/17/86 SecM86-482 22. Portugal Agricultural & Fisheries Credit .155 04/21/86 SecM86-481 (loan 1603-PO) 23. Kenya Second Livestock Development (Credit 477-KE) 6160 04/28/86 SecM86-485 24. Morocco Meknes Agricultural Development 6167 05/02/86 SecM86-527 (Loan 555-MOR) 25. Malavi Shire Valley Agricultural Consolidation 6171 05/05/86 SecM86-529 (Credit 823-MAI) 26. Sudan First Agricultural Rehabilitation 6170 05/06/86 SecM86-528 (Credits 1000/EEC-57-SU) 27. Kenya South Nyanza Sugar (loan 1389-KE) 6235 06/06/86 SecM86-676 28. Korea Second Rural Infrastructure (loan 1530-K0) 6219 06/06/86 SecM86-728 29. Qiinea Livestock Development (Credit 1063-GUI) 6243 06/13/86 SecM86-723 30. Zaire Ituri Livestock Development (Credit 697-ZR) 6254 06/13/86 SecM86-731 - 21 - ANNEX A Wae 3 Report Report Covering Memo Agriculture (Cont.) No. Date to Board 31. Pakistan Fourth Agricultural Development Bank 6261 06/17/86 SecM86-761 (Credit 957-PAK) 32. Thailand National Agricultural Extension 6263 06/18/86 SecM86-730 (Loan-1393-TH) 33. Malaysia Agricultural Research and Extension 6264 06/19/86 SecM86-760 (Loan 1115-MA) 34. Korea Agricultural Credit III (Loan 1974-K0) 6218 06/06/86 SecM86-805 35. Niger Irrigation (Credit 851-NIR) 6275 06/23/86 SecM86-849 36. India Orissa Agricultural Development (Credit 682-IN) 6281 06/25/86 SecM86-846 Madhya Pradesh Agricultural Extension and Research (Credit 712-IN) Assam Agricultural Development (Credit 728-IN) Coposite Agricultural Extension (Credit 862-IN) 37. Botswana Second Livestock Development (loan 1497-BT) 6245 06/13/86 SecM86-888 38. Indonesia National Resource Survey and Mapping 6316 06/27/86 SecM86-811 (Loan 1197-IND) 39. Senegal Eastern Senegal Livestock Development 6244 06/13/86 SecM86-780 (Credit 633-SE) 40. Yemen Arab Livestock Credit and Processing 6335 06/30/86 SecM86-879 Republic (Credit 662-YAR) Human Resources 1. Bangladesh First Population (Credit 533-BD) 6303 06/30/86 SecM86-864 2. Indonesia Second Population (Loan 1472-IND) 6276 06/23/86 SecM86-847 3. Jordan Third Education (loan 1781-JO) 6251 06/16/86 SecM86-727 4. Bangladesh First Education, Agricnltural & Rural 6259 06/18/86 SecM86-843 Training (Credits 407/621-BD) 5. Malaysia Third Education (loan 974-M) 6280 06/27/86 SecM86-806 6. Philippines Third Education (Textbook) (Loan 1224-PU) 6279 06/27/86 SecM86-781 - 22 - ANNEX A Page4 Report Report Covering Memo Human Resources (Cont.) No. Date to Board 7. Indonesia Fifth Education (First Teacher Training) 6315 06/30/86 SecM86-878 (loan 1433-IND) 8. Indonesia First Non-Formal Education (loan 1486-IND) 6304 06/30/86 SecM86-786 9. Indonesia Fourth Education (Loan 1237-IND) 6312 06/30/86 SecM86-858 DFCs and Industry 1. Morocco Seventh & Eighth loans to Banque Nationale 6058 02/10/86 SecM86-180 pour le Developpement Economique (BNDE) (loans 1061/1428-MOR) 2. Morocco Integrated Project for Small-Scale 6063 02/11/86 SecM86-181 Industry Development (loan 1687-MOR) 3. Mauritius Second Development Bank of Mauritius, 6206 05/27/86 SecM86-627 Third Development Bank of Mauritius and Fourth Development Rsnk of Mauritius (loans 979/1163/1481-HAS) 4. Pakistan Fauji Fertilizer (Credit 846-PAK) 6214 05/29/86 SecM86-678 5. Romania Brasov Bearings (loan 1436-RO) 6298 06/23/86 SecM86-798 6. Sri Lanka Third and Fourth Development Finance 6321 06/27/86 SecM86-889 Corporation of Ceylon (Credits 566/742-CE) 7. Caribbean First Caribbean Development Bank 6306 06/25/86 SecM86-845 (loans 1233/1234-CRG) 8. Niger Industrial and Artisan Sector Employment 6291 06/19/86 SecM86-850 Creation (Credit 809-NIR) Report Report Covering Memo Non-Project landing No. Date to Board 1. Philippines First and Second Structural Adjustment loans 5813 07/31/85 SecM85-975 (loans 1903/2266-PH) 2. Nepal First Technical Assistance 5935 11/25/85 SecM85-1284 (Credit 659-NEP) 3. Thailand First and Second Structural Adjustment loans 6085 02/28/86 SecM86-276 (loans 2097/2256-TH) - 23 - ANNX A Page 5 Report Report Covering Mem Non-Project (Cont.) No. Date to Board 4. Camroon First Technical Assistance 6114 03/24/86 SecM86-350 (Credit 673-04) 5. )iyana Structural Adjustmnt loan and Credit 6119 03/25/86 SecM86-358 (loan 1948/Credit 1098-4UA) and Second Technical Assistance loan (loan 1949-UA) 6. Pakistan First Structural Adjustmnt loan/Credit 6308 06/25/86 SecM86-822 (loan 2166/Credit 1255-PAK) Public Utilities and Energy 1. Romania First TSrcent 1hermal Power 5838 08/30/85 SecM85-1077 (Loan 1028-RO) 2. India First Bombay Water Supply and Sewerage 5875 10/04/85 SecM85-1144 (Credit 390-IN) 3. Nigeria Fourth 1ower (loan 847-UN1) 5936 11/22/85 SecM85-1300 4. Ethiopia Fifth Telecomnication (Credits 453/554-ET) 6165 04/30/86 SecM86-514 5. Nepal Kulekhani Hydroelectric 6177 05/08/86 SecM86-581 (Credits 600/600-1-NEP) 6. Republic of Secondary Centers Water Supply 6192 05/14/86 SecM86-579 COte d'Ivoire (Loan 1484-IVC) 7. Thailand Accelerated Rural Electrification 6269 06/13/86 SecM86-821 (Loan 1527-TR) 8. Yugoslavia Ibar Multipurpose Water Supply (loan 777-YU) 6232 06/03/86 SecM86-710 9. Sri Lanka First Water Supply (Credit 709-CE) 6228 06/02/86 SecM86-664 10. Ecuador Guayaquil and Qayas Province Water Supply 6289 06/20/86 SecM86-758 (loan 1030-EC) 11. Indonesia The Five CLties Water Supply 6256 06/10/86 SecM86-796 (Loan 1049-5/1049-IND) 12. Yugoslavia thbrovnik Water Supply and Wastewater 6313 06/27/86 SecM86-869 (loan 1066-VU) - 24 - ANNEX A Wge 6 Report Report Covering Memo iblic Utilities and Energy (Cont.) No. Da+e to Board 3. Yugoslavia Sarajevo Air Pollution Control 6231 06/03/86 SecM86-665 (loan 1264-1-YU) 4. Yugoslavia Sarajevo Water Supply and Sewerage 6227 06/02/86 SecM86-812 (Loan 1263-YU) 5. India First and Second Electrification 6307 06/26/86 SecM86-823 (Credits 572/911-IN) 5. Israel Sewerage (Loan 869-IS) 6320 06/27/86 SecM86-886 7. Syria Aleppo Water Supply (Loan 1458-SYR) 6319 06/27/86 Sec86-918 ransportation 1. Yemen PDR Aden Port Rehabilitation (Credit 584-YDR) 5982 12/20/85 SecM86-8 2. India Thirteenth Railway (Credit 582-IN) 6055 01/10/86 SecM86-154 3. Egypt First and Second Railway 6102 03/13/86 SecM86-394 (Credit 284-EGT/Loan 1098- 1) 4. Liberia Fourth Highway (Loan 1573/u. 1311-LBR) 6106 03/14/86 SecM86-318 5. Egypt Suez Canal Rehabilitation and Expansion 6122 04/01/86 SecM86-427 (loans 1064/1482-EGT) 5. Benin Cotonou Port (Credit 826-BEN) 6139 04/08/86 SecM86-426 7. Benin First and Second Feeder Roads 6105 03/21/86 SecM86-328 (Credits 717/1090-BEN) 3. Burundi Second Highway (Credit 773-BU) 6255 06/09/86 Sec86-797 9. Brazil Second and Third Railways 6295 06/20/86 SecM86-844 (loans 1074/1171-BR) ). Guinea-Bissau First Roads (Credit 878-GUB) 6328 06/30/86 SecM86-758 .ban 1. Jamaica Sites and Services (Loan 1003-JM) 5862 09/26/85 SecM85-1216 2. Republic of First Urban Development (Loans 1347/1348-IVC) 6200 05/14/86 SecM86-672 COte dIIvoire -25 - ANNEX A Pag & Report Report Covering Meu Urban (Cont.) No. Date to Board 3. Nicaragua Earthquake Reconstruction (Credit 3(19-NI) 6193 05/14/86 SecM86-612 Urban Reconstruction (Credit 965-NI) 4. Kenya Nairobi Site and Services 6272 06/16/86 SecM86-848 (Credit 543/loan 1105-KE) 5. Guatemala Earthquake Reconstruction 6284 06/19/86 SecM86-825 (loans 1314/1315-GU) 6. India First Madras Urban Development (Credit 687-IN) 6330 06/30/86 SecM86-857 7. Indonesia Second Urban Development (Loan 1336-IND) 6239 06/27/86 SecM86-917 8. Peru Sites and Services (loan 1283-PE) 6331 06/30/86 SecM86-916 B. PASS-THROUGB PROJECT COMPLETION REPORTS Agriculture 1. Haiti Post-Hurricane Agricultural Rehabilitation 5850 09/13/85 SecM85-111: (Credit 1106-HA) 2. Sri Lanka Tank Irrigation Modernization (Credit 666-CE) 5986 12/23/85 SecM86-57 3. Yugoslavia Agriculture and Second Agroindustries- 5998 12/26/85 SecM86-72 Macedonia (loan 1371-YU) 4. Mexico Tropical Agricultural Development 5997 12/27/85 SecM86-45 (loan 1553-ME) 5. Greece Vegetable Production and Marketing 6013 12/27/85 SecM86-88 (loan 1588-GR) 6. Nigeria Livestock Development (loan 1091-UNI) 5828 12/31/85 SecK86-89 7. Costa Rica Agricultural Credit and Development 6015 12/31/85 SecM86-23 (loan 1410-CR) 8. Philippines Fourth Rural Credit (loan 1399-PH) 6016 12/31/85 SecM86-25 9. Afghanistan Fruit and Vegetable Export (Credit 779-AF) 6103 03/21/86 SecM86-326 - 26 - ANNEX A Page 8 Report Report Covering Mem kgriculture (Cont.) No. Date to Board LO. Mexico Bajo Rio Bravo/Bajo Rio San Juan Irrigation 6207 05/30/86 SecM86-708 Rehabilitation (Phase I) - (loan 1111-ME) 11. Pakistan Hazara Forestry Pre-Investment 6202 05/23/86 SecM86-626 (Credit 755-PAK) L2. Korea Miho Watershed Area Development (loan 1319-KO) 6197 05/21/86 See86-611 and Yong San Gan Gang Irrigation II (Loan 1364-K0) 13. Malaysia Northwest Selangor Integrated Agricultural 6198 05/21/86 SecM86-592 Developeant (Loan 1522-MA) 14. India Periyar Vaigai Irrigation (Credit 720-IN) 6257 06/1/86 SecM86-704 15. India Rajasthan Canal Comnmand Area Development 6258 06/17/86 SecM86-715 (Credit 502 -IN) 16. Ecuador Technical Assistance (loan 1230-EC) 6327 06/30/86 SecM86-876 17. Mexico PIDER II Integrated Rural Development 6333 061'30/86 SecM86-877 (loan 1462-ME) L8. Honduras Guayape Regional Development (loan 1576-0) 6274 06/23/86 SecM86-866 9. Greece East Version Irrigation (Loan 1165-GR) 6266 06/19/86 SecM86-782 luman Resources 1. Algeria Third Education (Loan 1378-AL) 5797 07/24/85 SecM85-902 2. Brazil Nutrition Research and Development 5811 08/01/85 SecM85-989 (Loan 1302-BR) 3. Sudan Second Education (Credit 547-SU) 5863 09/30/85 SecM85-1189 4. Trinidad Third Education (loan 1722-TR) 5991 12/27/85 SecM85-1344 & Tobago 5. Quatemala Second Education (Loan 1212-GU) 5989 12/27/85 SecM86-133 6. Peoples's Republic Second Education (loan 1349-COB) 6273 06/24/86 SecM86-852 of Congo 7. Pakistan Third Education (Credit 678-PAK) 6252 06/16/86 SecM86-735 - 27 - ANNEX A Page 9 Report Report Covering Memo DFC and Industry No. Date to Board 1. Peru Centromin Expansic. (ian 1281-PE) 5902 10/25/85 SecM85-1188 2. Ecuador third and Fourth Development Banking 5946 12/02/85 SecM85-1283 (loans 1359/1731-BC) 3. Philippines Piso Development Bank (loan 1555-PH) 6006 12/27/85 SecM86-117 (formerly The 1hilippines Investments Systems Organization) 4. India Industrial Credit and Investment Corporation 6052 01/31/86 SecM86-166 of India (loans 1097/1475-IN) 5. Brazil Sergipe Nitrogen Fertilizer (loan 1406-BR) 6152 04/21/86 SecM86-464 6. Turkey Akdentz Forest Utilization (loan 957-TUR) 6153 04/21/86 SecM86-480 7. Nepal Industrial Development Corporation 6154 04/21/86 SecM86-478 (Credit 705-NEP) 8. Gambia Rural and Urban Enterprise (Credit 814-Gi) 6309 06/25/86 SecM86-799 9. Malawi Investment and Development Bank (INDEBANK) 6334 06/27/86 SecM86-871 (loan 16104fAI) 10. Korea Small and Medium Industry Bank (SMIB) 6302 06/24/86 SecM6-865 (loans 1507/1749-K0) 11. India Fertilizer Industry Credit (Credit 598-IN) 6297 06/23/86 SecM86-759 Non-Project lending 1. Senegal First Technical Assistance Project to 5956 12/02/85 SecM85-1318 the Parapublic Sector (Credit 764-SE) Public Utilities and Enegy 1. Kenya Mombasa Water Supply (loan 1167-KE) 5918 11/04/85 SecM85-1299 2. Cyprus Fifth Power (oans 1873/1873-5-Ci 5992 12/26/85 SecM85-1352 3. Thrkey Ankara Air Pollution Control (loan 5-15-TU) 6003 12/26/85 SecM86-6 4. Pakistan Second WAPDA Power (loan 1208-PAK) 6004 12/26/85 SecM86-13 5. Malaysia Seventh Power (loan 1178-MA) 6001 12/27/85 SecM86-5 - 28 - ANNEX A Report Report Covering Memo 1blie Utilities and Energy (Cont.) No. Date to Board 6. Syria Regional Electrification (loan 1531-SYR) 6007 12/27/85 SecM86-12 7. Brazil Northeast Power Distribution (loan 1300-BR) 5993 12/27/85 SecM85-1353 8. Costa Rica Fifth Telecommunication (loan 1532-CR) 6057 02/10/86 SecM86-165 9. Argentina Servicios Electricos del Gran Buenos Aires 6098 03/14/86 SecM86-317 Fourth Power (SEGBA IV) (loan 1330-AR) 10. Brazil Furnas Centrais Electricos, SA (FURNAS) 6099 03/14/86 SecM86-327 Itumbiara Hydroelectric (loan 923-BR) 11. Peru Fifth Power (loan 1215-PE) 6125 04/02/86 SecM86-442 12. Zambia First Telecommunications (loan 1131-ZA) 6149 04/21/86 SecM86-462 13. Thailand Mao Laem Hydroelectric (loan 1770-TH) 6157 04/25/86 SecM86-472 14. Jamaica Kingston Sewerage I & Water Supply 6186 05/13/86 SecM86-561 (loan 1146-JM) 15. Pakistan Second lahore Water Supply, Sewerage and 6204 05/19/86 SecM86-653 Drainage (Credit 630-PAK) 16. Pakistan Toot Oil & Gas (Credit 867-PAK) 6208 05/21/86 SecM86-677 17. Yugoslavia Naftagas Pipeline (loans 916/1264-2-YU) 6216 05/22/86 SecM86-628 18. India Second Bombay High Offshore Development 6237 06/04/86 SecM86-804 (loan 1925-IN) 19. India Third Trombay Power (loan 1549-IN) 6253 06/06/86 SecM86-725 20. Indonesia Sixth Power (loan 1365-IND) 6238 06/04/86 SecM86-726 21. Malaysia Eighth Power (loan 1443-MA) 6241 06/05/86 SecM86-729 22. India Uttar Pradesh Water Supply and Sewerage 6242 06/05/86 SecM86-724 (Credit 585-IN) 23. Tunisia Fifth Water Supply (loan 1702-TUN) 6288 06/20/86 SecM86-807 24. Algeria Water Supply Engineering (Loan S-17-AL) 6270 06/13/86 SecM86-757 - 29 - ANNEX A Report Report Covering Men Public Utilities and Energy (Cont) No. Date to Board 25. Republic Second Abidjan Sewerage & Drainage 6323 06/30/86 SecM86-919 COte d ' Ivoire (loan 1577-IVC) 26. Greece Salonica and Volos Sewerage (loan 1345-GR) 6285 06/19/86 SecM86-803 27. Botswana Second Water Supply (Loan 1763-BT) 6311 06/26/86 SecM86-872 28. Nicaragua Instituto Nicaraguense de Energia (INE) 6322 06/30/86 SecM86-851 Ninth Power (Loan 1402-NI) Tourism 1. Mexico Baja California Tourism (Loan 1420-ME) 5848 09/12/85 SecM86-1078 2. Barbados Tourism (loan 1685-BAR) 6158 04/23/86 SecM86-502 Transportation 1. Bolivia Third Railway (Loan 1422-BO) 6109 03/24/86 SecM86-351 2. Malaysia Second Sabah Ports (Loan 1580-MA) 6110 03/24/86 SecM86-428 3. Peru Lima-Amazon Transport Corridor (Loan 1196-PE) 6111 03/24/86 SecM86-334 4. Swaziland Third Highway (Loan 1619-SW) 6127 03/31/86 SecM86-430 5. Costa Rica Fourth Highway (Loan 1187-CR) 6123 04/01/86 SecM86-395 6. Madagascar Fourth Highway (Credit 641-MAG) 6148 04/18/86 SecM86-473 7. Thailand Fifth Railway (Loan 1662-TH) 6163 04/25/86 SecM86-496 8. Thailand Sixth Highway (Loan 1519-TH) 6205 05/20/86 SecM86-613 9. Yugoslavia Fifth Railway (Loan 1534-YU) 6168 05/06/86 SecM86-580 10. Zaire Fourth Highway (Credit 916-ZR) 6185 05/12/86 SecM86-562 11. Korea Seventh Railway (Loan 1836-K0) 6196 05/16/86 SecM86-582 12. Omn First Highway Maintenance (loan 1441-CM) 6210 05/21/86 SecM86-652 13. Rwanda Fourth Highway (Credit 769-W) 6209 05/21/86 SecM86-614 14. Somalia Third Highway (Credit 699-SO) 6222 05/30/86 SecM86-666 - 30 - ANNRM A wae 12 Report Report Covering Maw Transportation (Cant.) No. Date to Board 15. Somalia Fourth Pbrt (Credit 838-80) 6223 05/30/86 SecM86-813 16. Shrkey Ports Rehabilitation (loan 1741-TU) 6213 05/21/86 SecM86-643 17. Kenya Rural Access Roads (loan 1305/Credit 651-KE) 6271 06/17/86 SecM86-894 18. Nepal Second Highway (Credit 730-NEP) 6296 06/20/86 SecM86-824 Urban 1. Philippines Manila Urban Development 6141 04/11/86 SecM86-429 (loans 1272/1282-1-PH) 2. Thailand Bangkok Sites and Services and Slum 6174 05/07/86 SecM86-534 Improvement (loan 1556-TR) 3. Morocco Rabat Urban Development (loan 1528-MOR) 6184 05/12/86 SecM86-557 4. India Bombay Urban Transport (loan 1335-IN) 6239 06/04/86 SecM86-709 C. SPECIAL STUDIES 1. Impact Evaluation Report: Korea-Pyongtaek- 5941 11/19/85 SecM85-1258 Sugang Irrigation Project (loan 600-1D) 2. Impact of World Bank lending for Educational Development 5950 12/05/85 SecM85-1313 in Korea: A Review 3. The Smallholder Diamension of Livestock Development - 5979 12/18/85 SecM86-53 A Review of Bank Experience (2 Volumes) 4. Iapact Evaluation Report: Malaysia - First, Second and 5988 12/23/85 SecM86-36 Third Jengka Triangle (loans 533, 672 and 885-MA) 5. World Bank in Pakistan - Review of a Relationship, 1960-1984 6048 01/27/86 SecM86-119 6. The World Bank in Sri lanka - A Review of a Relationship 6074 02/24/86 SecM86-216 7. Bangladesh-Review of the Experience with Import Credits I-X 6080 02/24/86 SecM86-296 8. Sustainability of Projects Review of Experience in the 6073 02/26/86 SecM86-249 Fertilizer Subsector 9. World Bank ltanding Conditionality - 6283 06/25/86 SecM86-826 A Review of Cost Recovery in Irrigation Projects -31 - ANNE A Report Report Covering Mem No. Date to Board D. OTHER REPORTS 1. Anmal Review of Project Performance Audit Results 5859 09/27/85 135-3159 (three volumes) 315/1 2. Annual Report on Operations Evaluation, 1985 5836 08/30/85 R85-280 3. Concordance to Project 1erformance Audit Reports 5945 11/29/85 SecM86-35 Issued by Operations Evaluation Department, March 1972 - June 30, 1985 (2 Volumes) - 32 - ANNEX B Page 1 ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) OED POLICY REVIEWS AND EVALUATION STUDIES I. OPERATIONAL POLICY REVIEW: COMPLIANCE WITH LOAN COVENANTS 1. The Operational Policy Review: Compliance with Loan Covenants (Report #4090) dated September 1, 1982 was circulated to the Executive Directors on September 3, 1982 (SecM82-782). The Joint Audit Committee considered the OED Report on September 22, 1982 (JACM82-8) and the Management Response thereto (R83-2 dated January 5, 1983) on January 26, 1983 (JACM83-2). The OED Report and the Management response were considered at the Board Meeting of July 14, 1983. 2. The principal recommendations contained in the above report were set forth in the 1984's Annual Report (Annex C, Item IV) together with the Management Response. Since then the Management of the Bank has given extensive consideration to this subject which culminated in the formation of a Legal Department Task Force on "Simplification of Loan Documents." The Report of the Task Force (with one minor exception) has been fully supported by Bank Management and has been considered by the Managing Committee of the Bank. 3. Both the OED paper and the Management response highlighted the need for streamlining and simplifying the Bank's loan documentation and the Association's credit documentation. 4. In December 1984, the Executive Directors approved certain modifi- cations to the "General Conditions Applicable to Loan and Guarantee Agree- ments" and the "General Conditions Applicable to Development Credit Agreements", respectively, as proposed by the Legal Department, in order to achieve, in conjunction with a number of other measures so proposed, the required streamlining and simplification of loan and credit documentation proposed by the OED Report. The measures referred to above include the following: (a) A number of standard clauses which presently are repeated verbatim in each legal agreement have been transferred from the text of the agreement to the General Conditions. The revised General Conditions are applicable to all Bank loans and credit for which proposals are submitted to the Loan Committee after January 1, 1985. (b) The revised text of the "Guidelines for Procurement under IBRD Loans and IDA Credits", issued in August 1984 has taken into account the design for streamlining and simplification of legal agreements by incorporating many provisions which were previously required to be repeated in legal agreements. - 33 - ANNEX B Page 2 (c) The language of the loan and credit documentation has been simplified. (d) The legal agreements should not contain as covenants general- ized statements of intentions which are so broad in content or indefinite in nature that they cannot be monitored or "enforced". OMS No. 3.18 provides guidance on the use of the "Supplemental Letter" including the letter of intent "clearly not meant to constitute a covenant". (e) Wherever practical, the details of administrative and organi- zational arrangements for projects should be addressed in implementation schedules annexed to the legal agreements rather than as a series of "minor" individual covenants in the main body of the agreements. II. STRENGTHENING AGRICULTURAL RESEARCH AND EXTENSION THE WORLD BANK EXPERIENCE 5. The report on Strengthening Agricultural Research and Extension - The World Bank Experience (Report #4684) dated September 1, 1983 was distributed on September 13, 1983 (SecM83-879) and considered by the Joint Audit Committee on December 2, 1983 (JAC/M83-11). 6. The summary of major findings and principal recommendations con- tained in the above report were set forth in last year's Annual Report (Report #5247, Annex C, Item II). The actions taken by management based on the main recommendations of the report are as follows: (a) Action in line with the main recommendation that the Bank should increase its support in strengthening national research and extension programs is well demonstrated by the fact that of the 37 agricultural projects approved by the board as of March 31, 1985, 11 (or 30%) were for national agricultural research and/or extension projects in 10 countries while another is ready to go to the Board. The future pipeline is also very strong with a further 11 projects in FY86, and 10 in FY87. Many other projects, of course, also have a significant research and/or extension program component. (b) In advancing dialogue on extension and research, the proceed- ings of the 1984 Asian Regional Research-Extension linkage workshoph ave been prepared for publication, and two further workshops were held in Sub-Saharan Africa subsequently, one in Eldoret, Kenya and one at Yamassoukro, Ivory Coast. -34 - ANNEX B Extension and research leaders from many countries attended and a report on the major issues generated at the first and discussed at the second is being prepared for publication. It is worthwhile noting that four of the FY85 RE projects are in Eastern or Western Africa, five of those for FY86 and six of those for FY87, emphasizing the present focus being placed on Sub-Saharan Africa. (c) During the year two two-man units have been set up in the field to strengthen both preparation and supervision support on extension, one in Abidjan, and the other in Nairobi. On the research side plans are in hand to utilize some of the special funds for Africa on strengthening international and regional research activities outside the purview of the CGIAR centres. During the year EDI ran a course on research management for research directors in conjunction with ISNAR. (d) All projects are being prepared with full and active involve- ment of governments, and procedures are followed by which governments produce, sometimes with the help of consultants or ISNAR, their own strategies for research and extension before the Bank, in coordination with the many other donors active in this field, discusses possible projects. (e) The roster of consultants on extension (which had been weak relative to research) is being strengthened to ensure the Bank has well informed and experienced practitioners, especially when we are trying to help introduce new systems such as Training and Visit. (f) Emphasis is being placed on strengthening Governments' REM services in tha context of their existing services and environment: for instance in Africa for the moment the T and V approach is reserved for higher potential areas, and the Bank is advocating pilot trials of eppropriate extension methodol- ogies in marginal areas, where improved technologies are weak and carry higher risks. (g) To ensure that technology transfer messages are relevant to all types of farmers, special emphasis is placed on stratify- ing farmers into "recommendation domains" using diagnostic surveys or farm systems research techniques in several projects thereby obtaining clearer assessments of socio- economic and politico-cultural considerations. - 35 - ANNEX B Page 4 (h) In view of discouraging experience with attempts at sophis- ticated and expensive evaluation of impact by trying to measure yield trends, the Bank has moved towards putting major emphasis on the monitoring of adoption rates, which gives an essential input in revising extension and applied research programs, and making them more relevant to farmer needs. In this context AGRME has issued a publication on methods of estimating crop yields by crop cutting or by farmer survey. (i) As part of the Bank's action program for Sub-Saharan Africa, the two regional Projects Departments have initiated major region-wide reviews of agricultural research to strengthen the technology base, and widen understanding of the major issues involved as the Bank embarks on the program of expanded lending for R and E in these two regions. (j) Special focus has been placed on the problems of marginal areas in designing research and extension programs. Thus a special paper hs been prepared on desertification; the Western Africa Region is reviewing rainfed crop production in the Sahel; the Eastern and Southern Africa Region is develop- ing material on advances in improved water harvesting tech- niques; and EMENA has prepared a paper on the state of technology and its adoption in the rainfed areas of N.W. Africa. III. HARVESTING THE WATERS - A REVIEW OF BANK EXPERIENCE WITH FISHERIES DEVELOPMENT 7. Harvesting the Waters-A Review of Bank Experience with Fisheries Development (Report #4984 dated March 13, 1984) was circulated on March 19, 1984 (SeeM84-240) and was considered by the Joint Audit Committee on March 28, 1984 (JAC/M84-3). 8. The summary of major findings and recommendatioia contained in the aoove report were set forth in last year's Annual Report (Report #5247 Annex C Item III). Following are specific measures taken by Bank management in implementing the recommendations of the report: (a) Since ipsuing the Fisheries Sector Policy Paper and the recruitment of a Fisheries Adviser in AGR, a number of measures have been taken geared to the implementation of the Bank's Policy in the sector. In spite of the ample subject coverage and the wide variety of areas that the Policy - 36 - ANNEX B Page 5 encompasses, the main recommendations contained in the OED report are being implemented. (b) During FY85, there have not been major shifts in the lending trend as compared to the preceding years (e.g. one or two fisheries projects and 2 or 3 fishery components in other projects). However, there has been a substantial increase in sector work geared to future involvement particularly in Africa (seven countries and more are in the pipeline) where the recommendations of the OED document are being applied. (c) The actions being taken as referred to in para. 1 which are directly related to the recommendations of the OED's report, may be summarized as follows: (i) A review of the world fisheries situation geared to establish the possible role of the Bank is being prepared. Seven countries have been covered by sector studies or reconnaissance missions and more such studies/missions are in the pipeline. (ii) Closer working relationships have been developed between AGR and the Divisions responsible for fisheries compo- nents in other projects, with a consequent improvement in monitoring the development of these components and provi- sion of assistance and guidance. (iii) Close coordination with other advisers in AGR has allowed the early identification of fisheries implications in irrigation, hydro-electric and other related projects. Close coordination with the Environmental Unit in these matters has also lecome common practice. (iv) In-house expertise is being used to the full; consulta- tion between the Fisheries Adviser and other fisheries staff takes place regularly on a formal and informal basis. (v) A good working relationship with the FAO's Department of Fisheries and other multilateral and bilatezal agencies is making possible a rethinking of FAO's approach to projects to conform more to the needs of financing institutions. This should assure better follow-up during implementation. - 37 - ANNEX B Page 6 (vi) FAO's Department of Fisheries, the Norwegian Government and the Visibility Laboratory of the Scripps Institution of Oceanography (University of California) have agreed to collaborate with the Bank in resource assessment matters in areas where there may be potential for involvement. A program is underway in Africa. (vii) Institution building is being given high priority in all projects; some actions are being proposed in advance at pre-investment level with the participation of other agencies. (viii) In the areas that require clearly identifiable pre- investment work (e.g. prototype design of fishing boats OED's report indicates) our efforts to induce the participation of other agencies is beginning to move. The willingness of bilateral and multilateral agencies to work with the Bank at the pre-investment phase is beginning to materialize particularly in Africa. IV. INSTITUTIONAL DEVELOPMENT IN AFRICA A REVIEW OF WORLD BANK PROJECT EXPERIENCE 9. Institutional Development in Africa: A Review of World Bank Project Experience (Report No. 5085) dated May 17, 1984 was distributed on May 22, 1984 (SecM84-457) and was considered by the Joint Audit Committee on June 6, 1984 (JAC/M84-5). The summary of findings and the principal recom- mendations were set forth in last year's Annual Report (No. 5247, Annex C, Item IV). 10. A Management Response to the OED Report was distributed on Novem- ber 8, 1984 (SecM84-957). The Bank Management broadly agreed with the main conclusions of the report, although reservations were expressed about the conceptual framework and design of the study and some of its conclusions and recommenda-tions. The Joint Audit Committee considered the Management response on May 8, 1985 (JAC/M85-6). The Committee was not fully satisfied with the adequacy of the response and called upon Management to provide more comprehensive proposals for addressing institutional development issues within the Bank. 11. Since the report was distributed, however, a number of actions have been undertaken that are consistent with its major recommendations. The Public Sector Management Unit in the Projects Policy Department has been expanded and has provided over four staff years of operational support as - 38 - ANNEX B Page 7 well as some training to the two Africa regions. It has also undertaken a review of Africa public enterprise performance and a review of experience with privatization. 12. At the operational level, both Africa regions have now set up separate divisions responsible for institutional development, along the lines recommended in the report (para. 6.42 and Appendix 3, paras. 27-30). They are also giving increased attention to developing specific country and sector institutional deveiopment strategies-a process that has been facilitated by the increasing use of country teams in the work of the regions and the new geographical allocation of responsi-bilities at the assistant director level. 13. In the Eastern and Southern Africa Region, an Institutional Development, Training and Technical Assistance Division was established in 1984 and has since been strengthened both in size (now 9 staff members) as well as the range of specialist skills. So far the experience has been positive. The new divi-sion has been effective in focusing attention on the importance of institutional development at the various stages of the project cycle, and has provided training to regional staff and management. The transfer of project-related training (PRT) to the new division has helped to broaden the context of PRT work, as suggested by the report. 14. The Western Africa Region has also given increased attention to institutional development over the past year. It has already initiated two broad country institutional studies and is seeking UNDP funds to expand this work and to finance technical assistance for operational support. It has also gone further than other regions to develop and institutionalize the "twinning" approach to institutional development. There is also evidence that regional management is serious in its intent to enhance the effective- ness of institutional development work. Effective July 1, 1985, the region established a Development Management Division, with major responsibility for institutional development. Unlike its counterpart in Eastern Africa, however, the division does not yet have responsibility for project-related training and technical assistance. The region has also recently held a management seminar on institutional development, and is taking steps to strengthen the review of institutional components of projects. V. AGRICULTURAL SECTOR OPERATIONS REVIEW: BURKINA FASO AND MALAWI 15. The Agricultural Sector Operations Review: Burkina Faso and Malawi (Report No. 5387 dated December 28, 1984) was circulated on January 10, 1985 (SecM85-41) and was considered by the Joint Audit Committee on February 6, 1985 (JAC/M85-3). - 39 - ANNEX B Page 8 16. Following is a summary of the report's main findings and recommen- dations: (a) In both countries Bank assistance has been supportive of the smallholder sector, but reliance on extension-agricultural credit supported technical packages limited participation to farmers with land and labor capacities in excess of subsis- tence needs. Seed exchange programs for smaller farmers, special programs for women, and extending credit to the "junior" members of the extended farm families would permit wider participation in Bank-assisted projects. (b) Bank-assisted projects aiming at increased food production especially maize in Malawi and millet/sorghum in Burkina Faso, have been of limited success. Lack of suitable varieties and detailed knowledge of area - specific soil fertility problems, coupled with unsatisfactory pricing/marketing structures, have been the main reasons. (c) Marketing arrangements for staple food have been found deficient in both countries. There is a need for studying expansion of private trade in both countries, especially more socio-cultural analyses exploring the motivation of small traders would be helpful. (d) Socio-cultural research by the Bank prior to project appraisal, as in other countries and projects, was insuf- ficient, although major socio-cultural issues were identified before and during appraisal of several projects in both countries. (e) To gather information on unresolved socio-cultural problems Monitoring and Evaluation units hav,i been established through various projects. However, withot proper planning of tasks ahead and the main focus being presently on physical and financial progress of the projects, contributions of these units to understanding farmers' attitudes, reactions, invest- ment priorities, risk appreciation, etc. were marginal. (f) Limited efforts have been undertaken to expand responsibil- ities of farmer groups. There is scope for further expanding group activities, which, if successful, could take over some costly and burdensome responsibilities from the Governments. 17. Several sector issues merit early attention by the two Govern- ments, and by the Bank in future sector operations: - 40 - ANNEX B Page 9 (a) There is a need for improving manpower planning to anticipate the needs of Governments, organizations and the supporting services for future projects. Better coordination with Bank-assisted education programs is warranted. (b) In the absence of irrigation traditions in both countries, emphasis on training young farmers in irrigation husbandry would be more appropriate than to expect older farmers to change traditional dryland husbandry practices. (c) In both countries agricultural research priorities need to be clearly defined and established. To prevent duplication of efforts, inventories should be prepared of existing but filed away research results of past decades. (d) Timely land preparation is still an unresolved problem. Draft animal mechanization is only a partial solution and until now the benefits of seed/fertilizer packages cannot be maximized due to late planting and/or weeding. Research needs to focus on possible solutions such as plowing after harvest, mulching, etc.. (e) Programs aiming at improved animal health should receive priority, especially in Burkina Faso with its important livestock sector. (f) Preparation of land use plans is of great importance to prevent conflicts between agriculturists and pastoralists in Burkina Faso. In Malawi competition between livestock owners, foresters and truck farmers for the use of seasonally flooded lands would also call for land use plans. (g) In both countries farmers show little interest in maintaining soil erosion works. This problem could be overcome through early involvement of farmers in the design and construction of anti-erosion undertakings. (h) There is need for a strategy to develop rural institutions which would determine the role each future project could play in contributing to institution development. More efforts are also needed to improve the inter-sectoral/unilateral coordinating mechanism. (i) Assistance is needed to develop management support units within the Ministries concerned with rural development. These units could become instrumental in preparing annual work programs and monitoring project progress. - 41 - ANNEX B Page 10 18. Some recommendations with broader implications for operations emerge from this review: (a) On occasion the policy dialogues between the Bank and Govern- ments take place without proper coordination with other donors. The Bank should take a more active role in promoting better cooperation and coordination among aid donors to reach agreement on long-term development strategies and to enhance aid effectiveness. This could be done through Consultative Groups, or regular meetings organized by Resident Representa- tives. (b) A long-term development strategy for the smallholder sector is needed. With rapid population growth and dwindling land resources farm sizes of many smallholders become insufficient for cash crop production. Systematic efforts are needed in rural development projects to arrange reliable food supplies to rural areas - a prerequisite for permitting expansion and/or diversificaion into higher value cash crops. (c) More socio-cultural information has to be taken into consid- eration during the identification-preparation-appraisal cycle. Use of social antropologists during project identification/preparation is warranted and would ensure proper use and application of existing socio-economic data as well as pinpoint more accurately farmer needs and aspirations. (d) In encouraging price incentives to producers, consideration has to be given to cross elasticities of various commodities to prevent production imbalances. Impact on demand needs to be considered before linear subsidy reductions are recom- mended. (e) The Bank could take a leading role to coordinate agricultural research on a regional basis, i.e., in areas of similar ecological conditions. (f) Consideration should be given to improve the analysis of extension needs. Extension worker/farmer ratios should be based more on tie complexity and innovative weight of exten- sion messages bather than on standard, preconceived staff numbers. (g) With the emphasis of the Bank's lending operations on assisting smallholders, a more profound analysis of the individual country sectors has to be undertaken. - 42 - ANNEX B Page 11 (h) Consideration needs to be given to the promotion of indus- tries, especially smallscale agro-industries in rural areas to provide employment opportunities for underemployed mini- farmers or the landless. 19. The Joint Audit Committee in general commended OED for this report. OPS representatives at the meeting stated that they considered the review useful and that the recommendations in it were increasingly being taken into account in the design and implementation of agricultural projects. OPS had no major disagreements with the analysis contained in the report. Nevertheless, the JAC Chairman expressed doubts as to whether OED findings and recommendations were being adequately taken into account by the Bank in developing new projects and considered this an issue that should be addressed by the Committee. 20. The JAC, at the same meeting, requested OED to prepare a short action-oriented paper focussing on major policy issues and possible solutions drawn from both the agricultural sector operations review of Burkina and Malawi and the section on agriculture in the Tenth Annual Review of Project Performance Audit Results. The paper was distributed on May 15, 1985 (JAC85-13). The Committee also requested that the OED introductory statement concerning the report be distributed to the Committee for information, which was done on February 7, 1985 (JAC85-3). VI. THE EXPERIENCE OF THE WORLD BANK WITH GOVERNMENT-SPONSORED LAND SETTLEIENT 21. The Experience of the World Bank with Government-Sponsored Land Settlement (Report No. 5625 dated May 1, 1985) was distributed on May 9, 1985 (SecM85-560) and was considered by the Joint Audit Committee on July 29, 1985 (JAC/M85-9). 22. The Report is based on a desk study of 34 Bank-assisted projects promoting land settlement in some form. Its major Recommendations are as follows: (a) The Bank should clearly state its own policy on land settle- ment. At the very least agricultural production goals should be more effectively combined with goals for improved environ- mental management, for raising the living standards of beneficiaries above poverty levels, and for initiating regional development within the settlement and encompassing areas. - 43 - ANNEX B Page 12 (b) To alert Bank personnel to the nature and potential of land settlement as an investment option, an Operational Manual Statement (OE) should be prepared on land settlement, either incorporating or in addition to existing OMS 2.33. (c) Be--ause of some lack of understanding within the Bank about the nature, potential and implications of land settlement, training courses for Bank personnel should include material on land settlement. (d) During project planning and implementation, Bank staff should pay as much attention to issues dealing with the beneficiaries and iarming/production systems as with issues relating to procurement, disbursements, management and infrastructure. (e) Because land settlement areas are apt to be more fragile than habitats which have been farmed for centuries, the Bank's environmental policy as outlined in OMS 2.36 (Environmental Aspects of Bank Work) should be more systematically applied to lanA settlement projects. (f) The Bank's experience with land settlement as a development intervention should be applied to other types of projects (such as power and irrigation) which require the involuntary relocation of host (i.e., native) populations. (g) To reduce land disputes and other sorts of disruptive con- flicts between host population and settler communities, the Bank should ensure that during the appraisal process the rights of the host population are dealt with in an appropriate fashion. (h) To facilitate Bank assessment at different stages of the project cycle, core data should be collected and analyzed on specific key issues relating to institutional management, farming systems, the beneficiaries, physical and social infra- structure, regional development, host population incorporation and environmental management. Key issues related to each of these topics should be spelled out in an Operational Manual Statement, with specific recommendations included. (i) Because of their implications for project .uccess, Bank- assisted sectoral programs relating to such topics as improved site selection, research, extension, credit, improved institu- tional capacity for operation and maintenance, and settler participatory action organizations should be associated with specific settlement projects where possible. ANNEX B Page 13 (j) Where the numbers of settler households are not in the thousands, efforts should be made to link a number of smallseale settlements to adjacent marketing centers and rural towns so as to promote integrated area development. Target incomes for settler households should be high enough to catalyze such a development process. (k) Because the Bank's influence over project design and implemen- tation appears to be closely associated with Bank financing, the Bank should be prepared to fund key components which might otherwise be neglected, including urban development in or on the periphery of settlement areas lacking appropriate market- ing centers and rural towns. (1) To gain more experience with the urban aspects of land settle- ment, the Bank, on a pilot basis, should consider extending its assistance to non-farm enterprise development and employ- ment generation, and to the development of a range of urban functions for rural development. (m) The Bank should assess the development implications at all stages of the project cycle of its apparent bias toward parastatal settlement agencies as opposed to project manage- ment units and other management forms. Appraisals and covenants dealing with project management should pay more attention to the transfer of responsibilities from settlement agencies to suitable local organizations and other management issues associated with the later stages of the settlement process. (n) Because of the influence that good supervision appears to have on settlement project success, the Bank should provide more thorough supervision to Bank-assisted settlement projects. More emphasis should also be placed on monitoring and evalua- tion by appropriately trained staff of local research organi- zations and universities. (o) The Bank should be more prepared to insist that project considerations take precedence over country considerations, to request that unsatisfactory projects be redesigned, to withhold disbursements or to terminate projects if supervision missions so recommend. (p) Due to the long gestation period of land settlement which results in many projects just entering the important stage of economic and social development at the time of project - 45 - ANNEX B P ag e 14 completion, Project Performance Audit Reports should be supplemented by a larger number of Impact Evaluation Reports--posaibly undertaken later than the customary five years after project completion. (q) Because current appraisal methodologies are better at assessing the shorter term impacts of settlement projects than the longer term ones, consideration should be given to supplementing their use with other approaches. 23. The Committee in general commended OED for an important report on a very difficult subject. The report's conclusion was supported that, aside from economic rates of return, the Bank should pay iwre attention to benefits accruing to a majority of the settler households and to the potential of projects to catalyze a process of integrated area development. It was also suggested that land settlement would be a good area in which to apply the recommendations of OED's first review of the sustainability of projects. The Chairman agreed with the general recommendation in the report that the Bank needed to set out clearly its policy on land settlement; however, it was important for the Bank to be sensitive to the views of member governments and to socio economic issues involved in land settlement. VII. SUSTAINABILITY OF PROJECTS: FIRST REVIEW OF EXPERIENCE 24. Sustainability of Projects: First Review of Expe-rience (Report No. 5718 dated June 14, 1985) was circulated on June 19, 1985 (SecM85-730) and was considered by the Joint Audit Committee on July 29, 1985 (JAC/M85-9). 25. This study is based on the entire set of Impact Evaluations carried out by OED through April 1985. The study identifies and evaluates those factors which have an important tiaring on the sustainability of project benefits. The concern with sustainability factors arises from the radical change in the actualized and expected flow of benefits in several agricultural projects that were revisited some years after completion. These findings suggest an examination of the reasons leading to these changes and to a new look at the issue of sustainability and the factors that underlie it. More generally there is an issue of perspective and focus here in the approach to the design and implementation of projects, and whether the present methodology and practice carry within them the danger of unduly shortening that perspective and narrowing that focus to project completion. 26. The following is a summary of the concept and approach taken by the study and its findings and recommendations: ANNEX B Page 15 (a) Sustainability in this Review refers to the maintenance of an acceptable net flow of benefits throughout the economic life of the project. (b) The study reviews the experience of the entire set of impact evaluations (IBs) carried out by OED through April 1985. The selection of these 27 agriculture and 4 education projects for impact evaluation was not intended to be representative of sector, sub-sector or regional experience. Nevertheless, these Is provided valuable insight into the processes and influences which have an important bearing upon project sustainability in these two sectors. (c) A three-step process has been used in examining each of these 31 projects. First, an attempt has been made to determine which projects could be said to have achieved sustainability as viewed at the time of impact evaluation. Second, through an examination of the post-completion project experience, those factors have been identified whose presence appeared most likely to make successes endure and whose absence led to an erosion of benefits after the project had been viewed as being actually or potentially successful. Once identified, those factors ' re seen to have contributed to or detracted from sustainability were traced on a project-by-project basis from the design/appraisal stage through supervision, completion, audit and impact evaluation. Finally, lessons have been drawn regarding aspects of project planning, design and implementation which would seem to have an important bearing on sustainability. 27. Important factors underlying sustainable projects were identified as: (a) The development, performance, autonomy and flexibility of institutions responsible for project activities. (b) Participation of beneficiaries in design and implementation. (c) Adoption and adaptation of technology. (d) Attention to existing socio-economic behavior and traditions in project design. (e) Compatible borrower policies. (f) Adequate provisions for recurrent cost financing; and - 47 - ANNEX B Page 16 (g) Feasible objectives for human resource development. 28. Lessons and Recommendations derived from the report are: Design and Appraisal (a) The primary emphasis in appraisal reports was on implementa- tion features. At times, this worked at cross-purposes with sustainability. (b) It is necessary to plan considerab in advance for the transfer of responsibilities for project genecated activities from the project management unit (PMU) to a permanent institu- tion. (c) Transfer of responsibilities from a PMU should be gradual and coupled with institutional development measures to prepare permanent institutions for the assumption of additional responsibilities. (d) There may be an unintentional built-in bias working against sustainability in design and appraisal practices. Given the widespread use of ERRs for presentational purposes there is the possibility that their magnitude may influence project design features. The danger for sustainability lies in the fact that projects with very high estimated ERRs which result from early and rapidly accumulating benefits will heavily discount benefits accruing to sustainability. As a result, less than desirable attention may be given to the long-term implications of project design. (e) Designing permanent mechanisms into projects, which enhance flexibility and planning capacity is critically important for long-term sustainability. Such mechanisms should also be coupled with a capability to evolve objectives so that they remain relevant to the needs of a changing environment. (f) Beneficiary participation in project design would seem to be a way of factoring local behavior and beneficiary assessments of risks, costs and benefits into project design, and thereby enhance the inclination to adopt a proposed technology and to undertake new activities. Implementation (a) The Bank must retain flexibility in adapting project objectives and in assessing progress against those revised - 48 - ANNEX B Page 17 objectives throughout implementation. Ways should be sought to institutionalize such flexibiliLy so that the revision of objectives and subsequent changes in project activities do not become capricious. (b) Given the importance of institutional and socio-economic/ cultural factors for achieving institutional development and technology transfer, and hence, for enhancing the prospects for achieving sustainability, it would seem imperative for Bank supervision missions to give greater priority to these areas. Completion and Follow-Up (a) Where projects are financed in a series, there is a need for incorporating measures in later phas..s which are explicitly designed to ensure the sustainability of early phase activi- ties. (b) At the individual project level there is need for better planned Bank disengagement from projects nearing completion. One approach would be for the Bank to add a "graduation mission" to its project overview procedures. The "graduation mission" might recommend that the Bank finance an action plan to be undertaken in the post-completion period, which would consist of any measures deemed necessary to move the project onto a self-sustaining track. 29. In commenting on the report, the Committee noted that the impor- tance of appropriate technology and its adoption in agricultural projects was very important, especially in African conditions. The report's emphasis on the need for more advance planning for the transfer of responsibility from project management units was supported, as was the recommendation that the Bank needed to develop a process which would permit it eventually to disengage from projects without adversely affecting the sustainability of benefits. The Committee concluded that what was needed was for management to come up with some practical suggestions for improving the sustainability of project benefits. The Committee considered the issue of sustainability as one of great concern and looked forward to OEDs continuing work in this area. - 49 - ANNEX B Page 18 VIII. BUILT-IN PROJECT MONITORING AND EVALUATION: AN OVERVIEW 30. Built-In Project Monitoring and Evaluation: An Overview (Report No. 5781) dated June 28, 1985 was distributed to the Executive Directors on July 30, 1985 (SecM85-899) and considered by the OPS Directors' Meeting of September 18, 1985 and by the Joint Audit Committee on January 22, 1986 (JAC/M 86-1). 31. The summary of major findings and principal recommendations con- tained in the above report were set forth in last year's Annual Report (Report No. 5836, Annex C, Item VIII). 32. At the Joint Audit Committee meeting OED was commended for this report. OPS staff noted that steps had been taken to improve monitoring and evaluation (M&E) training for both Bank and borrower staff, and that discus- sions were underway concerning revisions to documents on the Bank's policy and procedures for M&E. It also was reported that Management was preparing a response to the OED report, and the chairman of the meeting stated that the report would be reviewed together with the forthcoming Management response. 33. Management's response entitled "Improving the Efficiency of Monitoring and Evaluation of Projects" (JAC86-4) was considered by the Joint Audit Committee on June 11, 1986 (JAC/M86-5). The actions proposed to be taken by Management based on the main recommendations of the OED report are as follows: (a) Strengthening of Bank advisory and specialist M&E inputs should take into account the needs of project staff in all sectors. (b) Regional M&E capabilities should be strengthened, while a central M&E unit within OPS should remain. A working group on M&E should be established, comprising sector representatives, regional specialists and the central unit. (c) Training of both Bank staff and staff of borrower agencies is proposed to be improved. (d) Internal project monitoring should be seen as a project management tool, and Bank staff need to pay adequate attention to this function during project appraisal and supervision. (e) The OPS M&E unit will construct and maintain a customized roster of approved M&E consultants for use by project and borrower staff in all sectors. S50- ANNEX B Page 19 (f) It has to be accepted by all parties concerned that limita- tions inherent in evaluation methodologies justify only modest expectations for impact analysis within the life-span of a project. 34. The Joint Audit Committee was reassured by an improved picture related to M&E in current Bank activities, as reported in the Management response, compared with the less favorable OED findings for completed projects. Aspects of sharing responsibility for M&E between the regions and the central OPS unit were discussed. It was also noted that the issue of M&E in policy based lending was currently being addressed. IX. THE SMALLHOLDER DIMENSION OF LIVESTOCK DEVELOPMENT A REVIEW OF BANK EXPERIENCE 35. The Smallholder Dimension of Livestock Development - A Review of Bank Experience (Report No. 5979) dated December 18, 1985 was distributed to the Executive Directors on January 14, 1986 (SecM86-53). 36. The report reviews the results of the Bank's involvement in lending for livestock development from the first Bank-financed project in 1959 through December 1983. Special emphasis is placed in the report on small- holder lending. In all, 330 projects were analyzed comprising 91 free- standing livestock projects and 239 projects with livestock components. Of the 330 total, 124 projects were completed and audited and 206 were ongoing. Of the 124 audited projects, 47 were smallholder; of the 206 ongoing projects, 86 were smallholder. The review was undertaken partly in response to the widely-held perception in the Bank that livestock development invest- ments were producing less-than-satisfactory returns. 37. The major findings and Conclusions of the study are: (a) Performance of Bank livestock development efforts was highly variable depending on region, being generally satisfactory in EMENA, LAC, EAP and SA and unsatisfactory in WA and, espe- cially ESA. Many of the countries in the latter regions were newly independent with poorly-defined policies and priorities, limited infrastructure, weak manpower and material resources, a high prevalence of drought and pestilence and overly- centralized and urban oriented governments. (b) Within overall livestock lending, the smallholder share increased continuously over the years, even though lending for livestock generally declined in the 70's and 80's - with time - 51 - ANNEX B Page 20 there was also a relative shift from mostly free-standing livestock projects to livestock components which were part of larger agricultural projects. (c) Rates of return on livestock projects were lower than component projects and there was a marked tendency for ex-ante rates of return to be unduly optimistic compared with ex-post - free-standing smallholder livestock projects invariably produced the most disappointing returns due to special difficulties in their implementation. Although every effort was made to develop livestock projects which were innovative, they were usually ambitious in scope and size, were often weak technically, and more often than not implemented in a largely unfavorable economic climate in countries where governments many times showed limited sensitivity to smallholder development potential and needs. Target groups sometimes failed to involve themselves in project design and implementation out of misunderstandings or from distrust of government intentions - or they were other times excluded either for paternalistic or political reasons. (d) In reviewing project outcomes, the factors which appeared most frequently related to project difficulties were: (i) weak government commitment to livestock development and/or smallholders, (ii) an economic context lacking producer incentives, (iii) political instability, (iv) non-availability of technological packages adapted to existing farming systems, (v) weak institutional capacity, (vi) inadequate tenure and land use rights, (vii) project design which not always realistically took into account country strengths and weaknesses, and (ix) weak involvement of target groups in project design and implementation. (e) The analysis suggests that livestock overall may be more difficult than other agricultural subsector projects and that livestock development assistance, especially to smallholders, will probably require greater design, implementation and supervision inputs in future - the report notes that there is evidence that considerable learning has already taken place and future projects should perform betterl/. 1/ Indeed, audits of several recently completed African smallholder livestock projects lately distributed to the Board (Senegal Eastern Livestock, Zaire Ituri) show increasing success in Bank-assisted smallholder livestock efforts; also, PCRs coming to hand for smallholder projects in Burkina Faso, Mali and Niger appear to confrm the trend. The draft report on a recent audit of four Indian smallholder dairy projects is also highly positive, the projects being concluded by the audit to be amongst the best to date in the Bank's overall livestock portfolio. - 52 - ANNEX B Page 21 (f) The study supports the view that the Bank should continue and probably increase its support for livestock development, especially to smallholders, given the high potential for raising the latter's incomes and living standards. Livestock are a key element in raising farm productivity and cash flow and it is difficult to conceive of sustained increases in smallholder agriculture in most areas in the., world without attention to livestock development. Demand for livestock products is also increasing rapidly in most developing regions, and livestock investments are expected to be increasingly economicsAly attractive. (g) The report proposes increasing Bank support for dairy development, particularly relative to beef production. Milk demand is growing rapidly in most developing areas, milk is a food often easily within the reach of the poor consumer, and it is usually readily produced by small farmers. Milk production utilizes excess family labor, makes more cost- effective use of better feeds, and provides a continuous cash income. The primary requirement for increased milk production in most developing countries is the organization of marketing (including incentive pricing), processing and distribution facilities. The report considers smallseale dairying to be one of the most promising avenues for increasing smallholder cash flows and one in which additional emphasis would provide satisfactory returns in future Bank livestock lending. (h) The report concludes that livestock projects have not generally been as unsuccessful as perceived and that they present the best possibility for raising smallholder incomes and living standards and, when properly implemented, can be an excellent instrument for poverty alleviation. 38. The Committee commented OED for a most valuable report. It also indicated the need to ensure Bankwide dissemination of the report's findings, particularly the conclusion that livestock projects did not appear to have been #s generally unsuccessful as perceived. Several speakers emphasized the need to ensure that the findings of the report were reflected in both Bank livestock development and in future operations. The need for ongoing analysis of Bank experience in the livestock subsector was generally agreed (the importance of breed improvement, pasture development, land tenure, environmental issues and the problem of bankrupt smallholders all being noted in this context). The effect on project outcome of international meat and milk dumping was also discussed, it being noted that Bank policy on the dumping issue would be addressed in the livestock policy paper currently - 53 - ANNEX B Page 22 under preparation. There was a Committee consensus that, as a follow-up to the report, consideration should be given to a joint OED/OPS/regional staff report to review the above aspects in particular and to identify in greater detail specific areas of success and failure in the Bank's livestock develop- ment efforts and to give increased emphasis to areas where the Bank's experi- ence has been most positive. X. WORLD BANK IN PAKISTAN: REVIEW OF A RELATIONSHIP, 1960-1984 39. The World Bank in Pakistan: Review of a Relationship, 1960-1984 (Report No. 6408) was distributed on January 27, 1986, and was considered by the Joint Audit Committee on February 19, 1986. Approach 40. This study, the first of a series initiated in 1985, examined the relationship between the World Bank and Pakistan over a period of twenty-five years. The interactions, instrumentation and the results achieved were considered at the macro-economic, sector policy, institutional and project aggregate levels in the major areas of Bank involvement in that coc.ntry. Findings 41. The Bank has maintained a remarkable continuity in its operations in Pakistan. Throughout it has proceeded on the assumption that its business is to help the borrower in the pursuit of three development goals: growth, equity and self reliance. While the borrower's relative emphasis on these objectives changed somewhat over time, growth has always been given pre- cedence. The sharper differences have been over the appropriate means for pursuing a particular goal in a particular period, e.g., in the mid-1970s over the extent to which growth should have been promoted by expansionist (inflationary) budgeting. 42. There has been continuity, evolution and flexibility in the instru- ments the Bank employed in the pursuit of these goals. The Bank's basic instrument in Pakistan has been the project loan and, since quite early on, the Bank's project design, appraisal and implementation have set a standard of professional, workman-like competence. Project lending has been supported by the use of in-depth development analyses, based on extensive economic and sector work, not only to underpin the Bank's one-on-one relation with Pakistan but also to support its leadership of the aid consortium. Likewise, the Bank funded technical assistance schemes in support of its other func- tions. The policy dialogue and non-project lending have also been part of - 54 - ANNEX B Page 23 the Bank's assortment of instruments. More recently, SAL and sector loans have helped establish a more sustainable pace of inter-linked policy discussion and non-project lending. 43. The Bank has been creative and resourceful in its Indus Basin work in the 1960s. It helped establish and led the aid consortium during the 1970s, and was instrumental in the 1974 debt rescheduling. The personal relations of its officers, officials, and experts with their Pakistani counterparts were close and effective. Its economic and sector work was regarded as the most authoritative outsider assessment of Pakistan's economic performance and policies. But in a sense, the Bank's own attitude and approach was too harmonized with that of the borrower. The Bank did not succeed in persuading the Government to undertake major trade reforms, and it lagged more than it led the Government in its perceptions of social need and redistributive opportunities. In agriculture, the Bank emphasized the need for better services to farmers and for improving the yields of small and medium farms by focussing on water management, research and extension, and agricultural credit. The social sectors did not receive the attention they deserved and anti-poverty initiatives were not promoted. In education, the Bank took some tentative steps in the fields of post-secondary and higher education, the training of teachers and agricultural extension agents; but it did not grapple with the vast needs of primary and/or basic education. Locked into a preconditions mode, the Bank was not prepared to provide flexible funding on a matching basis to induce efforts to raise local resources for expansion of rural and other anti-poverty programs. Throughout the early 1980s, the Bank focussed on needed improvements in macro-economic management including a major advance in the Government 's tax and savings efforts, advocated a better climate for private investment, and impressed the need for more export-led growth combined with efficient import substitution. Recommendations 44. The main recommendation of the study are: (a) Erratic Funding. Availability of Bank resources is critical for enlarging the sectoral span of the Bank's program and for enhancing its capacity for policy-based lending. Therefore, erratic annual financial allocations should be avoided. (b) Personnel Turnover. During the 1970s several key positions- regional vice president, country programs director, country economist--experienced a degree of turnover that, per se, was undesirable. - 55 - ANNEX B Page 24 (c) Headquarters, Missions, Field Office. To reduce the high frequency of the Bank's missions, the Resident Office should be strengthened by delegating broader functions and assigning more senior staff. (d) Aid Coordination. There is scope for more detailed and extensive policy discussions and for harmonization of donor programs within the present Consortium frame. While the Bank should take a stronger lead in aid coordination, it is only the associate leader. The lead must be taken by the Government--with the Bank assisting, as needed, in the exercise of such leadership. (e) Social, Institutional and Human Resource Development. The Bank should try to influence the Government's relatively low emphasis and priority on fertility restraiat, health, the status of women, literacy, and education, including higher and technical training, and research and technology development. (f) Federal-Provincial Finance. Provinces are denied the incen- tives and funding to join Bank-favored social and human- resource initiatives at the provincial level. The Bank should actively seek to impress on the Government the need for changing the status quo. (g) Resource Mobilization. The Bank should reinforce the Govern- ment's recent efforts to improve the mobilization of private and public savings by helping in the design of an appropriate analytical program and institutional framework. (h) Irrigation, Water Logging and Salinity. A long-run program should be developed for areas with saline water. Large investmeats will be required over the next two decades to maintain agricultural productivity in the Indus Basin and to develop the institutional capacity to conduct research and training on water resource engineering and management. (i) Agricultural Credit. The Bank should reconsider its views on interest rate subsidies which have facilitated tractor mechanization but have had limited impact on the growth of agricultural output; should place greater emphasis on invest- ments in private tubewells which have demonstrated their potential for enhancing agricultural production; and should encourage agricultural credit institutions in Pakistan to become genuine mobilizers of rural savings. - 56 - ANNEX B Page 25 (j) Rural Development and Human Capital. The Bank needs to intensify its efforts on rural reform and human resource development, capitalizing on the numerous projects in agricul- ture which it has supported. (k) Trade Polic The Bank should continue to emphasize the need 'or rationalization of incentives; greater reliance on tariffs and less on quantitative restrictions; appreciation of the effects of the trade regime on intersectoral terms of trade and Pakistan's comparative advantage; and assessing the implications of the temporary emigration of workers for savings accumulation. The Bank should also impress the need for expanding the market for manufactured products in the industrialized countries. (1) Energy Development. The Bank has a played an active role in undertaking technical studies and in coordinating bilateral and multilateral assistance in the energy sector. It should continue its efforts in this vital area. XI. SUSTAINABILITY OF PROJECTS: REVIEW OF EXPERIENCE IN THE FERTILIZER SUBSECTOR 45. The report No. 6073 on Sustainability of Projects: Review of Experience in the Fertilizer Subsector was distributed on February 26, 1986. It has not yet been considered by the Joint Audit Committee. Approach and Findiugs 46. The study aimed at identifying the factors and assessing their impact on sustainability based on post-completion project experience. The analysis focAssed on 14 fertilizer plants in seven countries, representing all projects financed by Bank/IDA in the early 1970s which had been operating by mid 1985 for at least four years after completion. Seven plants had sustainable levels of benefit flows, while five could become sustainable if technical or raw material problems were resolved. Two plants were judged to be unsustainable. Lessons of Experience 47. The major lessons of experience are: (a) Productive efficiency and capacity utilization were important elements in sustaining acceptable levels of benefit flows through the economic life of projects. - 57 - ANNEX B Page 26 (b) Given the sensitive interaction among process design, equip- ment and local conditions, the fundamental catalyst to combine properly all these elements is the human resource, with emphasis on managerial and technical skills, including con- tinuity of tenure. (c) Technology transfer and adaptation and the development of local technological capabilities can influence performance and, by extension, sustainability. (d) Government policies relating to such issues as managerial structures and autonomy were critical for enterprises in the public domain. (e) Factors outside the control of the enterprises affecting sustainability included adequacy and quality of raw material and/or energy supplies, pricing policies, and unanticipated changes in demand. (f) Institutional development was another critical factor impact- ing on sustainability. This involved inter alia establishment of management information systems and undertaking training programs for management, technical and operating personnel, combined with the use of technical assistance. But while training of local staff received considerable attention, there was less general acceptance of consultant use, particularly during implementation and the initial stages of operation. (g) Where projects are linked to the development of the local capital goods industry, the tradeoffs between the expected benefits of technology transfer and the potential costs in terms of longer delivery periods, higher procurement costs and mechanical failures in operation should be carefully weighed. Recommendations 48. The objective of sustainability could be enhanced by additional efforts to facilitate technology transfer, involving inter alia post- completion supervislon missions and interchange of experience among Bank- supported projects. Iaprovement in public sector management could also be dealt with on more general levels, including policy dialogues covering such issues as operational autonomy, appropriate incentives and adequate organiza- tional structures. Finally, more circumspect forecasts of demand and price trends could help weed out less promising ventures. - 58 - ANNEX B Page 27 XII. THE WORLD BANK AND SRI LANKA: A REVIEW OF A RELATIONSHIP 49. The World Bank and Sri Lanka: A Review of a Relationship (Report No. 6074) was distributed on February 24, 1986 and was considered by the Joint Audit Committee on March 19, 1986. Approach 50. This study, the second of a series initiated in 1985, examined the relationship between the World Bank and Sri Lanka over a period of thirty years. The interactions, instrumentation and results attained were examined at the macro-economic, sector policy, institutional and project aggregate levels in the major areas of Bank involvement in that country. Findings 51. Sri Lanka has had three main objectives since independence: economic growth, equity and self-reliance. Sri Lanka may be unique among mixed economies in achieving a massive transfer of resources from the better-off to the poor. The Bank's view of its own role evolved over time, but its approach remained attuned to Sri Lankan reality and the Government's objectives. Until the early 1970s, the Bank was primarily concerned with growth and only incidentally with self-reliance. The main focus was on specific projects executed with Bank support and on resource transfer. Equity and self-reliance did not receive due attention. But in the 1970s, emphasis was placed on equity, defined as alleviation of poverty and catering to basic needs. There was also a gradual appreciation and greater concern with devising appropriate policies and developing an appropriate institu- tional framework as a means to achieve growth with equity. 52. The Bank's implicit, and occasionally explicit, position with respect to Sri Lanka has generally been that only a growing economy could support substantial welfare transfers. In its economic and sector work the Bank tended to assess welfare policies in terms of their effect on growth, and paid less attention to their benefits in terms of equitl. It was therefore supportive of the Government's economic reforms in 1977, which aimed at stimulating growth of output, efficiency and employment. Less attention was given to the possible adverse consequence on the poorest of the sharp reduction in food subsidies. On the whole, the Bank has played a minor role in supporting, or even analyzing, welfare and equity issues and policies in Sri Lanka. 53. The Bank has had a significant and positive role in Sri Lanka's success in increasing growth and in improving the efficiency of the economy during the market-oriented periods, especially in 1965-70 and from 1977 - 59 - ANNEX B Page 28 onwards. The Bank was relatively effective because it, and most Aid Group donors, supported policies espoused by an important group of officials within the Sri Lankan Government, who had developed their own views of desirable policies largely from their own anaiysis of the Sri Lankan needs, experience and reality and through interaction with Bank staff. Lessons and Recomendations 54. Major lessons and recommendations of the study are: (a) Policy Reforms. The Bank needs to focus its program more clearly (e.g., tree crops, manufacturing) and to relate its size more plicitly to policy changes in the targeted sectors; macro-economic nditionality should be rationalized; and efforts in the Aid Group mobilize additional resources for Sri Lanka should be intensified. (b) Timeliness of Intervention. Unlike Pakistan, there has been discontinuity in the Bank's relations during the dirigiste regimes. There is much to commend a low-keyed approach and presence when conditions are not favorable for major Bank initiatives. An integral element in this strategy is the Resident Mission, which needs to be strengthened, especially with respect to its capacity to facilitate the policy dialogue and to formulate an assistance strategy. (c) Improving the Policy Dialogue. An amalgam of socio-political factors, vested interests, and the intricate workings of policy formulation can often act as genuine constraints in the implementation of the Bank's recommendations. The Bank therefore needs to exercise patience and to appreciate the reasons for the Government's lack of receptivity. A better understanding could lead to a more productive dialogue. Greater involvement of local officials in economic and sector work could improve understanding of the problems involved and facilitate the "learning process" for Sri Lankan officials. (d) Aid Group Coordination. The problem of aid coordination remains a difficult one, especially since the Government prefers to have a one-on-one relationship with donors. If the Bank were to assume a leading role, many donors may mistake the move as subordination. The Government should therefore be encouraged to take the lead in aid coordination, with the Bank ensuring coordination at the operating levels. - 60 - ANNEX B (e) Lending for Agriculture. Bank lending for agriculture has generally been in le with the strategy of supporting rural development and rehabilitation and as a vehicle for engineer- ing policy changes. The Bank's concentration in the past on institutional development has had limited success, since the Bank was insufficiently aware of the politically sensitive nature of existing institutional arrangements. While Bank assistance in institution building has had satisfactory results at the project level, it has not had a tangible impact at the sector level. The Bank should avoid pursuing agricul- tural sector work in isolation from the Government's own activities and priorities, and should concentrate on those areas where the potential impact on institutional and policy reforms can be greatest. (f) Industrial Policy Formulation. The protagonists of market- oriented industrial policies which were instituted in 1977 should be assisted in their efforts, and "textbook solutions" which have proved to be ineffective in the past should b2 avoided. (g) Energy Development. The Bank's virtual withdrawal from the power sector between 1971-1979 has had a high cost; the intended institution building was not achieved, and the Bank could not influence forward planning and the adoption of least cost solutions. In later years, the Bank's sector work has had a strong positive influence on the formulation of energy policies by the Sri Lankan Government. XIII. BANGLADESH: REVIEW OF EXPERIENCE WITH IMPORT PROGRAM CREDITS I-X 55. The report (No. 6080-BD) on Bangladesh: Review of the Experience with Import Program Credits I-X was distributed on February 24, 1986, and was considered by the Joint Audit Committee on July 30, 1986. Approach 56. The report discusses the genesis and evolution of program lending in Bangladesh; the initiatives taken to streamline the imports control system and their impact; the design and effectiveness of the action programs and technical assistance schemes enacted to rehabilitate the jute, textile and pulp and paper industries; the progress made in instituting an export development program and the results achieved; the impact of policy reforms in - 61 - ANNEX B Page 30 agriculture, focussing on foodgrain production, procurement and distribution, pricing and distribution of agricultural inputs, and agricultural sector financing; the experience with the disbursement of IDA funds and procurement arrangements; and the impact of the credits on individual sector performance and the balance of payments. Finally, the report attempts an overall assess- ment of program lending to Bangladesh and draws the lessons learned. Findings 57. Despite the difficult political and institutional climate, pro- gress has been made between the mid-1970s and mid-1980s in advancing an array of objectives. However, such progress has been modest and uneven. Achieve- ments have been very closely associated with activities where initiatives and commitment have been sustained by political will and where the implementing mechanisms put in place continued to enjoy institutional support. The impact of the action programs in key areas of concern in the industrial sector, such as increasing operational autonomy, strengthening management control systems, improving production efficiency, and restoring financial viability, has been limited. Certainly, they have had no success in inducing the preparation of a development strategy for the two crucial industries, namely jute and textiles. On the whole, the results achieved so far fall short of original expectations, with the exception of the partial denationalization of the jute and textile mills. In agriculture, results have been encouraging in efforts to stimulate foodgrain production and to improve the marketing and distribu- tion of inputs, but very limited progress has been made in rationalizing the agricultural credit system. In general, implementation has been relatively more forthwith, although not necessarily more successful in terms of results, in the area of institutional arrangements and policy reforms, e.g., export promotion incentives, organization and administration of the imports control regime, pricing of agricultural inputs, and rationalization of public procurement and distribution schemes. Lessons of Experience and Recommendations 58. As an instrument to support sectoral adjustments and institutional reforms, the experience with program lending affirms the following conclu- sions: (a) Circumspect and focussed economic and sector work is of critical importance in appreciating the modus operandi of key economic and institutional parameters, identifying deficien- cies in performance, institutional arrangements and policies, and in enhancing the Bank's capability to design meaningful and monitorable action programs to address specific problems. - 62 - ANNEX B Page 31 (b) Deeper understanding and appreciation of the cultural back- ground, political realities, institutional arrangements, attitudes and, more generally, of the way decisions are taken and implemented in a country prior to proposing organiza- tional, institutional and policy reforms can lead to less dogmatic approaches and more workable solutions. (c) Action programs and policy reform packages should be designed in a fashion that reflects and articulates the underlying analyses of the issues involved; defines each action with a degree of specificity that makes it easy to understand, implement and monitor; and ensures that programs and packages are staggered over a reasonable period of time to take into account the borrower's (and/or entity's) institutional and administrative capacity. Excessive optimism in this respect could lead to overloading a weak bureaucracy and the premature disintegration of such programs. (d) Concurrent involvement of bilateral and multilateral aid agencies in technical assistance programs necessitates careful coordination of the total effort, to ensure congruity of purpose and synchronization of actions, to prevent the trans- mission of conflicting signals and recommendations, and to avoid duplication and dissipation of effort. (e) Non-project lending in support of well-designed action programs can contribute to sectoral adjustment and institu- tional and policy reforms, provided there is political will, firm commitment and unswerving support on the part of the Government and the entities involved. (f) Inertia and tentativeness on the part of the Government, and continuation of the status quo by adopting palliatives or fitful measures, only helps to perpetuate a morbid situation, to postpone the hard political choices that inevitably will have to be made, and to make the eventual decisions even tougher. (g) Institution building is as important as the transfer of resources and extension of technical assistance; but it is a particularly difficult and drawn out process, and commitment, patience, and perservance are all required to achieve long- lasting results. Furthermore, sustainability of effort is crucial in solidifying any progress already achieved. - 63 - ANNEX B Page 32 59. The experience with program lending involved with actions that addressed more specific industrial sector issues provides useful insights on a number of issues: (a) Nationalization of industrial activity has not contributed to the solution of the deep-seated problems of the sector and probably has exacerbated them. In the search for workable solutions to improve public enterprise performance, privatiza- tion deserves to be thoughtfully considered as an expedient and viable alternative. (b) The attainment of corporate objectives in the public sector can be vitiated by the absence of managerial autonomy over key decisions and the ubiquity of government controls and inter- ventions built around a firm's operations. If the public enterprise is perceived as a serviceable institution, develop- ment of ex post, as opposed to ex ante, forms of controls and accountability seems to be a more sensible approach to achiev- ing these desiderata. (c) The caliber of top management and its commitment is a powerful driving force behind the development of an agency or the growth of an enterprise; appointment of unqualified executives or frequent change of competent managers, resulting in instability and lack of leadership and direction, tend to undermine the effectiveness of such entities. (d) Featherbedding, staff compensation, and inadequate reward of high performance remain vexing issues in public enterprises. (e) The importance of adopting a professional approach toward preventive maintainance programs as a means of sustaining operating efficiency cannot be emphasized enough. 60. The experience with program lending in the agricultural sector leads to the following observations: (a) Lack of realistic and effective resource planning and manage- ment, slow progress in institutional reforms, and budgetary constraints have been major obstacles in achieving foodgrain production targets. Nevertheless, the strategy emphasizing small, low-cost, quick-gestation irrigation and flood control schemes, as well as improvements in infrastructure, has had a high pay-off in terms of production growth. - 64 - ANNEX B Page 3 (b) Regular and timely adjustments of foodgrain support prices can provide farmers with the requisite incentive to use modern inputs associated with high-yielding technologies. (c) The substantial reduction of unit subsidies has not affected fertilizer demand, suggesting the presence of other potent facto-i which encourage greater fertilizer use, including expansion of irrigation, increased use of fertilizer by farmers using substantially less-than-optimum levels, con- tinued shifting of cultivation from low to high-yielding varieties, and access to credit. (d) Poor organization, weak management, and injudicious financial policies hampering the ability of the financial institutions to cater effectively and equitably to the legitimate needs of a major segment of service recipients, as well as frequent abuse of the existing institutional arrangements, can have devastating effects on production and productivity growth in agriculture and undesirable social repercussions. 61. Attempts to streamline the imports control regime and to devise arrangements to promote exports yield some interesting findings: (a) Restrictive import policies, and the attendant reliance on administrative regulations and controls to ensure their effectiveness, tend to adversely affect production and resource allocation, are open to abuse, take a heavy toll on the country's scarce administrative resources, and stifle entrepreneurship. As a result, such policies should be used sparingly and only as an interim expediency until a rational tariff structure and a well-designed incentive system can be put in place. (b) Speedy import liberalization may not* be expedient, and is likely to be resisted in the absence of a sufficiently broad export base and a rather stable inflow of foreign receipts. (c) The existence of a transparent rationale for instituting an export incentives scheme is very important. Incentives should basically be compensatory in nature, counterbalancing discriminatory domestic levies and/or equalizing advantages of competitors due to subsidies, and should not aim at making up for production inefficiencies. (d) The effectiveness and serviceability of well-conceived incen- tives can be easily stultified by poor implementation practices. - 65 - ANNEX B Page 34 (e) The strong link between an efficient production framework and export potential, in view of the constraining effects of supply inelasticities, has to be duly appreciated in efforts to improve export performance. (f) Reform of export procedures and documentation can be distress- ingly slow in implementation, as it affects deep-rooted attitudes and vested interests, entails far-reaching changes in the internal procedures of agencies involved and the flow of documents, necessitates the establishment of new controll- ing mechanisms, and requires the development of inter-agency trust. XIV. WORLD BANK LENDING CONDITIONALITY: A REVIEW OF COST RECOVERY IN IRRIGATION PROJECTS 62. World Bank Lending Conditionality: A Review of Cost Recovery in Irrigation Projects (Report No.6283) dated June 25, 1986 was distributed to the Executive Directors on July 11, 1986 (SeeM86-826). 63. The Bank's policy on cost recovery was very general up to 1976, aiming to recover all project costs from beneficiaries. For irrigation projects, recovery of operation and maintenance (O&M) costs was considered a minimum objective, with capital costs to be recovered in addition as practicable. The policy changed in 1976, requiring that water charges be levied progressively on project farmers in relation to incremental incomes derived from the project. The policy was further revised and clarified in 1984. 64. The present report is based on a review of documents for 48 com- pleted irrigation projects, and on the findings of field investigations in four countries. It does not focus on any particular project or on Bank experience in any particular country, but presents a global view of the Bank's experience with conditionality in the irrigation sector. 65. Following is a summary of conclusions and lessons learned: (a) Cost recovery in irrigation projects has not been good. The proportion of O&M costs recovered frequently has been between 15% and 45%, and a portion of capital costs has been recovered in very few cases. (b) O&M of irrigation systems has been satisfactory in only about one-half of the projects. - 66 - ANNEX B Page 35 (c) There has been slow progress in establishing farmers' organi- zations for operating and maintaining tertiary water distribu- tion systems, and cost recovery for O&M of tertiaries frequently has been inadequate. (d) Poor operation and maintenance of irrigation systems by irrigation agencies and inequitable water distribution, especially within tertiary systems, have been important factors in farmers' unwillingness to pay water charges. In contrast, in several instances farmers have been willing to pay high water charges for water supplied efficiently from private sources. (e) Adherence to cost recovery covenants was low, largely because of lack of government commitment, unreliable water supply due to poor O&M of irrigation systems, and the often heavy burden of other taxes imposed on farmers. The Bank's response to non-compliance with cost recovery covenants has varied among regions and countries. (f) The Bank's recovery objectives have not always been in harmony with borrower country socio-economic conditions and cost recovery politics. A continuous dialogue with governments might be a better approach than project covenants for ensuring sound cost recovery policies. (g) Covenants may be dropped or loosely interpreted, as has been the case, for sound reasons, conflicts between historic and contemporary policy goals being among them. Concluding from this, covenants on non-technical socio-political matters that are subject to change possibly should be of limited duration, subject to review or excluded. (h) Adequate O&M of irrigation investments is necessary for a reliable water supply, which in turn is a necessary but not a sufficient condition for profitable farming. Profitable farming is a precondition for cost recovery. Consequently, the Bank should insist on borrower plans for assured funding of O&M. Provisions for O&M in existing projects should be examined prior to implementing new projects. (i) Direct charges, based on nominal price of water, have proven inappropriate on many occasions. Bank policy should therefore be applied flexibly, taking into account various technical, socio-economic and political factors as well as other taxes already imposed on farmers. - 67 - ANNEX B Page 36 (j) There should be a visible link between water charges and the quality of irrigation services provided, hence, irrigation agencies also should collect water charges. To counteract erosion of water charges due to inflation, such charges should be indexed. (k) Applied cost recovery principles should be understandable to farmers. A Bank policy mixing cost recovery income distribu- tion objectives and focussing upon incremental incomes runs the risk of confusion. A better policy basis are fairness and reliability of water distribution among farmers in line with appropriate systems designs. XV. THE IMPACT OF WORLD BANK LENDING FOR EDUCATIONAL DEVELOPMENT IN KOREA: A REVIEW 66. The Report No. 5950 on the Impact of World Bank lending for Educa- tional Development in Korea: A Review was distributed on December 5, 1985, and was considered by the Joint Audit Committee on March 19, 1986. Approach and Findings 67. The study aimed at assessing the Bank's role as a lender for educa- tion within the context of the economic ' ielopment of Korea and the evolu- tion of the education system. Between 1968-1984 the Bank lent US$965 million for five education projects and 36 project related training components in non-education projects. 68. The report notes that the economic development strategy adopted by the government, which concentrated on rapid industrialization, produced significant changes in and placed significant pressures on the education system in terms of producing a highly skilled technical labor force. During the time under review, government expenditure for education: (a) increased relative to both GNP and the total government budget and (b) shifted propor- tionately from lower to higher education, reflecting the importance attached to an increasingly differentiated and responsive education system. Foreign financing, and particularly the World Bank, represented a major source of funding: of the US$464 million borrowed for education development, the Bank's share was on the order of 44%. 69. Bank lending focussed and had the grestest impact on technical/ vocational education at all levels, which also the segment of the education system were the greatest achievements have been recorded. The education projects reveal an ougonig and fruitful dialogue between the Bank and the - 68 - ANNEX B Page 37 Borrower and a carefully constructed development strategy that encompassed equipment provision, teacher/instructor training, curriculum revision and program evaluation. The largest proportion of Bank financing was used to purchase equipment for the technical/vocational programs, which, despite unresolved problems of equipment management and utilization, was an economic use of the foreign exchange and contributed directly to the quality of the programs. 70. The study also revealed areas for concern in the patterns of educa- tional development in the period under review, including a certain neglect of basic education and a tendency toward creating new institutions without due regard for those in existence or the balanced development of the sector. Lessons of Experience 71. By concentrating external resources on an educational sub-sector identified as having priority, and systematically working toward upgrading various aspects of that sub-sector over a given period, Borrowers can achieve sustainable progress. 72. Project effectiveness and was enhanced not only by pursuing a given investment strategy which encompassed various critical elements (equip- ment provision, teacher and instructor training, curriculum revision, research and evaluation activities, and institutional development), but also by building linkages to employers and monitoring evolving manpower needs. 73. Developing a non-formal network of institutions to provide employ- ment-oriented training, occupational analysis of skills which orients the non-formal training programs, and externally-based occupational certification strengthens the technical vocational education system. 74. In technical-vocational programs, it is very important to strike a balance between concepts of terminal and lifelong education. The Korea experience demonstrates the utility of using junior colleges to produce technicians while leaving the door open for some technicians to undertake further training to become engineers. - 69 - ANNEX C Page 1 ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) MONITORING AND EVALUATION IN PROJECTS 1. Following a review by OPS directors of an OED report entitled, "Built-in Project Mon .oring and Evaluation: An Overview" (SecM85-899), the Monitoring and Evaluation Unit in the Agricultural and Rural Development Department (AGRKE), in collaboration with other sectors, conducted a Bank-wide survey of the current status of project M&E. The survey's basic findings were endorsed by the Operational Vice Presidents in January 1986. AGRME was mandated to coordinate M&E work of the OPS sector departments and link it with that of the regional projects departments. An informal coordinating committee, with representatives of all these departments, was established. A first task, as recommended by OED, will be to revise the M&E sections of the Operational Manual in the light of accumulated Bank experience. 2. Much more attention is presently given to M&E in Bank projects than before, especially in agriculture and rural development, urban development and education projects, although M&E emphasis in other sectors is also on the rise, as evidenced by the sections added this year on Transport. This Annex provides details, on the basis of information provided by OPS, on the present status of M&E activities in major sectors. A. Agriculture and Rural Development 3. AGRME has functional staff responsibilities for M&E in agricultural and rural development projects. It also has a pool of expertise available to support regional departments. AGRME continues to review M&E provisions in agriculture and rural development projects at the appraisal stage. In line with the changing advisory role of OPS, review of project-specific M&E is taking place earlier in the project cycle, although this is made difficult by the fact that for such components, little detail is available prior to appraisal. However, AGRME also made direct contributions for the design, review and technical assistance on buit-in M&E in agriculture and rural development projects. During the year under review, AGRME supported the Regional Projects Departments in appraisals and supervisions to Thailand, Indonesia, China, Ivory Coast, Malawi, Kenya and Mali. In addition, substantial support was also prnvided for the analysis of longitudinal data pertaining to Malawi's National Rural Development Program. This culminated in a seminar attended by all program managers and evaluation of`icers in Malawi. - 70 - ANNEX C Page 2 4. Consistent with the recommendations in the above mentioned OED Review and the OPS Response, M&E training--both internal and external-has received particular attention in the past year. Two residential regional seminars for Bank staff were held during the year, one for the African Regions, the other for the Latin America and Caribbean Region. In each case, special training materials, including case studies on actual implementation experience, were presented. AGRME also contributed to training for borrower country officials, conducted through EDI, Particularly noteworthy in this context were EDI's Microcomputer Applications Course held in Washington, the Agricultural Project Planning and Implementation Course in China, an Irrigation Management Course in Sri Lanka and the Agricultural Management Training for Africa Program. 5. AGRME undertakes research and development in order to incorporate modern techniques into M&E programs. Examples include work on the use of microcomputers for monitoring progress as well as for in-depth analysis of data collected for evaluation purposes, including one data set for Nigeria on the reponse of farm output to fertilizer use. 6. A Handbook on Monitoring and Evaluation of Agriculture and Rural Development Projects was published in 1982. This will be replaced by a three-volume work on Monitoring and Evaluation of Agriculture and Rural Development Projects which is being prepared by AGRME in collaboration with IFAD and FAO. These manuals will incorporate experience gained to date and will further elaborate M&E concepts and approaches that have evolved over the years and amplify them for application at the subsectoral level also. Submission to the Bank's Editorial Sub-Committee is planned for the Fall of 1986. 7. In addition to this international endeavor, AGRME continues to maintain other international links regarding M&E in agriculture and rural development projects. It continues to participate in the U.N. Task Force on Rural Development and more in particular its Sub-Committee on Monitoring and Evaluation. During the year, AGPME staff also held discussions on M&E concepts and practices with German and French development agencies. B. Water Supply and Urban Development 8. in projects in these sectors, the establishment of a monitoring system early in the project cycle continues to be a standard appraisal requirement. Under these systems, progress toward agreed physical, financial and institution-building targets is reported on for use by national agencies and Bank staff. Monitoring may be done by separate project coordination units, usually at the national level, or undertaken within an existing management structure. - 71 - ANNEX C Page 3 9. Monitoring in urban development projects has expanded beyond keeping track of narrow project performance indicators to monitoring of adherence to agreed upon sectoral policies (as for example, the Chile Public Sector Housing Project approved in FY85). We can expect to see much more of this sectoral monitoring as urban development projects become increasingly concerned with city investment strategies, and overall urban efficiency. 10. Monitoring in water supply projects continues to be concerned with improving the operational and financial efficiency of water authorities. Appraisal reports contain detailed monitoring indicators to ensure adherence to targeted schedules for contracting, project implementation, and disburse- ment. 11. Evaluations tend to be selectively done rather than being a built- in feature of these projects. Socio-economic studies are done on an ad hoc basis in response to the particular needs of an evolving project. Among the urban development projects, for example, special surveys of families and businesses are done in selected sites to determine the extent to which the anticipated progressive development is occurring. At times, evaluations are carried out to assess the economic impact of land tenuring and reconstruction activities to explore, for example, ways of linking short-term programs with long-term development goals. In water supply projects, evaluations are undertaken to determine a community's and an individual's capacity to pay to determine the level of cost recovery that could be expected from consumers of the services, particularly rural ones. 12. The Urban Development Departments 1985 publication, "Monitoring and Evaluating Urban Programs: A Handbook for Program Managers and Researchers" has been widely distributed. C. Education 13. The incidence of monitoring and evaluation components in education projects has increased. All 20 education projects approved in FY86 included monitoring and evaluation components compared to only 70% of the education projects approved in FY85. Regional staff have continued to give particular emphasis to the development oi borrower's institutional capacity for monitor- ing and evaluation. Where needed specific assistance in this respect was provided more in particular through development of detailed performance indi- cators (for 10 of the 20 projects approved in FY86). 14. A review of M&E for the period FY80-85 was conducted by the Educa- tion Department (EDT) during FY86. Statistics show that 85% of projects approved during these six years included M&E components. This review indicated that monitoring was strongest for financial and physical implemen- tation, and less strong for activities at the school or beneficiary level. - 72 - ANNEX C Page 4 It also showed that it is more common to monitor project progress than to evaluate project impact and that project staff find evaluations less useful than monitoring, for the design of subsequent projects. A principal reason for this has been projects are generally completed before evaluations are available. 15. An evaluation specialist was recently appointed in EDT and the department continues-as a priority activity-to provide support to Regional staff in improving M&E in education projects. In FY86, assistance from the Education Policy Division (EDTEP) expanded to cover monitoring and evaluation issues as part of work to strengthen education project management. This work was disseminated to Regional staff in two staff seminars during the fiscal year. EDTEP also completed evaluative reviews of Bank experience in non- formal education and in distance education. A number of activities of the Research Division of the Education and Training Department (EDTRS) during FY86 were also directly relevant to operational evaluation, such as a cost- benefit analysis of Colombia's well-known SENA (Servicio Nacional de Aprendaje) job training program, and a longitudinal study of school quality and the cost effectiveness of increasing school resources in Northeast Brazil's rural primary schools. D. Transport 16. M&E activities take different forms in the various transport modes. For simplicity, we are dealing with five groups, as follows: (a) Railways (b) Ports and Shipping (c) Highways (d) Rural Roads (e) Construction Industry (a) Railways 17. Railways, like all revenue-earning organizations, maintain compre- hensive financial monitoring systems, including income statements and balance sheets. In addition, they monitor an exte 'sive array of physical indicators, relating to traffic, state and utilization of assets, and staff produc- tivity. The financial health of a railway can be monitored using such indicators as the ratios of revenues to expenditures, return on assets, the debt/equity ratio or the ratio of subsidies to total revenues. Indicators of staff productivity are staff per km of line, and number of traffic units per employee. Equipment utilization is measured by locomotive km per year, locomotive and car availability as a percentage of total fleet, train load - 73 - ANNEX C Page 5 per locomotive, and turn-around time per car. The number of derailments per year and the extent of speed restrictions indicate the state of the track. The adequacy of maintenance is indicated by the availability of traction and rolling stock (ratio of days for which a vehicle is available for service, to number of days in period considered). 18. Evaluation of such indicators is at the heart of the management process. However, much more detailed information is needed for day-to-day action, depending upon the responsibilities of individual managers. Many Bank projects support management information systems. Their development involves an assessment of what information is needed by different managers, and on what time-scales. It may also involve acquisition or enhancement of data processing and data capturing equipment, and of telecommunications-the latter because of the railway's geographic spread and of the large quantity of data they generate. 19. Thus, the development of Management Information Systems (MIS) is a component of many proiects. In Mexico, the process of introducing the MIS system has spanned the life of several Bank projects, the latest Loan 1929-ME. It is now operational. In Colombia, consultants are in the process of developing systems in costing, financial planning, and accounting (Loan 2090-CO). In China, India and Brazil where railway systems are very large, the Bank is supporting the development of operating information systems to improve utilization of rolling stock and cost reduction in operations. 20. In complex projects, in the West Africa Region, they include a Project Coordinator for M&E purposes (e.g., Nigeria-Transport Parastatals; Ghana-Railways). These coordinators set up the machinery needed for M&E which could be conveniently continued from project to project. Besides, they incorporate an MIS component in projects in order to generate the requisite M&E material on a permanent basis for the management and, incidentally, to help generate the reports to the Bank. The MIS is planned in a modular fashion: first state is the financial MIS (costs, revenues, financial ratios); next, the operational MIS (traffic, operating and staff efficiency indices, etc.) and last, the maintenance MIS (failures meantime to repair, staff productivity, cost of maintenance, spare parts management). Recent examples where this modular strategy has been adopted are Nigeria railways and Ghana ports. (b) Ports and Shipping 21. Bank-funded port and shipping projects do not generally have explicit M&E components and supervision is usually concentrated, to a large extent, on physical and operational aspects of the project. Training components are often amenable to be monitored and evaluated, although mea- surement of training effectivenes is sometimes missing, as training is - 74 - ANNEX C Page 6 generally monitored on a performance basis alone. Institutional development, although part of most recent Bank-funded projects, does not always avail itself to meaningful M&E, as institutional development objectives are often too broad or general. Nevertheless, in the LAC region, an important com- ponent of ongoing (and proposed) projects is the development and implementa- tion of planning and control systems. Examples include, in Panama (Loan 2181-PA), technical assistance in planning and control, including upgrading of accounting, costing and budgeting systems; a system of equipment and inventory control; and improved collection of cost and port operating statistics. Improvements to cost accounting systems are also included as components in Uruguay (Loan 1798-UR) and in Mexico (Loan 2450-ME). 22. Port and shipping projects usually have specific requirements with respect to pricing and regulation of port or shipping services. Although covenants often include requirements for the adjustment of port prices to cost reimbursing levels, M&E procedures for the actual pricing adjustment process are difficult to design and are usually absent. The same applies to the development of management information systems (MIS) which are often incorporated in port projects. While the monitoring of the design of MIS is usually accomplished quite well, evaluation of the effect of the MIS on the port institution and port management is often lacking. Post evaluation done in the Bank of the effects of port MIS systems is often weak because of the lumpy nature of port projects. Unlike highway and railway projects where the Bank often has a long-term relationship with the Borrower, port projects are not always immediately followed up by successor projects. By the time the project is completed, the MIS system may well be in only its incipient stages, and evaluation of its effects premature. Still, in the West Africa Region, they have instituted up-stream monitoring and management and ask for management restructuring wherever necessary in the very early stages of the project, and a supporting MIS is set up (e.g., Gambia Ports where consultants are reviewing management performance and providing feedback to the managers). Ports management is receiving an increasing degree of emphasis in the M&E context, just as it has been in the railways. 23. Most port projects M&E emphasize the physical implementation of the project with institutional, environmental and socio-political or organiza- tional factors given only minor, if any, attention. In fact, there is no formal qualitative or quantitative method in use for the general analysis of port projects and the assessment of their success or failure in meeting their objectives. The approach taken is usually a narrow quantitative financial and economic evaluation and qualitative physical and operational assessment of the projects performance. 24. Impact assessment of port projects again is usually concerned only with the operational, financial and economic impact on the immediate hinter- land and port users. Environmental and socio-political impacts of such proj- ects are, if at all, only peripherally addressed and hardly ever assessed qualitatively. - 75 - ANNEX C Page 7 (c) Highways 25. Highway projects normally do not have M&E components attached to them for the sake of monitoring and evaluation per se. However, for the sake of institutional development, and to serve the borrower's needs, the elements of monitoring can be classified as follows: (i) Road maintenance management systems which would include a basic data bank about all the roads in the network, as well as a system for monitoring of road condition, history of maintenance interventions, and traffic and load conditions. This serves as the basis for determin- ing road maintenance strategies, maintenance intervention levels, and prioritizing maintenance activities, road by road, year by year. (ii) Equipment management systems, which would include a basic data bank on all the equipment owned, as well as a system for monitoring equipment condition and maintenance history, as well as keeping inventory of spares and tools. This serves to keep track of equipment avail- ability and utilization, as well as operational costs, and helps decide on needs for equipment renewal and scrapping, and for ensuring that stocks are kept up-to- date. (iii) Operational management systems, which relate inter alia, to: o traffic counts; o axle load control and taxation; o highway safety. Such systems are directed at monitoring of how well the road system functions. 26. Certair elements of these systems have been pursued for a long time under Bank projects. Traffic counting is of long standing, and practically all our borrowers now have systematic counting, in some cases very comprehen- sive systems. The data are a basic ingredient for determining road upgrading or improvement and maintenance, for monitoring adequacy of the roads, and the evaluation of project achievements. Axle load control has been attempted in m-st borrower countries, but with rather poor results, except for a few not- able cases (e.g., Brazil and Chile). Equipment management has been a major concern, and various systems for improving it have been tried, including plant hire schemes, but much remains to be done in this area. Road main- tenance management systems are of later vintage, and have grown in importance over the last decade--a development that has been made possible through improved knowledge of road pavement performance and vehicle operating costs - 76 - ANNEX C Page 8 as a function of road condition-a further indispensable ingredient is com- puterization of the process, now being feasible in even the poorest of countries through the use of PCs. In this area, a rapid development can be expected, and Bank project tend to be at the cutting edge in their introduc- tion and use in developing countries. Finally, highway safety elements: Highway safety concerns are of rather late date in most of our client countries. Accident data are scant, but enough to show that highway accidents represent a serious problem. As highway safety components have been introduced in Bank loans over the last 6-8 years, accident recording and monitoring systems have been set up, and methodologies are being established to evaluate the effect and effectivity of highway safety interventions. 27. Installing MIS in the highway sector is sometimes difficult due to the dispersed nature of the operations and maintenance. A communication network and regular transmittal of information even by conventional means to a central procesing point has to be established first. Nevertheless, in several countries in West Africa, systems have been installed which provide an annual inventory of the roads, with an evaluation of their condition, albeit only qualitative. In Niger, a road management unit has been set up for MIS, but not specifically for M&E. In Ghana, a transport planning cell has started setting up a data base of vehicles and their characteristics; the Highway Authority is being helped by a Project Management Unit to monitor road maintenance, productivity of gangs, utilization of equipment and evaluating the relative efficiencies of force account and contractors. Another three projects in that region, vis. Highway III in Guinea, and High- ways IV and V in Mali, include M&E elements since for those projects dis- bursements were tied to output measured in kilometers of road works instead of to purchase of materials and equipment. The intent, however, was to establish a measure of discipline in expenditure control, rather than to monitor implementation and evaluate project impact. 28. Most highway projects approved in the LAC region between FY80-84 include one or more of the elements outlined in paras. 11 and 12. Some are first-time efforts, such as a road maintenance management system in Belize (2273-BEZ) which is technically operating but which has not yet been incor- porated into the Ministry decision-making process, and the Barbados (Loan 2432-BAR) maintenance and equipment management systems currently being developed by consultants. Because of the Bank's long-term relationship with the highway sector authorities in most LAC countries, most ongoing systems have evolved through efforts under a series of projects. An example of a positive experience is the maintenance management system operational in Argentina, which works reasonably well. In Chile, a pavement management system was developed under Loan 2297-CH. In an innovative move, this monitoring system will be managed by a University body. But not all such systems have been successful. In Peru, equipment and maintenance management systems were developed under Loan 1196-PE and were to have been improved - 77 - ANNEX C Page 9 under the ongoing Loan 2091-PE. The systems are not being used. This is mainly due to the weak capacity of the Ministry of Transport and Communica- tions (due to staffing problems caused by inadequate civil service salaries) coupled with low road maintenance budgets. The Ministry is now planning to revitalize these systems, and a payoff to earlier work may be realized. (d) Rural Roads 29. Rural Road projects are often designed with explicit or implicit organizational development objectives as well as objectives for physical outputs, instrumental outcomes, etc. M&E of these organizational objectives, which heretofore has seldom been done, would focus on a number of non-quanti- fiable variables such as, for example, the quality of leadership or the rela- tions between the implementing agency and its external environment. Such objectives call for a qualitative approach to project evaluation, one in which beneficiaries as well as project staff should be involved. Qualitative evaluations are potentially more flexible and may be better able to trace the unexpected and idiosyncratic causes of project outcomes. Socio-political or organizational factors are not likely to be emphasized if evaluation teams do not include personnel with training or experience in these areas. 30. The differences in organizational objectives between the administration of capital-intensive road projects by a public works ministry and the coordination of labor-based construction by a rural roads organization are very substantial. The first is largely concerned with the operations of a single bureaucracy; the second is laregly concerned with interorganizational relationships and problems of organizing or adapting to the environment. A qualitative evaluation by appropriately trained field staff and beneficiaries may provide the best approach to assessing project success or failure with respect to such organization objectives. In this context, an M&E system was set up for the Kenya Rural Access Roads Project. The draft PCR is now available. The report confirms that the attempt has not been very successful. 31. Most of the impact evaluations done on Bank-funded rural road projects have addressed situations where road investments have been combined, either formally or informally, with investments in the agricultural sector. In Upper Volta, road improvements were planned in parallel with complementary investments in cotton and foodcrops projection through other projects. In Tunisia, complementary investments in credit and extension services were planned as part of a road project. Because there is no praccical way of separating road investments from related agricultural investments, even in these situations, project evaluations must always address the effects of the total investment package. In LAC, rural road investments are either an integrated component to an agriculture/rural development project, in which case M&E is handled under that project, or a stand-alone rural roads project - 78 - ANNEX C Page 10 managed by the Transportation Division. In the latter case, M&E systems usually are developed to monitor physical and financial implementation of the program. An example is the MIS system developed in Brazil (Loan 2224-BR) designed to assist BNDE's management and to monitor the project. 32. In the past, it has been difficult to attribute causality to empirically determined relationships among input and output variables on rural road projects, because the design of evaluation studies has not been sufficiently rigorous to exclude alternative explantions of these effects. Successful monitoring and evaluation or rural roads projects would either have to remain in place or an extended period of time, say ten years or move, or else would have to take into account changes over time in comparable control groups as was attempted without success, in the Kenya Rural Access Program. Both approaches are expensive to implement and governments are often reluctant to allocate adequate resources for such studies. Thus, projects usually fall back on monitoring progress in terms of output measurements, accounting and audit procedures, etc. (e) Construction Industry 33. Construction industry components have not had a formal "M&E System" attached to them. However, project supervision by the countries themselves is following up on the results of Bank-financed interventions, for example: (i) By regularly keeping track, through the national employ- ment and labor unions statistics, of the number of trainees remaining in the industry (e.g., the education project for training construction industry personnel, Sri Lanka). (ii) By keeping track of the number of indigenous contractors in business (e.g., the construction industry components of highways projects in C.A.R., Ghana and Kenya); and (iii) By monitoring the performance of construction and maintenance units under training, against previous levels of production or tender prices (e.g., Burma, Guinea, Ethiopia). 34. Regarding monitoring systems related to construction industries, the following are recommended: (i) A contractor's registry, doubling up as a prequalifica- tion system for small to medium construction contracts tendered locally (this is usually an office within the ministry of public works). (ii) An indexing system for basic inputs, which can be used in escalation formulae (as part of the office in the minis- tries of finance or planning, in charge of tatistics); and - 79 - ANNEX C Page 11 (iii) Trade or skill statistics (a part of the ministry of labor statistics). 35. Many countries already keep the above records; they are usually in the middle to upper reaches of development (for example, most LAC count- ries). A considerable amount of work remains to be done, particularly in LDCs, and it cannot be tackled solely as a construction industry effort: all three systems have wide spectra of applicability, either for procurement for all sectors or for national statistics. Conclusions 36. M&E activities in the transport sector have been defined essential- ly as management tools for the Government agencies concerned. This is the right thrust, but principles and procedures need to be tightened in several respects. The areas where the M&E effort has to be intensified are: (a) costing o' maintenance works; (b) rigorous accounting and post auditing of road funds; (c) equipment maintenance; (d) spare parts inventorying and stocking; (e) monitoring aid funds and their eventual application. Thus, the most appropriate types of M&E activities in the transport sector are those which relate to the implementation of management tools which do not look simply at the progress and effectiveness of the narrowly defined project, but which instead allow the borrower (be it a railway company, port authority or Ministry of Transport) to have ready access to the data and evaluation systems needed for decision-making. 37. M&E in transportation projects may be improved by paying more attention to the design of data collection and processing procedures based on the assessed capacity of project management. The data to be collected must enable projects management to take appropriate actions with regard to adjusting on-going activities and approaches. Prior to the design of data collection and processing procedures, management must also decide on the impact it wishes to measure. The qualitative evaluation of organization objectives would be interesting and useful but may be time-consuming and require expertise which is lacking in most developing countries. 38. The consensus among transport divisions in most regions is that specific guidelines are not needed for M&E systems--indeed because of the varying degrees of complexity and sophistication of our borrowers, such guidelines would have to be so broad as to be of little practical use. Where M&E could be beneficial, such as in rural road projects, they should be tailored to the specific circumstances of the project and the borrower so as to avoid the creation of additional bureaucratic layers in project execution. To the extent possible, one should use data from borrowers' systems set up for operational management needs, and avoid project units, which often collapse when funding ends. - 80 - ANNEX C Page 12 39. One region believes that continued assistance from OPS, perhaps in the form of case studies of successful (and not so successful) MIS systems (particularly in railways and ports), would be useful for Project staff who advise borrowers in the development of such systems. E. Energy 40. Most power and oil and gas projects are implemented by commercially oriented entities. As a result the sector's approach towards monitoring and evaluation is somewhat different from that of sectors where projects are the responsibility of government departments, be it at national, provincial or municipal level. 41. For most energy projects, the tendency has been to avoid having project-related specialized M&E units but to ensure that the general management system provides for the monitoring and evaluation of the entity in all its activities. At an early stage of the project cycle, a detailed analysis of the capability of the implementing agency for running its busin2ss in general, and for completing and operating the project in particular, is prepared. This results in a diagnosis which is the basis for a set of recommendations aimed at improving any weakness detected. These recommendations are expected to be implemented either through actions undertaken by the staff of the entity or by consultants. Through this process energy projects frequently contain actions that range from improvements in the borrower's accounting system to more sophisticated management information systems for che more complex operations. 42. Recently the Energy Department completed the preparation of a manual to help design performance monitoring systems for power utilities. Since each utility has its own characteristics and uniform monitoring systems have limited application, the new guidelines will assist the user in defining performance indicators better suited for its operation, will provide instructions on how to collect them and, finally, will give guidance on how to use them. - 81 - ANNEX D Page 1 ANNUAL REPORT ON OPERATIONS EVALUATION (FY86) PROJECTED CONTRIBUTION OF BORROWERS TO PCR PREPARATION THROUGH CY87 I. EASTERN AND SOUTHERN AFRICA A. Countries with five or more PCRs Burundi: Seven PCRs will be prepared in the next two years. Draft PCRs will be prepared by Borrowers for three projects (Nickel Explora- tion, Local Construction I and Highways III). The Borrower will provide background information and data as input to PCRs prepared by Bank staff in four cases (DFC I, Forestry 1, Education I and II). There is no centralized arrangement for the preparation and review of PCRs in the country. Ethiopia: Five PCRs are planned in this period. Draft reports for two of the three agriculture projects will be prepared by the Borrower. A third agriculture PCR will be prepared with data and background material provided by the Borrower. The Government has been asked to prepare a draft PCR for Education V. The executing agency for the First Transport Sector Credit prepared a PCR which is being revised by Bank staff. There is no centralized arrangement for the preparation and review of PCRs in the country. Kenya: Twelve projects will be due for PCRs during the next two years of which four are in Agriculture. Bank staff will prepare PCRs for five projects - three in agriculture and one each in Telecomnuni- cations and Power. The Ministry of Agriculture is expected to prepare the draft for one project, Narok. The Borrower will provide data and background information for three projects in Transport IDF (2). The Borrower will prepare PCRs for three Water Supply and Urban Projects. There is no centralized arrangement for the preparation and review of PCRs in the country. Madagascar: Nine PCRs will be prepared during he next two years. The Borrower will prepare the first drafts in eight cases. Two of these projeats are in agriculture two are in transport and one each in water supply and power. The PCR for education II will - 82 - ANNEX D Page 2 A. Countries with five or more PCRs (cont.) be prepared by the UNESCO Coorperative Program with assistance provided by the project implementation unit. There is no centralized arrangement for the preparation and review of PCRs in the country. Malawi: Seven PCRs are being processed in the period; two of which are SAL's which will be prepared by Bank staff. The Borrower is expected to prepare the first draft PCR in two cases (Education I and Blantyre Water). The PCR for NRDP I will be prepared by Bank staff with assistance from the Ministry of Agriculture. There is no centralized arrangement for the preparation and review of PCRs in the country. Sudan: Five PCRs are being processed during this period; four of which are in Agriculture. The Borrower will provide data and back- ground material as input for two of the Agriculture PCRs, the rest will be prepared by Bank staff. The Borrower will prepare the draft PCR for Highways II. There is no centralized arrange- ment for the preparation and review of PCRs in the country. Tanzania: Eleven PCRs are due to be processed in the next two years; three in IDF and two each in Agriculture, Transport, Water/Urban and Industry. All of the Agriculture PCRs will be prepared by the Borrower. Given the limited resources of the executing agency (TLAI) for the two Industry projects (Morogoro Industrial Estates and Mufindi Pulp and Paper) the Bank expects to prepare the PCRs for these two projects. There will however be some Borrower participation in data collection for the Mufindi project. There is no centralized arrangement for the preparation and review of PCRs in the country. Zaire: Six PCRs will be processed in the period; two of which are in Agriculture. The Borrower is expected to prepare the first draft in four cases (Kwango Kwilu I, Sugar, and Railways I). Bank staff will prepare the PCRs for two IDF projects (DFC IV and V) on the basis of data to be provided by the Borrower. There is no centralized arrangement for the preparation and review of PCRs in the country. B. Countries with less than five PCRs Botswana: Four PCRs will be prepared during this period. All four (Educa- tion II, Highway V, Water Supply II and III) will be prepared by the Borrower. There is no centralized arrangement for the pre- paration and review of PCRs in the country. - 83 - ANNEX D Page 3 B. Countries with less than five PCRs (cont.) Djibouti: The Borrower is expected to prepare the PCR for the Highway Main- tenance Project one of the two PCRs which will be prepared in Djibouti during the period. There is no centralized arrangement for the preparation and review of PCRs in the country. Lesotho: There will be two PCRs processed during the period. Both (BASP, Highways 1II) will be prepared by the Borrower. There is no cen- tralized arrangement for the preparation and review of PCRa in the country. Mauritius: Four PCRs will be processed during the next two years. The Borrower will prepare the first draft for three of these (Urban, Education II and Power Distribution). In the case of Education II, the project implementation unit will prepare the draft with assistance from the Evaluation unit of the Department of Curri- culum Research and Development. The final report will be pre- pared by the UNESCO Cooperative Program. There is no centralized arrangement for the preparation and review of PCRs in the country. Rwanda: Four PCRs are being processed; two of which are in Agriculture. The project units will prepare the reports for the two agricul- ture projects. The PCR for Education I will be prepared by the UNESCO Cooperative Program with assistance by the project imple- mentation unit. To improve local participation in reviewing experience with Technical Assistance the Bank is considering the "workshop" technique in the case of the PCR for the First Techni- cal Assistance Project. There is no centralized arrangement for the preparation and review of PCRs in the country. Somalia: Three PCRs will be prepared in the next two years. The Borrower is expected to provide data and background information for two of the PCRs (North West Agriculture I and Education III). There is no centralized arrangement for the preparation and review of PCRs in the country. Swaziland: Two PCRs, will be prepared in the next two years. One Power PCR will be prepared by the Borrower, the other for Education III, will be prepared by the Bank. Although it was stipulated in the Loan agreement for the Government to produce a completion report, the project implementation unit was dissolved as the project was closed in December 1985 and the key staff have been recalled to the civil services, leaving no one accountable for the task. There is no centralized arrangement for the preparation and review of PCRs in the country. - 84 - ANNEX D Page 4 B. Countries with less than five PCRs (cont.) Uganda: The Borrower is expected to be involved in two of the four PCRs which will be processed in the next two years. The PCRs for two Reconstruction Credits will be prepared by the Borrower. There is no centralized arrangement for the preparation and review of PCRs in the country. Zambia: Three PCRs will be processed during the period. Two of the PCRs (Cof fee I and Highways III) will be prepared by the Borrowers. The Borrower will provide data and background information for the preparation of the third PCR (DFC II). There is no centralized arrangement for the preparation and review of PCRs in the coun- try. Zimbabwe: Two PCRs will be prepared during the period. The Borrower is expected to prepare both reports (Transport Rehabilitation and Manufacturing Export Promotion). There is no centralized arrangement for the preparation and review of PCRs in the country. II. WESTERN AFRICA A. Countries with five or more PCRs Benin: Five PCRs will fall due through CY87 (Cotonou Sanitation Engineering, Regional (Benin/Togo) Nangb4to Power Engineering, Industrial development, Fourth Highways, and Petroleum Technical Assistance). The completion report for the Cotunou Sanitation Engineering Project will be prepared by the Sociftf Bdninoise d'Electricitg et d'Eau and will form the basis for the Bank,s PCR. The PCR for the Regional (Benin/Togo) Nangb6to Power Engineering Credit will be prepared by the Coordinating Committee of the three project implementation agencies with the assistance of consultants, and reviewed by the technical ministries in Benin and Togo. The Banque Bfninoise de Ddveloppement will prepare the initial completion report for the Industrial Development Project which will provide the basis for the Bank's PCR. The PCR for the Fourth Highways Project will be prepared by consultants supervis- ing the project and reviewed by the Ministry of Transport; the Bank will prepare its PCR based on the report. The initial com- pletion report for the Petroleum Technical Assistance Project will be prepared by the project unit and will form the basis for the Bank's report. - 85 - ANNEX D Page 5 A. Countries with five or more PCRs (cont.) Cameroon: Six PCRs are scheduled through CY87 (Feeder Roads, Highways IV, Railways IV, Livestock II, Water Supply II, and ZAPI East Rural development). A draft project completion report for the Feeder Roads Project has been prepared by the consultant. this will be updated by the responsible ministry to include project items being carried out undar a later extension of the closing date. A project completion report for the Highways IV Project will be prepared by consultants on behalf of the Government. the comple- tion report for the Railways IV Project, to be prepared by the Borrower, will serve as a basis for the PCR which will be pre- pared largely by a Bank-financed consultant with assistance from Bank staff. In the case of the Livestock II Project, a series of consultant studies financed by the Borrower on aspects of the performance of this Project have been initiated and will be followed by consultant preparation of a project evaluation. The Report for the Water Supply Project will be prepared by SNEC (Project executing agency) and finalized by Bank staff. For the ZAPI East Rural Development Project, a consultant is in the field to prepare a completion report on the basis of terms of reference issued by ZAPI management. In the absence of a formal review process, draft PCRs are reviewed by the project implementation agencies ard centrally by the Ministry of Planning for overall project imp±ementation and policy issues. Nigeria: Six PCRs are expected through CY87. A PCR for Forestry I will be prepared by the Borrower. A combined PCR for Oil Palm II and Oil Palm IV will be prepared by a consultant hired by the Borrower. The Federal Agriculture Coordination Unit (7ACU) will prepare reports for Bida ADP and Ekiti-Akoko ADP. In the case of the Fertilizer Imports I Project and the Agricultural and Rural Management Training Institute Project, consultants financed by the Project will prepare the PCRs under the supervision of FACU. The relevant project implementing agency, the participating state governments, and relevant ministries at the Federal level are involved in the work on the review of PCRs. Senegal: Eight PCRs are scheduled through CY87. A PCR for the Petite CMte Tourism Project and a PCR for the Power Engineering Project will be prepared by their respective responsible project implementa- tion entities. A PCR for the Education III Project will be pre- pared by UNESCO with the Ministry of Education providing inputs. A PCR for the Deli Lampsar Irrigation Project will be prepared by technical assistance staff attached to the project PCRs prepared by the Borrowers in the transportation sector, Bank staff have - 86 - ANNEX D Page 6 A. Countries with five or more PCRs (cont.) found that they have to do the preparation work themselves with a couple of exceptions. Work has started in the Bank on the pre- paration PCRs for four projects in the transportation sector (Port, Aviation, Third Railway, and SEFICS). The PCR review will primarily involve the technical ministries and project execution agencies. Tog>: Five PCRs will fall due through CY87 (Structural adjustment credit I, Technical Assistance I, Power Engineering, feeder Roads, and Education I). The completion reports for the struc- tural adjustment and technical assistance credits will be pre- pared by Bank staff on the basis of preliminary reports prepared by the Directorate General of Planning of the Ministry of Planning and Industry. The PCR for the Regional (Benin/Togo) Nangb4to Power Engineering Credit will be prepared by the Per- manent Coordinating Committee of the three project implementation agencies, with the assistance of consultants, and reviewed by the technical ministries in Benin and Togo. An initial completion report for the Feeder Roads Project wi11 be prepared by the proj- ect consultants, and, together with an audit report, will form the basis for the Bank's PCR; the Ministries of Rural development and Planning will participate in the review process. UNESCO will prepare a PCR for Education Project; the Ministry of Education will review the report along with the Director of Planning, the Director of Primary Education, the Director of Vocational educa- tion, the Director of DIFOP, and the Director of the PIU. B. Countries with less than five PCRs Burkina Two PCRs are scheduled through CY87. The one for First urban Faso: Project is currently being prepared. The Government's contribu- tion, while not legally required, consisted of two PCR drafts, one for Bobo-Dioulasso, and one of Ouagadougou. These were pre- pared by consultants contracted under the Project. The other PCR for the Small and Medium Enterprises Project will be prepared by Bank staff based upon inputs provided by the three Borrower agencies involved in the Project's execution. The PCRs will be reviewed by the Ministry of Finance and concentrated technical ministries and project implementation agencies. - 87 - ANNEX D Page 7 B. Countries with less than five PCRs (cont.) Central African Republic: A PCR for Livestock will be prepared by consultants financed by the Borrower; the terms of reference and report content will be reviewed by the Bank supervision mission. The Ministry of Planning, Commerce, Statistics, and International Cooperation will review the Report. Chad: A PCR for the Third Education Project will be prepared by UNESCO-CP ;_ad reviewed by the project implementation unit, the Ministry of Education, and the Ministry of Planning. Congo: Three PCRs are scheduled through CY87. Reports for the Railways II and River Transport Projects will be prepared by the Borrower, and will serve as the basis for the PCR which will be prepared largely by Bank-financed consultants with assistance from Bank staff. The report for the First Technical Assistance Project will be prepared by Bank staff on the basis of a completion report draft by the Project coordinator and reviewed by the Minister of Finance. There is no formal central unit to process and review the PCR in the Congo. COte d'Ivoire: Four PCRs are scheduled through CY87 (First Highways, Third Education, SAL I, and technical Assistance). The report for the First Highway Project is to be prepared by the Borrower. The PCR for the Third Education Project will be prepared by UNESCO-CP with considerable support from PIU and reviewed by the Ministry of Education, and the respective Directors of PIU, teacher Train- ing, Technical Training, and Planning. There is no clearly- defined process for reviewing the experience of Bank projects within the Government and sectoral ministries; the Minister of Planning and Finance normally reviewed the Project and writes his comments on Bank-prepared PCRs and PPARs. Ghana: A PCR for the Reconstruction Import credit I (RIC) will be drafted by the Bank of Ghana and the procurement agents, and updated and completed by Bank staff. Guinea: A PCR for the Guinea Rice Project is expected to be completed in the first half of CY87. Bank staff will prepare the report based upon the information provided by the Project under the Ministry of Rural development. A formal review process does not exist within the Government. - 88 - ANNEX D Page 8 B. Countries with less than five PCRs (cont.) Guinea- Bissau: A PCR for the Rehabilitation Import credit and the Supplement to it (financed under the Special African Facility) will be prepared in CY87. The National Bank of Guinea-Bissau, the implementing agency, will play the lead role in the preparation of this Report. So far, no arrangements exist at a central point for processing PCRs. Liberia: A PCR for the Forestry Project will be prepared by a consultant hired by the Borrower, and reviewed by the Ministry of Agricul- ture. Mali: Three PCRs are scheduled through CY87. One PCR in the agricul- ture sector (ODIPAC Technical Assistance Project) is expected to be issued in CY87 based on a preliminary report prepared by the project implementation agency. The Ministries of Agriculture and of Natural Resources and Livestock normally review draft PCRs, whether prepared by the Projects or by the Bank. As for the Fourth Highway Project, the Borrower has agreed to prepare a Report by January 1987, with the assistance of consultants and Bank staff. In the case of the First Urban Project, the Borrower is legally required to prepare a PCR before June 30, 1987. A framework for preparation will be discussed with the project unit next fall. Mauritania: A PCR in the agriculture sector (Gorgol Irrigation) is expected through december 1987, based on a preliminary report prepared by the project implementation agency. The draft PCR will be reviewed by the Planning Unit of the Ministry of Rural Develop- ment. Niger: Two PCRs for Dosso rural development and Feeder Roads are planned through CY87. A draft completion report for the Dosso Rural Development Project will be prepared by the Monitoring and Evaluation Unit of the Ministry of Rural Development. The Report on the feeder Roads Project will be prepared by Bank staff based on past progress reports. The review process is rather informal; the reports will be sent to the concerned technical ministries for review and comments. Sierra Leone: A PCR for Power Sector Engineering and TA Credit will be prepared by the executing agency (NPA) with Bank staff assistance, if necessary, once the Bank decides whether or not to postpone the - 89 - ANNEX D Page 9 B. Countries with less than five PCRs (cont.) credit closing date. This would depend on tariff actions by the Government that would be satisfactory to IDA and carried out before June 30, 1986. III. EAST ASIA AND PACIFIC A. Countries with five or more PCRa Indonesia: Twenty-two PCRs are planned for completion in Indonesia from the beginning of FY86 through end of CY87: Eleven in agriculture sector, two each in the IDF and education sectors, one each in the energy, industry, technical assistance, population and health, urban and water supply sectors. Most PCRs in agriculture sector will be prepared either by the Bank with assistance from the borrower or by the borrower with assistance from the Bank and FAO. All PCRs in the IDF and urban and water supply sectors will be prepared by the borrower and those in the education sector by the borrower with assistance from UNESCO. The PCRs in the energy and population and health sectors will be prepared jointly by the Bank and the borrower, the remainder will be prepared by the Bank. Review arrangements for draft PPARs exist for all agricul- ture, population and health and urban and water supply sectors: the drafts are sent to the Planning Agency (BAPPENAS) and the technical ministries for review and comments. Korea: Fifteen PCRs are planned for completion during this period: Six are in the agriculture sector, four i1 the IDF, three in urban and water supply sector and one each in the education and trans- portation sectors. All PCRs in the IDF, three in the agriculture and two in the urban and water supply sectors will be prepared by the borrower. Only two in the agriculture and the one in the urban and water supply sectors are prepared by the Bank, the remainder jointly by the Bank and the borrower. Draft PPARs are forwarded to the Economic Planning Board, Ministry of Finance and the line ministry concerned for review and comments. Malaysia: Twelve PCRs are planned for completion during this period: Six in the agriculture, two in the education and one each in energy, transportation, population and health, and urban and water supply sectors. All but one PCR each in the agriculture and education sectors will be prepared by the borrower and those in the popula- tion and health, and urban and water supply sectors by the Bank - 90 - ANNEX D Page 10 A. Countries with five or more PCRs (cont.) with the remaining three jointly by the Bank and the borrower. The Economic Planning Board and technical ministries concerned review draft PPAR9 and comment on them. Philippines: Seventeen PCRs are planned for completion during this period: Eight in the agriculture, four in the urban and water supply, two in IDF and one each in the energy, industry and transportation sectors. All PCRs in the agriculture and IDF, and one in the urban and water supply sectors will be prepared by the borrower. The one in the industry and three in the urban and water sectors will be prepared by the Bank and the remainder jointly by the Bank and the borrower. Draft PPARs are followed to the National economic Development agency and technical ministries concerned for review and comment. Thailand: Eighteen PCRs are planned for completion during this period: Four in the agriculture, four in the industry, three in the urban and water supply sector, two each in the IDF, transportation and one each in the energy, education and population and health sectors. All PCRs in the agriculture, IDF, telecommunications sectors and the one in the urban and water supply sector will be prepared by the borrower. Three PCRs in the industry, population and health and urban and water supply sectors will be prepared by the Bank and the remainder jointly by the Bank and the borrower. Review arrangements for draft PPARs exist for education, population and health and urban and water supply sectors - Draft PPARs are routinely reviewed by the National Economic and Social Development Board and the technical agencies concerned. B. Countries with less than five PCRs China: Only one PCR is planned for completion from the beginning of FY86 through the end of CY87: this first PCR in China is in the education sector and will be prepared by the Bank. Draft PPAR will be reviewed by the State Planning Commission and the Ministry of Education. Fiji: Two PCRs are planned for completion during this period: Both PCRs are in the energy sector and will be prepared by the borrower with assistance from the Bank. Draft PPARs will be sent to the Ministry of Finance and the technical agency concerned 'or review and comment. - 91 - ANNEX D Page 11 B. Countries with less than five PCRs (cont.) Laos: Only one PCR is planned for completion during this period: It is in the agriculture sector and will be prepared by borrower. The Ministry of Finance and the Ministry of Agriculture will be asked to review the draft PPARs. .apua New Guinea: Three PCRs are planned for completion during this period: One each in the energy, transportation and agriculture sectors. The PCR in the transportation sector will be prepared jointly by the Bank and the borrower and the remaining two by the Bank. The draft PPARs will be reviewed by the Ministry of Finance, Planning agency and the technical ministry concerned. Solomon Islands: Only on PCR is planned for completion during this period: It is in the IDF sector and will be prepared by borrower. The draft PPARs will be forwarded to the Ministry of Finance and the DFC for review and comment. IV. SOUTH ASIA A. Countries with five or more PCRs Bangladesh: Fourteen PCRs in five sectors (Agriculture, Industrial Finance, Transportation, Water Supply, and Program lending) are under pre- paration. Basic data are collected by the agencies directly (6) or with help of consultants (4). In four cases Bank staff are collecting the basic data. Coordination within the Government of Bangladesh is provided by ERD for Program Lending and Industrial Finance, but is left to implementing agencies in other sectors. All PCRs are expected to improve heavy input from Bank staff. Quality of the data provided by the Borrower is generally good. Burma: Six PCRs, all in agriculture sector, are under preparation. Basic data is collected by the Ministry/Corporation concerned. There is no central agency providing coordination of PCR prepara- tion. All PCRs require heavy assistance form Bank statf. the quality of the data is generally poor. - 92 - ANNEX D Page 12 A. Countries with five or more PCRs (cont.) India: Twenty-one PCRs in five sectors (Agriculture, Industrial Finance, Transportation, Water Supply and Urban development) are under preparation. In most cases collection of initial date and coordination of the preparation of PCR is the responsibility of the Project Authority. A significant number of PCR (11, 52%) require heavy assistance from Bank staff to complete PCR draft. PCR draft data provided by the Project Authorities is generally good. Pakistan: Twelve PCRs in four sectors (Agriculture, Education, Industrial Finance, and Transportation) are under preparation. FAO is preparing four PCRs in Agriculture and UNESCO will prepare a PCR in Education. The initial data is collected by the Government ministry/corporation concerned or with the help ofCooperative Programs (2) or consultants (1). There is no central agency responsible for the coordination of PCR work. The assistance reauired from Bank staff is generally heavy. Quality of the initial data is generally poor. Sri Lanka: Five PCRs in three sectors (Agriculture, Industrial Finance, and Transportation) are currently under preparation. Initial data is collected by the Project Authority or the department concerned. The Central Ministries concerned generally provide the coordina- tion within the Government. the assistance required from Bank staff has been generally heavy and the quality of data is reason- ably good. Nepal: Two PCRs in Education and Transportation Sectors are under pre- paration. The ministries concerned collect the data directly with the help of consultants. The assistance required from the Bank staff is reasonable. In one case (Transportation) for which the data have been collected, the quality is good. V. EUROPE, MIDDLE EAST AND NORTH AFRICA A. Country with five or more PCRs Egypt: Six PCRs are scheduled. Two in each of the Agriculture and IDF sectors and one in the Water Supply and Transportation sectors, respectively. The Agriculture Ministries will use local consul- tants to assist in PCR preparation. The borrower (DIB) in the IDF sector will provide the data needed to prepare the reports. - 93 - ANNEX D Page 13 A. Country with five or more PCRs (cont.) DIB has been very cooperative and interested in PGR pruparation for previous projects. In the Water Supply and Transportation cases, the borrowers are not covenanted to prepare a PCR but will provide necessary data to input to Bank staff work, B. Countries with less than five PCRs Algeria: Two PCRs are scheduled in the Educational and Transportation sectors. The borrowers are not obliged to prepare PCRs, however, both are expected to make contributions either through a review of the implementation of major components or by providing neces- sary data. Completion missions will be mounted by Bank staff who will prepare the final PCRs. Cyprus: Only one PCR is expected to be prepared in CY87. This will be done by an expatriate project manager. The borrower will also provide background information and participate fully in the report review. Greece: There are two PCRs due in 1987, both in the Agriculture sector. The operating Ministries are responsible for their preparation and review. They are generally cooperative in preparing the reports although delays can be expected. Lebanon: Civil disorder has precluded completion of all the components of the two projects still in the Lebanon portfolio. Limited PCRs will be prepared by Bank staff based on information provided by the borrowers. No missions will be undertaken. Morocco: Three PCRs are expected to be issued in CY87 in the Agriculture, Power and Transportation sectors. In each case, the borrower will prepare the report based on d'scussiori with and guidelines provided by Bank supervision missions. We expect these reports to be of good quality given the borrowers' previous experience in this respect. Oman: The borrower is required to prepare a PCR for the project in the Transportation sector. The Ministry of Communications will pro- vide data and subwit a caaft PCR to the Bank for finalizing. Portugal: Two PCRs in the Education and Power sectors are due. The report on the Education II project will. be prepared by the project implementation agency. Bank staff will review and comment on the - 94 - ANNEX D Page 14 B. Countries with less than five PCRs (cont.) draft report. EDP, the borrower in the Power sector has provided good PCRs in the past and is expected to do so again. Romania: Three PCRs are expected to be prepared by the staff of the Agri- cultural Bank (BAFI) with assistance from the Ministry of Agriculture and the Design Institute for Agriculture and Food Industry (IPSAIA) following the standard PCR format. Previous experience indicates good reports will be received after careful review of the draft reports by the government. Syria: An Irrigation project PCR will be prepared in CY87. The Ministry has been urged to commence report preparation. However, since the Ministry responsible was unable to prepare a previous PCR, it appears likely that substantial Bank staff involvement will be required. Tunisia: A PCR in each of the Power and Water Supply sectora is expected. STEG, the borrower in the Power project will prepare the report which we expect will be adequate given previous experience. Itn Water Supply, Bank staff will prepare the PCR with substantial inputs expected from the borrower. Turkey: Three PCRs in the IDF sector and two in kgriculture are due in CY87. The borrowers have always cooperated fully in the IDF case in the preparation of data. Although Bank staff will finalize the reports, these mature financial institutions will participate actively. Unfortunately, the Agriculture sector lags behind, while in some cases the government prepares initial draft comple- tion reports or prepares information for this purpose, the qua- lity to date has been poor. In practice Bank staff prepare the PCRs almost exclusively. However the Ministry of Agriculture, Forestry and Rural Affairs recently established a Department of Externally Financed Projects with a Monitoring and Evaluation Unit which is a promising development for future tasks of this nature. Yemen AR: Three Agriculture PCRs and one in the Water Supply sector will be issued. Expatriate consultants have been engaged in the Agricul- ture cases and comprehensive draft PCRs are expected. Past experience suggests that analytical parts of the reports may require some further investigation by Bank staff, leading to delays. In Water Supply, Bank staff will prepare the report with data proviled by the borrower. - 95 - ANNEX D Page 15 B. Countries with less than five PCRs (cont.) Yemen PDR: Four reports in three sectors (Agriculture, Power and Water Supply) are anticipated. In Agriculture, a report on the second Fisheries project will be prepared with expatriate consultant assistance. The first report prepared in this manner was compre- hensive though relatively weak on economic analysis. In Power, the borrower (PCEP) is inexperienced in report preparation but we expect to receive all the necessary data for Bank staff to issue a report on the two projects concerned. The Water Supply PCRs will be prepared by Bank staff with data provided by the bor- rower. Yugoslavia: Three PCRs will be prepared in CY87, all the Agriculture sector. Previous experience indicates that a variable performance pattern will emerge. Borrowers in less developed regions cannot fully prepare PCRs often due to language difficulty. However, all borrowers feel obligated to fulfill loan covenant requirements for PCRs and in many cases have engaged the Economic Institute in Belgrade to assist in either gathering data for Bank staff or in preparing draft PCRs. This type of contiiued improvempit can be expected to persist. VI. LATIN AMERICA AND THE CARIBBEAN A. Countries with five or more PCRs Brazil: Eighteen PCRs are forecast throughout CY87. In all cases, major contributions, usually in the form of a draft report, are expect- ed to be made by the agencies in the sector concerned. A recent- ly approved Public Sector Management project provides for the strengthening of planning, project appraisal, and evaluation capabilities within the Ministry of Planning. This will provide the basis for enhanced interaction between the Government and the Bank on the evaluation of Bank financed projects. Colombia: Ten PCRs are forecast through CY867. In all cases, contribu- tions, usually in the form of a draft report, are expected from the executing agencies involved. The National Planning Depart- ment (DNP) has established a project monitoring unit. It has initiated a project review scheme with the Bank's Resident Mission to monitor projects on a regular basis and to analyze - 96 - ANNEX D Page 16 A. Countries with five or more PCRs (cont.) systemic issues that affect project implementation such as slow procurement, complicated budgetary issues, frequent movement of staff and other factors impeding progress. Mexico: Eight PCRs are scheduled through CY87. In the majority of cases, contributions from executing agencies are expected in the form of ,-aft reports which would then be reviewed by Bank staff and dis- -ased with the executing agencies. Experience with recent draft rCRs prepared by implementing agencies shows an improving trend, although the capacity of evaluation units of the borrowing inter- mediary institutions will have to be continuously strengthened. Paraguay: Five PCRs are forecast through CY67. The executing agencies are expected to prepare draft reports or make major contributions to the preparation of all but one of these. B. Countries with less than five PCRs Bolivia: Four PCRs are forecast through CY87. Bank staff are expected to prepare all of these reports with significant contribution from executing agencies. Uruguay: Four PCRs are forecast through CY87. In all cases major contri- butions, usually in the form of a draft report, are expected from the executing agencies. The quality of available data is good and staffing of the executing agencies is adequate. There is no separate project evaluation unit at the Central Planning Depart- ment level, although this Department reviews the draft PCRs prior to their issuance.