Document of The World Bank FOR OFFICIAL USE ONLY Reprt No. 7958 PROJECT PERFORMANCE AUDIT REPORT THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (LOAN 2402-TH) JUNE 30, 1989 Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents .nay not otherwise be disclosed without World Bank authorization. Currency Equivalents Appraisal Year US$1.00 - Baht 23.0 Final Year US$1.00 a Baht 26.6 Fiscal Year of Borrower (RTG) October 1 - September 30 Weights and Measures Metric System Abbreviations ARD Office of Accelerated Rural Development BOB Bureau of the Budget CGO Controller General's Office DOA Department of Agriculture DOAE Department of Agricultural Extension DOF Department of Fisheries DOLD Department of Livestock Development ERR Economic Rate of Return FAO United Nations Food and Agriculture Organization HYV High-yielding Variety IPIED Information Processing Institution for Education and Development JICA Japanese International Cooperation Agency LDD Land Development Department M&E Monitoring and Evaluation MOAC Ministry of Agricultural Cooperation MOI Ministry of Interior NESDB National Economic and Social Development Board NRDCC National Rural Development Coordinating Center NRDC National Rural Development Committee NRDP National Rural Development Program PAP Poverty Areas Program PCR Project Completion Report PPO Provincial Project Office PRDCC Provincial Rural Development Coordinating Center R3GP Rural Employment Generation Program RID Royal Forestry Department RTG Royal Thai Government TDRI Thailand Development and Research Institute UNDP United Nations Development Program USAID United States Agency for Internation.'. Development VDC Village Development Committee FOR OFrfiIL OLY THE WORiD BANK Washifgton, D.C. 20433 U.S.A. Opwtaokm &Ewuluan June 30, 1989 MEIRAlDUN TO THE MEXCUTIVE DIRECTORS AMD THE PRESIDENT SUBJECT: Project Performance Audit Report ot hEailand NationRra l Development Proect (Loan 242-TH) Attached, for information, is a copy of a report entitled 'Project Performance Audit Report on Thailand - National Rural Development Project (Loan 2402-TH)" prepared by the Operations Evaluation Department. Attachment This document has a restricted distribution and may be used by recipients only In the pefformance of their offical duties. Its contents may not otherwise be disclosed without Worfi Dank authoration. FOR OFFlCLUL USE ONLY PROJECT PERFORMANCE AUDIT REPORT THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (LOAN 2402-TH) TABLE OF CONTENTS Pare No. Preface .......................................................... i Basic Data Sheet ... ............................................... ii Evaluation Summary ......... ...............*........ iv PROJECT PERFORMANCE AUDIT MEMORANDUM I. PROJECT BACKGROUND ...................... 1 A * Introduction ...................................... 1 B. Objectives ........................................... 2 C. Project Formulation, Design and Content .............. 3 D. Project Cost and Financing ........................... 5 E. Implementation Arrangements .......... ............... 6 F. Project Documents and Clarity of Objectives .......... 8 II. PROJECT IMPLEMENTATION AND OUTCOME ............. .......... 9 A. Implementation History ...... ....................... . . * . . . . . . 9 B. Project Results ...................................... 13 C. Institutional Impact ................................. 16 D. Economic and Financial Performance ................... 17 III. FINDINGS AND ISSUES .............. ...................... . 18 A* General .............................................. 18 B. The Bank's Role and the Project Concept ...... ........ 19 C. Timing and Duration of the Bank Project .............. 21 D. Program Management Improvement .............. ......... 21 E. Reaching the Rural Poor ......... . . . . . . . . . . . . . . . . . . . . . 23 F. Perceptions and Reality .............................. 24 G. Conclusions ..... ..................................... 24 Box 1: Performance of Investment Subprograms ............ 14 Box 2: Rationalized Project Subprogram Investments ...... 18 Box 3: Impact Evaluation and Village-Level Indicators ... 22 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Table of Contents (Cont.d) Pate No. REGIONAL OVERVIEW INTRODUCTION ********************************* ............ 27 PROJECT OBJECTIVES AND CONTENT ........ ................... 27 PROJECT IMPLEMENTATION . ...................... .. . .. . .. . . . . 28 General * ............................................... 28 Monitoring and Evaluation .............................. 29 Special Studies ........ 31 Poverty Areas Subprograms .............................. 31 PROJECT COSTS AND FINANCING . ........... 36 PROJECT RESULTS .................. 37 Economic Re-evaluation .*.. ........... . . . . . . . . . . . . . . . . . . . . 38 Sustainability ......................................... 40 BANK PERFOPRMANCE ...................... 40 LESSONS LEARNED .........**........... * ** ** ** * ** **** * * * * 41 COtMENTS FROM THE BORROWER ATTACHMENT: Comments from the Office of the National Economic and Social Development Board ....................... 43 Map: IBRD 17589 PROJECT PERFORMANCE AUDIT REPORT THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (LOAN 2402-TH) PREFACE 1. This is the Project Performance Audit Report (PPAR) for the Thai- land National Rural Development Project, involving an IBRD Loan in the amount of US$50.0 million to the government, with the objective of (1) strengthening planning. budgeting and management aspects of the newly ins- tituted National Rural Development Program, and (2) alleviating poverty in rural areas. The loan was approved on March 29, 1984 and became effective on Aug. 14, 1984. The Bank loan was closed on schedule on June 30, 1988. The final disbursement was made on October 25, 1988 and the unutilized balance of US$5.2 million was cancelled on November 25, 1988. An amount of US$8.5 million was cancelled earlier, making total cancellations US$13.7 million. Parallel financing was provided by the Japanese International Cooperation Agency (JICA) (US$2.1 million), the United Nations Development Program (UNDP) (US$0.5 million), and the United States Agency for International Development (USAID) (US$5.0 million). 2. The PPAR consists of a Project Performance Audit Memorandum (PPAM) prepared by the Operations Evaluation Department (OED), an Overview of the Project Completion Report (PCR) prepared by the Bank's regional office for Asia, and a PCR prepared by the government's National Rural Development Coordinating Center (NRDCC), with the assistance of the World Bank/FAO Cooperative Program (FAO/CP). The government also submitted with the PCR an ex-post evaluation study prepareA by staff of Kasetsart University.!/ The PPAM is based on the PCR, the Overview, the Staff Appraisal and the President's Reports, the loan documents, a study of project files, and on discussions with Bank staff. Impact studies of rural development in the North and Northeast by staff of Chiang Mai University and Khon Kaen University respectively have also contributed to the PPAM. An OED audit mission, which visited Thailand in October 1988, held discussions with staff of the concerned agencies and inspected a sample of project activities and sites in the North and Northeast Regions. The Bank expresses its appreciation to the officials of the ministries and other agencies of the borrower involved in the coordination and implementation of the project, Kasetsart University, the PAO/CP and the cofinanciers, for their valuable assistance in preparing this report. 3. The PCR and Overview provide an account and assessment of the project experience, and discuss the performance of the Bank and the project executing agencies. The PPAR elaborates on particular aspects such as the Bank's role and project concept, the timing of the Bank assistance, program management improvement, poverty targetting and beneficiary participation, and the evaluation of NRDP results. 4. Following standard OED procedures, copies of the draft PPAR were sent to the Government, the Borrower and to USAID, UNDP and JICA on May 12, 1989. Comments received from the government, dated June 13, 1989, are attached. 1/ The government's PCR and the accompanying evaluation study are too bulky for general circulation but copies are on file in OED for use as required. - ii| - PROJECT PERFORMANCE AUDIT REPORT THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (LOAN 2402-TH) BASIC DATA SHEET KEY PROJECT DATA Appraisal Actual or Actual as X of Exp ctstion Curront Estimate Appraisal Estimate Projoet Cost (USS million) 167.2 91.6 58.2 Loan Amount (USS million) 60.0 36.8 La 72.6 Cofinancing (US$ million): - USAID 6.0 - JICA 2.1 - UNDP 0.5 Economią Rate of Return (X) 12.4 - Financial Rate of Return (X) - - Institutionsl Performance Adequato Technical Porforance- Adequate Number of Direct Benoficearies (households) 2 millon 2 million (b) 100 KEY PROJECT DATES Date of Board Approval 08/20/84 Date of Efftctivoness - 06/14/64 Date physicol coeponente comploted 09/80/8U (a) 18/31/67 Proportion t-sn completed (X) 100 91 01 Closing Date 06/80/s 06/80/86 100 CUMULATIVE DISBURSEMENTS FY86 FY86 FY87 FY88 Appraisal Estimate (SS million) 0.8 O9.0 50.0 0.0 Actual (USS million) 5.2 18.8 26.8 88.8 Actual as X of Estimate 85.60 44.8 58.6 72.6 Amount Cancelled: USSUS million on October , 1986 USS6.2 million on November 25, 1698 18.7 Date of final disbursements October 25, 1906 Principal repaid to (mo/d.y/yr) (US$ million) - - - STAFF INPUTS (staff weeks) FY62 FY88 FY04 FY65 FY68 FY67 FY66 TOTAL Preappralsal (c) 7O.1 6. 0 126.0 App sisal 20.2 22.1 42.2 Nogotiations 7.6 7.6 Supervision 4.8 37.0 21.8 24.1 16.4 164.3 Other .1 14.2 3.0 1.3 16.7 Subtotal 70.2 90.3 87.1 86.9 21.0 24.1 16.4 20"0. (a) Loan Agreemnt shows 06/80/67 as Completion Date. (b) Audlt was unable to verify this figure. (c) Doo not Include the first two Identification missions (see over) for which time recording date were not traced. - ,ii - MISSION DATA Date No. of Staff/Days Sp.cialization Performance Types of to/vr) Porsona in flold roeroesnted /b Ratina /c Trend Id Problem. sr Idontification t 07/79 5 106 csi,s,oob tt 11/79 2 S2 c,s UII 1/82 5 196 a,ze,zi Preparation I 4/02 7 112 3a,b,xI,o tI 7/82 2 20 r,h III 11/02 9 166 g,ts,b,tid,g,t IV 01/03 2 35 2d Appraisal 04/8S 6 a6 a,b,2d,t Subtotal 632 Supervislon I 01/66 5 So 2*pb,c,d, 2 2 0 II 11/86 6 so 2&,b,c,d, 2 - F,M II 96/"8 4 40 2a,b,c 2 - F,M IV 04/07 8 85E 2a,b 2 - F,M Subtotal 175 Total OTHER PROJECT DATA sorrop r Kingdom of Thailand Extcuting aency National Economie and Social Development Board (NESMO) Fiscal year of borrower October 1 - September 80 Name of currEncy (abbroelvtion) Baht (8) Appraisal yjr aver9ge (108M) S.81.00 a I 28. Intervening yers average (19"4-67) US1.900 a O 26.7 Completion year avorge9 (1968) USf1.09 a O 26.0 Follow-on projet: NONE La Cancelled USJ18.7 million. lba a agriculturiet; b a aqricultural economlt; c * financitl analyst; d a MAE specialist; * a engineer; f * livestock specialist; 9 a anthropologist; h a health; I a organization specialist; t a training. /a 1 a proble-ree, or minor problems; 2 a *oderato problems; and 8 a major problems. Id 1 * Improving; 2 a stationary; 8 a detortorating. L. F a financial; N a managerial; and 0 a other. - iv - PROJECT PERFORMANCE AUDIT REPORT THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (LOAN 2402-TH) Evaluation Summary Introduction 1. This su mmarizes the Project Performance Audit Memorandum (PPAM) for the Thailand National Rural Development Project, for which the Bank approved a loan of US$50 million on March 29, 1984. The project was supported also by cofinancing of US$7.6 million from the USA, Japan and UNDP. The Bank funds supported 18 investment subprograms under the govern- ment's ongoing Poverty Areas Program (PAP) in designated poverty districts within the existing National Rural Development Program (NRDP) of the Fifth Five-Year Plan (1982-1986).1/ Cofinancing was for development of the moni- toring and evaluation (M&E) system, studies, technical assistance and training. The Bank loan was closed on schedule on June 30, 1988 and US$13.7 million of unused funds were cancelled. 2. Poverty is overwhelmingly a rural problem in Tha4l%n1. About three-quarters of Thailand's population of about 53 million live in rural areas and earn their living principally from agriculture. Over one-third of the rural population is regarded as living in a state of absolute po- verty. Average incomes vary greatly by regions, with the North at one- third, Northeast one-fifth, and South at half of the Central Region. Objectives 3. The project's main objectives were to: (1) strengthen planning, budgeting and management aspects of the newly instituted National Rural Development Program, (2) to alleviate poverty in rural areas. 4. These objectives were to be achieved through three main components: (1) Strengthening of che monitoring and evaluation system (USAID and JICA assistance). 1/ The reader should note especially that this report is not an ex-post evaluation of the whole of NRDP or PAP, but of the much smaller Bank- assisted project. This was funded by Loan 2402-TH over a limited period between mid-1984 and mid-1988. There is also some reference to Bank involvement in strategy aspects of rural development during earlier preparation, to parallel sector mission discussions and in brief to developments since the project was completed. - v - (2) Special studies to identify and correct weaknesses in NRDP (UNDP and USAID assistance). (3) Support for 16 agricultural and two infrastructure subprograms (Bank assistance). The project was presented in the Bank as the first of a series to support poverty alleviation, including through health and education aspects of NRDP. Implementation Experience 5. The project was implemented through the organizational framework of the NRDP established in 1982 under the Fifth Plan. One of the four programs in NRDP was the Poverty Areas Program, which included the 18 subprograms to which the Bank loan funds were allocated. With some exceptions (detailed in the PCR and PPAM) the subprograms and related activities received most of their budget allocations. Imp.' .tation of the subprograms and related institutional components proceed.. with mixed success. Some programs were modified during implementation as intended at appraisal, when a flexible approach was advocated. 6. The monitoring and evaluation component was financed by USAID and JICA (with the latter's contribution, a mainframe computer, arriving before the Bank loan became effective). The M&E component was not designed in detail at appraisal nor was the project agreement with USAID signed at the time the Bank loan was approved. A combination of delayed recruitment of technical assistance (which was to work out operational details of the component) and doubts about the feasibility of the M&E plan by USAID, led to the component not starting up as planned and not producing the evalua- tion outputs expected at appraisal. This has made impact assessment dif- ficult, but M&E performance has since improved and is now making a valuable contribution to planning and implementation. Financing of the M&E compo- nent by USAID continued through to early 1989. 7. Signing of the UNDP agreement with government was delayed such that the 12 studies relating to policy and program implementation were late, although the Bank Overview of the PCR reports that a majority of the study recommendations on management aspects were accepted and implemented. A planned study relating to preparation of the Sixth Five-Year Plan was not carried out under the project. 8. Training of farmers and staff (including under the Bank-assisted subprograms) went largely according to plan or exceeded targets. Planning and implementation capability of the program staff were thereby improved. 9. Not surprisingly for an ongoing program with strong political support and participation at the highest levels, construction and physical aspects of investment subprogram implementation went quite well, except for some redesign and adjustments in several which had technical problems but Improved during implementation, and two which had littlo success. Tech- nical problems with some investment subprograms and resultant redesign or modification are detailed in the PPAM and PCR. - vi - 10. During implementation three main problems were of concern to Bank supervision stafft the slow rate of disbursement of Bank funds, delayed start-up of the monitoring and evaluation component and the delayed annual meeting of project donors. Disbursement &.lays were caused by late imple- mentation, delayed establishment of the special account, documentation problems and in the audit's view probably because there was no direct linkage between the project's investment activities and the flow of Bank funds. The annual donors' meetings, to review progress, were intended at appraisal as a main means of coordinating donor roles but the first of these meetings was delayed, only one other is on record and meetings were less substantial than originally intended. 11. Supervision by the Bank during implementation was heavy but defi- cient in some respects. There were only four missions at an average of nine-month intervals, but with the first not starting until five mcnths after effectiveness of the Bank loan (during a period which was marked by the delayed start and problems of both the UNDP and USAID components). Although staff inputs were very low in the first year of supervision they then became unusually high. The first two missions were led by a staff member without previous experience of the project and neither of the two principal staff who prepared and appraised the project participated in supervision. The PCR notes that although the M&E component was encoun- tering difficulties, the Bank did not send an M&E specialist on the last two missions. Results 12. In the absence of M&E data the results and impact of the 16 pro- ductive subprograms assisted by the Bank are not entirely clear. Audit analysis of impact and sustairibility aspects of 16 productive subprograms, based on PCR accounts, indicates six were without major problems and are regarded as successful, eight had moderate problems but achieved some or most of their objectives, and two had very little success to report. (Box 1 in the PPAM summarizes physical achievements and PCR comments on subpro- grams.) The PCR estimates overall physical achievements of the subpro- grams, including the two infrastructure subprograms, at 91X, with some variation (See Box 1). The audit notes that these achievements relate to the means to achieve objectives rather than objectives themselves. Com- ments in the PCR indicate that the production objectives of the subpro- grams, and therefore their poverty alleviation impacts, are either uncertain (because of the lack of objective M&E data or analysis) or that they are less in some cases than physical achievements would suggest. Overall therefore the audit has concluded that the achievements of objec- tives by the Bank-supported subprograms were somewhat less than both the PCR and Overview suggest. 13. Achievement of project institutional objectives was mixed as follows: (i) The M&E system was not established as intended and therefore its use by management to improve and refine the program was limited. (Later promising development of M&E is discussed in the PPAM.) (ii) The system for managing and controlling Poverty Areas Program activities has been improved considerably through unifying NRDP procedures, the review role of the NRDCC and the links with the Comptroller and Auditor Generals. Subprogram expenditures can now be traced. (iii) The institutional struc- ture is well established (Village Development Committees, the Township - vii - Councils, and the District and Provincial Development Committees). (i-i) Bottom-up planning, however, has not yet been introduced, although the compromise system in place does allow for more local level influence than before on investment decisions. The PCR, project evaluation reports and other sources indicate, however, that the system has to be developed further before the needs of the rural poor will 4 fully reflected in planning decisions. (v) Training and special studies were carried out largely according to plan, although with some delay. 14. Information on production presented in the PCR is based largely on circumstancial evidence and consequently the reestimated rates of return were rejected in the Overview as being based on inadequate data. Sustainability 15. The brief reviews in the PCR of each subprogram include comments on aspects of 11 of the productive subprograms suggesting that sustaina- bility is uncertain (Box 1). In sorae other cases, although there is no comment in the PCR, field observations and other evidence suggest that sustainability is also uncertain. Many of these comments relate to inadequate technical designs (poor data) and poor farmer interest. The sustainability of one of the two infrastructure subprograms (Village Water Resources Development), which was one of the larger subprograms, is com- promised, according to the PCR, by innppropriate design and poor pond location suggesting that full use of facilities is uncertain in some cases. (The Bank's regional office suggests that it is too early to assess sustainability prospects.) Findings 16. The project was prepared and processed at a time when there was growing reluctance on the part of the government to borrow Bank funds, although the Bank, conversely, was at that time pursuing an aggressive lending program given Thailandt's good absorptive capacity. Even so, the Bank make a valuable contribution to the formulation of Thailand's rural poverty program through financing consultants (under another loan), preparation discussions and the dialogue during sector work over the early 1980s. The project had ambitious objectives including the wider implica- tions for further Bank lending. The relatively small scale of the Bank's intervention suggests now that these objectives were over-optimistic (a point noted during processing). The Bank project was an attempt to join an ongoing rural development program, and change and improve some aspects of the management of that program, although related institutional aspects were funded by other donors. In that respect, however, the project was disap- pointing, especially given the relatively large amounts of Bank staff resources used. 17. The audit memorandum raises particular concerns with the Bank's role and project concept, the timing of the Bank assistance, program management improvement, poverty targetting and beneficiary participation, and the evaluation of NRDP results. 18. In respect of these issues the audit questions whether the timing of the Bank loan was appropriate; whether the expected management changes to an ongoing program were feasible in the circumstances (especially given - viii - that the Bank had little influence over these cofinanced activities); whe- ther the loan had incremental impact; notes that poverty targetting was inadequate in that PAP had less impact in the poorer villages and benefits were sometimes unduly concentrated, and agrees with the PCR and Overview judgements that 'bottom-up" planning (or beneficiary participation), in the full sense, have yet to be introduced. The audit also draws attention to the different perceptions conveyed by the PCR/Overview and the findings of the three independent impact evaluations (which raise questions about the effectiveness of NRDP's poverty alleviation efforts with respect to poverty targetting and program coverage). 19. The audit concludes that from the Bank's point of view the project experience was satisfactory in terms of the rural development policy assistance the Bank was able to provide in the run-up to the project, but not fully satisfactory in terms or project results, since: (a) it did not fulfill the program management objectives, (b) the subprogram investments had mixed results and sustainability is uncertain to doubtful for parts of the majority of subprograms, and (c) it did not lead to further Bank assisted poverty alleviation operations. In relation to the long period of project identification and preparation, the outcome in lending terms was unsatisfactory compared with the large amounts of Bank staff resources employed. PROJECT PERFORMANCE AUDIT MEMORANDUM THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (Ln. 2402-TH) I. PROJECT BACKGROUND A. Introducrion 1. About three quarters of Thailand's population of about 53 million live in rural areas and earn their living principally from agriculture. There are about 6.6 million farm households. The agriculture sector accounts for 21% of GDP, 60% of exports and 76% of employment. Through the 1960s to mid-1970s the sector growth rate exceeded 5%, but this has slowed to 3.5% since 1975. Poverty is overwhelmingly a rural problem in Thailand. Over one third of the rural population is regarded as living in a state of absolute poverty with annual incomes of less than $165 per capita (SAR para. 1.08). Average incomes vary greatly by regions compared with the best off Central Region, as follows: North--one third, Northeast--one fifth, and South--at half of the Central Region. 2. The National Rural Development Project (Loan 2402-TH approved on March 29, 1984) was the first and, to date, only broad-ranging rural deve- lopment project assisted by the Bank in Thailand. The loan of US$50.0 million was about 32% of the total project cost of US$157.2 million equi- valent. Bank funds were for village-level production investments and infrastructure in the poorer North, Northeast and Southern regions under the Poverty Areas Program (PAP) component of the government's National Rural Development Program (NRDP).l/ Parallel financing of NRDP activities was provided by the Japanese International Cooperation Agency (JICA) (US$2.1 million), the United Nations Development Program (UNDP) (US$0.5 million), and the United States Agency for International Development (USAlD) (US$5.0 million). JICA assistance was fully utilized before Bank loan effectiveness, and the UNDP-assisted component was completed in September 1986. USAID assistance continued through March 1989. The Bank- assisted component of the project was completed in March 1987, three months ahead of schedule. The loan was closed on schedule on June 30, 1988, and 1/ This audit memorandum attempts to avoid the classification confusion evident in the files and project documents by the following conventions: The government's National Rural Development Program under the Fifth Five-Year Plan is wNRDPH; one of NRDP's four component programs is the Poverty Areas Program (PAP), to which the Bank funds were allocated, and the National Rural Development Proiect under audit is either "the project' or for variety "the NRD project". the final disbursement was made on October 25, 1988. US$13.7 million (27X) of the loan was cancelled. 3. Previously the Bank had supported development of rural areas by twelve loans and credits.2/ Two of these projects were regional rural development projects, but this NRD project was intended to be the first in a series of projects directly supporting alleviation of rural poverty in the poorer areas ,the relatively better off Central Region was not included in the project). The Staff Appraisal Report (SAR) links the design of the project with the Bank's earlier experience with rural development in Mexico (the PIDER Program), Colombia (Integrated Rural Development) and the Korean Sae-Ma-Ul program. The Bank helped Thai senior staff arrange visits to these programs during formulation of the government's National Rural Deve- lopment Program in the early 1980s. 4. The SAR notes that national rural development programs date back to the early 1940s in Thailand, but that these programs were often under- funded, short-lived, poorly targetted toward the poor, and that the focus of such programs tended to shift with each new administration. Generally the main objective of such programs was the supply of inputs *qith little emphasis on program management and efficient implementation. The PCR reports that these earlier programs ran into difficulties caused by over- centralized decision making and lack of coordination among implementing agencies which led to low program achievements. 5. The reader should note especially that this report is not an ex- post evaluation of the whole of NRDP or PAP, but of the much smaller Bank- assisted project. This was funded by Loan 2402-TH over a limited period between mid-1984 and mid-1988. There is also some reference to Bank involvement in strategy aspects of rural development during earlier prepa- ration, to parallel sector mission discussions and in brief to developments since the project was completed. B. Oblectives 6. The objectives of the project were twofold: to improve the administration of the National Rural Development Program and to relieve poverty and provide employment through direct investments at the village level. Bank funding was allocated to the Poverty Areas Program (PAP), one 21 Population and Health (Cr. 767-TH), Education (Ln. 1771-TH), Agricultural Extension (Loans. 1393-TH and 1752-TH), Agricultural Research (Ln. 1922-TH), Northern Agricultural Development (Cr. 929-TH), Land Reform (Ln. 2198-TH), Livestock (Ln. 1199-TH), Northeast Rural Development (Ln. 1198-TH), Electrification (Loans. 1527, 1817, and 2312-TH). of four parts of NRDP,31 which was designed by government principally to attend to 'the needs of the poor population who have missed out on the development benefits available so far" (Fifth Five-Year Plan). The project objectives were described in the Staff Appraisal Report (para. 3.04) as follows: "M(i) to strengthen the planning, budgeting and management aspects of the newly irstituted National Rural Development Program, in or- der to develop a more effective mechanism for identifying local needs, for channelling funds to rural areas and to increase the cost effectiveness of Thailand's rural development efforts; and (ii) to alleviate poverty in rural areas through simple projects to increase food production, basic nutritional and infrastructure needs, and to provide employment opportunities......." The Bank in addition had a longer-term ambition since 'the project is per- ceived to be the first in a series to support poverty alleviation through the Rural Development Program and subsequent projects may well support health and education aspects of the Program" (President's Report, Report No. P-3744-TH, Loan and Project Summary). C. Proiert Formulation, Design and Content 7. The genesis of the project dates back to a request from NESDB in January of 1979 for the Bank to send a mission to assist the government with formulation of its rural development strategy. The mission that went some seven months later, however, was charged primarily with reviewing the merit of a project proposal (the New Village Development Program) which had been submitted by government in late 1978. The report arising from that mission and a subsequent mission was released internally in mid-1980 but concluded that there was no need for Bank support of the program at that stage since it was being funded by the Japanese government. The Bank did not follow up on government's request for a strategy mission until March 1981 when a sector mission focussed on growth and equity issues in the agricultural sector. This mission and related discussions helped the formulation of the poverty program. In addition some assistance to the rural development planning process was provided through funding under existing projects for related technical assistance and studies. Project preparation work finally got underway in earnest in January 1982, following a request for assistance from National Economic and Social Devnlopment Board (NESDB), with a mission to identify both "Rural Development I' and 'Rural Development II". Two preparation missions then followed in Novem- ber 1982 and January 1983, and appraisal in April 1983. 3/ The other three were the Rural Employment Generation Program (REGP), the Security Areas Program (SAP) and the Water Resources Development Program (WRDP). - 4 - 8. Meanwhile in 1982 the government had reorganized its national rural development effort into a new structure and in October 1982 initiated the Poverty Areas Program (which absorbed both the village development ac- tivity and the earlier rural infrastructure proposal, except for rural roads). The files indicate that Bank staff contributed specifically to- wards developing an objective methoology for identifying the poorer vil- lages and thus to targetting investments more directly at poverty at the village level. Previously the district had been the planning focus (SAR, para. 2.13). The Bank does not appear, however, to have had any impact on rationalizing the overlapping subprograms included in PAP (see further below). The Bank also helped senior Thai staff visit relevant rural deve- lopment programs in other countries. 9. The project comprised three main components: (a) Strengthening of the monitoring and evaluation (M&E) system (for which USAID and the Japanese government provided funding) including periodic analyses of the poverty alleviating impact of NRDP. (b) Support for special studies to identify and correct weaknesses in the NRDP through a UNDP grant and part of the USAID funds provided for monitoring and evaluation. Terms of reference for these studies were not presented in the SAR but were subject to agreement between the Bank and UNDP. The studies were to: (a) identify management skills needed to implement NRDP, (b) to design training programs for NRDP management, (c) to analyze administrative procedures, policy and effecti- veness as a basis for formulating the rural development pro- gram for the Sixth Five-Year Plan, and (d) to identify and correct design problems in the investment subprograms. (c) Selected agricultural and infrastructural subprograms of the PAP to be supported by Bank funds. The 16 productive sub- programs were selected from 33 in the PAP for their potential impact on production and poverty alleviation. The selected subprograms were as followst Villate-Based Subprograms (i) Village Fisheries Development (ii) Village Woodlots (iii) Village Coastal Fisheries/South (iv) Village Small Animal Improvement (v) Village Cattle and Buffalo Banks (vi) Village Agricultural Water Utilization Farmer Participation Subprograms (vii) Food Production for Nutrition (viii) Rainfed Rice Improvement (ix; Upland Crop Improvement (x) Highland Rice Improvement - 5- (xi) Fruit Tree Development (xii) Livestock Health and Management (xiii) Soil Improvement/Crop Residues (xiv) Saline Soil ImprovementINE (xv) Acid and Saline Soils/South (xvi) Soil Erosion Prevention/North Infrastructure Subprograms (xvii) Bridge Construction and Replacement (xviii) Village Water Resource Development From the above list it is apparent that all but four of the subprograms were offered in all three regions. Two features of the subprograms deserve comment. Firstly, as discussed further below, many of the subprograms overlapped in objectives and means, which suggests that the overall program was much less extensive than appears from above list. Secondly, the size of the subprograms (measured in cost terms) varied greatly--the smallest being only under 3% of the largest two and four others being under 7% of the largest two (see below on costs). This is significant in judging results. The Bank's support for these investment subprograms included substantial funds for technical training of government staff, village leaders and farmer participants (US$8.9 million or 5% of project cost). Health and education components were also considered for inclusion but dropped out before appraisal because the Bank preferred a free-standing health project and preparation of a rural education strategy was not sufficiently advanced. 10. The M&E component was to provide for recruitment of specialists to improve the ongoing M&E system (it had been started some two years ear- lier), to increase data collection and processing, provide associated equipment (including a mainframe computer), train M&E staff, and for "special studies" of (a) aspects of the M&E system and (b) analysis of selected technical subprograms. The Japanese (JICA) assistance was for the supply of the mainframe computer which arrived before the Bank project became effective. D. Proiect Cost and Financing 11. Total project costs were estimated at US$157.2 million and were expected to be financed by the Bank loan of US$50.0 million (32%); USAID US$5.0 million (3%); JICA US$2.1 million (1.3%); UNDP 0.5 million (0.3%.); and government $99.7 million (63%). Prior to appraisal it had been thought that the International Fund for Agricultural Development (IFAD) would also cofinance the project, but the files record that shortage of funds pre- vented IFAD from participating. The project was intended to cover the last three Thai financial years (1984-1986) of the Fifth Five-year Plan. Four subprograms dominated the project in terms of being allocated 62% of the project's funds as follows: Village Water Resources (222 of total funds), Rainfed Rice (14%), Village (inland) Fisheries (13%), and Bridge Cons- truction (13%). Over a third of project costs were allocated to the two infrastructure subprograms for Bridge Construction and Village Water Resources. 12. In terms of both cost and project content the Poverty Areas Pro- gram and the Bank's financing were a relatively small part of the total National Rural Development Program coordinated by the NESDB. (A senior Bank reviewer commented on file during project processing that the objec- tives seemed very ambitious in comparison with the small scale of the pro- ject.) Contrary to the expansive sounding title of the Bank project - The National Rural Development Proiect - the Bank's financial assistance was intended for only 18 out of 156 subprograms in the government's ongoing National Rural Development Program (NRDP), In half of the country's pro- vinces (i.e. about 11% of the cost and number of subprograms in NRDP). Bank assistance was to fund 35% of the cost of selected village-level subprograms in one of the four special rural development programs - the Poverty Areas Program - in the government's national program. The proposed cost of the national program for the Five-Year Plan 1982-86 was Baht 54,209 million, of which the PAP was to receive a relatively small portion, Baht 6,077 million or 11.22 of total costs. For the last three years of NRDP, while Bank funds were available, Baht 4,305 million was allocated to PAP, of which under the project the Bank provided Baht 1,053 million equivalent (24 percent of the PAP funding for the period, but only 1.9 per cent of NRDP investment costs for the Fifth Plan period). E. Implementation Arrangements Production and Infrastructure Subprograms 13. The project was to be implemented through the organizational framework of the National Rural Development Program established in 1982. This was headed by an interministerial National Rural Development Com- mittee (NRDC), chaired by the Prime Minister, which had its secretariat, the National Rural Development Coordinating Center (NRDCC), within the NESD3. The structure then spread down through Provincial Development Committees and District Development Committees, to Township Councils and Village Development Committees. The Township Councils are comprised of members elected from the villages which make up townships. 14. The planning process for rural development was intended to create a 'bottom-up" planning system for small village-level projects. However, the system did not provide for a free choice of investment projects at the village level, but offered villages options among a number of subprogram models approved at national level. (While this arrangement may have been attractive to administrators in the center of a centralized program, later comments 41 about lack of farmer interest, poor design fits with local features and the need to better tailor subprograms to the specific needs of 4/ In the PCR, evaluation studies and to the audit mission. the poor demonstrate the weakness of the system. The project included under the UNDP-funded component, however, subprogram reviews to identify and correct design problems, and changes were made - see below.) 15. Budget allocations were approved by Cabinet, while the NRDC approved provincial allocations. These provincial allocations are then split among individual subprograms and Township Councils are informed of the approved subprograms from which villages may select. (This system was introduced under NRDP and while it goes some way to decentralizing plann- ing, and introducing "bottom-up" planning, later comment drawing on the PCR and evaluation studies addresses some issues arising.) At this stage the lead in implementation would be taken by the functional line agency con- cerned with a given subprogram. The 16 production subprograms were im- plemented by the Ministry of Agriculture and Cooperatives and the two infrastructure subprograms by the Ministry of Interior through its Acce- lerated Rural Development Department. In the many cases where subprograms require inputs from more than one agency, coordination is achieved at the Township Council level. Monitorina and Evaluation Component 16. The M&E component was finar.ced by USAID and JICA, although the Bank provided staff assistance to the NRDP subcommittee on monitoring and evaluation during project preparation in drawing up a framework for strengthening the system. Operational details were to be worked up during implementation with the help of technical assistance to be funded by USAID. It was expected that complementary M&E systems would be strengthened in each of the technical ministries responsible for subprograms, in addition to focussing the M&E system on the needs of the Townships, Districts, Pro- vinces, Bureau of the Budget and NRDCC. The emphasis and prominence of this part of the project in the documents indicates that it was regarded as of great importance by the appraising staff, although one senior Bank re- viewer commented that its importance was overstated. (The project docu- ments are ambiguous, however, in describing the M&E component, as to whether it focuses purely on the Poverty Areas Program or the NRDP as a whole.) Training 17. At the time of appraisal, training activities were under review. It had been the practice for individual departments within the implementing ministries to undertake training, but a study financed by USAID had looked at the possibility of MOAC establishing a specialized training unit. The SAR supported "some rationalization' of training courses and there was a provision in the project documents that the Budget Bureau would review training costs as part of its overall review of the current costs associa- ted with the Poverty Areas Program. - 8 - F. Project Documents and Clarity of Oblectives 18. Whether by intent or oversight the project documents, including the Loan Agreement, sometimes have difficulty distinguishing between the umbrella National Rural Development Program and that part of it, the Poverty Areas Program, being supported by Bank funds. Thus the SAR uses the term "program" almost indiscriminately as though the NRDP and PAP were synonymous, while the President's Report defines "program" as being the NRDP, but then describes features of PAP under that title. Treatment of the monitoring and evaluation component is especially confusing. Although it apparently was intended to focus on PAP's activities, in some references it was also expected to improve management information for the whole of NRDP, a quite different order of work. Even the Loan Agreement is confused on whether M&E conditionality applied to PAP alone or the whole NRDP (see para. 3.04 (c) (i) and Schedule 2, (a)). (Complexity was not restricted to Bank documents, however, as shortly after the project started, changes in NESDB resulted in responsibility for proiect M&E being in different hands from NRDP M&E.) 19. This lack of clarity in the project documents gives rise to concern that at the time there may have been an imperfect correspondence between what the Bank and the government thought were the major objectives of the Bank's assistance. Comments on file by both Thai and Bank staff on the draft SAR indicate that this point was raised during the appraisal process, but the documents do not resolve the issue. 20. Vagueness or ambiguity in project documents on objectives could have two important operational consequences: firstly it is more difficult to monitor and evaluate progress and achievements and secondly it can only complicate the relationship between the Bank and the government. In this case the relationship was already complex enough, given the several govern- ment agencies involved in PAP, as well as the four cofinanciers. The files suggest that the Bank also had had broader intentions for this project than government. These intentions included pursuing a decentralization strategy for the NRDP structure, largely through the beneficiary participation as- pect, as well as leading into future Bank loans in the agriculture, health and education sectors. 21. In preparing and appraising the Bank project, the Bank selected some of the subprograms of PAP for support but did not apparently attempt to rationalize the program's strurture to avoid overlap and competition between components. Furthermore, during supervision it was noted that there were overlaps between project activities and those of the Rural Employment Generation Program (another of the four NRDP programs) and various schemes undertaken by the Community Development Department. 22. Overlaps in the list of subprograms (see above) were numerous. For example, one component (Village Water Utilization) was to assist villagers to better use existing water resources (to grow high value crops), another subprogram (Food Production for Nutrition) was to help - 9 - villagers grow legume and other vegetable crops (the same "high value crops") for nutritional reasons, while a third component (Soil Improvement through Use of Crop Residues) was to encourage farmers to use compost (to produce 'vegetables and other high value crops"). Meanwhile for livestock there was a "Livestock Health and Management Improvement Subprogram", a separate "Village Small Animal Improvement" subprogram for breeding, management and disease prevention in poultry, and a "Village Cattle and Buffalo Banks' subprogram which was concerned with breeding, management and disease prevention of animals loaned to villagers. With respect to water resources, rehabilitation or construction of village reservoirs was pro- vided for under both the Village Fisheries Development subprogram and the Village Water Resource component within the infrastructure part of the project. The somewhat artificial division of the various production sub- programs into 'Village-Based" and 'Farmer Participation' categories also tends to add further confusion since all the Village-Based schemes depend on farmer participation, and some of the Farmer Participation schemes would require cooperation and participation from most of the village to be fully effective (e.g. livestock health and soil erosion). 23. Such overlapping is the source of adverse comment in the files, both from inside and outside the Bank, and must have made it difficult for both project implementers and villagers to understand what was intended. Furthermore, monitoring and evaluation of such an intertwined set of acti- vities (and by no means all the overlaps are described above) would be unusually difficult. Accounting for project funds was clearly a complex and difficult exercise and this may have contributed to delays in submis- sion of reimbursement requests to the Bank (see below). Implementation efficiency must also have been affected, resulting in higher costs than necessary. 24. An explanation for design overlap, offered by Bank staff familiar with Thailand at the time, is that the multitude of PAP subprograms re- flects the many different agencies with rural development interests and the inability of the government to rationalize the competing proposals of these interests into an efficient overall set of activities. While this may be the reality, it reflects poorly on investment management and tends to por- tray the NRDP, in part at least, as an institutionally fragmented system which the Bank was too late, unable or unwilling to influence in the name of efficiency. II. PROJECT IMPLEMENTATION AND OUTCOME A. Implementation History 25. The Bank loan was intended to finance PAP investments and aspects of NRDP management functions over the last three years of the Fifth Five- Year Plan period (1982-1986). This was from the third fiscal year of the Plan (October 1, 1983 to September 30, 1984) through to September 30, 1986, - 10 - but because the project was not approved until March 1984 disbursements were to be made against expenditures over only the last two and a half years of the Five-Year Plan. An earlier proposal for retroactive financ- ing was dropped--without explanation on the files. In the event the go- vernment did not request reimbursement for any expenditures in FY84 and the Bank agreed to finance expenditures delayed from FY86 during Thai FY87. The date of pro4ect completion is recorded as March 31, 1987. Whether this was six months behind schedule or three months ahead of schedule is a moot point since the SAR refers to the project covering expenditures up to the end of the Fifth Five-Year Plan (Sept. 30, 1986), whereas the Loan Agree- ment shows June 30, 1987 as the completion date. 26. From the outset progress on the investment subprograms financed by the Bank under the project was reported to be generally satisfactory, as might be expected since these activities had been started earlier, 2 1/2 years before the Bank project was approved. (The issue in the circums- tances of whether the Bank funded incremental activity is raised below.) Budget allocations generally exceeded 851 of SAR estimates for 13 of the subprograms with the five subprograms being allocated less as follows: Village Small Livestock (75%), Village Woodlots (73X), Village Coastal Fisheries (69%), Saline and Acid Soil Improvement (551) and Soil Erosion (53%). 27. Supervision reports record, however, that early problems indicated that the design of some subprograms needed to be changed. Consequently, some components were altered to some degree during implementation, follow- ing technical reviews. This was foreseen as a possibility at appraisal, since it was known that the various sub-components were not fully field- tested, and the UNDP-assisted component included provision for the tech- nical reviews. The ability and willingness to adapt designs as experience was gained was one of the strengths of the implementation arrangements. Thus three programs included at appraisal (Village Coastal Fisheries in the South, Soil Erosion and Saline and Acid Soil Improvement) were cut back in the FY84 budget (the first year of the project) because of design problems and weak farmer participation. The first of these subprograms had to be redesigned to reflect farmer capabilities. Problems relating to design, climatic constraints, labor shortages and other implementation constraints were also reported for three other subprograms (Bridge Construction, High- land Rice Production and Saline Soil Improvement). 28. Three aspects of project implementation were of concern to su- pervision staff through the early life of the project--the slow rate of disbursement of Bank funds, the delayed start-up of the monitoring and evaluation component financed by USAID and the delayed project donors' meeting. Bank disbursements always lagged well behind appraisal estimates (at around half or less of the expected rate and eventually US$13.7 million of the loan was cancelled). Reasons were delayed implementation, delayed establishment of the special account, documentation problems and in the audit's view, probably because there was no direct linkage between the project's investment activities and the flow of Bank funds (since these funds were provided through the budget in any case). Informed project - 11 - staff suggested that the Bank funds had no impact on NRDP's scale of operations. 29. The M&E component was slow moving because processing of USAID financing was not on the same time path as the Bank loan (or closely linked with it). Also USAID developed reservations about the component's design and there were delays and changes in the provision of technical assistance and procurement of equipment. Inopportune senior staff changes, on both the Thai and USAID side, compounded the component's inherent problems since the new staff either were less committed to the M&E approach or queried the M&E design concept, respectively. USAID conducted a mid-term review of the M&E operation in late 1986 which was highly critical.5/ (The Overview reports, however, that the review resulted in improved performance of this component, but at no time was M&E fully effective.) The USAID review comments that the original component agreement was concluded in June 1984 under pressure from both governments but with reservation-z by both NESDB on loan funded technical assistance and by USAID on the centralized focus and (complex) coordination requirements of the M&E project. (Design of this system was aided by technical assistance funds from the Bank and one of the Bank's few M&E specialists was involved in the process.) USAID's strong reservations were conveyed in the interim reviews' report with the general conclusion that "Major RDME Project 6/ design flaws, particularly unwar- ranted confidence in a complex, centralized grand design to "rationalize' NRDP monitoring and evaluation and reliance on undemonstrated and tenuous administrative arrangements, resulted in a project with very h5g: risk of failure." The audit notes that no evidence of such strong reservations was on file earlier at the time of appraisal of the Bank project in 1983. 30. During the project many data were collected, but little was pro- cessed into usable output and passed to those who could use it (a common problem of many M&E components supported by the Bank directly elsewhere, as reported by OED 71 and others). The impact studies of subprogram perfor- mance were not done promptly enough to provide management input at an early stage when it would have been useful. USAID continued to finance the com- ponent into early 1989, although the emphasis had changed towards locally recruited technical assistance after arrangements for overseas assistance foundered over contract terms. 31. The poor performance of the M&E component weakened the planning, management and execution of the project, and also has made it difficult to draw firm conclusions on the overall impact of the project. The poverty 5/ Rural Development Monitoring and Evaluation, Interim Review, December 12, 1986, draft, USAID. 6/ i.e. the USAID project name for the M&E component. 7/ See Rural Development - World Bank Experience. 1965-86, A World Bank Operations Evaluation Studs, OED, April 1988, ISBN 0-8213-0983-8, page 34 and Annex 2. - 12 - impact studies were delayed until 1988 when separate studies of the North, Northeast and Southern regions were completed (of which the first two were obtained by the audit mission from NESDB).8/ The importance of esta- blishing a solid M&E system (given the complexity of the NRDP) was cor- rectly foreseen at preparation and appraisal (SAR, para. 3.06). With parallel financing of the M&E component, however, there was little the Bank could do to influence matters, although the Bank urged USAID and go-ernment to move faster. The audit believes, however, that a better defined M&E component (a criticism during supervision) and perhaps with fuller commit- ment to the objectives of M&E on the government side, early problems could have been overcome more swiftly. The Bank's regional office notes that "work on the M&E component has continued after project completion; the system is undergoing continuous refinements and its usefulness for planning and monitoring of poverty alleviati.on programs in Thailand has been widely acknowledged (projects being prepared by the EEC, US.ID, AIDAB and ADB all use the socio-economic data base developed by the M&E component)". 32. The PCR provides training data on nine of the 16 production sub- programs showing that for five of the subprograms training of staff and farmers was close to, or as much as double, SAR targets. For the other four subprograms (Nutrition, Upland Crops, Woodlots and Saline and Acid Soils) achievements were around 50-60X of targets. Some 940 staff were trained and thereby the planning and implementation capability of the program staff was improved. UNDP assisted the completion of 12 studies relating to rural development policy and program implementation aspects, although these were completed late because signing of the UNDP agreement was delayed by a year. The Overview reports that the majority of the study recommendations on management aspects were accepted by NRDC and Cabinet and were implemented. The planned studies relating to preparation of the Sixth Five-Year Plan were not carried out under the project. 33. The first annual donors meeting to revieu s..e NRDP annual report and program progress was delayed beyond the original covenanted date of November 30, 1984 and was held, after much urging by the Bank, only in May 1985. A second weeting was held in November 1985, but there is no further reference to annual donor meetings in the files, although there was regular contact between donor and project staff. 34. Supervision by the Bank during implementation was heavy but deficient in terms of frequency and the staff employed during the early period. The first of only four missions did not occur until ten months after loan approval and five months after effectiveness and was led by a staff member who was not involved in the appraisal process. (Neither of the two principal staff who prepared and appraised the project participated 8/ (a) An Assessment of the Impact of rural DeveloPment in the Northern Region, UNISERV, Chiang Mai University, July 1988. (b) Summary Report of the Evaluation of Local Impact of Rural Development in the Northeast Region, the Research and Development Institute, Khon Kaen University, October 1988. - 13 - in supervision.) Staff inputs for the crucial first year of the project totalled only 4.3 staff weeks compared with a planning average of 13. Thereafter inputs increased sharply to 37.6 in the second year and averaged a high 21 staff weeks over the next three years. These high staff inputs after the first year are attributable in part at least to an unusually large number of people on supervision missions, namely 5, 5, 4 and 3 for each of the four missions. Miss_.n intervals were nine, eight and ten months compared with six months recommended by the first two missions. The second mission was led by the same staff member who led the first mission, but with participation by another staff member who led the next and then the final mission. The project was rated as having 'moderate problems' by each of the four supervision missions. Although it is unclear whether a more adequate supervision effort would have improved project results, supervision received lower priority in some respects than the project deserved. 35. The PCR notes in connection with Bank supervision that "individual specialists were sent when difficulties in disbursements and implementation of the M&E component were encountered. Mission compositions were satis- factory and mostly consisted of an agriculturist, an engineer and an eco- nomist. However, there was no field participation by an M&E specialist during the last two missions even though very little progress had been made under this component. Supervision missions succeeded in accelerating dis- bursements, overseas training and initiation of special studies." B. Proiect Results 36. In the absence of M&E data the results of the productive sub- program investments are indeterminate in an objective sense, and the audit mission's field visits can add only impressions of such a disparate pro- gram. The audit's analysis of PCR comments on impact and sustainability aspects of each component is presented in Box 1 (note that column (a) of this box refers to physical achievements, which were lmpressive in many cases--but see pars. 37 below on objectives). These indicate that of the 16 productive subprograms, six were without major problems and were regarded as successful, eight had moderate problems tut achieved some or most of their objectives, and two had very little success to report (Box 1). In the case of 11 subprograms the PCR makes comments suggesting that sustainability of some investments may be uncertain.9/ 91 The Banks's regional office disagrees with this assessment on the basis that it is too early to judge. - 14 - BOX 1: PERFORMANCE OF INVESTMENT SUWPROGRAMS (PCR data and e_mento unloss stated) PCR Reservatione Co) (b) (c) Vii Iage-Based Suboroeram PCR Coments 1 Village Fisherlos Development 198 a I In earlier schee_s low fanrmr participatlon, planned technical standerds not achieved because poor siting of pond axnd budgotsry coastraints, production lovolo uncertain. Probloem overcome for loter schoes, but returns vory low. 2. Village Woodlote 115 x x Initial probloe wore unoxpectedly high seedling demand (no prlor survoy), inadquate supply and low survlivl rate. Most probloes wore resolved but survlval rate still low (SOX). 8. Village Coastal Fisherles/South 84 x x Subprogrm modiflod and greatly reduced, lnadequately prepared, achievemsnte vry low. 4. Village Smll Animal Improvemnt i1O Subprogram modifled--poultry Improvement dropped, poultry vaccination Successful. S. Village Cattle axnd Buffalo Banks go x Rates of return low, borrowing term modified. 8. Village Agric. Water Utilization 68 x a Competing use of water, low benfilts, overlaps with other subprogrom. Farmer Particisgtlon Subproaroms 7. Food Production for Nutrition 100 x ; well targetted to poor, poor sed quality, but signiticant contribution to child nutritlon. S. Rainfod Rice Improvoeont 94 x x Poor targettino, low quality sed, low dmand by farmers. Soed bank proposl moditiod. 9. Upland Crop lmprovemnt 44 x x Duplicates other subprograms-poor sed supply. 10. Highland Rice Improvement 71 x x Serious shortage of Improved sed. 11. Fruit Tre Dovelopment 113 Achlovemsnt refers to sedling distribution, no data on survivol. 12. Liveotock Helth an Mana gemnt 1ts Cattle program succesful. Pig coverogo very low cf. SAR's unreolistic populotion assumptions. 18. Soil Improvement/Crop Reidues iN x Farmer partlelpation very good, supplies and rice straw shortage (in e_ms areas). Ovarlaps with No. 14)4/ 14. Saline Soll Improvemont/NE 116 x x Several technieal problems--inadequate technical preparation, lmpet confused with other Subprograms. 15. Acid and Saline Solls/South 9 x x Serious technical problems becaue of Inadequate site sirveys and design faults, Impct confused with other subprŽgrsm. 'Very high subeidy *lement'. 18. Soll Erosion Prevention/North 9 Targets about met, except for low cover crops seed produetion and distribution (but weak PCR coverage). Infrastructure SuboParame 17. Bridge Construction & Replacement 62 DoesIgn change ralsed cost for 1SX of bridges, budget cute, constr. standards good and within SAR costs. 18. Village Wator Reoure Dev. lSO x x Inadequate site date led to inappropriate doeigns and pond location, which my conflict with villagers' ned therefore full use of facilities Is uncortain In som ses. Column Identificotlon (a) Physical Achlevemento X. From PCR Tablo 1, Key Indicators, with some &udit judgem_nto to obtain representative single figure when more than one physical activity por subprogrm. When weighted by SAR cosot audit's analysis indicate SOX overall physical achievement of. 91X quoted In the PCR. (b) Performnce Problems. (e) Sustainability Insues, at loset for part of subprogram. Both (b) and (c) are audit claseificat:ons as follows: x a significant reforne In PCR; blanks indicate no probloem/no clear reference In PCR. / Kaseteart University report detalIs more nrious problems and recoemnds against oxpanding this subprogram. - 15 - 37. The independent evaluation by Kasertsart University 10/ provides an alternative view of the subprograms (See Chapter 5). This report recom- mends, based on results to date, that eight subprograms be expanded, that six be expanded subject to modification or pre-conditions (including that two overlapping ones be merged), and that three not be expanded. No direct recommendation was made on the bridge construction program. The suggestion not to extend the crop residues subprogram is based on serious problems which are not reflected in the PCR or therefore In Box 1. 38. Physical implementation achievements are reported by subprograms in the PCR in detail in terms of both outputs and costs. The PCR states that the project achieved 91% of its physical targets. These results are heavily influenced, of course, by performance of the four subprograms which were allocated 62% of the project funds, as follows: (a) Village (inland) Fisheries slightly exceeded targetted construction performance (although distribution of fingerling fish was low at only 62% of target). (b) Rain- fed Rice Improvement exceeded the target by about a third. (c) By quantity only some 62% of the planned Bridge Construction was completed because of budget cuts and changes in objectives. (The audit does not find this a major failure to meet project objectives since this component was in any case poorly targetted towards poverty alleviation. Construction of timber bridges in village localities would have had more direct impact on the poor than their replacement on district roads by two-lane concrete bridges.) (d) In the case of Village Water Resource Development, the PCR reports that construction of ponds and reservoirs was above 90% but has strong reser- vations on whether, in the pursuit of quantitative targets and with ina- dequate site data, the facilities constructed meet village's needs and will be fully used. Notable poor achievers were Acid and Saline Soils (techni- cal problems /overlap with other subprograms) at 9%, Village Coastal Fishe- ries (high costs and low farmer interest) at 34% and Upland Crops Improve- ment (unclear objectives/overlap with other subprograms/inadequate seed supply) at 44%. 39. The audit notes, however, that reporting achievement of physical targets is only a partial and sometimes even spurious indicator of success in meeting objectives. In most cases physical components of a project are only the means to achieving an objective. In this case, for example, construction of ponds for fish farming is only the means to raise protein consumption and increase village income towards the ultimate objective of raising the quality of life of the poor. Monitoring the achievement of this objective requires carefully designed field surveys and analysis of data, not just the cataloguing of physical activities. (See also Box 3 below in this connection.) 40. The PCR reports a remarkable impact on poverty during the project period in that out of 286 "poverty districts' targetted at apprsisal in _ / "An Ex-post Evaluation of Villate-based and Farmer-Participation Schemes in Rural Poverty Alleviation Program", Kasetsart University for NRDCC, November 1987. - 16 - 1983, 122 (or 43%) are listed as having bean promoted to the non-poverty category by 1985.11/ This is an excellent result, if verifiable, but the audit finds it difficult to comprehend that a significant and sustainable change on such a widespread basis could have occu.red in so short a time. In any case it is doubtful whether this result is attributable solely to the project (cause and effect is not demonstrated in the PCR), and data anomalies may contribute to the result. C. Institutional Impact 13. Institutional results are summarized here again for convenience. Achievement of project institutional objectives was mixed as follows: (i) The M&E system was not established as intended and therefore its use by management to improve and refine the program was limited. (ii) The system for managing and controlling Poverty Areas Program activities has been improved considerably through unifying NRDP procedures, the review role of the NRDCC and the links with the Comptroller and Auditor Generals. Sub- program expenditures can now be traced. (iii) The institutional structure is well established (Village Development Committees, the Township Councils, and the District and Provincial Development Committees). (iv) Bottom-up planning, however, has not yet been introduced, although the compromise system in place does allow for more local level influence than before on investment decisions. The PCR, project evaluation reports and other sources indicate, however, that the system has to be developed further before the needs of the rural poor will be fully reflected in planning decisions. (v) Training and special studies were carried out largely according to plan, although with some delay. Clearly, establishing the NRDP and its institutional structure was a major step towards correcting the flaws in earlier rural development programs. The government, NESDB, and the Bank, to the extent that the Bank was of assistance in designing the system, can all take credit for this major achievement. Under the project the USAID and UNDP assisted compo- nents focussed on the M&E and management aspects, respectively. Unfortunately, for the reasons given above, thr1 M&E component did not progress as assumed at appraisal (and there was a suggestion that plans and ambitions at the time were optimistic) with the result that less infor- mation was available for project management than was desirable. There is also some difficulty assessing the impact of the subprograms and further evaluation work is needed to follow up the conclusions on specific issues of the university evaluation studies. 111 The Bank's regional office reports also that "a preliminary study by Kasertsart University and Khon Kaen University, recently commissioned by the Department of Fisheries, found that PAP activities increased per capita fish consumption from 10 kg to 25 kg in the past seven years'. - 17 - Although there was a delayed start, progress on the UNDP funded management aspects was more even and had a valuable impact on the mana- gement system, as intended at appraisal. Most of the management recom- mendations were taken up except, as the PCR states (para. 4.20), for strengthening of NRDCC and the program for decentralization to the pro- vincial level. The recommendations adopted are expected to reduce NRDP administrative costs and to increase efficiency. On one aspect, introduction of bottom-up planning, opinions differ. The evaluation reports state that bottom-up planning, per se, has not been introduced. The audit's understanding of the system now in use confirms this view, with the qualification that at least there is more opportunity now, in theory, than before, for villages' needs to be heard. In practice, however, offering poor villagers a choice of approved sub- programs through the (intermediary) village leader may not result in real needs being properly served and provides opportunity for concentration of benefits in a few influential hands. Several PCR and evaluation report comments on subprograms not matching villagers' wishes attest to this problem. (This issue is well recognized in academic literature, including for Thailand.) Clearly, therefore, there is further to go before true bottom-up planning, as intended at appraisal, is reflected in day-to-day practice. The Bank's regional office, on the other hand, places emphasis on the current compromise between top-down and bottom-up a planning being a major step in the right direction and does not see the project as having fallen short in this respect. D. Economic and Financial Performance 41. The impact evaluation report completed by staff of Kasetsart University 12/ presents a range of economic rates of return for the dif- ferent subprograms (some startingly high), but the Overview by the regional office correctly does not accept these on the grounds that data were inadequate. 42. A good example of the weakness of the impact analysis stemming from poor data (attributable to the slow start of the M&E program) is that eleven of the subprogram production estimates (which are used to calculate the rates of return) are not actual results from field records or sample surveys, but are based on counts of farmers attending field days, visiting demonstration plots or receiving improved seeds. No evidence is presented of production results, objectively measured. (Furthermore the audit mis- sions' visits to a limited number of sites in the field included a number of subprogram activities with mild to serious problems.) Rates of return calculations are also not always supported by consistent accounts of sub- program results in the PCR text. 12/ Op. Cit. - 18 - III. FINDINGS AND ISSUES A. General 43. The project was prepared and processed at a time when there was a growing reluctance on the part of government to borrow Bank funds, although the Bank's Lending Program was showing more not less lending over forward years. (The Bank at one time programmed US$75.0 million for the project, but a higher loan smount was not acceptable to the government.) The Bank was successful, during the project preparation phase and parallel sector work, however, in providing support and technical assistance (financed from other Bank loans) in critical areas of program formulation. More ambitious objectives and potential spin-offs were loaded onto the project than were justified in the lending climate of the time. The project now generally appears somewhat overblown, which was a sentiment expressed by reviewers during project processing. It was overblown in terms of its too optimistic institutional objectives in a short period of three years, its selection of subprograms for Bank support--many of which overlapped or were very small (Box 2 suggests a list of six investment categories which rationalizes where Bank support was intended) and its promises as the entree for futuro poverty alleviation lending in three sectors at a time when the government was trying to contain or even reduce external borrowing. In the event the uncertain sustainability (of parts of some components) and institutional shortcomings, notwithstanding its physical achievements, now confirm that earlier promises of things to come were over-optimistic. BOX 2: RATIONALIZED PROJECT SUBPROGRAM INVESTMENTS (in each of M., N.E.& S. Regions where not otherwise indicated) 1. Crop Production Development (a) Annual Crops Increase production through use of improved seeds and other inputs and better water use. Crops includet irrigated and rainfed lowland rice, highland rice in the North; upland crops in the North (maize, sorghum, peanut, mung bean and kenaf); and others suCcu as legumes, sesame and vegetables. (b) Tree Crops Establishment of fruit tree groves, and woodlots in the N. & N.E. Regions. 2. Land Management Improvement of saline and acid soils in the N.E. & S., use of crop residues and erosion control in the Northern Region. 3. Livestock (a) Management, breed improvement and veterinary services for cattle, buffaloes, goats and poultry. (b) Establishment of cattle and buffalo 'banks' for poor villagers. 4. Fisheries - 19 - Development of inland fisheries in the N. & N.E. and coastal fisheries in the Southern Region. 5. Nutrition Special assistance to malnourished areas in the production and preparation of nutritious foods. 6. Infrastructure Construction of road bridges and replacement of timber road bridges, and development of village ponds and reservoirs. 44. The failure to establish an operational M&E system at an early stage of implementation leaves results more uncertain than they should be, although the original M&E design may now be considered unrealistic. (Even so, given the crucial importance of the M&E component, it is difficult to understand now how the government and USAID allowed relatively minor con- tractual issues to hold up M&E activities for so long. With full commit- ment on both sides, such matters should have been resolved more speedily.) Physical data are presented in the PCR on activities completed under the project but few of the reported production results are based on objective field data, and there is som, inconsistency between the PCR and evaluation study findings. The widespread use of attendance figures at extension meetings and demonstration plots as evidence of technology transfer (and supposed production increases) is an unsupportable proxy for field surveys and analysis of real change in the countryside. References in tho files to benefits in some cases being too concentrated among few families raises questions about the effectiveness of poverty targetting, even when invest- ments are located in poorer villages. A functioning M&E system could have allayed fosrs raised by such concerns and allowed program managers to take corrective action. Reservations in the impact studies that have been com- pleted to date for the North and Northeast,13/ with USAID finance, question the effectiveness of the current poverty targetting efforts and are discussed further below. 45. Some more particular concerns raised by this aud't are discussed below and relate to the Bank's role and the project concept, the timing of Bank assistance, program management improvement, poverty targetting and beneficiary participation, and the evaluation of NRDP results. B. The Bank's Role and the Proiect Concept 46. The request for Bank assistance for rural development made by NESDB in early 1979 (some five years before the project was eventually approved) was related to a Bank mission reviewing the government's ongoing rural development programs and making a contribution to strategy aspects of rural development planning. No such broad focus mission was mounted by the Bank until March 1981, but apparently with a view to an early lending 13/ Op. Cit. - 20 - opportunity the operational division concerned focussed mainly on project identification, but without a project materializing. The agricultural sector mission of March 1981 reviewed growth with equity issues and made recommendations with respect to rural development implementation for decentralized planning, including taking into account local needs, and for rationalizing and coordinating line agency roles. Project work was then in abeyance, by intent, to give the upper levels of government time to decide what it would do for rural poverty alleviation under the Fifth Plan. Time passed and eventually preparation in earnest of the project began only in the first year of the Fifth Five-Year Plan, after NESDB expressed renewed interest in Bank involvement in the poverty subprograms. In consequence the project had to link up with an ongoing rural poverty alleviation component in the Plan, with the Plan period half over before the project was approved.14/ The presentational compromise in the documents was that although the project was costed over the last three years of the Plan period, disbursements would be against only the last two and a half years. 47. This sequence of events suggests that there may have been very little, if any, incrementality of development activity associated with the Bank loan.15/ (Consequently, for some agencies and staff in Thailand the Bank-assisted project was seen as a distracting and ephemeral event.) Disbursements of Bank funds were in any case not linked directly with project activities, hence in part the delays in processing reimbursement claims. Whether the budget cuts that occurred would have been greater in the absence of Bank funds cannot be judged now. 48. The Bank, after contributing to program formulation through financing consultants and through sector work discussions, thus joined in to support NRDP financially in the area of rural poverty alleviation and related institutional aspects, but only after these programs were well underway. The Bank did not finance institutional activities (M&E, stu- dies, technical assistance and training of senior staff), as these were funded by other donors over which it had little control. Furthermore, at the time the Bank loan was approved these two donors had yet to conclude agreements with government.16/ This reduced the effectiveness of the Bank's dialogue with the government on rural development issues since the 14/ A knowledgeable reviewer of this report has suggested that Joining an ongoing program, after the wrinkles had been ironed out, was one of the strengths of the Bank project. 151 A reviewer of this report has characterized this as "the price of entry into the program'. 16/ The cofinancing arrangement was not a Bank decision, however, but the Government's, which preferred to fund such "soft" investments from low cost sources. The point remains that proceeding with the Bank loan before two cofinanciers had processed their funds, and without cross effectiveness clauses, left the Bank high and dry with no control over two activities it considered vital to the success of the Bank project. - 21 - Bank's participation was relatively minor in importance in the overall NRDP budget and in the work program of the major agency--the National Economic and Social Development Board (NESDB). (At the working level in Thailand the relatively minor role of Bank funds in the program, but high profile of the Bank's operation, combined with disproportionate time demands on NESDB staff, was a cause of concern.) C. Timing and Duration of the Bank Proiect 49. The feasibility of changing the management system of NRDP through externally funded interventions was compromised by the late arrival of the project and the relatively small funds involved (in the jargon--the Bank's "leverage" was not great). In the audit's view it was by then too late to expect a complex multi-agency M&E process to be developed into a function- ing system for introducing changes to program design and implementation (but see also footnote 10/ above). The time horizon was unrealistically short, even if all parties had been fully committed to the outline system design, as they apparently were not at the time. (The short time horizon might now look less of a problem if there had been a follow-on project.) In particular the change over to a truly beneficiary centered planning and Implementation process could not be achieved at all, in the audit's view, so long as the "approved subprogram' menu approach to village project selection was imposed by the centralized management system. This change, even if it had been agreed as an objective in principle, would take some years to introduce through a gradual process of consensus building among administrators and demonstration of its effectiveness to villagers them- selves, not just their leaders. The audit concludes, therefore, that both the general institutional objective (the main selling point justifying the Bank's financial assistance for subprograms) was unrealistic, especially with respect to the timing and duration of the project (similar concerns were clearly also behind the comments of some reviewers during the ap- praisal process). D. Program Management Improvement 50. In the project financing plan, all the Bank funds were allocated to the investment subprograms, although these appear almost incidental to the main objective of improving the management and implementation of the program. Unfortunately this institutional objective was largely outside the control of the Bank since it was being financed by other donors (there were no cross effectiveness clauses in the Bank's legal documents). With respect to USAID's support of M&E, there appears from the files to have been a lack of full agreement with the appraisal design (at least, later, when USAID staff changed). To that extent the project design, or at least its financing plan, were faulty if the Bank was most interested in the wider program aspects of poverty alleviation in Thailand. (Possibly the Bank could have done little to change the arrangement since the government chose the cheaper sources of funds to finance "software" components.) 51. Program management remained centralized although there was long standing pressure by the Bank during preparation to decentralize the system - 22 - (decentralization was one of the recommendations of UNDP-funded management studies which vas not accepted by the government). It is evident from the documents and field visits that such a dispersed program as PAP cannot be controlled efficiently from the center. The consultation process on investment selection between the various levels of the NRDP structure, down to the village, is an improvement on the past totally 'top-down system", but needs further development. Greater devolution of the decision-making process was advisable and some has since occurred. With such devolution, provided the right incentives and transparency (the M&E system) are in place, the problems of targetting the poor and beneficiary participation (see below) are likely to be more readily resolved. 52. Box 3 outlines one management tool that has been introduced through the M&E program and which holds great promise for the future. BOX 3: IMPACT EVALUATION AND VILLAGE-LEVEL INDICATORS One aspect of the management of the National Rural Development Program reported to the audit mission holds great promise for dealing with the targetting and impact evaluation deficiencies exposed during the pro- ject. This is the village-level socio-economic surveys and updates that have been introduced for all the about 53,000 villages in Thailand. Data were first collected in 1984 and are updated every 2 years for 34 indica- tors in five categories as follows: infrastructure--10, agricultural production--10, public health--8, education--4, and water resources--2. Data are processed and stored on the mainframe computer financed by JICA at the Information Processing Institution for Education and Development. The system also provides for individual family returns to be updated regularly, although this information is not processed beyond district level. The audit mission did not inspect the system closely, but provided that the indicators have been carefully selected and the incentives and controls are in place to ensure accurate recording of data, the system could be very helpful in planning and monitoring poverty alleviation pro- grams. Thus the system provides a means of readily identifying different categories of villages (and eventually families) for targetted poverty alleviation programs, and then the means to track the impact of such pro- grams objectively. To preserve the integrity of the system, however, careful "fild-truthing' of data, on at least a random sample basis and by disinterested staff, is essential to avoid biased recording of progress, which would be the natural tendency of a management system whose perfor- mance is measured by such indicators. Provided such controls are in place, or can be put in place, this village-level indicator system has great potential for improving Thailand's rural development programs. - 23 - E. Reaching the Rural Poor 53. The achievements of PAP are qualified by the following two obser- vations on NRDP poverty targetting and beneficiary participation. The PCR notes in this respect that "vested interests of people at the field level and lack of adequate monitoring and coordination resulted in many project benefits accruing to non-poverty areas and to better-off beneficiaries". 54. (a) Targetting was clearly faulty if the main objective was alleviation of the worst poverty, as numerous references to basic needs and subsistence production in the SAR seem to suggest. The evaluatior. impact studies of the North and Northeast Regions quoted above note that 'he pro- gram has relatively less presence, and therefore results, in the more remote and poorer villages (the Bank was right at the preparation stage to encourage government to redefine poverty areas in terms of villages rather than districts but this alone may not have been sufficient to get the pro- gram into poorer villages). There are also sufficient references (in the files, the PCR (see para. 52 above), evaluation studies and the Overview) to benefits in some cases being undesirably concentrated, to send a warning signal that mechanisms have not yet been established to design projects and allocate funds in a fully equitable way. 55. The evaluation reports note also that major distortion of bene- ficiary participation tends to occur when villagers are expected to con- tribute free labor to scheme development. This results in participation shifts (and therefore it is assumed in realization of benefits) away from the poor, who can least afford to contribute free labor. (In poverty al- leviation efforts elsewhere, payment for labor inputs, not necessarily at the full market rate or in cash, is regarded as essential to involving the poor in village level development schemes.) 17/ 56. (b) Beneficiary Participation or "bottom-up" planning was intended to ensure that investments met the real needs of the poor and consequently that facilities provided were more likely to be sustainable. Unfortunately the approach adopted, of offering standard approved sub- programs to village leaders on a take-it-or-leave-it basis, does not meet this requirement. This is especially so since as the files note, "village' usually means "village leader" with respect to how a subprogram is select- ed. Thus as several observers record in the files and impact studies, the introduction of "bottom-up" planning has yet to be achieved in practice, no matter how the current process is presented on paper. Some of the subpro- grams visited by the audit mission had sustainability problems which pro- bably resulted in part from lack of meaningful participation by beneficia- ries in their design. 171 For example, see Aqa Khan Rural Support Protram, An Interim Evaluation, OED, June 1987. - 24 - 57. The Overview reports that both poverty targetting and bottom-up planning are being strengthened under the Sixth Five-Year Plan. F. Perceptions and Reality 58. The audit notes the different perceptions conveyed by the PCR and Overview (by the numbers and qualitative judgements on project performance) and some of the findings of the independent impact evaluations done by Kasetsart, Chiang Mai and Khon Kaen Universities. Two facts stand out from the regional evaluations of the North and Northeast: (a) that the remoter (and usually smaller and poorer) villages have participated less in the program than larger well off villages, and furthermore, (b) that in the North 43% of village leaders were reported not to know of NRDC projects. These findings correspond with observations that because of their remo- teness, smallness and relative lack of access and facilities, those vil- lages most in need of assistance see government officials the least. (Un- fortunately, this is also true of audit missions.) Again positive steps and appropriate incentives are required to deal with such perverse features of poverty program coverage and benefit distribution. G. Conclusions 59. The project was approved some five years after the Bank was asked by NESDB to assist with rural development strategy on a national scale. In the interim some individual project proposals were submitted by the govern- ment and considered by the Bank operational division concerned, but without results in lending terms. Response by the Bank to the original request for a strategy mission did not occur until March 1981 when an agricultural sec- tor mission visited Thailand. This was understandable at the time, given that in addition to the usual pressure to lend, there was added urgency in Thailand's case because the country was seen as having good absorptive ca- pacity (notwithstanding reluctance to borrow). (This sector work, combined with later project preparation discussions, did provide the government with valuable assistance as the approach to rural development was being refin- ed.) The audit suggests, however, that had the Bank been able to take the longer-term view in 1979 and provide the requested strategy assistance earlier, it might have avoided the various project-specific false starts and ended up with a tidier, more timely operation in support of NRDP under the Fifth Plan.18/ This would have been preferable to the late starting project eventually supported, which then slipped further. Even this project was processed before all the parts had fallen into place, since 18/ An alternative view offered by the regional office is that the Bank was right to stand back, as it chose, to let the government decide how to proceed with rural development before the Bank became directly involved as a lender. - 25 - neither USAID nor UNDP had finalized their support of the vital institu- tional components to which the Banks's project documentation gave center stage. Bank staff refer to lending pressure at the time as an explanation of the untidy arrangements surrounding cofinancing, since clearly the Bank loan was processed eventually on a faster track than was the support of the two remaining cofinanciers (JICA had already financed the supply of the mainframe computer and thus was barely a "cofinancier"). 60. Not surprisingly, for an ongoing program with strong political support and participation at the highest levels, construction and physical aspects of subprogram implementation went quite well, except for some redesign and adjustments (as was expected and provided for at appraisal) in several which had technical problems but improved during implementation and two which had little success. Benefits and results are not well document- ed, however, because of the failure of the M&E component to function as quickly and comprehensively as expected at appraisal. (Experience now suggests that these M&E expectations were over-optimistic--a point made by Bank reviewers ex-ante and noted by USAID ex-post.) Consequently the audit had to rely on judgements presented in the PCR, on the three evaluation studies and on interviews and limited field visits in Thailand, rather than on objective M&E data. 61. The audit concludes that from the Bank's point of view the project experience was satisfactory in terms of the rural development policy assis- tance the Bank was able to provide in the run-up to the project, but not fully satisfactory in terms or project results, since: (a) it did not fulfill the program management objectives, (b) the subprogram investments had mixed results and sustainability is uncertain to doubtful for parts of the majority of subprograms, and (c) it did not lead to further Bank as- sisted poverty alleviation operations. In relation to the long period of project identification and preparation, the outcome in lending terms was unsatisfactory compared with the large amounts of Bank staff resources employed. - 27 - PROJECT PERFORMANCE AUDIT REPORT THAILAND NATIONAL RURAL DEVELOPMENT PROJECT (LOAN 2402-TH) REGIONAL OVERVIEW OF THE PROJECT COMPLETION REPORT Introduction 1. This Overview by the Asia Region (AS2AG/ASTAG) is based primarily on the PCR for the National Rural Development Project, financed under Loan 2402-TH. The PCR was prepared by the National Rural Development Coordination Center (NRDCC) of the Royal Thai Government (RTG), with assistance from staff of the FAO-World Bank Cooperative Program. The Regional Overview expands on the PCR by incorporating the results of the final evaluation report ("An Ex-post Evaluation of Village-Based and Farmer Participation Schemes in Rural Poverty Alleviation Program") of the Faculty of Economics and Business Administration, Kasetsart University, Bangkok. It also clarifies some apparent inconsistencies in the PCR by expanding the description of subprogram performance, and reassesses the economic evaluation of the project and its subprograms. Project Obiectives and Content 2. The National Rural Development Program (NRDP) of the RTG had begun operations in 1982. Within the overall framework of the NRDP, there were four special rural development programs: (a) the Poverty Areas Program (PAP), which focussed on the longer-term development needs of the country's poorest rural areas; (b) the Rural Employment Generator Program (REGP), intended to provide off-season, labor-intensive employment so as to stem seasonal migration to Bangkok and other major urban centers; (c) the Security Areas Program for villages near the Khampuchean conflict zone; and (d) the Water Resources Development Program. Bar,k support for the NRDP was proposed by an identification mission in May 1982; project preparation took place during the following year, and the project was appraised in April 1983. Loan 2402-TH was presented to the Bank's Directors for approval on March 29, 1984 and became effective on August 14, 1984. Parallel co- financing was provided by USAID, UNDP and JICA. 3. A consultant study commissioned by the NRDP Secretariat (NRDCC) and financed under IDA Credit 929-TH had reviewed initial implementation experience under the NRDP and made extensive recommendations; many findings were incorporated into the project design. Also, in appraising Loan 2402-TH, the Bank mission adopted the approach that the NRDP was an evolving and innovative program, and that the project therefore needed to be kept simple and flexible in design. Some lessons would be learned only in the course of actual project implementation and they would be applied gradually, so that the framework and content of the NRDP would be continuously evolving and improving. 4. The Bank-supported project concentrated on only two aspects of the NRDP. First, the project financed institutional development activities - 28 - to strengthen the glanning. budgeting and management of the overall N4RDP, in order to establish a mechanism for identifying local needs and channeling official funds in a cost-effective manner to rural areas. These activities included: (a) development of a monitoring and evaluation (M&E) system to provide timely management information, program progress and budget analysis, and periodic analysis of the impact of the NRDP on alleviating poverty; and (b) special studies to identify and correct weaknesses in the NRDP before the M&E system became fully functional. USAID and JICA provided parallel financing for the M&E component, and UNDP for the special studies. 5. Second, in order to support the national Roverty alleviation effort. Bank financing was channelled to 18 of the Droductive "self helD" subgrograms of the PAP in some 12,600 villages located within 286 poverty districts (40% of the country total, spanning 38 provinces), including: (a) six village-based subprograms requiring the involvement of the community (village fisheries development, village woodlots, village small animal improvement, village cattle and buffalo banks, village coastal fisheries in the south, and village agricultural water utilization); (b) ten farmer-based subprograms for individual farmers or groups (food production for nutrition, rainfed rice improvement, upland crop imprt-vement, highland rice improvement, fruit and other productive tree planting, livestock health and management improvement, soil improvement through crop residues, saline soil improvement in the Northeast and the South, and soil erosion in the North); and (c) two infrastructure subprograms (bridges and water supply). Details concerning targets and accomplishments, by subprogram, are provided in Table 1 of the Borrower's PCR. Project Implementation 6. General. The administrative framework for the NRDP was grafted onto the then existing national planning system in Thailand. At the center, the NRDP was coordinated by a Cabinet-level National Rural Development - 29 - Committee (NRDC), chaired by the Prime Minister ar.d including high-level representatives of all concerned Ministries. The NRDC set policy and allocated the budget committed by Cabinet for rural development. It was assisted by a permanent secretariat known as th.e National Rural Development Coordination Center (NRDCC), established within the central planning unit (National Economic and Social Development Board, NESDB). Although counterparts to NESDB had already existed for some time at the provincial, district and township levels, RTG planning arrangements were in fact heavily centralized. One of the primary objectives of the NRDP was to revert this process by fostering a "bottom-up" planning system for small rural development activities. Rural development sub-committees or working groups were therefore established at the provincial, district and township levels to assist the general planning units, and considerable training was built into the NRDP to develop local rural development planning and monitoring capabilities. 7. With regard to the Bank-supported project, at national level the NRDCC was responsible for overall coordination and for implementation of the M&E and special studies components. The Ministry of Agriculture and Cooperatives (MOAC) was responsible for implementation of the 16 productive poverty area subprograms, and the Ministry of Interior (MOI) for the two infrastructure subprograms. Some initial activities were delayed by the need to identify staff, establish institutional working relationships and set up the first Thai Special Account. Also, the NRDCC was inexperienced in working with several co-financiers, each with its own procurement and reporting requirements. Nonetheless, Ministerial performance was generally satisfactory, and many lessons were learned and applied. Individual subprograms started promptly (indeed, many were already underway, since the NRDP had initiated operations in 1982), with potential beneficiaries variously involved. 8. Government officers usually discussed possible activities with township councils or village committees; the objective was for the village committee to identify development constraints, rank them in order of need, and request help in implementing specific subprograms. At the outset, RTG officers often provided a short list of activities and villages either chose or were assigned a subprogram, but this situation later improved as more attention was given to strengthening provincial, district and local governments and to the need for public participation in planning and implementation activities. Members of the township councils and village committees were usually selected by the village headmen and were often representatives of the better-off families, which may have influenced selection of specific subprograms. There was no special effort to ensure participation of only the poorest families, since all rural citizens of the 286 districts in the designated poverty areas were considered eligible beneficiaries. 9. Monitoring and Evaluation. This component included: (a) technical assistance to improve linkages between NESDB/NRDCC, the - 30 - implementing agencies, and the provincial, district and township governments; (b) equipment and transport facilities; (c) local and overseas training; and (d) a mid-term and final evaluation of M&E activities. JICA assistance, in the form of a grant for the purchase of a central mainframe computer housed at the Information Processing Institution for Education and Development (IPIED), with terminals at NRDCC and other agencies, was fully utilized in FY1984. USAID assistance will continue through March 1989 (para. 11 below). 10. Despite the fact that all parties considered M&E to be crucial for proper planning, management and evaluation, the component achieved very few of its objectives during the first years, due mainly to a lack of agreement over procedures for contracting technical assistance. RTG favored direct contract, while USAID felt that technical assistance should be secured through the U.S. Bureau of Census. The latter insisted on having diplomatic privileges and, after breakdown of protracted negotiations between RTG and the Bureau of Census on this issue, RTG finally hired a Senior Advisor directly in 1985. A second advisor was not available until October 1986, and a Thai consultant joined only in 1987. The failure to act promptly to place an M&E team leader, recruit local and foreign staff, gain the support of advisory and participating organizations, and procure essential equipment all served to restrict severely the effectiveness of this component. 11. Besides the difficulties in bringing on board technical assistance, effectiveness of the component was constrained by lack of clarity from the outset as to the kind of M&E system and the information requirements that would be appropriate for the NRDP. During the early stages of the project, large quantities of information were collected and forwarded to the NRDCC for central processing and interpretation, Without standardization of definitions and formats, staff and computers available to handle the volume of information were inadequate, and it was not until after the mid-term review of the component that some of these problems began to be resolved. With the exception of the village survey form used in developing the Sixth Plan, many of the questionnaires which were regularly completed and forwarded by participating agencies were of limited value in identifying the needs of target groups in specific geographic areas. On the other hand, many people received training in M&E, rural development planning and project management, and this was perhaps one of the most important achievements of the M&E component. It is also important to note that USAID is continuing to provide assistance to RTG through March 1989 to support Thai management consultants, purchase microcomputers for all provinces, and fund additional training for provincial and line agency headquarters staff. 12. Certainly the delays in implementing M&E activities weakened the management, planning and execution of the project. Specifically, they made it difficult to assess subprogram benefits anid identify in a timely manner modifications needed to improve project performance. For instance, after - 31 - 1985 some subprograms had to be revised in scope and content because of national budgetary constraints; the timber bridge replacement, soil erosion, and acid sulfate and saline soil improvement activities were the most affected. Unfortunately, because of the problems with the M&E component, there was inadequate management information to assess project performance and benefits. In the absence of such feedback, the Bank and the Government were forced to consider target revisions on the basis of rather limited reporting and occasional field inspections. Similarly, as discussed in para. 22 below, at the completion stage it has been extremely difficult to re-estimate the ERRs of individual subprograms or otherwise document the project impact in quantitative terms. 13. Special Studies. Studies and project-related training were more successful. In all, the UNDP-assisted component, which was completed in September 1986, supported 12 studies to aid in rural development policy formulation, program administration, management, staff training and evaluation. rRDC and the Cabine* accepted and implemented the majority of the recommendations. Some study proposals for both a stronger NRDCC and increased provincial control might have been conceptually contradictory but, partly due to the inability of the centralized NRDCC staff to carry out all administrative functions, responsibilities for many activities ultimately shifted to the provincial level. Through extensive staff training (918 in local courses and 25 under international training programs) the functional capabilities of the provincial planning officials (PPOs) in planning and coordinating the implementation of rural development activities by line agencies was greatly developed. The competence of the PPos was also increased through the development and installation of the USAID-funded computer-based management information system. A study on management improvements recommended merging the numerous committees and subcommittees at provincial level into a single planning and coordinating entity under the Governor, so as to improve efficiency, avoid duplication and increase dialogue. This organizational scheme is now in place in many provinces, and the remaining provinces are expected to follow suit. Other special studies addressed public relations and the use of the media in fostering rural development programs, and people's participation in the NRDP did increase in areas where short-term tangible benefits were obvious and well-publicized. 14. Poverty, Areas Subprograms (a) Village-based Subprograms (i) Village Fisheries DeveloRment. The objective of this subprogram was to raise consumption of protein-rich fish by the construction of approximately 285 community ponds and 300 school ponds, rehabilitation of approximately 900 catchments, and the conduct of some 300 farmer training courses in fish production. Targets for building new community and school fish ponds and rehabilitating existing ponds were fully met, but the final distribution of fingerlings fell below (62%) - 32 - appraisal targets. Fish yields per ha were initially low because of limited farmer participation and the fact that villagers did not restock at recommended rates; some ponds were poorly located where seepage was a problem; and technical construction standards were relaxed because of budget constraints. It was also difficult to ascertain actual production because of out-of-season fishing and escape of fish during the rainy season. The Department of Fisheries (DOF) overcame most of these problems by greater involvement in site selection, development of more suitable pond designs, and modification of the specifications for large ponds. DOF selected 20% of the ponds for more detailed analysis and hired biologists, economists and supporting staff to provide more efficient technical and managerial support. As a result, farmer participation improved, and the survival rate of fingerlings increased to an average 80% (i.e., surpassing the appraisal estimates of 60-70%). (ii) Village Woodlots. The objective was to introduce rural villagers to a means of increasing their wood supply without resorting to deforestation, through the establishment of a"pproximately 5,500 ha of nurseries and development of approximately 35,000 woodlots. This subprogram achieved or exceeded its targets in the number of nurseries established, seedlings produced and area planted to woodlots. Some 95% of the target for training participating villagers and farm leaders was also achieved, but only 64% of the targeted nuiber of government officials were actually trained. Initial problems included difficulties in meeting the demand for seedlings, low survival rate of seedlings and poor maintenance because of supervisory staff shortages in the Land Development Dupartment (LDD). However, these problems were eventually overcome by better liaison with the vill&gers and involving staff of other projects in the implementation of this subprogram. (iii) Village Small Animal Improvement. The main objective was to increase the availability of small animals for home consumption by distributing improved breeding stock, disseminating information on poultry rearing, and preventing poultry diseases through training and vaccinations in approximately 12,000 villages. Due to low survival rates, the distribution of improved poultry breeding stock was discontinued. Efforts were concentrated instead on the more successful training and vaccination activities, with trained villagers participating in some 125 million vaccinations. (iv) Village Cattle and Buffalo Banks. This subprogram aimed to promote livestock production through the establishment of "banks" consisting of approximately 19,000 head of cattle and buffalo; by assisting farmers to obtain draft cattle; and by providing technical assistance in breeding, management and disease prevention. Farmers would obtain draft cattle or buffalo either by means of interest free loans repayable over five years, or through a traditional system under which the farmer received a breeding animal on loan during the period necessary to produce two calves. The first calf would be returned to the animal bank for subsequent - 33 - sale (if male) or loan (if female) and the mother would be returned when the second calf was one year old. The credit alternative was soon abandoned in favor of the traditional scheme. A total of 17,600 calves (92% of target) were produced by the cows distributed to farmers. Farmers retained the mother cows for a longer period than originally planned, keeping the second of the three calves and handing over the other two to the bank. The actual number of calves and cows returned is not available. There were no major problems with this component, although there we I some reports of lack of personnel and delays in purchasing animals for distribution. (v) Village Coastal Fisheries in the South. The objective was to encourage farmers to raise sea bass and cockle; approximately 20 sites were to be developed for seabass culture and three sites for cockle production, and facilities were to be established for fishermen's training. This subprogram faced problems of site selection, as well as diseases and theft. Also, farmer participation rates were low because of high operating and marketing costs, difficult access to sites, and competing demands for labor from other farming and nonfarming activities. Training was hampered by the lack of funds for teaching aids. Gradually, the orientation of this subprogram changed from a subsistence to a commercial undertaking: the total number of sites was reduced from 20 to 7 for sea bass culture and from 3 to 1 for cockle culture, while 3 sites for more profitable freshwater prawn and 2 for green mussel culture were added. (vi) Village Agricultural Water Utilization. The objective was to enable the inhabitants of 10,000 villages to better utilize existing reservoirs by encouraging them to grow protein-rich vegetables and food crops, through distribution of leguminous and vegetable seeds. Village participation rates and the total area cropped were 87% and 83%, respectively, of appraisal targets. However, reliable crop production records were not kept, and it is reported that only about 37% of participating farmers continued to the final year. The main problem seems to have been the fact that most reservoirs were small and served multiple purposes. (b) Farmer Participation Schemes (i) Food Production for Nutrition. The objective was to raise the nutritional level of rural households, particularly that of 32,000 preschool children, by promoting production of animal protein and food crops on 17,000 ha of vegetable plots. By FY86 all targeted villages were participating in the subprogram. About 14,500 ha were planted to beans, sesame and vegetables, but production of chickens reached only 53% of the appraisal target. Problems cited included poor quality and untimely procurement and distribution of seeds, unfavorable climatic conditions during the first two years, and difficulties in obtaining adequate water and timely advice for crop production. - 34 - (ii) Rainfed Rice Improvement. The main objective was to promote improved rice cultivation practices by providing advice and training through, inter alia, the use of demonstration plots, and to establish seed banks and construct about 1,900 stores of 10 ton capacity and 700 stores of 30 ton capacity. The effort to raise rice output and productivity was very successful; the target for seed plots was met; 93% of the total area targeted for rice production was covered; and training targets were met. The ex-post evaluation study reports that yields increased by about 400 kg/ha (i.e., double the appraisal target). On the other hand, only 63% of rice seed loans to farmers were returned. The original concept of establishing seed banks was modified, in that seed storage facilities were used also as village rice banks to store grain for consumption. The most frequently cited problems were late budgetary transfers, drought and poor seed quality. (iii) Upland Crop Improvement. The subprogram aimed to increase the output of upland field crops by enabling farmers to have good quality field crop seeds in reserve for planting, through training and advice in efficient cropping practices and seed selection and preservation. The subprogram concentrated on maize, sorghum, jute, peanuts and mungbeans. Problems included overlap with other subprograms, such as the one for Food Production for Nutrition. At some sites too many projects were implemented, which may have reduced farmer interest in individual activities. Also, supply of good quality seed was inadequate and planning was complicated by the fact that farmers often modified their seasonal demand for the five different crops in response to changing market conditions. (iv) Highland Rice Imirovement. The objective was to identify and breed high yielding rice varieties (HYV), by distributing approximately 300 tons of seed and promoting improved cropping practices. Despite the fact that the upland rice research substation had only a limited area for seed production, distribution of improved seeds achieved appraisal targets; the total area benefitted was about 85% of target; and farmer training 79%. (v) Fruit and Other Productive Tree Planting. The objectives were similar to those of the Village Woodlot subprogram, except that the nurseries established and trees planted would yield fruits for home consumption and sale, rather tharn wood. This subprogram was relatively problem free, and all major targets were achieved. Nurseries were established in all 286 districts and a total of 6.9 million seedlings were distributed to farmers (i.e., 18% above the appraisal target), although actual planted areas, survival rates and volume of fruit production were not reported. (vi) Livestock Health and Management Imrovement. The objectives were to improve livestock health through vaccinations and - 35 - deworming, improve public pasture, upgrade livestock quality by expanding artificial insemination services, and provide farmer training in livestock management. About 75% of the disease prevention and deworming targets were met, although pig vaccinations reached only about 55% of target because the original estimate of the number of pigs proved unrealistic. The four sub- activities of pasture development (legume seed distribution, repurchase of seed from farmers, seed production and hay-making demonstrations) attained 111%, 51%, 11% and 87% of targets, respectively. Livestock quality improvement achieved about 60% of target, and training targets were fully met. (vii) Soil Improvement through Crop Residues. The subprogram called for production of approximately 375,000 tons of compost for production of vegetables and other high value crops on sandy soils, through training and provision to farmers groups of compost starter-cultures, cover crop and green manure seed. In some areas there was a serious shortage of rice straw for compost to be used in paddy fields, but participation was good in paddy areas near forests and in nonpaddy areas. There was also a shortage of compost starter supplies, as the private sector was slow in entering the market and LDD facilities for compost starter production were inadequate; in these circumstances, farmers tended to apply less than the recommended rates of compost. Four of the seven supporting activities (training of government personnel, organization of farmer groups, seed procurement and research) achieved appraisal targets; green manure extension achieved 90% of targets; farm leader training 84%; and green manure production 34%. (viii) Saline Soil Improvement in the Northeast. The objectives were to improve production in approximately 500 ha of areas of low salinity, and to reduce the spread of salinity through research and technology development, training and legislative action. The subprogram succeeded in demonstrating techniques for growing plants in slightly saline soil through extension work in farmers' fields, promoting the use of saline-resistant rice seeds, and training government personnel and local leaders in saline-soil improvement methods. The main problems included inadequate identification of soil constraints, shortages of salt-tolerant varieties and uncertainties about the response of different varieties under varying soil conditions, the difficulty of distinguishing the effects of variety from soil treatment, and farmers' preference for sticky rice (which is less salt-tolerant than non-sticky varieties). Also, construction of drainage schemes was postponed pending findings of research being undertaken with the help of Australian Technical Assistance. (ix) Acid and Saline Soil Improvement in the South. The objective was to improve rice yields through promotion of desalinization methods, including application of approximately 90,000 tons of marl and 900 tons of rice seed by approximately 18,000 families. The achievements of this subprogram were negligible, due mainly to technical difficulties. Although farmer training exceeded targets by 5%, indicating strong wnterest - 36 - in the topic, application of marl was actually demonstrated on only 1,280 ha (v. 14,400 ha planned) and only 96 tons (v. 900 tons) of rice seed were produced. Sea dikes which were constructed to prevent salt water intrusion failed; variations in the degree and extent of acidity/salinity at specific locations were unknown; because earth-moving machinery was used, demonstrations were carried out on larger, less typical plots (15-30ha) pending completion of designs for smaller plots; treatment effects overlapped with those of composting and high fertilizer application rates, making it difficult to isolate the impact of liming; and there were problems in transporting materials to field sites and in coordination between government officers and villagers. (x) Prevention of Soil Erosion in the North. The objectives were to promote soil erosion c.'itrol measures, including contour planting, strip cropping, tree planting and earth bank construction; arid to provide approximately 800 tons of seeds of suitable plant species and farmer training. Except for cover crop production, all activities achieved more than 90% of targets: 94% of planned farms were visited by extension agents, and 90% of contour bunds and 93% of demonstration plots were established. The main difficulties had to do with accessing the sites and interesting farmers in an activity for which the benefits could not be explicitly quantified. (c) Infrastructure Subprograms (i) Bridee Construction and Replacement. This subprogram was designed to provide all-year access to poverty areas through construction and/or replacement of 11,000 meters of bridges and replacement of low-level flood-damaged and deteriorated timber bridges which were costly to maintain. Ultimately, only 6,779 meters of bridges (i.e., 62% of target) were constructed due to budget cuts beginning in 1985 and to new instructions, also in 1985, to the effect that 15% of the bridges should be (more costly) dual-purpose structures including a concrete weir under the bridge to take advantage (for water storage) of dams created by road embankments. (ii) Village Water Resource Development. This subprogram aimed to provide water supplies for domestic uoe, livestock, irrigation of vegetable crops and fishponds, by deepening existing natural ponds or constructing small reservoirs with a capacity of about 10,000 m3. Performance was satisfactory: the number of reservoirs constructed exceeded the target by 10%, and 92% of the targeted ponds were developed. The main problems encountered were the inadequacy of geological and hydrological datta for site selection and the need to develop location- specific solutions for different sites. Project Costs and Financing 15. The total project cost was estimated at appraisal to be US$157.3 million (Baht 3,616 million), with external financing of US$57.6 million - 37 - (World Bank US$50.0 million, JICA US$2.1 million, UNDP US$0.5 million and USAID US$5.0 million). Although no major changes were made in project content or timetable during implementation, US$8.5 million was cancelled on October 8, 1986, and another US$5.2 million on November 25, 1988. Final project costs were US$91.5 million (Baht 2,328.9 million - Borrower's PCR, Tables 2 and 3), or about 58% of appraisal estimates. The shortfall was due to depreciation of the Baht, reduced expenditures for some activities, and ineligibility of other expenditures (US$14.5 million) incurred before loan signing. Proiect Results 16. The essential question remaining at completion was whether the 18 selected PAP subprograms were the correct ones for overcoming the root causes of underdevelopment in the designated poverty areas. Unfortunately, lack of a fully functional M&E system from the outset makes it difficult to analyze the effect of individual subprograms, although the Borrower's PCR indicates that production increases were higher than anticipated at appraisal except in a few cases (PCR para. 5.7), and that 121 (42%) of the 286 participating districts have graduated from the poverty category (PCR para. 5.9). At the same time, it is important to note that project activities were situated where nationwide data identified groups of poor villages within districts, but little was done to differentiate among the poor and better-off members of the target groups. Hence, the more aggressive farmers and village hierarchy were probably among the major beneficiaries. Overall, success was better with the village-based subprograms than with the individual farmer programs; probably group activities were relatively more visible and therefore received more attention and support. 17. Certainly another important accomplishment of the project was to encourage and develop the philosophy and administrative procedures for bottom-up rural planning, which has now been accepted in principle and hopefully will become more effective with time. An effort was made to respond to the constraints perceived by the beneficiaries; while there was not always a perfect match between perceived needs and the subprograms available, this situation improved gradually. Also, the assignment of several activities to particular areas sometimes had a synergistic effect in generating benefits and greater public awareness of the NRDP. 18. While called an "integrated rural development" scheme, this project focussed almost exclusively on the productive subprograms of the PAP (i.e., without supporting social sector development, feeder roads, land regularization, etc.). In that sense, it was a less complicated operation than many other multi-sectoral IADPs supported by the Bank, and implementation by the line agencies was generally satisfactory. Most activities were free-standing, with limited need for coordination among the subprograms. Line agencies had to liaise principally with local political leaders and the NRDP committees, but less so with each other. On the - 38 - question of scope, the PCR does reflect the Borrower's views on the importance of also developing infrastructure, public health and education activities in conjunction with agricultural schemes, in order to effectively reduce poverty and improve nutrition and living standards. 19. Budgetary constraints limited some subprogram achievements and this was partly attributable to a weakness in the administrative arrangements for the NRDP. NRDP planning and implementation were the responsibility of the NRDCC, local governments and executing agencies, but financial controls were exercised by the Controller General's Office (CGO) and the Bureau of Budget (BOB). An inadequate understanding of the NRDP, and the financial agencies' caution in budgeting and releasing funds, delayed implementation of some activities and compromised the quality of others. Because the financial agencies were independent, it would have been important to involve them more closely in NRDP planning activities on a regular basis. A functional M&E system might also have increased confidence in the efficacy of allocating limited resources to the various subprograms. 20. Although the project was less complex than some multi-sectoral IADPs (para. 18 above), coordination still posed a major challenge, given the very broad geographic coverage of the NRDP. In this regard, the project demonstrated the shortcomings of trying to direct such a large program from the national level, particularly given the lack of a M&E system which would generate key management information. Many physical and financial reports were required and sent to the NRDCC, where they were only superficially analyzed because of sheer volume and inadequate staff resources. It soon became evident to all that planning, management and monitoring should be decentralized to the provincial level and, as district governments became more capable, to the district and subdistrict levels. To th-.s end, procedures and manuals were developed and training courses held for over i,000 provincial/district/sub-district staff, and many provincial offices have been strengthened. 21. Economic Re-evaluation. At appraisal the overall project ERR was calculated at 12.4% for 14 of the 18 subprograms (i.e., excluding the Food Production for Nutrition, Livestock Development, Soil Erosion Prevention and Village Agricultural Water Resources subprograms). Based on an ex-post evaluation study by the Faculty of Economics and Business Administration of Kasetsart University, the Borrower's PCR re-estimated the ERR for 12 sub- programs: eight had positive ERRs (including four in excess of 100%), two - 39 - had marginal ERRs, and three ERRs were low (including one negative ERR), as summarized in the Table below: Table: ECONOMIC RATES OF RETURN BY PAP SUBPROGRAM ERR (%) _ Subnrogram SAR Borrower' PCR Village-Based Subprograms 1. Village Fisheries Development 15 - 9 2. Village Woodlots 10 >100 3. Village Coastal Fisheries/South 13 11 4. Village Small Animal Improvement 13 >100 5. Village Cattle and Buffalo Banks 11 5 6. Village Ag. Water Utilization 10 - Farmer ?articioation Subprograms 7. Food Production for Nutrition - >100 8. Rainfed Rice Improvement 15 >100 9. Upland Crop Improvement 17 59 10. Highland Rice Improvement 17 41 11. Fruit Tree Development 14 21 12. Livestock Health and Management - - 13. Soil Improvement/Crop Residues 13 - 14. Saline Soil Improvement/NE 16 9 15. Acid and Saline Soils/South 10 5 16. Soil Erosion Prevention/North - Infrastructure Subprograms 17. Bridge Construction/Replacement 16 18. Village Water Resource Dev. - - Aggregate ERR 12.4 NA 22. Having reviewed the data in the ex-post evaluation study, the Region questions the usefulness of re-estimating the ERRs for this project/subprograms. The study itself states that "the collection of data regarding changes in product, production cost, income and employment was not mandatory for all subprograms, resulting in some subprograms collecting some data and some subprograms none at all. This results in missing data for the evaluation of financial and economic rate of return, impact on income and employment-..Moreover, the data...are not wholely suitable for the financial and economic evaluation of the subprograms, and data on some items are incomplete." (Ex-post study, Section 5.3). Clearly, this situation is attributable to the lack of consistent baseline data at the start of the project, and the absence of an effective M&E system throughout most of the implementation period. In these circumstances, despite the heroic attempts of the NRDCC and Kasetsart University to reconstruct - 40 - relevant data by drawing on a variety of sources, it is simply not possible to assess the impact of the project in quantitative terms with an acceptable degree of confidence. On the other hand, the re-estimated ERRs offer some indication of what were probably the more and less successful NRDP subprograms, from an economic standpoint. This information, combined with the ex-post evaluation study's analysis of unit costs and farmer continuation rates for each of the subprograms, provides valuable guidance as to those activities which may be worth continuing and/or expanding. 23. Sustainability. For the aforementioned reasons, it is also difficult to reach firm conclusions regarding sustainability of the various subprograms. In 12 of the 18 cases farmers were expected to make some in- kind repayments (e.g. for seed, planting stock, livestock, etc.). Repayments were received by the Village Development Committees (VDCs), but the extent cannot be documented due to lack of uniformity in obligations and in physical ability to receive, record and utilize the repayments. On the other hand, the Borrower's PCR raises no major issues of principle regarding the repayment obligations, presumably reflecting RTG's concurrence that some cost recovery should be attempted in appropriate cases. For the future, more adequate provision should be made for collecting and utilizing the repayments, particularly for recycling activities at the local level. 24. The RTG is continuling its rural development efforts under the Sixth Plan, although it has not requested World Bank financing because of a general reduction in external borrowing. The PAP concept is being replaced, in favor of more careful targeting of NRDP activities to low- income households, which can now be identified from the provincial data base. The emphasis will remain on economically viable activities and local level, bottom-up planning will be further strengthened. The need for an adequate M&E system is well understood and continuing efforts are being made towards its development. Bank Performance 25. The Borrower's PCR credits the Bank with making important contributions to the NRDP at the stage of project design and appraisal, * particularly with regard to its emphasis on the need to strengthen the N&E -. system and execute special studies to complement information coming through that system (PCR, para. 7.1) and on the importance of decentralizing : : planning decisions to the provincial and district levels. However, while it may have recognized the need for effective M&E for the management of such a widespread and diverse project, the Bank did little during the implementation stage to help speed up execution of M&E activities. This was partly due to the fact that the M&E and special studies components were financed by the cofinanciers rather than the Bank; however, there was also no participation of an M&E specialist in the last two (of a total of four) Bank supervision missions, even though very little progress had been made under this critical componen_. With regard to decentralization, while - 41 - conceptually sound, the Bank was overly optimistic about the rate at which this could realistically take place and did not adequately foresee the staffing and equipment inputs needed. Although the Bank's interaction with .TICA and UNDP was limited, liaison with USAID grew stronger towards the conclusion of the project. Finally, in retrospect it appears that none of the parties involved adequately anticipated at appraisal the difficulties which RTG would face in dealing with the different operating procedures of the four cofinanciers. lessons Learned 26. Bottom-up rural development planning, an important goal of this project, is evolutionary and requires considerable guidance and close implementation assistance. The assumption that such a system could be developed within three years was not realistic. Further, the failure to prepare adequately the M&E component before the start of project implementation was a serious barrier to success, as was the subsequent failure of those involved to give enough priority to overcoming quickly the bureaucratic and procedural issues which plagued this component, in the interests of achieving fundamental project objectives. 27. Beneficiaries and local authorities must be involved in a project from the outset, in order to develop a consensus about perceived constraints to rural development, productivity and family welfare. It is important that some actions show immediate results, in order to build the confidence of beneficiaries in the project. Also, pre-testing of technology to overcome identified constraints is important before full- scale implementation, since beneficiaries' disappointment or possible loss of investment are likely to have lasting consequences for acceptance of project activities. Finally, sustained popular support will be forthcoming only when promised developments are clearly and successfully demonstrated and when the techniques are both acceptable and accessible to most beneficiaries. _43 _ATTACRMEN C3~~~~~~~3 Office of the National EconoLzic (NRDcc) 0911/ ,~~~ and Social Development Board No. (NRDCC) 0911/ 962 Krung Kasem Road Bangkok 10100 June IJ , 1989 Dear Mr. Graham Donaldson: This letter is in response to your letter dated May 12, 1989 regarding the Project Performance Audit Report sending to us for comments. After reviewing through this PPAR, we found out that it has already covered all critical points concerning project implementation. As a project coordinating office, we would like to conclude that the key factor contributing to the successful of implementing such project is the close coordination among doners (iorld Bank, USAID and UNDP) which seem to be lack in this project. This should be initiated and take action by brld Bank at the begining through the end of project life, but in fact there were only four missions at an average of nine-month interval to supervise during project implementation. However, in our point of view, the project performance was satisfactory in term of assistance provided to run-up the project, to strengthen the M & E activities and to build up the strong management system for rural development in Thailand Yours Sincerely, Sumet Tantivejkul Deputy-Secretary General NESDB Mr. Graham Donaldson Chief Agriculture, Infrastruclture and Human Resourees Division Operations Evaluation Department World Bank 1818 H. Street, N.W. Washington D.C. 20433 U*S.A. T HA IL AN D rw ~~NATIONAL RURAL DEVELOPMENT PROJECT B U R M A e - Poverty Areas (by district) -20 O *- District Boundaries (poverty areas only) 20'- r A 3 ,%\y --- ~~~~~~~~~~~~~Changweot (Province) Boundaries f Fi 91 R ,< t 4w w. ~~~~~~~~~~~Regional Boundaries Ab uH -gns Paa . International Boundaries \ | - - Ną Tl->\ \J n < , q > LAO PEOPLE'S ,,, AS; -~~~~~~~~~~~~~~~~~~~C DEMOCRATICTL: s !19 ~ ~~~~~~ '.- f Xs H - S--N 4, S -I ' l '--" %'V'.-j; tgJetS /X^t'j-o--C-', A~~~0L- 7_ _16e i < vS Sa \ I f { 9 rA v pq /vSlthotA . it J- ._Th|,,ic, LJf / 8 Ot joAV"t?t; r->n z n :L-L.X- t-~~~~~~~~' ~ hi( tpr,xt;J phi Blf ; 1 PO° wbb 8NkO _4 > tR- z ,*okt _ j p C ^-rn,sl) > C OVRYAES YRGO - , >,,,/ @\NOTHESTPholC5DmCSsBihkR0T 16-I ' \ tttBr (NRHI 0r \ SC E N T R A L-; z oR .}ohriJ -TOTAL- 36 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 242 ^4~~~~7 C -, t>~~~> -- -^f ~~- - -.^ o S)~ Rs ' . . ,^ o,tL t) '.' _}' t ' .. .;W 12' : *l;rg KhAd Khtn _ \ 12' S - . ./ KAMPUCHEA, Su ._ >~~~~~~~~~~~~~~~~~~~~OET ARA BY hsREGhION rbr , \ . CbOn trs~~~~C, 0 ~~ ~ Orl h5 rehD rS rtr irp.ty. 0 lV P~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~t -W ftf b.- PO'D t A V5 Chp: ,,eEw - o1 hW. -. -a-.* -' -W{ A P.,Fj" ' _. 0 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~~~~~~~-.h Th. 200 30 ._ Kiloteelen ) > t M A L A r S I A ffi~~~~~Sh 0~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~t 6 00(>r* eSnd;R III 0~ ~ f°°MALAY Ow".10'St I A. ,< s