ICRR 12991 Report Number : ICRR12991 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/26/2008 PROJ ID : P042237 Appraisal Actual Project Name : Provincial US$M ): Project Costs (US$M): 31.1 30.2 Infrastructure Project Country : Laos Loan/ US$M ): Loan /Credit (US$M): 27.8 27.9 Sector Board : TR US$M): Cofinancing (US$M ): Sector (s): Roads and highways (58%) Water supply (21%) Sub-national government administration (16%) Aviation (4%) Ports waterways and shipping (1%) Theme (s): Rural services and infrastructure (100% - P) L/C Number : C3131 Board Approval Date : 09/22/1998 Partners involved : Closing Date : 11/30/2006 06/30/2007 Evaluator : Panel Reviewer : Group Manager : Group : Roy Gilbert John R. Heath Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: To assist the Borrower strengthen the local institutional capacity and rehabilitate and upgrade critical basic infrastructure of its remote northern Provinces of Oudomxay and Phongsaly, so as to reduce the poverty conditions and improve the standard of living and socioeconomic potential of the poor inhabitants of these provinces and gradually integrate them into the Borrower's national economy . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Part A. Province of Oudomxay 1. Transport Access: incl. rehabilitation of 191 kms of roads, including a program of road emergency spot improvements (appraisal cost US$10.0m.; final cost per province not reported ) 2. Township Water Supply in Muang Xai : incl. upgrading, installation of a water treatment plant and extension of local utility office. (appraisal cost US$2.0m.; final cost per province not reported ) 3. Rural Water Supply: incl. provision of latrine, sanitation and washing facilities by local community groups . (appraisal cost not reported; final cost per province not reported ). 4. Institutional Strengthening: incl. provincial government capacity to budget, manage, implement, monitor and evaluate provincial investments, through consultants' services, workshops, scholarships and training . (appraisal cost US$0.3m.; final cost per province not reported ) Part B. Province of Phongsaly 1. Transport Access: incl. rehabilitation of 88 kms of roads, including the airfield access road and a river crossing approach on the provincial border, as well as a program of road emergency spot improvements . (appraisal cost US$6.4m.; final cost per province not reported ) 2. Township Water Supply in Muang Khoua : incl. upgrading, installation of a water treatment plant and extension of local utility office. (appraisal cost 0.8m.; final cost per province not reported ) 3. Rural Water Supply: incl. provision of latrine, sanitation and washing facilities by local community groups . (appraisal cost not reported; final cost per province not reported ). 4. Institutional Strengthening: incl. provincial government capacity to budget, manage, implement, monitor and evaluate provincial investments, through consultants' services, workshops, scholarships and training (appraisal cost US$0.4m.; final cost per province not reported ) d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The ICR provided the following consolidated cost figures per component, not detailed by province, as follows : Component : US$m ) Appraisal cost (US$m) US$m .) Actual cost (US$m.) I. Transport Access 19.26 23.47 II. Township Water Supply 3.31 3.57 III. Rural Water Supply 2.44 1.94 IV. Institutional Strengthening 0.74 1.19 contingencies: 5.35 - Total . 31 .10 31. 30. 30 .18 At completion, overall project costs were similar to those estimated at appraisal, as they were for each major component. The consolidated cost data in the ICR does not allow an assessment of the cost performance per province, however. The project closing date was extended twice but only for a total of seven months . The first extension (of four months) was to allow the completion of several components, and the second extension (of three) months was to give the provinces time to finalize their ICRs . 3. Relevance of Objectives & Design: The project objective formulation is complex . Although consolidated into a single statement, IEG identified four distinct objectives for evaluation, namely : (i) strengthening local institutional capacity; (ii) upgrading critical infrastructure; (iii) reducing poverty conditions and improving the standard of living; and (iv) gradually integrating the two provinces into the national economy . The project objective formulation infers that (i) and (ii) are the intended direct results of the project interventions, while (iii) and (iv) are the expected results from achieving (i) and (ii). Within this framework, IEG concludes that the overall project objective is substantially relevant to current country and Bank priorities as summarized in three of the four objectives of the current (2005) Country Assistance Strategy (CAS), particularly its aims of regional integration, strengthening service delivery better targeted to the poor, strengthening provincial capacities . Overall, the design of the project was substantially relevant for achieving the stated objectives . On the positive side, concentrating upon just three subsectors helped keep the operation simple . The priority focus upon roads (that accounted for two-thirds of the total project costs ) made sense for two remote northern provinces struggled to retain linkages with the rest of the country as well as access to their own hinterlands . Also positive was the participatory approach to involving local communities brought a demand driven element into the project design . One weakness of the design was the limited control over the quantities and unit costs of sub -projects, leading to large overruns in some cases. Another was that there had not been a realistic assessment of the capacity of the provinces themselves to manage the project, resulting in unduly high expectations of what the provinces could be expected to contribute . 4. Achievement of Objectives (Efficacy): - first line objectives: (i) strengthening local institutional capacity; substantial : Altogether, 1,930 staff from provincial government departments were trained under the project, ten times the target . Most of these staff could bring their newly acquired knowledge to bear directly upon the implementation of the project . As a result, both provinces' transportation departments especially developed a capacity for effective project management that they did not have before this operation. There was progress too with the water departments, but marked deficiencies, especially in project implementation oversight and M&E. Both provinces now have a core of competent staff capable of managing externally funded projects with less reliance upon central government than before . (ii) upgrading critical infrastructure . modest : The rehabilitation of two national road sections --that led to 70% time savings in one case--had a significant impact in improving access to the region, including that by traders and tourists from China and Vietnam. But the damage to roads by overloaded trucks and the lack of maintenance reported by the ICR is likely to undercut the benefits of this component . The poor design of project interventions to upgrade the water supply of Muang Khoua led to the failure of the project water supply system that was able to provide water for only two hours or less per day, and the water tariffs failed to cover operation and maintenance (O&M). The rural water supply and sanitation component had mixed results, reaching less than percent of its targeted delivery to primary schools and health clinics in both provinces, but the overall number of people served exceeded the target by 38 percent for water and 38 percent for sanitation). The Bouan Neua airport access improvement yielded positive economic benefits, but only insofar as the road from Pak Nam Noi to Ban Yo remained unimproved . - second line objectives: (iii) reducing poverty: modest : The ICR notes the reduction of time poor villages need to collect water . It also mentions improved health outcomes and reduced medical expenditures, but provides no data or evidence of this . The beneficiary survey showing 70% of Muang Xay's and 33% of Muang Nampapa's respondents attributing their increased incomes to the project's Township Water Supply components are not convincing . It is not clear how many beneficiaries were interviewed, nor over what period the increases of income occurred . It also appears that beneficiaries surveyed may have been asked a leading question about their incomes, requiring them to simply agree or not with the proposition that the project had indeed increased their income . A more rigorous evaluation could have asked in the first instance if a respondent's income had changed (up or down), and in a second instance asked what, in their opinion, caused their income to change . (iv) integrate the two provinces into the national economy modest : Although not systematically and with little hard data, the ICR does provide some evidence that the project helped bring the provinces' isolation from the national (and international) economy to an end. 5. Efficiency (not applicable to DPLs): The ERR reported in the ICR probably overstates the benefits from the roads components . The cost benefit analysis leading to this estimate assumes a 15 year useful life of the roads, an unlikely scenario given the lack of funds for maintenance and the damage to the pavement caused by overloaded trucks . (The figures given below refer to road components only . ICR did not provide ERRs for other components . Appraisal ERR is a simple average of eight ERRs in the 14.4% - 62.3% range estimated for eight separate sub -projects. The weighted mean cannot be estimated, as the PAD does not provide the costs for each sub -project for this analysis.) ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal Yes 33% 58% ICR estimate Yes 24% 73% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: There is evidence that the project achieved its relevant objectives, but with moderate shortcomings . The greatest achievement was in strengthening the local institutional capacity of the two provinces . Since this was from a very low base, there is still much more to do . Some infrastructure was upgraded, although short of target in urban water supply: Muang Xai 19,330 actual beneficiaries versus 22,000 targeted and Muang Khoua 2,376 actual beneficiaries versus 7,000 targeted. Rural water supply, apart from that specifically intended for schools and clinics, on the other hand performed better than expected; 71,000 actual beneficiaries in 193 villages versus 42,000 targeted in 165 villages. Whether the upgrading will be sustainable is in serious doubt for the lack of operations and maintenance and the damage to the road pavement by overloaded trucks . The project did help integrate the provinces into the wider economy, according to ICR reports of traffic in the region, but the scale and scope of this integration is not well documented. For that the evaluation would require baseline data on the degree of integration prior to the project (measured by traffic flows, investments and trade ) that is not available. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Current levels of funding are inadequate to pay for the operation and maintenance of the road infrastructure provided through the project . Moreover, the roads are subject to use by heavily overloaded trucks with axle weights that exceed the design specifications of the road foundations and pavement . Muang Khoua water supply tariffs have remained unchanged since 2001, resulting in persistent deficits for the local utility . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: While thorough in the technical aspects, project preparation did not make a realistic assessment of local capabilities of the two provinces to effectively implement the project . The provinces' weaknesses soon became apparent during implementation, resulting in delays and shortfalls of project achievements . Also, the Bank proceeded with project preparation, estimating costs and quantities by rapid field assessments without more accurate estimates that preliminary engineering designs could have provided . Supervision was regular and benefited from inputs from nearby Bank country offices in Vientiane, Bangkok and Hanoi . While supervision dealt adequately with most implementation issues, it was not able to give the attention due to water supply issues for the lack of an appropriate specialist on the Bank side . Nevertheless, QAG rated supervision as satisfactory . at -Entry :Moderately Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: The central government's role in the project was only limited as it had effectively decentralized responsibilities for project implementation to the provinces . The national government and the two provincial governments had no experience of an externally funded project of this kind . Their decision making was sometimes cumbersome and their financial management was not always in well qualified hands . Nevertheless, both provinces carried out procurement effectively and kept adequate project accounts that were audited and found to be satisfactory by the Bank. On the other hand the project's administrative costs were 61% higher than foreseen, because of the lack of local expertise, according to the ICR . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: The ICR does not explicitly assess the performance of the design, implementation and utilization aspects of M&E . Nonetheless, IEG notes that the M&E design was weak . Baseline data for the number of schools and clinics was not estimated accurately. It reported much higher levels of service than were actually the case . Provincial water departments were particularly weak in implementing their M&E . There is no evidence in the ICR that provincial utilities systematically utilized M&E of project implementation and progress toward meeting its objectives . a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Shortcomings in upgrading Satisfactory infrastructure particularly water supply . ERRs of road infrastructure are likely to be lower in view of likely short life of road investments resulting from limited O&M and truck axle weights exceeding the design standards of the pavement . Risk to Development Significant High Lack of operations and maintenance Outcome : and excessive axle weights of overloaded trucks increase the risk of curtailing the useful life of the road investments. Bank Performance : Satisfactory Moderately Design shortcomings especially Satisfactory through inadequate assessment of the weak capacity of the provinces to implement the project, and basing overall project design estimates on inadequate cost and quantities information in the absence of the necessary detailed designs of sub-projects. Borrower Performance : Satisfactory Moderately Lack of prior experience with the Bank Satisfactory and weak in-house capacity led to delays. Provinces' decision making was cumbersome and financial management weak, resulting in overruns of project administrative costs. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Key lessons from the ICR included : Project funding for components designed to meet the project objectives (including poverty reduction ) should be allocated among local provinces according to performance -based criteria. Especially when local capacity for project preparation is weak, the design of key components should be well advanced prior to project start -up if not appraisal. Consideration has to be given early --within the same project, or a follow-on operation--to solutions to the increased production of waste water that inevitably follows the expansion of water production and distribution . In addition to providing road (or any other) infrastructure, there have to be appropriate measures to prevent its use by vehicles whose weight exceeds the design limit of the pavement . IEG adds: In designing road infrastructure, it is important to take into account the likely demand . This would include projected volumes of traffic and vehicle types . If the latter will include heavy trucks and buses, then the standards of the design should be able to meet that demand . Especially in remote areas of countries that have had no experience of working with the Bank, a thorough appraisal of the local institutional capacity should be part of project preparation, leading to proposals within the project itself to remedy shortcomings . Inadequate financial management and weak local planning are key shortcomings that need to be addressed . 14. Assessment Recommended? Yes No Why? To review performance ratings discussed by the Region . 15. Comments on Quality of ICR: IEG rates the report, that includes a good Borrower ICR, as satisfactory overall, despite having twice the length recommended by the guidelines. The Bank ICR is informative about project implementation and candid about shortcomings and risks faced by the project, but the self evaluation could have been strengthened in a number of respects. Instead of evaluating actual results and identifying the causes of weak outcomes, the ICR reads in parts more like an appraisal document proposing new solutions and remedies (leaving the reader to wonder why they were not tried out by the project under review ). ICR reporting of project costs would have been more complete had it given a breakdown of final costs per province as the PAD had done in accordance with a project design with clearly separate parts for each province . This data is essential for an assessment of the relative efficiency in the performance of each province, and also necessary to support the ICR lesson that provincial access to project financing should be on a competitive basis . The lack of detailed cost data also does not enable the ICR to explain how the scale of actual project training exceeded planned training ten -fold. The ICR also provides little precise information about the design, implementation and utilization of the project's M&E . Aspects of the self-evaluation would have been strengthened by comparing actual outcomes with intended outcomes (e.g. airport traffic and Muang Khoua urban water supply), and providing an ERR estimate for water supply at completion . a.Quality of ICR Rating : Satisfactory