ICRR 12620 Report Number : ICRR12620 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/06/2007 PROJ ID : P040085 Appraisal Actual Project Name : Participatory Rural US$M ): Project Costs (US$M): 87.3 83.8 Investment Project Country : Bolivia Loan /Credit (US$M): Loan/ US$M ): 62.8 58.2 Sector Board : RDV Cofinancing (US$M ): US$M): 2 grants, 2.2 respectively: 0.9 &1.5 Sector (s): Sub-national government administration (25%) Irrigation and drainage (20%) General transportation sector (20%) Other industry (20%) Other social services (15%) Theme (s): Rural non-farm income generation (40% - P) Indigenous peoples (20% - S) Gender (20% - S) Participation and civic engagement (20% - S) L/C Number : C3065 Board Approval Date : 05/12/1998 Partners involved : Swiss Development Closing Date : 06/30/2004 06/30/2006 Corporation Evaluator : Panel Reviewer : Group Manager : Group : Keith Robert A. Oblitas Kris Hallberg Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: Assist the Borrower to promote economic development of rural communities and eligible municipalities through sustainable productive investments based on local demand which is generated through a participatory planning process. (The intended target group of the project was the population of the 200 poorest rural municipalities.) b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): 1. Rural Investments (Planned costs (base costs): $59.4 million; Actual costs $54.1 million): investments based on demand-led and participatory processes in : (i) improvement of access roads and construction and rehabilitation of bridges; (ii) community small-scale irrigation; (iii) markets and produce-storage facilities; (iv) river embankment improvements; (v) natural resources management; (vi) technical assistance to rural producers; and (vii) cultural heritage rehabilitation. 2. Institutional Strengthening (Planned costs: $20.0 million; Actual costs: $29.7 million): through: (i) technical assistance to municipalities, indigenous districts, rural communities and NGOs in municipal management, participatory planning and implementation, sub -project planning, preparation, implementation and maintenance; and sub-project financial management and administration; environmental management and municipal revenue generation; (ii) formal training for staff of municipalities and indigenous districts in municipal management, and non-formal training of municipal councils, vigilance committees and community leaders; and (iii) consultants, office equipment, vehicles and incremental operating costs for the Vice -Ministry of Popular Participation and Municipal Strengthening, the Vice-Ministry of Rural Development and the Small Farmer Development Fund (FDC) (later replaced by the National Fund for Productive and Social Investment (FPS). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: A major issue, was the Bank's identification in December 2000 of corruption connected with some of the FDC sub-projects. The Bank's Integrity Unit (INTIU) was brought in to investigate and sub -project approvals were suspended. In December 2001 all disbursements were suspended, and remained so till August 2003. The Government refunded the Bank $ 2.8 million, and $3.0 million of the Credit was cancelled. Project implementation effectively halted during this 20 month period. The Credit was amended (without going to the Board) in August 2003. The amendment comprised: (i) handover of project implemention responsibility to a new agency - the National Fund for Productive and Social Investment (FPS); (ii) an extension of the Credit Period by 2 years (to June 30, 2006) to make up for the hiatus in project implemention caused by the suspension of the Credit; and (iii) reallocation of $3 million from the Rural Investment Component to the Institutional Strengthening Component . The reallocation was to support an expansion of the project's institutional strengthening activities to include greater support to sound fiduciary management by the municipalities and to enhance monitoring by FPS . Counterpart funding was planned to be $ 23.5 million at appraisal, almost entirely from local governments ($13.0 million) and local communities ($9.7 million). There was a slight reduction in community contributions (from $9.7 million to $7.7 million) in project implementation, compensated by equivalent additions from government . The overall counterpart funding remained unchanged at $ 23.5 million, 28 percent of actual project costs . The Swiss government financed part of the Institutional component through 2 grants: $0.9 million provided at appraisal and an additional $ 1.5 million allocation in 2000. Total disbursements from the grants amounted to $2.2 million. The appraisal estimates of the costs of the components that are cited in section 6.5 of the ICR include contingencies, whereas Annex 3 presents base costs with separate contingencies . 3. Relevance of Objectives & Design: Relevance of Project Objective: The project's objective has substantial relevance to Bolivia's needs; low economic growth and high levels of poverty need addressing . The latest CAS (January 2004) comments on Bolivia's high poverty levels as well as the substantial inequality between the middle class and the poor, particularly in the impoverished rural areas and for the country's indigenous peoples . Enhancing economic growth, especially where growth benefits the poor, is seen as a key need by both the Bank and the Government . Relevance of Project Design: Project design substantially supported the project's objective . Investing in productive rural infrastructure, using a community based approach and devolving government support to be as close as possible to field implementation was a good approach . Investments chosen by communities and implemented with substantial community involvement would likely have been the only practical way of assuring investment suited to their needs. The project's emphasis on strengthening municipal governments was an important project feature . It provided the necessary direct exposure to community needs and the capacity for implementing the project . Additionally, municipal governance would benefit generally from the project's institutional strengthening component . However, the choice of FDC as project coordination agency, despite its poor track -record, was a weak design feature with major consequences (section 8); and fiduciary management arrangements were seriously inadequate (Sections 2d and 11). 4. Achievement of Objectives (Efficacy): The project's objective to promote economic development of rural communities and municipalities had several contributing facets: (i) the productivity of rural investments; (ii) the sustainability of these investments; and (iii) the use of demand-led participatory processes to achieve the investments . Achievement of the project Objective can be examined using these three facets (this procedure was also used in the ICR ). (i) promotion of productive investments : Achievement was overall Substantial . Although the number of municipalities involved with the project was 156 compared with an appaisal target of 200, the overall physical investment could be considered more, primarily because a new form of "spot-improvements" for rural roads, the project's largest component, enabled road rehabilitation to increase from the appraisal estimated 1480 km to 2372 km. In the project period, productive investments by the municipalities increased by 7.7 percent annually, compared with the appraisal target of 5 percent. The productivity of the investments also increased; some 91 percent of the sub-projects are assessed in the ICR to have had positive net present values . By comparison, the ICR estimates that only a half of rural investments achieved positive returns prior to the project, and the appraisal target for productivity improvement was 80 percent of all sub-projects. (One indicator which is likely to have not been met, however, is the PAD's target for municipalities to reach economic growth rates of 5 percent per annum - as discussed in section 10, this target was unrealistic, and, in terms of the impact on growth of the project alone, not measurable .) (ii) Sustainability of Investments : This was the weakest aspect of the project, with only Modest achievement. Village level investments may be sustainable because of the direct interest of the villagers . However, rural roads, which were 67 percent of the project's overall investment costs, may deteriorate as benefits are indirect and geographically dispersed. Formal mechanisms for O&M of roads, including clear responsibilities, were not established under the project . (iii) Demand-led, participatory processes : Achievement was Substantial. The ICR reports that all investments used a participatory and demand -led process in selection and implementation of the sub -projects. The capacity of the municipalities to promote participatory processes was improved overall, with benefits going beyond the project itself . The Overall Efficacy of the project is assessed as Substantial. This takes into account the resilience of the project's ERR to O&M shortfalls. Thus, while facet (ii) was only partially achieved, shortfalls in O&M could still yield a positive ERR (Section 5). While not stated as part of the project's development objective, and, hence, not included as part of the assessment of efficacy, another important aspect of the project was institutional strengthening , The positive achievements here bear mentioning. The project reached 246 poor rural municipalities (76 percent of all municipalities in Bolivia ) and most numerical targets for training were reached or exceeded . Government departments were also strengthened and communities received training relevant to the concerned investment activity . The bottom line is that sufficient human capacity was created to implement the investment program . The ICR also refers to spin-off benefits for municipalities in the general strengthening of their planning and financial management capabilities . 5. Efficiency (not applicable to DPLs): The costs of the project's institutional component were not included in the ERR calculation . As a project starting a new participatory approach to rural investment, institutional costs (35 percent of project costs) can be expected to be relatively high . Over time, such overheads would reduce . Inclusion in the ICR of overhead costs, at least the estimated costs once the program is familiar to municipalities and communities, would have been desirable . Based on aggregation of models of typical investments, a 29 percent ERR is estimated in the ICR. The higher ERR estimated in the PAD is primarily due to a difference in analysis methodology . Noteworthy is the high resilience of the project ERR to inadequate maintenance . Thus, if investments become completely non-functional in 10 years due to poor maintenance, the ERR remains substantially the same as the 20 year model used in the ICR. Even if only a 5 year life for investments is assumed, the ERR (12 percent) would still be (very marginally) viable. Despite the generally positive indications above, efficiency could have been better . The corruption issue resulted in a two-year delay in project implementation, and needed considerable effort by the Bank and government to resolve. And there would have been extra costs associated with this and with establishing a new implementing agency. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 37% 75% ICR estimate Yes 29% 65% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The outcome of the project was satisfactory as concerns the project's relevance and achievement of objectives . But the major corruption in the earlier part of the project was highly unsatisfactory, influencing the assessment of efficiency and the project overall . Nevertheless, the project ended with a new agency and better fiduciary monitoring . This enables a moderately positive outcome rating, but with evident strong qualification regarding the corruption issue. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The main risk is the possibility of rapid deterioration of infrastructure due to lack of maintenance . O&M would likely be more assured for infrastructure directly benefitting a village . However, O&M of such infrastructure over the medium and long-term is only peripherally discussed in the ICR; yet there may be need for continuing technical assistance support (minimum level) to the communities. Assessing whether this is needed would be desirable . The adequacy of funding for O&M of more general infrastructure, especially roads, may be the most risky, although there would be political pressures on municipalities to maintain roads . Risk can be reduced by establishing a more systematic planning and management process for infrastructure . The Moderate assessment of Risk to Development Outcome is because only extremely rapid deterioration of infrastructure would result in a negative Net Present Value (Section 5). Notwithstanding, keeping infrastructure in good order is clearly a much better outcome, and placing emphasis on O&M in designing for future rural investments would have important development pay -off. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Ensuring Quality at Entry: In most respects design and detailed preparation of the project's implementation features were good. There were two aspects which could have been better : (i) The project's Development Objective was too broad and the associated monitorable indicator (section 10) was overambitious. (ii) The M&E design concept was broadly satisfactory, but preparation lacked detail and M&E was far from ready to be implemented at project effectiveness . The critical shortfall was in the preparation and appraisal of project fiduciary management . First, the choice of FDC for project coordination is questionable given its history of corruption and political interference (though the ICR comments that FDC management had been improving ). Second, especially given FDC's record, a clear and ready for implementation fiduciary management system with strong internal controls and external oversight would have been highly desirable. (The ICR expresses these points as follows :" The institutional assessment of FDC and the mitigationg actions taken with respect to the history of corruption and political interference of the FDC were not sufficient") Quality of Supervision: Supervision was strong. In particular, major and ultimately successful effort was devoted to rectify the fiduciary management weaknesses . Supervision of ground implementation was good, leading to achievement of most project targets . In summary, Bank performance was strong in most aspects; but weak design of fiduciary management results in a less than satisfactory rating for quality at entry and a moderately rather than fully satisfactory Bank performance overall. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Government Performance: In most core areas Government was effective : Commitment and ownership was strong, counterpart contributions were made, the participatory approach was personally led by the Vice Minister of Popular Participation, and, despite the unstable political environment, the project's core staff were maintained throughout implementation. However, the critical fiduciary issue was inadequately handled - proper controls were not adopted, and when the issue was found, redressment actions were slow . Implementing Agency Performance: Highly Unsatisfactory can be the only rating for the disastrous FDC performance. Neverthless, FDC was closed down and the subsequent project coordinator, FPS, performed well, as did the ministries of decentralization and participation . The project ended well, with rapid implementation in the last two project years. Overall, taking account of the corruption issue and initially slow response in dealing with it, an otherwise satisfactory performance is rated Moderately Unsatisfactory . a. Government Performance :Moderately Unsatisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: M&E Design: The M&E system envisaged at appraisal was to continue with the already ongoing monitoring system established in the preceeding project - the Rural Communities Development Project, which primarily monitored physical and financial project implementation; and to supplement this by intensive surveying of a sample of municipalities, concentrating on measuring project outcome . In actuality, there was little in the way of project outcome indicators in the PAD, and the only significant indicator was flawed; it was for an economic growth of the municipalities of 5 percent annually. The ICR appropriately comments that this indicator is impractical . First, the project's investments are a small part of overall municipal budgets; second, it is difficult to assess the specific contribution of the project where a large number of other influences are impacting on growth; and, third, it appears from the ICR that the municipalities did not have such data in the first place . M&E Implementation: Little progress was made during project implementation . The second system only got underway in the second year of the project . It then stopped after government and the Bank had decided on having only one municipal information system for the entire country and for all development programs . M&E Utilization: The physical and financial monitoring system was a useful management tool, but primarily a management information system rather than for M&E . Data measuring outcome was very scarce . Some consultancy surveys were used at the end of the project to gather outcome data . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project's major issue was the finding of fraudulent procurement and disbursement practices involving the initial project coordination agency and various private and other parties (Section 2d). FDC had major deficiencies in fiduciary management and control . These included deficient accounting records, inability to prepare financial statements, and deficient internal control mechanisms . Remedial fiduciary management actions were agreed and put into place as part of the Credit Amendment agreement, and a new agency - FPS - was created to take over project management from FDC. The ICR indicates that considerable Bank resources and supervision attention was provided to establish and capacitate the new financial management system . Concerning environmental aspects, there were short -lived compliance problems with respect to the Environmental Management Guidelines. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Moderately A mostly satisfactory Bank Satisfactory performance, especially during supervision, is qualified by inadequate preparation and appraisal of the project's fiduciary management arrangements (Section 8). Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The project illustrates several familiar findings from projects implemented by multiple local governments and communities: that demand driven, participatory and community managed approaches can be successful relative to top-down planning and implementation; that training and capacity building for local governments and communities is a major need (in this project, one-third of project costs), and that a systematic M&E system measuring outcomes as well as inputs is needed. Specifically, the project experience yields two major lessons : 1. A strong financial management system, including internal and external fiduciary oversight, is a particularly critical need for a decentralized community development program : Such a system was not developed during project preparation, and the initial implementing agency had a poor fiduciary management record . Inadequate financial management and controls provided a fertile medium for fiduciary irregularities until financial management was revamped. Development at the outset of a strong financial management and oversight system should be a central part of project preparation . 2. An operation and maintenance strategy, with accompanying detailed implementation features, needs to be a formal part of a local government and community implemented investment program : There were no formal arrangements for O&M by municipalities of larger infrastructure such as roads . The ICR comments on the need for a special account for O&M in municipal budgets . Additionally, a specific O&M plan for each municipality would have been helpful. At the level of communities, preparing a simple O&M plan with expected activities, costs and revenue sources included, should be part of sub -project preparation and be openly discussed and agreed to by the community. 14. Assessment Recommended? Yes No Why? To contribute lessons relevant to the county's and the Bank's intended long -term program for participatory rural development in Bolivia, including the follow -on to this project. 15. Comments on Quality of ICR: A generally strong report. The report is informative, thoughtful in its discussion, and candid in its evaluation (eg. the assessment of Bank performance ). The economic analysis is effectively used to discuss risks to development outcome. Given their importance, two areas might have been further developed to increase the learning value of the ICR: (i) the detailed actions taken, and any further actions desirable, to provide adequate fiduciary management; and (ii) similar elaboration of proposed detailed actions, for both municipalities and communities, to help ensure adequate O&M. a.Quality of ICR Rating : Satisfactory