ICRR 12985 Report Number : ICRR12985 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 10/24/2008 PROJ ID : P045182 Appraisal Actual Project Name : Rural Water Supply US$M ): Project Costs (US$M): 21.42 23.54 And Sanitation Project Country : Rwanda Loan/ Loan /Credit (US$M ): US$M): 20 22.45 Sector Board : WAT US$M ): Cofinancing (US$M): 0 0 Sector (s): Water supply (62%) Sanitation (38%) Theme (s): Rural services and infrastructure (34% - P) Access to urban services and housing (33% - P) Water resource management (33% - P) L/C Number : C3368 Board Approval Date : 06/06/2000 Partners involved : Closing Date : 12/31/2006 12/31/2007 Evaluator : Panel Reviewer : Group Manager : Group : Emily O'Sullivan Roy Gilbert Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: The objectives of the project were to assist the Borrower in : (i) Increasing the availability and sustainability of water supply and sanitation (WSS) services in rural areas; (ii) Strengthening the capacity of (a) communities to plan water supply and sanitation investments and manage water supply and sanitation services and (b) agencies in charge of water supply services, communes, water users and the private sector to carry out their respective responsibilities under the Borrower's water sector strategy; and, (iii) Mobilizing community support for (a) the rehabilitation or expansion of the major regional water systems, and (b) their operation by the private sector . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project had four components : (A) Community Subprojects Grants (Appraisal- US$10.43 million; Actual- US$14.1 million): Grants to communities and schools through their Community Development Committees (CDCs) for construction of water and sanitation facilities; finance technical assistance and community development activities to strengthen community capacity to plan, implement, operate and maintain water and sanitation facilities in an effective and sustainable manner. Using a demand-driven approach, this component was to start in four communes and then extend to 12 other communes. Eligible physical works included : protected springs; spring catchments (gravity or pumped) with distribution network, storage and standpipes (house connection would be paid for by users ); rehabilitation of small piped systems with standpipes; expansion of distribution networks connected to regional trunk systems; boreholes equipped with handpumps; rainwater catchment; household, school, health center and, where feasible, markets and high density rural settlement latrines; and, drainage and wastewater collection . (B) Major Water Supply Systems Component (Appraisal- US$3.40 million; Actual- US$5.08 million): Restoration and expansion of water services in the area supplied by the Mayaga system; provide support for the rehabilitation of the Lava region systems through technical assessments and community development services . This component was designed (i) to demonstrate that the decentralized and demand -responsive approach is suitable for rehabilitating major systems; and (ii) to establish sustainable management arrangements for those systems. (C) Capacity Building Component (Appraisal- US$2.79 million; Actual- US$1.90 million): Support to the providers of goods and services through training and simplification of procedures to encourage participation of small-scale entrepreneurs; support Community Development Committee (CDC) strengthening through (i) orientation courses for CDC personnel; (ii) training for the Communal Water Supply committee; (iii) training for Water Users' Associations; and (iv) logistical and operational support . (D) Project Management (Appraisal- US$1.35 million; Actual- US$1.78): Incremental costs of managing the project and providing technical assistance to the CDCs . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Dates: The closing date was extended by one year to December 31, 2007 in order to allow more time to complete the rehabilitation and extension of a water supply system (the Mayaga system). Financing: Allocation of funds was modified for the purchase of equipment and works for the Mayaga system as the actual cost of rehabilitation turned out to be higher than expected at appraisal (actual: US$4.55 million; appraisal: US$3.5 million). Borrower Contribution: Appraisal- US$0.49 million; Actual- US$0.48 million. In addition, beneficiary communities financed US$0.60 million (actual), a lower figure than proposed at appraisal (US$0.93 million). 3. Relevance of Objectives & Design: Relevance of Objectives: Substantial . In the wake of the 1994 genocide, water supply and sanitation coverage declined in rural areas with only 44% of the rural population having access to a water delivery point serving less than 200 users at the time of project appraisal . The project objectives are still relevant to the latest CAS (2008). The CAS is framed around two themes: first, promotion of economic transformation and growth; and second, reduction of social vulnerability. The foci of the first theme include access and quality of infrastructure services, agricultural production and capacity to manage public resources, all of which are relevant to the project objectives . The second theme deals with reducing child and maternal mortality rates for which improved water supply and sanitation services are critical. Relevance of Design: Substantial . Design was relevant to project objectives . The outcome indicators served as good measures of project success . Benefits were clearly linked to project activities . The decentralized design of the project is in line with the Government's decentralized, participatory approach to development embarked upon in 1998. The Community Development Committees (CDCs) were elected in 1999 and provided with administrative and financial autonomy. The Bank assisted the Government in financing the development of an institutional and financial framework for community level development activities in order to ensure their functioning . The project design follows the new rural water supply and sanitation (RWSS) strategy that involves decentralized planning and management via water users' associations (WUAs) as decision-makers, owners and managers of their facilities . The CDCs work with the WUAs to help improve services . Much of the project design focuses on assisting these groups in transitioning from the former centralized framework to a demand -responsive approach. 4. Achievement of Objectives (Efficacy): Objective 1: Increasing the availability and sustainability of water supply and sanitation (WSS) services in rural areas. Substantial . The performance indicators for this objective were largely exceeded such that 163% of the PAD target was provided access to water services (408,000 people, excluding those covered by the Mayaga water system ). In spite of significantly exceeding the target, the low EIRR for this component points to a potentially weaker demand for it than foreseen at appraisal. All systems were able to finance their operation and maintenance compared to the 80% PAD target. The target for school facilities was fully achieved . Water users were able to consume 10.7 liters per capita per day on average, or 7% above the PAD target, in seven districts . While the original target was to raise average consumption to 10.0 liters per capita per day in 16 communes, during implementation under the national decentralization policy the communes were renamed districts and eventually redrawn . Objective 2: Building the capacity of water agencies, communes, users and the private sector to assume roles consistent with the new water sector strategy . High. High . The appraisal target for this objective was to enable nine Community Development Committees (CDCs) to program and execute WSS investments and six partner organizations able to assist communities in programming and executing investments. In 2001, during implementation, the GoR extended the national decentralization policy and communes were renamed districts and boundaries were redrawn . Whereas prior to the change there were 194 communes throughout the country . After the policy was implemented 30 districts covered the entire country, each covering a much larger area than the individual communes . By project completion, there was success in seven districts. Taken as a percentage of the total number of local governments, districts with the capacity to provide water supply and sanitation services was 23% of the total (30 districts). Although exceeding the appraisal target of 8% of all communes, these shares are not directly comparable . The ICR notes that local operators took over the delivery of water supply and sanitation services well beyond the project area with 850 water systems (25% of the country total) came under the management of public -private partnerships. Objective 3: Mobilizing community support for the rehabilitation /expansion of major (regional) water systems and their operation by the private sector . Substantial . Despite some delays during implementation, the results were largely successful . The Mayaga water system met the PAD target for service coverage (120,000 served). Rehabilitation of the Mayaga system production facilities (water catchment, treatment plant and headworks ) and main trunk were completed in December 2007. The private operator has been selected and the public -private partnership (PPP) for the system operation will be effective shortly, however was not active at project close . Technical studies of the completion of the Lava Region water systems were carried out in order to determine the investments required to achieve the initial service targets . These studies were used successfully in attracting funds (AfDB and Dutch financing) for the execution of works. The water supply and sanitation systems were operated by local private operators (Aquavirunga and Electrogaz) by project completion. 5. Efficiency (not applicable to DPLs): Substantial : At appraisal, a cost-benefit analysis was carried out . Two types of economic benefits accruing to beneficiaries were estimated: (a) the cost savings associated with a decrease in the use of a more time -consuming traditional source; and (b) the increase in consumer surplus on the increased quantity of water consumed . The coverage for the appraisal analysis was not available in PAD . The EIRR for "protected sources" of water (springs) was 19.4%, for "rehabilitation of piped systems " it was 19.1%, and that for "new piped systems" it was 8.3% (see table below). This much lower figure is likely due to the much higher per capita cost of new systems . The overall EIRR was estimated to be 12.6% at appraisal. At project close, an ex-post analysis was carried out for the water supply investments . The methodology was similar to that used at appraisal- type of improved water supply system and the extent to which it is used by the community . The analysis covered 72% of actual project costs (the Subprojects component excluding the sanitation facilities and the Major Systems component excluding the Lava Region studies ). Below is a table of the EIRRs calculated for various water supply systems as reported in the ICR . Water Suppl y System EIRR (% ) Te chnical O ptions P AD ICR Springs 19.4 65.4 Rehabilitated pipe d systems 19.1 9.8 Ne w piped systems 8.3 6.9 - gravity-fed systems n/a 13.6 - pumping systems n/a 3.6 Mayaga system 11.3 9.0 Global 12.6 10.6 The analysis found the EIRR for springs and rehabilitated piped systems to be 65.4% and 9.8% respectively. Thus the benefit of protected sources was much higher than expected, however that of rehabilitated systems was lower . For new systems, the analysis was split between "gravity-fed systems" and "pumping systems" which had EIRRs of 13.6% and 3.6% respectively, with an overall EIRR of 6.9%, or slightly lower than expected . The overall EIRR was estimated to be 10.6%. While the EIRRs were lower than the PAD estimates, the spring catchments ended up serving two times as many people as expected, hence the much higher EIRR in the ICR . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 12.6% 0% ICR estimate Yes 10.6% 72% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Satisfactory . The project was successful in achieving its relevant objectives . Despite some implementation delays and inadequate monitoring and evaluation at the start of the project, the outputs, outcomes, and impacts were satisfactory. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: With regard to sustainability, the ICR notes that key measures to maintain capacity are being implemented . These include continued technical assistance from the Directorate of Water and Sanitation (DEA) and from provincial engineers. A national association was established by the system operators to provide members with support for financial and commercial management. Additionally, the Water Supply and Sanitation Program - Africa (WSP-AF) is planning an intervention to assist private operations in their activities as part of its program in Rwanda . Finally, the districts are accountable to both the communities they serve as well as the Government, which, according to the ICR, closely monitors their performance . Steps have been taken to mitigate environmental impacts through the addition of water resources protection measures to the Poverty Reduction Support Credits matrix . a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: The Bank supported the decentralized, community -driven approach of the project . However, at the appraisal and entry stages greater focus could have been placed upon designing monitoring indicators that allowed assessment of intermediate results and the pace of the implementation, and less upon the likelihood of private sector engagement and affordability . The absence of process indicators may have caused delays that were not corrected until after the Mid-term Review (MTR). However, during implementation the Bank performance in supervision improved markedly. During missions implementation workshops were held with partner organizations, districts, provincial engineers, contractors, the Directorate of Water and Sanitation (DEA) and the Project Coordination Unit (PCU). Brainstorming sessions were held in the field with the project team . The staff comprising the supervision team remained the same throughout the project which allowed an effective partnership to develop between them and the Borrower . The candid nature of the Interim Status Reports ultimately led to the downgrading of the Implementation Progress rating which then prompted the project participants (DEA, PCU, MINERENA) to work with stakeholders in making the necessary changes to fix existing problems. While safeguard reviews were not carried out as often as they perhaps should have been, the inclusion of water resources protection measures in the Poverty Reduction Support Credit (PRSC) matrix helped counterbalance this shortcoming . As noted in the ICR, the Borrower was very pleased with the flexibility provided by the Bank in dealing with necessary changes to the project design and implementation . at -Entry :Moderately Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The Government effectively supported the decentralization process and showed commitment to the project by providing adequate financial and human resources to the districts, allowing them to take over the water supply and sanitation services. The Government paid attention to project results and put special emphasis on accountability (according to the ICR several district mayors were removed for poor performance ). The State Minister of Water and Natural Resources prompted the Project Coordination Unit (PCU) and the Directorate of Water and Sanitation (DEA) to deal with implementation issues causing delays and helped set up a group to prepare the shift to a sector -wide approach while still managing the rural water supply and sanitation projects . The implementing agency consisted of a project team (the Project Coordination Unit and the Directorate of Water and Sanitation) and the districts. The project team struggled at the start of the project with implementation delays related to the slow pace of disbursements, monitoring deficiencies and insufficient provision of counterpart funding from the budget. After the revisions following the Mid -term Review, these issues were resolved . The Project Coordination Unit held workshops with stakeholders to address the implementation issues . In addition, the Project Coordination Unit conducted a review of the community mobilization process and community management in order to make further improvements . The project team remained committed to community involvement throughout. The districts took over implementation in 2001 and focused on the restoration of water supply and sanitation services and their expansion . a. Government Performance :Highly Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: M&E Design: Modest The monitoring and evaluation indicators were appropriate, measuring access to water supply and sanitation services, use and sustainability of the services, and assessments of capacity building activities . The shortcoming in M&E design was that monitoring indicators did not adequately account for intermediate results or the pace of implementation. These issues were resolved following the Mid -term Review. M&E Implementation & Utilization: Modest . The Project Coordination Unit implemented the M&E . During design a deputy project coordinator was appointed and placed in charge of M&E and the establishment of semi -annual reports. However, prior to the Mid-term Review, reports were not produced on time and the Project Coordination Unit did not aggregate data provided by the partner organizations . While unsatisfactory before the Mid -term Review, monitoring became a sectoral priority afterwards . The Government provided financial support with Bank assistance through the Poverty Reduction Support Credits . The M&E system was used to measure project outcomes successfully, though less so in monitoring implementation . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project did not involve significant negative social or environmental impacts . Two reviews of the mitigation plan were conducted, one in 2003 and one at project close. The second review raised concerns regarding the protection of water resources in spring catchments and water reservoirs . Although water resources protection measures were included in the Poverty Reduction Support Credits matrix, safeguard reviews perhaps should have been conducted more than twice, especially since many of the water supply technologies used in the project had not been used widely in Rwanda before. Financial reports were not prepared adequately or on time prior to the Mid -term Review; however, afterwards they were satisfactory . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Highly Satisfactory Satisfactory The project EIRR is too low for a highly satisfactory rating. There were also significant delays in implementation and M&E prior to the Mid-term review. Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: (1) Public-private sector partnership (PPP) has a strong potential for the delivery and development of improved water services. Community-based management seems more appropriate for point sources (springs or hand pumps) than for piped systems. The Rwandese experience showed that (i) Water users' associations offer limited accountability, (ii) maintenance and cost recovery is quite challenging, and (iii) local private operators can be found anywhere. In situations where the operating conditions and actual demand are uncertain, it is not advisable to try to impose the use of standard contracts . The Rwandese districts decided instead to design themselves very simple contracts for candidates offering a minimum of qualifications, which resulted in quite satisfactory arrangements . (2) Self-standing rural water supply and sanitation (RWSS) projects can also aim at facilitating harmonization and preparing to scale up and to shift to a sector -wide approach and budget support by (i) providing a complete set of tested guidelines and procedures for preparing and implementing investments and for managing and delivering services; and (ii) enabling sector staff, rather than consultants, to elaborate objectives and policy matrix actions . As it was the case in Rwanda, only a strong sectoral leadership at the ministerial level can create incentives for staff and donors to internalize the change in approach and to ensure harmonization . (3) Despite some resistance by users to the principle of paying for water, experience shows that without user fees the delivery of quality water services on a sustainable basis is extremely difficult . Through establishing special arrangements for the poorest users and the design of systems based on user's ability to pay can make such schemes more acceptable to the population . For example, at the Mid-term review it was found that over 50% of the water users' associations were no longer operational due in part to users' low willingness to pay . In order to tackle the problem, community contribution rules were revised to emphasize in -kind and labor contributions . In addition, to alleviate the financial burden for the poorest part of the rural population, the districts keep a list of vulnerable households (widows, poor single-parent households), who get free access to water points . 14. Assessment Recommended? Yes No Why? The subject of this project is of interest to the IEG Water Study . 15. Comments on Quality of ICR: The ICR is satisfactory being both candid and concise . There is ample evidence of the project's success and the target indicators are all addressed in the discussion of the project outcome . One improvement that could be made would be a greater discussion of the process of working with the communities to establish acceptable user pays schemes as this is often a problem area in other projects . a.Quality of ICR Rating : Satisfactory