Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) Report Number: ICRR0022593 1. Project Data Project ID Project Name P120026 GH:Sustainable Rural Water & Sanit Serv Country Practice Area(Lead) Ghana Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-47890,IDA-61280 30-Jun-2016 120,527,501.45 Bank Approval Date Closing Date (Actual) 23-Jun-2010 30-Jun-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 75,000,000.00 0.00 Revised Commitment 120,690,814.45 0.00 Actual 120,527,501.45 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Ihsan Kaler Hurcan Ramachandra Jammi IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) as stated in the Financing Agreement (Schedule 1, page 5) and in the Project Appraisal Document (PAD, page 4) was: "To expand access to, and ensure sustainable water supply and sanitation services in rural and small town communities in six regions of Ghana." Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) This review is based on the three sub-objectives: (1) To expand access to water supply services in rural and small town communities: (2) To expand access to sanitation services in rural and small town communities: and (3) To ensure sustainable water supply and sanitation services in rural and small town communities. A split rating was not used for this review, as the PDO was unchanged, and the targets were revised upwards with the Additional Financing (AF) increasing the ambition of the project. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 29-Nov-2017 c. Will a split evaluation be undertaken? No d. Components There were three components (PAD, pages 5 - 7). 1. Rural and Small Town Water Supply. The estimated cost at appraisal was US$47.6 million. The revised estimate with AF was US$77.6 million. The actual cost was US$85.2 million. This component planned to increase access to water supply in six regions in Ghana. Activities to be financed in this component were as follows: (i) construction of boreholes with hand pumps for rural communities: (ii) construction of medium-sized systems, small sized elevated tanks and standpipes based on mechanized boreholes for medium-size small towns: (iii) construction of mechanized boreholes, larger sized elevated tanks for larger-size small towns: (iv) construction of piped schemes to connect with transmission lines of the Ghana Water Company Limited (GWCL) systems for rural and small towns : (v) rehabilitation of broken down "orphan" boreholes in rural communities and small towns (old boreholes, usually over thirty years old and not claimed by any donor): and (vi) rehabilitation of non-optimal performing water supply systems in small towns transferred from GWCL. Additionally, this component was to finance technical advisory services for the design, appraisal, supervision, and monitoring and evaluation of the above-listed investment activities. 2, Rural and Small Town Integrated Sanitation and Hygiene Promotion. The estimated cost at appraisal was US$18.6 million. The revised estimate with AF was US$29.8 million. The actual cost was US$21.0 million. This component planned to finance investments in sanitation systems for accelerating the pace for attaining Ghana's Millennium Development Goals (MDGs) to reach open defecation free status. Activities to be financed in this component were as follows: (i) construction of core elements of basic household latrines, and latrines, urinals, and hand washing facilities for schools and health facilities; and (ii) implementing behavioral change communication campaigns. Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) 3. Institutional Strengthening and Project Management. The estimated cost at appraisal was US$11.1 million. The revised estimate with AF was US$15.6 million. The actual cost was US$14.2 million. This component planned to finance technical assistance activities for building capacity of stakeholders in the water and sanitation sectors. Activities in this component were as follows: (i) Capacity building of public sector institutions at the national, regional, district and community levels and an assessment of the roles of the Ministry of Sanitation and Water Resources (MSWR), Ministry of Local Government and community level institutions, training Regional Coordination Councils for supervision of the District Assemblies (DAs) for water and sanitation subprojects, strengthening capacity of the key directorates in MSWR for planning and monitoring, provision of technical advisory services, training and logistical support to the DAs, non-governmental organizations and public service providers. (ii) Operationalization of a Sector Information System (SIS) to facilitate information sharing, coordination, and monitoring and evaluation in water and sanitation operations. (iii) Capacity building of the Community Water and Sanitation Agency (CWSA). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated cost at appraisal was US$77.3 million. The revised estimate with AF was US$120.6 million. The actual cost was US$120.5 million. Project financing. The project was financed by an IDA credit of US$75.0 million. AF of US$47.5 million was approved in November 29, 2017. With this, the Bank financing for the project was US$122.5 million. The amount disbursed at closure was US$120.5 million. Borrower contribution. Counterpart funding of US$2.3 million from District Assemblies (DAs) was planned at appraisal. There was no contribution from DAs during implementation and funds were allocated from the AF to cover the five percent counterpart funding that were to be paid by the DAs. Dates. The project was approved on June 23, 2010 and became effective on December 6, 2010. It was scheduled to close on June 30, 2016. AF was approved on November 29, 2017. The project closed on June 30, 2020, four years behind the original schedule. Other changes. There were three Level 2 restructurings. The changes made during the project lifetime in sequential order are as follows. The following changes were made through the first restructuring on June 27, 2017.  The closing date was extended by one year from June 30, 2016, to June 30, 2017, to allow time for the completion of some water supply systems that were behind schedule (as their costs had been underestimated at design), and for completing the sanitation activities that were behind schedule. The second restructuring on June 2017 extended the closing date by six months from June 30, 2017 to December 31, 2017 to provide time for signing the Credit Agreement for AF. Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) The following changes were made with the AF for the project on November 29, 2017.  The AF was meant to achieve the original targets (particularly related to the sanitation component).  The target values of the key outcome indicators for water supply and sanitation services were increased. The targets that were increased are as follows: (i) Increase in the number of people with access to improved water services: (ii) Increase in the number of boreholes to small town water supply systems, and new water supply systems for small towns: and (iii increase in the number of toilets in project areas.  Funds were allocated to cover the five percent counterpart funding that were to be paid by the District Assemblies (DA).  Technical Assistance activities were added to support the District Environmental Health Officers (DEHOs) and the Regional Health Environment Officers (RHEOs) for the campaign to construct household toilets with the aim of attaining open defecation free status in an additional 490 communities.  Logistical support for rolling out the Sector Information System, and capacity building support for the Ministry of Sanitation and Water Resources (MSWR), the Regional Coordinating Councils (RCCs), and the DAs. The third restructuring on December 2019, extended the closing date by six months from December 31, 2019, to June 30, 2020, to allow time for completion of the civil works in Northern and Northeast regions that were delayed because of the boreholes vandalized during local communal conflicts. 3. Relevance of Objectives Rationale Government strategy. The PDO was well-aligned with the Government's ongoing National Community and Water Sanitation Program (NCWSP) launched in 1994. The PDO was consistent with the goals of the National Water Policy of 2007, and the pillars of the second Ghana Poverty Reduction Strategy (GPRS II) for 2006 -2009. The PDO was aligned with the United Nations (UN) Sustainable Development Goals (SDGs) relating to access to safe and affordable water and adequate sanitation for all by 2030. Bank strategy. The PDOs were well-aligned with the Bank strategy. The Bank's Country Partnership Strategy (CPS) for 2013-2018 focused on improving health outcomes in a sustainable manner, and specifically referred to the lack of progress made in Ghana for improving delivery of sanitation services. The Bank's Ghana Systematic Country Diagnostic (SCD) published in 2018, advocated the need for further investments, especially in the sanitation sector. The Bank's Country Partnership Framework (CPF) for 2020-2026 is currently under preparation. Prior Bank experience. The Bank has executed prior projects in the water and sanitation sectors (Phase one and two of the Community Water and Sanitation project), and the Community-Based Rural Development Project. This project aimed at building upon gains achieved thus far while adding activities at aimed at supporting the Government's decentralization policy framework in the sectors through holding the District Assemblies (DAs) responsible for implementing subnational level subprojects. This project also aimed at adding value in the sanitation sector, through providing Community-Led Total (CLT) sanitation, to Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) create awareness for sanitation services (an approach that had been effective in parts of Bangladesh and India), and to help the communities achieve an open defecation free status. Given the limited success in improving delivery of sanitation services thus far in the country, and the need to increase access to improved water in rural areas, the project objectives were sufficiently challenging. However, while the objective remained relevant throughout the project cycle and was a necessary response to a development gap in Ghana, a significant shortcoming was the lack of clarity in the objective formulation around what outcomes would be achieved through increasing access to water and sanitation services. The causal chain between funding and outcomes was clear, albeit with most targets at output level (excluding the one related to the achievement of open defecation free status in the project area), as the objective was closer to the output level, rather than the outcome level. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To expand access to water supply services in rural and small town communities. Rationale Theory of change. The causal links between project activities, outputs and outcomes were logical. The intended outcomes were attributable to project activities and monitorable. The outputs of activities such as, providing boreholes with hand pumps for rural communities, medium-sized systems based on mechanized boreholes for small-size small towns, large-size mechanized boreholes for larger-sized small towns, connecting transmission lines from Ghana Water Company Limited (GWCL) for small towns, and rehabilitating broken-down "orphan" boreholes and non-optimal performing water supply systems were expected to increase access to water supply services in rural and small town communities. Outputs (ICR, pages 13 -14 and 35 -37).  1,373 boreholes that were fitted with hand pumps were operational at closure, exceeding the target of 1,200.  58 new mechanized boreholes for small towns were operational at project closure. This was marginally short of the revised target of 60.  75 new water systems were operational in small towns when the project closed, exceeding the revised target of 60.  376 orphan boreholes were rehabilitated, exceeding the revised target of 300.  15 former Ghana Water Company Limited (GWCL) systems were rehabilitated and transferred to the communities, as targeted. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026)  Nine Community piped water system from the GWCL were rehabilitated, exceeding the target of five. Outcomes (ICR, pages 13 and 31). The project activities were expected to increase the number of people with access to improved water supply.  1,356.340 people had access to improved water supply at closure. This exceeded the revised target of 800,000. The beneficiaries included 524,700 people benefitting from boreholes with hand pumps in rural areas and 831,640 people in small towns. Fifty two per cent of the beneficiaries were estimated to be women.  The reliability of water services improved in the project areas to 16 hours of service per day as targeted.  The team clarified that the quality of water was regularly tested for quality assurance. The ICR (paragraph 47) notes that a beneficiary assessment was conducted through a mini-census in each beneficiary community. The assessment noted that 90 percent of the respondents were satisfied with the increased access to potable drinking water, and reduction in time to fetching water. The beneficiary assessment however noted some lapses in operation and management of some small towns water systems due to inadequate oversight. Given that the outcomes were attributable to the project activities, efficacy of this sub-objective is rated as substantial. Rating Substantial OBJECTIVE 2 Objective To expand access to sanitation services in rural and small town communities. Rationale Theory of change. The causal links between project activities, outputs and outcomes were logical. The intended outcomes were attributable to project activities and monitorable. The outputs of activities aimed at constructing basic household latrines, latrines for schools and health facilities, coupled with communication campaign aimed at behavioral change towards better hygienic practices, were expected to increase access to sanitation services in rural and small town communities. Outputs (pages 37-38).  56,228 improved latrines were constructed, exceeding the target of 40,000.  568 institutions (schools and clinics) were provided with improved latrines, exceeding the revised target of 400.  1,168 communities were trained in areas such as personal hygiene and sanitation practices, linkages between sanitation, water and diseases, and environmental sanitation, exceeding the target of 600. Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) Outcomes (ICR, page 16). The project activities were expected to increase the number of people with improved sanitation and the number of open defecation free communities in the project area.  672,676 people in the project area had improved sanitation services, exceeding the target of 600,000.  1,052 communities in the project area were declared free of open defecation, almost double the target of 550 communities.  The ICR (paragraph 34) notes that qualitative assessment of beneficiaries conducted in the household survey indicated that less than seven percent of children under five years experienced symptoms of recent diarrhea (in a 24 hour period during two weeks before the survey) in some project areas, suggesting that the incidence of diarrhea might have gone down for children in that age cohort as compared to in the previous years. The community surveys also indicated that 95 percent of the respondents practiced hand washing at critical times (before eating and after defecation). The beneficiary assessment indicated that majority of respondents noted an increase in their level of knowledge about personal hygiene practices. The assessment however indicated that just over half (51%) of the respondents had access to improved latrines in the community and in schools. Given that the outcomes pertaining to expanding access to sanitation services in rural and small towns were realized, the project’s efficacy in achieving this objective is rated substantial. Rating Substantial OBJECTIVE 3 Objective To ensure sustainable water supply and sanitation facilities in rural and small town communities. Rationale Theory of change. The theory of change had gaps. The activities that were expected to ensure the sustainability of water and sanitation services were not clearly defined. Establishing a Comprehensive Water Supply and Sanitation Information System (SIS) for information sharing and monitoring and evaluation of the water and sanitation operations, capacity building of the Community Water and Sanitation Agency (CWSA) and of the District Assemblies in (DAs) for monitoring the District Monitoring and Evaluation System (DIMES) were aimed at increasing the capacity of the CWSA and district assemblies in using relevant sector information. The outputs of these activities were expected to contribute to the financial and physical sustainability of water supply and sanitation facilities in rural and small-town communities. It is not, however, clear whether these outputs were by themselves sufficient for realizing the expected outcomes. The project activities did not fully address the barriers to the sustainability of water and sanitation services, except behavioral change education and campaigns for improved sanitation and hygiene. The theory of change implicitly assumes that beneficiaries would adequately take care of the latrines and that there would be adequate financing and efficient supply chains of spare parts for the maintenance of boreholes and hand pumps. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) Outputs (ICR, pages 14 -15).  94 District Assemblies (Das) were operating the District Monitoring and Evaluation System (DIME) as targeted.  The comprehensive Water Supply and Sanitation Sector Information System (SIS) was not operational when the project closed. According to the team, at the end of the parent project a validation workshop was held, and a web server system was installed at the Water Resources Commission to serve as a hub for the information system. A list of facilities including computers and hardware facilities were procured to facilitate operationalization of the system, and staff of the relevant agencies were trained on using the system. While these activities were completed under the parent project, AF for the project was intended to support the operationalization of the system. The team clarified that the Water, Sanitation and Hygiene (WASH) sector was going through institutional changes when this project closed. Given this, the Community Water and Sanitation Agency and the Ministry of Water Resources Works decided to delay the operationalizing the SIS system, until the institutional studies and actions were completed.  Given that the default rate of water users was significantly higher for water systems that utilized a monthly flat rate for water tariffs when compared with Pay-As-You-Fetch modality, the project introduced “smart” technologies, such as token activated pre-paid public water stand pipes and controlled solar-powered pumping systems for improving service delivery.  The project introduced plastic latrine slabs as an alternative to basic sanitation option for households. These plastic toilet slabs are installed over a pit and can easily be removed and relocated once the pit is full. The team stated that no report of filled pit was received to date. Outcomes  There were no key outcome indicators aimed at monitoring the sustainability of sanitation services. As indicated above, only half of the beneficiaries were satisfied with the access to improved latrines and in schools.  Full cost recovery including depreciation and effective fee collection are essential for the long-term sustainability of water systems. The financial sustainability of the systems was to be monitored through comparing their revenues to costs. Specifically, the revenues raised by the water supply systems were to either be equal or exceed their operational and maintenance costs. This ratio of the water supply systems could not be determined when the project closed. The ICR (paragraph 35) notes that several of the water systems were just completed towards the end of the project and at that time the government was providing free water for the population, due to the exceptional circumstance of the on-going COVID -19 pandemic (from April 2020 to December 2020 and beyond), a situation which was expected to last for a while even after the project closure date. Given that the water and sanitation systems can be adversely affected if they are not properly operated or maintained, it is not clear to which extent the activities contributed to ensuring the sustainability of the water systems.  The beneficiary assessment highlighted challenges around a few cases of inadequate oversight of water systems by some District Assemblies and the risks associated eventual rundown or collapse of the systems. The ICR notes that there were challenges with supply chains for handpump spare parts, borehole replacement, and inadequate handpump mechanics in some communities. The ICR notes that 23 schemes under the CWSA benefitted from increased capacity of CWSA but provides no information about the other schemes.  It is not clear if there are adequate arrangements for the sustainability of the plastic latrine slaps or behavioral change in hygiene. The ICR does not provide any quantitative or qualitative evidence that Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) the Open Defecation Status (ODS) will likely be maintained. Given that the likelihood of slippage to open defecation following rural sanitation interventions is high, the project did not adequately address this issue. Because of the significant shortcomings in ensuring the sustainability of water and sanitation services and lack of evidence, the project’s efficacy in achieving the third objective is rated Modest. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The efficacy of the project in achieving objectives pertaining to expanding access to water supply and sanitation services are rated Substantial. The efficacy of the project in achieving the third objective to ensure sustainable water supply and sanitation services is rated Modest for lack of credible evidence. Taking these ratings into account, overall efficacy of the project is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic and financial analysis. A cost-benefit analysis (CBA) was conducted at appraisal and at closure to estimate the financial returns from water supply investments. This component accounted for 62% percent of the appraisal estimate and 70% of actual cost. The financial benefits were assumed to come from the revenue from the sale of (incremental) water production, connection fees for new customers, and consumer surplus form incremental sales of water. The Net Present Value (NPV) at 12% discount rate at closure was US$7.5 million as compared to the NPV of US$26.4 million at appraisal at the same discount rate. The ex post Internal Rate of Return (IRR) was 17.2% as compared to the ex-ante IRR of 20.5%. The ex post IRR was lower than the ex-ante IRR due a combination of factors including the long implementation period, increase in costs of water supply systems, and delayed project benefits. However, despite the extension of project duration and cost overruns which resulted in AF, the IRR exceeded the discount rate. The economic analysis of sanitation investments was not undertaken as they were not income-generating activities. Administrative and Operational Issues. The parent project suffered from shortcomings in preparation due to the large expansion of target communities and the correspondingly larger populations, which led to cost overruns. The costs of water supply investments of the original project were underestimated, with the costs of rehabilitating water systems proving to be higher than estimated at design due to a combination of technical factors including: (i) original designs had to be replaced by concrete reservoirs to withstand weather conditions: Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) (ii) longer pipelines were deemed necessary to connect communities in high saline areas: and (iii) some communities did not have electricity from the national grid (especially in the Upper East region), and the cost of connecting these communities to the national grid was higher than anticipated. The cost overruns were covered by funding from AF for the project. There was lower than anticipated demand for sanitation services in the initial years due to affordability concerns (resolved eventually through direct support to the rural households). Initial progress from the District Assemblies (DAs) was slow and this led to delays in sub-project starts. The operational issues were eventually resolved with the AF for the project. And given that the project exceeded its revised end targets both in terms of people provided with access to improved water supply (169% of the revised target), with improved sanitation services (112%), and communities which attained open defecation free status (191%), efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 62.00 Appraisal  20.50  Not Applicable 70.00 ICR Estimate  17.20  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the PDO is rated as Substantial. Efficiency is rated substantial given the economic justification for the project. The project’s efficacy in achieving the objectives to expand access to water supply and sanitation services is rated Substantial. The project’s efficacy in achieving the objective to ensure the sustainability of these services is rated Modest because of lack of credible evidence. As per the Bank guidance (p.38), overall outcome is rated Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) Institutional risk. There is risk to development outcome associated with the sustainable operation and management of the completed water supply and sanitation systems, given that it is not clear if there will be adequate resources for maintaining these systems. Stakeholder risk. The ICR (paragraph 93) notes that the beneficiary assessment report indicated challenges associated with oversight of water systems by District Assemblies. There is risk to development outcome given that lack of oversight could lead to the eventual rundown of the systems. 8. Assessment of Bank Performance a. Quality-at-Entry The strategic relevance of the project was well-articulated at entry and matched the client priorities. The project was prepared based on the experiences from prior-Bank financed projects in Ghana (Phases one and two of the Second Community Water and Sanitation Project), and from the experiences from Bank- financed projects in Bangladesh and India. Lessons incorporated at design included adopting Community Ownership and Management (COM) approach, decentralized service delivery through holding the District Assemblies (DAs) responsible for implementing sub-projects, and unlike prior projects where funds allocated for sanitation services were limited, allocating 17 percent of the project funds for Community- Led Total Sanitation (CLTS) activities. Given the Government's decentralization program framework for the sectors, the implementation arrangements were appropriate, with the Community Water Supply and Sanitation Agency (CWSA) responsible for overall coordination and the DAs for implementing sub- projects at the subnational level. The financial management capacity of CWSA was deemed to be adequate at appraisal. Several risks were identified at appraisal, including substantial risks with weak implementation capacity, governance and fiduciary risks, and concerns about the financial sustainability of water systems. Mitigation measures at design, included support to DAs on fiduciary compliance. Appropriate arrangements were made at appraisal for monitoring and evaluation, and safeguards and fiduciary compliance (discussed in sections nine and ten). There were moderate shortcomings at Quality-at-Entry. It is not clear if the design was based on in-depth studies or assessments and if there was adequate due-diligence at design, given the large expansion of target communities and the corresponding larger population. This led to cost overruns (discussed in section 5). These cost overruns necessitated AF for the project. The design of the sanitation component could have befitted from additional studies and assessments (to better understand and define the works and optimal implementation arrangements. This could have helped in preventing the low achievement of the delivery of household latrines and open defecation free status for the rural communities during the first years of the project. (The issue was eventually resolved through providing direct material support to rural households). Quality-at-Entry Rating Moderately Satisfactory Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) b. Quality of supervision Supervision missions were held twice a year during the project lifetime. The supervision teams included multidisciplinary teams with expertise in fiduciary management and safeguards. This aided in safeguards and fiduciary compliance (discussed in section 10). The supervision team was proactive and resolved issues, such as issues associated with limited demand for sanitation activities in the initial years of the project. Given the cost overruns associated with water supply activities and challenges associated with mobilizing counterpart funding from the District Assemblies, the supervision team was instrumental in resolving these issues through the AF for the project. This aided in completion of all project activities at closure. According to the clarifications provided by the team, with a field-based Task Team Leader (TTL) up until six months before the revised closing date, and a third headquarters based TTL since then. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results chain was logical, and the key outcome indicators (the number of people with access to water and sanitation services, reliability of water supply and the number of communities declared free of open defecation in the targeted areas, were attributable to the project activities and monitorable. The indicator associated with the ratio of revenue over operation and maintenance (O&M) was appropriate for monitoring the institutional strengthening project components. There were no appropriate indicators to monitor the sustainability of the sanitation sector. The baseline for the indicators were to be established at appraisal. The M&E was to be through a system that was to be established. Since assessment of system performance was not done in the prior Bank- financed projects in Ghana, the M&E design envisioned assessment of the system through a performance contract signed between the District Assemblies and the service providers of the Water and Sanitation Development Board (WSDB) (PAD, paragraph 39). b. M&E Implementation The District Monitoring and Evaluation (DIMES) - the official database platform of the Community Water and Sanitation System (CWSA) was established as envisioned at appraisal. The ICR (paragraph 71) notes that where it was difficult to establish a clear baseline, the project considered the baseline to be zero and measured additionality. The data collected by the District Assemblies (DA) for monitoring subprojects at the district level, was submitted to the Regional Coordination Councils Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) (RCC) and the Regional Water and Sanitation System (RWST), and the databases of the central Community Water and Sanitation Agency (CWSA). The indicator associated with the ratio of revenues over Operational and Maintenance (O&M) costs could not be determined for lack of data. The activity associated with the installation of a comprehensive water and sanitation information system was repeatedly delayed, and the system was not operational when the project closed. The weaknesses in the M&E system, especially relating to sustainability of the water and sanitation systems were not corrected during implementation. c. M&E Utilization The M&E system was used to support decision making and for monitoring implementation progress. The ICR (paragraph 74) notes that the M&E system was likely to be used by CWSA and the DAs to influence subsequent interventions in the near term. Overall, the M&E quality is rated modest because of the shortcomings in the M&E design and implementation in capturing the sustainability of services. The M&E system was, otherwise, sufficient to capture the achievements in increasing access to water and sanitation that were closer to the output level. M&E Quality Rating Modest 10. Other Issues a. Safeguards This project was classified as a Category B (partial assessment) project under World Bank safeguard policies. Two safeguard policies were triggered at appraisal: Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12) (PAD, page 25). No other safeguard policies were triggered with the AF. Environmental safeguards. The PAD (paragraph 72) notes that the environmental impacts of the project, were expected to be minimal and site-specific. An Environmental and Social Management Framework (ESMF) was prepared and publicly disclosed at appraisal. Following the AF for the project, the ESMF was updated to reflect the changes in the project. The ICR (paragraph 76) notes that the project was in compliance with environmental safeguards during implementation. Involuntary Resettlement. A Resettlement Policy Framework (RPF) was prepared and publicly-disclosed at appraisal to address involuntary resettlement issues (PAD, paragraph 65). This framework was updated with the AF. The ICR (paragraph 77) notes that the project was in compliance with social safeguards during implementation. Twenty three site-specific RPF plans were prepared during implementation to address resettlement issues. Grievance Redressal Mechanisms (GRMs) were established across all project locations and all grievances were resolved before the project end date. Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) b. Fiduciary Compliance Financial management. An assessment conducted at appraisal concluded that the financial management arrangements were satisfactory. The financial risk was rated as moderate at appraisal (PAD, paragraph 62). There were no major financial issues during implementation (ICR, paragraph 80). The Community Water and Sanitation Agency (CWSA) submitted acceptable periodic financial reports, including audits, although with some delays. The external auditors noted some internal control and accountability issues, most of which were rectified by closure. The most recent audit report for the year ending December 2019 was submitted in September 2020, after the deadline. This was accepted by the Bank, in view of the restrictions on social activities and delays caused by the COVID-19 pandemic. Procurement. A procurement assessment was conducted at appraisal. CWSA, the agency in charge of overall coordination, had executed Bank-financed projects and was familiar with the Bank's procurement procedures (PAD, paragraph 9). The procurement risk was however rated as substantial at appraisal, in view of the large number of project entities under the project, and many of the Regional CWSA's and District Assemblies (DA's) unfamiliar with Bank procedures. The mitigation measures incorporated at design included recruitment of additional procurement specialist and training the DA staff. The ICR (paragraph 81) notes that there were no major issues with compliance on procurement procedures. The ICR reports no case of mis-procurement. c. Unintended impacts (Positive or Negative) An unintended positive impact of the project was its contribution to managing the COVID-19 pandemic. The project implemented a hygiene and handwashing promotion, which were considered critical elements to mitigate the spread of COVID-19. The Community Water and Sanitation Agency (CWSA) championed the hand washing with soap campaign in Ghana and provided veronica buckets, free water and other hand washing facilities to households and institutions, to enable them to adhere to the hand washing with soap protocols. The protocols implemented by the government helped to mitigate the spread of the COVID-19 pandemic. d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment The project’s efficacy in ensuring Outcome Satisfactory Moderately Satisfactory the sustainability of water and sanitation services is rated Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) modest due to insufficient evidence. The project’s efficacy in achieving the other two objectives was rated substantial. The relevance of objectives and the project’s efficiency in achieving the objectives are both rated substantial. Overall, in accordance with the Bank guidance (p.38), the outcome is rated moderately satisfactory because of modest achievement of one of the objectives, i.e., to ensure the sustainability of services. Moderately Bank Performance Moderately Satisfactory Satisfactory There were shortcomings in the M&E design and implementation Quality of M&E Substantial Modest in capturing the sustainability of services. Quality of ICR --- Substantial 12. Lessons The ICR draws the following lessons from the experience of implementing this project with some adaptation of language. 1. Projects need to be prepared with due diligence to avoid implementation delays. In this project, the costs of water supply systems were underestimated due to technical considerations, and there was less than anticipated demand for sanitation services during the initial years. These issues were eventually resolved through the AF for the project. The lesson is that rigorous project preparation and pre-project due diligence is required for avoiding implementation delays. 2. Technology improvements and innovative methods could help enhance access to affordable and sustainable water supply and sanitation systems. Small town piped systems in Ghana had used electrical systems. For a typical small town power system, the cost of power supply could represent 50 percent of the operational cost. To address this issue, the project introduced solar panels (instead of the grid powered systems) for improving the operational efficiency of the water systems. Likewise, for sanitation issues, the project pioneered plastic latrine slabs to assist low income communities in the construction of household toilets. 3. An effective fee collection system for full cost recovery is essential for the long-term sustainability of the water systems. The beneficiary assessment in this project noted that in some areas the default rate of water users was significantly higher for water systems that used a monthly Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH:Sustainable Rural Water & Sanit Serv (P120026) rate for water tariffs when compared with a Pay-As-You-Fetch modality. These difficulties were addressed in this project through installing pre-paid smart meters for standpipes. 4. Behavior change communication strategies delivered by relevant stakeholders, coupled with material support, can help in achieving open defecation free status. In this project, there was demand for household toilets. The limited results were due to inadequate consultants and targeted communities. This issue was resolved in this project through a more intensive outreach by the District Assemblies and the targeted communities and providing direct material support to the rural households. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well-written and for the most part, the evidence for increasing access to water supply and sanitation services that were attributable to project activities, is from a credible source. The evidence provided about the sustainability of water and sanitation systems is, however, sparse. The quality of analysis is sufficient, and to the extent possible, the ICR includes evidence from a beneficiary assessment. The lessons from the experience of implementing this project are based on evidence and appropriately respond to the specific experiences for the project, and they are sufficiently linked to the narrative. However, the ICR’s rating for the project development objective of ensuring sustainability of water supply and sanitation systems is not backed by sufficient quantitative or qualitative evidence. The main body of the text at 25 pages is more than the recommended length of 15 pages. a. Quality of ICR Rating Substantial Page 16 of 16