Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Report Number: ICRR0021698 1. Project Data Project ID Project Name P054221 DR Water & Sanit in Tourist Areas Country Practice Area(Lead) Dominican Republic Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-76800 30-Sep-2013 27,500,000.00 Bank Approval Date Closing Date (Actual) 02-Apr-2009 31-Mar-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 27,500,000.00 0.00 Revised Commitment 27,500,000.00 0.00 Actual 27,500,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Chikako Miwa Fernando Manibog Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objectives of the Project are: (i) strengthening and consolidating the policy framework of the Borrower’s water and sanitation sector; (ii) improving and expanding access to sanitation and wastewater treatment and disposal services in the Puerto Plata region; (iii) improving the financial and operational performance of CORAAPPLATA, the region’s water utility; and (iv) enhancing operational and commercial performance of other regional utilities and preparing them to participate in the second phase of the Program (Loan Agreement dated September 18, 2009, Schedule 1, page 5). Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Regarding the PDO (iii), Puerto Plata Water and Sewerage Company (Corporación de Acueductos y Alcantarillados de Puerto Plata: CORAAPPLATA) was a provincial utility established in 1997 to decentralize and replace the National Institute for Drinking Water and Sewerage (MAPA). CORAAPPLATA was responsible for managing water supply and sanitation systems in the community and province of Puerto Plata. Regarding the PDO (iv), other regional utilities that were targeted under the Project were: Moca Water and Sewerage Company (Corporación de Acueductos y Alcantarillados de Moca: CORAAMOCA) and La Romana Water and Sewerage Company (Corporación de Acueductos y Alcantarillados de La Romana: CORAAROM). The second phase of the APL did not materialize due to the delay in implementation of this project, which was originally planned as the first phase. The PDO (iv) was revised to drop the second phase of the APL at the sixth restructuring on January 5, 2018. IEG concurred with the ICR that “a ‘split evaluation’ was deemed unnecessary, as the revision of the PDO statement was not substantive, did not change the key outcomes, and simply removed language that referenced a second phase of the APL” (ICR, para 24). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 05-Jan-2018 c. Will a split evaluation be undertaken? No d. Components Component 1: Technical Assistance for Sector Development (Appraisal cost: US$3.52 million, Actual cost: US$4.62 million) At appraisal, key activities planned to be financed were: (a) National Strategy for the Water and Sanitation Sector, (b) Institutional Modernization and Strengthening of Utilities, (c) Watershed Management Plans and Strategic Environmental Assessment, (d) Environmental Education and Public Participation Campaign (PAD, page 8-9). At the project closure, key activities financed were: (i) sector development, (ii) metering for CORAAROM and CORRAAMOCA; and (iii) business plans and commercial systems for CORAAROM and CORRAAMOCA (ICR, page 42). Component 2: Rehabilitation and Expansion of Sanitation Services in Puerto Plata (Appraisal cost: US$26.33 million, Actual cost: US$24.70 million) Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) At appraisal, key activities planned to be financed were: (a) Rehabilitation and Improved Access to Sanitation, (b) Wastewater Treatment and Disposal, (c) Technical Assistance to CORAAPPLATA (PAD, page 9-10). At the project closure, key activities financed were: (i) sewage network expansion (Sosua/Montellano), (ii) wastewater treatment plant, (iii) pumping stations and force lines Sosua/Montellano, (iv) pipeline connecting to Playa Dorada, (v) marine outfall Puerto Plata, (vi) marine outfall in Cabarete, (vii) metering for CORAAPPLATA, (viii) technical assistance for CORAAPPLATA (ICR, page 42). Component 3: Program Administration (Appraisal cost: US$4.08 million, Actual cost: US$6.06 million) Key activities planned and financed were: costs associated with program management such as Project related audits, equipment, training and technical assistance to strengthen the State Secretariat of Economy, Planning and Development (SEEPYD - Secretaria de Estado de Economía, Planeación y Desarrollo) and CORAAPPLATA, as well as individual consultants (PAD, page 10; ICR, page 9). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: At appraisal, the original cost estimate was US$34.00 million. The actual cost was US$35.44 million (ICR, page 42). The increase in project cost was covered by the Borrower's contribution. Financing: At appraisal, IBRD loan financing was estimated at US$27.50 million. The actual IBRD disbursement was US$27.50 million, the same as the estimate at appraisal. Borrower Contribution: At appraisal, the Borrower was expected to contribute US$6.50 million. The actual contribution was US$7.95 million. Dates: The project experienced six restructurings since its approval on April 2, 2009. There were six restructurings: the first (July 18, 2013), the second (June 30, 2015), the third (September 15, 2015), the fourth (November 22, 2016), the fifth (June 14, 2017), and the sixth (January 5, 2018). The key changes made among the first to the fifth restructurings were to extend the project closing date, in order to: get contracts signed for the main sewerage works, allow for the completion of the contracted works, complete the construction of the sewage collection and treatment works, and allow the completion of the submarine outfall. The fourth restructuring also reallocated US$641,000 provided under the Works Category to the Goods Category. The sixth restructuring removed all reference to the second phase of the APL in the PDO and results framework since a phase II of the project will not be initiated. Phase II was not carried out due to delays in executing the first phase (ICR, footnote 23, page 19). The cumulative amount disbursed by that time was US$26.36 million. The project closed on March 31, 2018, after five closing date extensions totaling four and a half years of delay from the original closing date of September 30, 2013. 3. Relevance of Objectives Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Rationale Country Context. Environmental contamination poses a threat to the tourism industry, which was a traditional driver of macroeconomic growth. According to the WTO, receipts from foreign visitors represented 12% of the country’s GDP (PAD, para 2). Insufficient water and sanitation system worsened pollution and degradation of natural segments in coastal areas, which negatively affected tourism. Improved services for safe drinking water and sanitation were essential for the competitiveness and sustainability of tourism as well as for sustainable economic growth. Sector Context. The main issues in WSS sector were watershed degradation, pollution, and water scarcity. These WSS issues were prominent in tourism areas, where informal settlements of low-income households were formed to provide services to hotels and tourists. Expansion of these communities increased water demand, worsening watershed degradation and water scarcity. Lack of basic water and sanitation services in these communities led to more pollution in rivers and coastal areas. Relevance to Government Strategies. At appraisal, the PDOs were relevant to the National Development Plan (END) 2010-2030. At present, the PDOs remain relevant to the END, as well as the Medina administration's 2016 statement to set the next four years as the "Quadrennium of Water" (ICR, page 11). The project supported the government to develop the 2016 National Sanitation Strategy to improve the quality of sanitation services. Under the social development pillar in the END, the project contributes to END General Objective 2.5, Dignified housing (and water) in healthy environments, where the WBG had both comparative advantage and demand (CPS, page 21). Relevance to Bank Strategies. At appraisal, the PDOs were relevant to the FY 2010-2013 Country Assistance Strategy (CAS)'s Strategic Objective II: Promote Competitiveness in a Sustainable and Resilient Manner and the government's National Development Strategy’s Pillars of (i) Economic Growth and Sustainability and (ii) Territorial Development (ICR, page 7). The project was strongly aligned with government priorities to promote sustainable economic growth. At present, the PDOs remained relevant to the FY2015-2018 Country Partnership Strategy (CPS), which was the latest available CPS at the time of project closure. FY2015-2018 CPS set out five results areas: 1. Improving the investment climate and fostering private sector development; 2. Improving access to efficient and reliable electrical distribution networks, ICT and other infrastructure; 3. Supporting the Government in building resilience to external shocks; 4. Promoting equitable, efficient, transparent and sustainable management of public resources; 5. Strengthening social services delivery. The main results areas that the projects contributed were: social services delivery and resilience. The project's PDO (ii) was explicitly included under CPS Results Area 5 and CPS Outcome 5 on social services delivery. The objectives of the project had been aligned with the Bank’s strategies from the appraisal to the closure (i.e. FY 2010-2013 Country Assistance Strategy and FY2015-2018 Country Partnership Strategy); however, the results areas that the project was classified changed from sustainable economic growth to strengthening social delivery. Institutional Capacity and Realism. There was a technical gap between the Dominican Republic’s previous sector experience on water submarine outfalls and the technical expertise required to design deep water submarine outfalls in the project area. The infrastructure portion of the project was designed based on the experience of the Learning and Innovation Loan (LIL) closed in 2015, which constructed small-diameter submarine outfalls that required less technical expertise compared to this project. At the time the project Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) started, the counterparts who worked on the LIL were not in the PMU, indicating a loss of accumulated knowledge and experience. The deep-water submarine outfalls for this project required advanced technical knowledge that the Borrower and the Implementing Agency did not possess at the time the project started. Moreover, geotechnical surveys were not conducted in the project area at the design stage (ICR, page 21). This caused the project to start its implementation without sufficient analysis on technical feasibilities and costs, which resulted in delay in construction of the infrastructure (ICR, page 19). Previous Sector Experience. The Wastewater Disposal in Tourism Areas LIL closed in 2015 led to: “(i) the initiation of discussions amongst key stakeholders on sector regulatory and institutional functions and of INAPA's role with that of the Corporaciones; (ii) application and testing of a practical and cost- effective technology for environmentally sound disposal of treated wastewater for small and medium size coastal towns; and (iii) discussion on private sector participation as an option to increase efficiency of the sector” (PAD, page 4). The Project was designed to expand on these initial advances. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective (i) strengthening and consolidating the policy framework of the water and sanitation sector in the Dominican Republic Rationale Theory of Change (TOC) for the Project: The Project had six restructurings, with extensions of Project duration for 51 months over the original closing date. The Project maintained the original TOC, with adjustments to the results framework. The TOC showed a logical, causal chain for suppliers to improve coverage, quality, efficiency, and sustainability of water and sanitation services. In the TOC, (ICR, page 8, Figure 1), there were three paths to contribute to the achievement of the Project’s intended impact, which was the improved provision of efficient and sustainable water and sanitation services in the Dominican Republic’s tourist areas. The first path was to increase access to sewerage and to improve wastewater treatment and disposal in the Puerto Plata region. The second path was to strengthen the capacity of utilities. The third path was to strengthen the sector’s policy framework. TOC for PDO (i): The Project financed technical assistance to establish the Water Sector Council, an inter- agency entity responsible for developing a national strategy, which stated official goals and objectives for the sector. The Project supported the stakeholders to reach a consensus on roles and responsibilities for the provision, management, and regulation of water and sanitation services, to be reflected in the final version of Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) the water and sanitation law. It was aimed that establishment of the water and sanitation law and strategy would strengthen the WSS sector’s policy framework. Outputs:  Technical assistance (i.e., through consultants), a draft national strategy, an update of a draft water and sanitation law (ICR, para 28). Outcome: Achievements of the PDO indicators were summarized below:  The Water Sector Council was established by presidential decree on August 11, 2011.  The Water Sector Council developed and adopted a National Strategy for the WSS sector. The PDO indicator on the national strategy was not appropriate to measure outcome-level achievements; because, the indicator was limited to approval of the strategy, and did not call for its implementation. The national strategy had not been implemented pending legislative approval of the draft water and sanitation law by Congress (ICR, para 28). The draft water and sanitation law was under review by congressional sub- committees as of June 2019, according to the meeting between the ICR Team including the last TTL of the Project and IEG held on August 21, 2019 (TTL meeting). IEG concurred the ICR Team’s acknowledgement that ideal indicators would have focused on outcomes such as implementation of the national strategy (ICR, para 29). Implementation of the national strategy was critical to formally advance the sector reforms, even though there was a consensus among the local stakeholders on technical aspects of the draft water and sanitation law. On balance, the establishment of the Water Sector Council and the adaptation of the National Strategy for the WSS sector by the Water Sector Council were achieved. Though there was a delay in approval of the new water and sanitation law by the Congress, the Project supported the stakeholders to arrive to a consensus on the technical contents of the WSS sector’s new policy framework. On this basis, the Project’s efficacy in achieving PDO (i) is assessed as substantial. Rating Substantial OBJECTIVE 2 Objective (ii) improving and expanding access to sanitation and wastewater treatment and disposal services in the Puerto Plata region Rationale TOC for PDO (ii): The Project financed construction of a pre-treatment plant, a submarine outfall, and a sewerage network. It was aimed that the infrastructure would improve sanitation and wastewater treatment and disposal services in the target area. Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Outputs:  The construction of the pre-treatment plant was completed.  The construction of the submarine outfall was partially completed. After the project closing, the remaining works of the submarine outfall were financed through the Government’s counterpart funds (ICR, para 83).  The construction of the sewerage network was partially completed with a reduced scope. The target area was reduced from poor neighborhoods in Puerto Plata region to that of Puerto Plata City due to the lack of resources. Outcome: Achievements of the PDO indicators were summarized below:  142,952 consumers received access to improved or new sanitation and to wastewater treatment and disposal services in the Puerto Plata region, exceeding the target of 128,000.  3,679 consumers received access to network sanitation in the Puerto Plata region, which was significantly below the target of 18,000 (20 percent of the original target).  79% of consumers and 78% of hotels were connected to the sanitation network in Puerto Plata, slightly missing the targets of 80% for consumers and 70% for hotels. Under PDO (ii), there were two distinct groups of target beneficiaries: (1) consumers and (2) hotels. (1) Consumers: There were two distinct results which targeted consumers as beneficiaries: (i) sanitation sewerage results, and (ii) wastewater treatment and disposal results. Regarding the first and the second PDO indicators, the PAD clearly distinguished targets for the two results as: mostly poor consumers gaining new access to network sanitation (18,000 people) and consumers receiving new access to wastewater treatment and disposal (110,000 people) (PAD, page 39). For (i) sanitation sewerage results, 3,679 consumers received new access to network sanitation in the Puerto Plata region, which was significantly below the target of 18,000 (20 percent of the original target) (ICR, page 29). The scope of the Project area was reduced from the Puerto Plata region to the City of Puerto Plata, because of the cost overruns of the submarine outfall, which used “almost 60 percent of the entire World Bank funding” (ICR, para 45) and the increase in the predicted costs for sewerage expansion in Sosua and Montellano. For (ii) wastewater treatment and disposal results, there was no separate reporting against the target in the results framework of the ICR, which reported only a combined number of 142,952 consumers who received access to improved or new sanitation and to wastewater treatment and disposal services in the Puerto Plata region (ICR, page 29).The reported number, 142,952 consumers, was calculated from 39,709 residential connections to the sewerage network in the city of Puerto Plata documented in the registry (ICR, para 31). It was unclear how many consumers received new access to wastewater treatment and disposal attributable to the Project due to lack of evidence. Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) (2) Hotels: Regarding the third PDO indicator, the PAD included a target for hotels, that was, 70% of occupancy capacity of hotels in the Puerto Plata region were connected to communal sanitation networks (PAD, page 39). The ICR reported that, based on the registry, 78% of hotels were connected to the sanitation network in Puerto Plata. The IEG concurred with the ICR that the Project “made no material impact” on the expansion of the sanitation network connection to the hotels and that the third PDO indicator should have been modified to omit the reference to hotels during one of the restructurings (ICR, para 34). On balance, the construction of the submarine outfall was partially completed at the project closing but was eventually completed. While its cost overruns caused the reduction of the scope in the construction of the sewerage network, the completed infrastructure directly contributed to the improvement in the poor households’ access to the sanitation network and waste water treatment and disposal services. On this basis, the Project’s efficacy in achieving PDO (ii) is assessed as substantial. Rating Substantial OBJECTIVE 3 Objective (iii) improving the financial and operational performance of CORAAPPLATA, the regional's water utility Rationale TOC for PDO (ii): The Project financed the installation of micro-meters and technical assistance to update a cadaster and to develop a business plan. It was aimed that the micro-meters and the cadaster would support a reduction of non-revenue water, which would increase the own-sources revenue and in turn improve the utility's financial performance. It was aimed that the business plan and the cadaster would support improvements in its operational performance. Outputs:  4,000 new micro-meters were installed for CORAAPPLATA.  The cadaster, CORAAPPLATA’s customer registry, was updated. Intermediate Results:  Business and modernization plans were completed for the three regional utilities, which achieved the target.  Metering increased from 3 percent to 24 percent, which surpassed the target of 12 percent.  Non-revenue water was reduced from 57 to 45 percent between 2015 and 2017. Outcome: Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Achievements of the PDO indicators were summarized below:  CORAAPPLATA identified 100 percent of its potential consumers and made operational an active registry of 41,657 users (surpassing the PDO-level target of 40,000), which covered 65 percent of its operational areas (surpassing the PDO-level target of 60 percent). Under PDO (iii), there were two distinct results to be discussed: (1) financial performance and (2) operational performance of CORAAPPLATA. (1) Financial Performance: The original PDO indicator on the financial performance was dropped at the 2013 restructuring. To measure the financial performance, the ICR team reported instead on two other indicators: the ratio of total own-source revenues to total operating expenses, where (i) expenses included energy costs and (ii) expenses excluded energy costs (ICR, para 39). When (i) expenses included energy costs, the ratio of total own-source revenues to total operating expenses improved from 0.38 to 0.45 from 2014 to 2017 (ICR, Table 2, page 15). When (ii) expenses excluded energy costs, the ratio of total own-source revenues to total operating expenses improved from 1.07 to 1.20 from 2014 to 2017 (ICR, page 15, Table 2). The rationale for the exclusion of energy costs from the calculation were: (1) energy costs for regional utilities were heavily subsidized by the central government and (2) energy costs could make up a significant portion of overall operating costs depending on the provider (ICR, para 39; TTL meeting). The staff costs were the other main driver of the operational expenditures, but the volume of subsidies for the staff costs were much smaller compared to the energy costs. For CORAAPPLATA, 87 percent of energy costs and 17 percent of staff costs were financed by transfers from the central government (ICR, page 45). The reduction of non-revenue water (NRW) contributed to the improvement in the financial performance. The reduction of NRW was achieved through: (1) installation of micrometers and (2) update of the cadaster. The Project played a catalytic role to initiate installation of micrometers to improve cost recovery. The project contributed to an increase in metering from 3 percent to 24 percent over the course of the Project (far surpassing the intermediate outcome indicator of 12 percent) through financing the installation of 4,000 micrometers (ICR, page 35). Then, CORAAPPLATA engaged in a private public contract with a firm to manage their commercial functions and expand metering. The financial performance of CORAAPPLATA improved largely due to installation of micrometers that were funded outside of the Project. (2) Operational Performance: The PDO indicator on the operational performance was achieved, as summarized in achievements for outcome above. The Project missed an opportunity to measure improvements in the operational performance of CORAAPPLATA through systematic customer satisfaction surveys by specialized firms due to the delay in the infrastructure construction (ICR, page 23). According to the consumer feedback survey conducted by CORAAPPLATA in 2018, of 489 responses to the survey question, “How would you rate the operational services provided by the institution?,” 86 consumers (18%) responded Excellent, 81 consumers (17%) responded Good, 170 consumers (35%) responded Fair, and 126 consumers (26%) responded Poor (ICR, Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) page 57). Over the half of the customer rated the operational services of CORAAPPLATA as below the average. The result of the survey suggested that the improvements in CORAAPPLATA’s operations had not been felt by its customers yet. On balance, the operational performance of CORAAPPLATA improved in general but there is little or no evidence that it improved customers’ satisfaction level. The Project played a catalytic role for the installation of micro-meters to reduce the non-revenue water, which contributed to a modest improvement in CORAAPPLATA’s financial performance. On this basis, the Project’s efficacy in achieving PDO (iii) is assessed as modest. Rating Modest OBJECTIVE 4 Objective (iv) enhancing operational and commercial performance of other regional utilities and preparing them to participate in the second phase of the Program (APL II) Rationale PDO 4 was revised to exclude the preparation for the second phase of the program due to the delay in implementation of this project. Achievement of Objective (Efficacy) was reviewed for revised DPO 4 as shown below. Rating Not Rated/Not Applicable OBJECTIVE 4 REVISION 1 Revised Objective (iv) enhancing operational and commercial performance of other regional utilities Revised Rationale TOC for PDO (ii): The Project financed installation of administrative and financial software and micro-meters and technical assistance to develop business plans. It was aimed that the micro-meters supported a reduction of non-revenue water, which increased the own-sources revenue to improve the utilities' financial performance. It was aimed that the business plan and the administrative and financial software would support improvements in their operational performance. The Project supported the regional utilities to develop and sign compliance agreements, which included agreements on targets-setting and reporting on financial and operational performance. Outputs:  The new administrative and financial software was implemented at the regional utilities. Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221)  The new strategic business plans were created for the regional utilities.  3,000 new micro-meters were installed for CORAAMOCA. Intermediate Results:  The coverage of metering increased from 4 percent (2012) to 42 percent (2017) for CORAAMOCA and from 8 percent (2012) to 13 percent (2017) for CORAAROM. CORAAMOCA was able to expand the coverage of metering more than CORAAROM; because, CORAAMOCA secured resources outside of the Project to expand their metering program (ICR, footnote 22).  Non-revenue water was reduced from 83 to 68 percent between 2013 and 2017 for CORAAROM. Outcome: Achievement of the revised PDO indicator is summarized below:  Compliance agreements were signed on improving the cadaster, commercial, administrative and financial systems of CORAAPPLATA, CORAAMOCA and COAAROM, as well as a sewerage systems Operation and Maintenance (O&M) strengthening program for CORAAPPLATA. Under PDO (iv), there were two distinct results to be discussed: (1) financial performance and (2) operational performance of CORAAMOCA and CORAAROM. (1) Financial Performance: To measure the financial performance of CORAAMOCA and CORAAROM, the ICR team reported on the ratio of total own-source revenues to total operating expenses, where (i) expenses included energy costs and (ii) expenses excluded energy costs (ICR, para 41). When (i) expenses included energy costs, the ratio of total own-source revenues to total operating expenses improved from 0.46 to 0.72 from 2014 to 2017 for CORAAMOCA (ICR, page 16, Table 3). Considering that the Project invested in only 3,000 micro-meters for CORAAMOCA, the improvement of the financial performance of CORAAMOCA was largely due to the expansion in the metering program financed outside of the Project. The Project played a catalytic role to initiate installation of micrometers to improve cost recovery. For CORAAROM, the ratio increased from 0.42 to 0.51 in 2017 (ICR, page 16, Table 3). When (ii) expenses excluded energy costs, the ratio of total own-source revenues to total operating expenses improved from 0.72 to 0.90 from 2014 to 2017 for CORAAMOCA (ICR, page 16, Table 3). For CORAAROM, the ratio increased from 0.70 to 0.91 in 2017 (ICR, page 16, Table 3). (2) Operational Performance: Regarding operational performance, the regional utilities strengthened its compliance system through the development and the signing of the compliance agreements. The regional utilities submitted periodic operational and commercial performance reports to comply with the country’s regulations (ICR, page 49). The sewerage O&M strengthening program was signed for CORAAPPLATA, which was expected to support the regional utility to sustainably manage its infrastructure. IEG concurred with the ICR team that the revised PDO indicator did not permit measuring the PDO’s outcome-level achievement PDO (ICR, page 49); because, the indicator did not consider what were achieved by the activities carried out under the compliance Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) agreements and the O&M strengthening program. Even though the signed agreements and program were solid achievements to strengthen the regional utilities’ operational performances, monitoring their implementation status would be essential to measure the achievements of the outcome. As PDO (iv) explicitly targeted “other regional utilities,” namely, CORAAMOCA and CORAAROM, the signing of sewerage O&M strengthening program for CORAAPPLATA would be more relevant to PDO (iii) than PDO (iv). On balance, the financial performance of CORAAMOCA and CORAAROM improved in general from 2014 to 2017. The regional utilities submitted operational and commercial performance reports in line with the national regulations. On this basis, the Project’s efficacy in achieving PDO (iv) is assessed as substantial. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The Project aimed to improve accessibility, efficiency, and sustainability of WSS services in tourist areas in Dominican Republic. The achievement of PDO (i) was substantial. The WSS strategy was developed but was not implemented on the ground as its implementation was contingent on the legislative approval of the water and sanitation law by the Congress. The ICR team confirmed that the technical contents of the water and sanitation law were agreed by the stakeholders which started their own internal processes to prepare for the implementation of the WSS strategy once the water and sanitation law comes into effect (TTL meeting, August 21, 2019). The achievement of PDO (ii) was substantial. The installation and rehabilitation of infrastructure was significantly delayed, and its scope was reduced; however, upon completion of the infrastructure, the access to the wastewater treatment and sewerage network services were widely expanded in the poor neighborhood. The achievement of PDO (iii) was modest. The Project partially contributed to a gradual improvement in CORAAPPLATA’s financial performance from 2014 to 2017 as a catalyst of change. The effect of the Project’s intervention to improve CORAAPPLATA’s operational performance had not reached the level that could improve the customers’ satisfaction, according to the customer feedback survey held in 2018. The achievement of PDO (iv) was substantial. The financial and operational performances of CORAAMOCA and CORAAROM were in a trend of improvement from 2014 to 2017. The regional utilities submitted operational and commercial performance reports in line with the national regulations. Overall, the Project’s efficacy in achieving its objectives was substantial. Overall Efficacy Rating Substantial Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) 5. Efficiency Economic Analysis: At appraisal, the Project’s economic internal rate of return (EIRR) was estimated at 36 percent and the net present value was US$31 million using a discount rate of 12 percent. These analyses used two parallel methods to estimate direct and indirect benefits of sewerage expansion - willingness to pay (WTP) and revealed preference technique. WTP was estimated during Project preparation through direct information obtained from beneficiaries; this estimation was updated and used to evaluate the sewerage component. The revealed preference technique was applied as a proxy to estimate demand for sewerage services, by using the market price of septic tanks, which are built and operated by many potential beneficiaries. Both approaches were used for comparison reasons and the results show higher returns when using WTP. The ICR conducted a cost-benefit analysis (CBA) using a modified methodology compared to the one used at appraisal. In the ICR, the cost benefit analysis for the Project was carried out for the following components: (i) sewerage expansion benefits, including health benefits in the sensitivity analysis; and (ii) final disposal of wastewater and treatment systems, including the value of wastewater treated and environmental benefits. The Project CBA assumed a 30-year life span and values of costs and benefits were discounted using a 6 percent discount rate as per World Bank guidelines. All benefits and costs were estimated based on the components of the restructured Project. According to the ICR, upon commissioning of the submarine outfall, the Project’s EIRR was 11.2 percent. Using a discount rate of 6 percent, the benefit-cost ratio was estimated at 1.61. Net present value of cost and benefits over the life of the Project were estimated at US$28.7 million and US$46.2 million, respectively; and the net present value of net benefits was estimated at US$17.4 million. Financial analysis: Financial analysis was inconsistent between the appraisal and the ICR. While the appraisal assessed the financial viability of each component, the ICR assessed commercial efficiency of regional utilities, that needed to be assessed under the efficacy section. At appraisal, the Project showed a financial rate of return (FRR) of 14% (PAD, page 21) Operational and Administrative Efficiency: The Project experienced six restructurings, with extensions of Project duration for 51 months over the original closing date. The Project maintained the original Theory of Change (TOC), with adjustments to the results framework. The TOC showed causal chain for suppliers to improve coverage, quality, efficiency, and sustainability of water and sanitation services. Design: Regarding the design aspect, there were two points to note: (1) ambiguity in the Project’s intended impact (refer to the paragraph on TOC and Relevance to Bank Strategies above), and (2) insufficient technical analysis. Technical analysis was not sufficiently conducted at the design stage. It caused delays in Project implementation, procurement, and cost overruns. Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Implementation: Percentage of administration costs out of total Project costs increased from the original estimate of 10 percent to 17 percent, due to Project implementation delays (ICR, page 19). Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 68.70 Appraisal  36.00  Not Applicable 72.70 ICR Estimate  11.20  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives was substantial: the Project was closely aligned to national priorities in the WSS sector, the Bank’s comparative advantages, and the Bank’s country strategies. The Bank’s previous sector experience as the LIL was not enough for the stakeholders to be technically ready for the Project. Efficacy was substantial: three out of the four PDOs were substantially achieved, while one PDO was modestly achieved. The Project contributed to improve accessibility, efficiency, and sustainability of WSS services. Efficiency was modest: the Economic Rate of Return (ERR) was reduced significantly to 11.20 percent at closure. The financial aspects and implementation performance were also modest. On this basis, the overall outcome of the Project is moderately satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The risk to development outcome were high at Project completion. Technical and Operational Risks: Substantial. Significant efforts would be needed to improve meter connections and revenue collection as well as technical maintenance and repair of outfall and pre- wastewater treatment plant. Economic and Financial Risks: High. The regional utilities were heavily subsidized with respect to operational costs (on average, 92 percent of the energy costs and 22 percent of the staff costs for these Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) three CORAAs) (ICR, page 26). If the country experiences an economic shock, the regional utilities could be faced with cuts, thus compromising their ability to deliver services including the operation and maintenance of the sanitary infrastructure built under this Project (ICR, page 26). Institutional Risks: Substantial. The legislative framework to implement the national strategy on water and sanitation was not in place at the Project’s completion. Environmental Risks: Substantial. Despite the Bank’s recommendation, the platform to prevent spillage and soil contamination by solid wastes had not been constructed by CORAAPPLATA (ICR, page 24). Though the Environmental and Social Management Framework (ESMF) was implemented satisfactorily with the Bank’s supervision (ICR, page 24), it was unclear whether its implementation would be continued after the Bank’s supervision ended. 8. Assessment of Bank Performance a. Quality-at-Entry On the positive side, relevance of the Project was high at entry, as the Project was well aligned with the Country Assistance Strategy FY2006-2009, contributing to the pillar on sustainable economic growth (Please refer to Section 3. Relevance of Objectives in the ICR Review). On the negative side, the Project was premature at the time of entry, in terms of feasibility studies, implementation arrangements, and M&E arrangements. The significant shortcomings are discussed below. First, the Bank did not conduct feasibility studies thoroughly at entry. When the Project started, feasibility studies for most of the major construction works had not been completed. For the feasibility study that was conducted, assumptions on cost and physical locations of the infrastructure were made in 1999 (ICR, para 67), which were outdated. Between 1999 and 2013-2014, the cost assumption increased due to the inflation; moreover, the physical landscape changed, requiring a more complex design of the submarine outfall (ICR, para 67). When the project re-estimated the costs for proposed infrastructures in 2013-2014, the estimated costs increased from twice to 2.6 times of the estimate made at appraisal; therefore, the sewerage works in Sosua, Montellano, and the Playa Dorada hotel complex were not constructed due to the lack of resources (ICR, para 67). Moreover, geotechnical survey for the submarine outfall was not conducted in the Project area at the design stage by the design firm. It required the construction firm to conduct the geotechnical survey and re-design the submarine outfall based on the survey results, as the original design had technical shortcomings (ICR, para 68). Second, the Bank did not design implementation arrangements appropriately. From the beginning, the Bank recognized that the PCU’s capacity needed to be strengthened prior to implementation; however, there were no mitigation measures taken to address the capacity gap. Moreover, the implementation arrangement was designed in a way that the technical capacity for the Project heavily relied on the PCU, which was supposed to delegate technical staff to CORAAPPLATA to support the implementation of Component 2 (PAD, page 47). The shortcomings in the implementation arrangements caused the Project to start its implementation without thorough analyses of technical feasibilities and costs, which resulted in delay in the start-up of construction. Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Third, the Bank did not design a strong M&E framework and operational arrangements. There was a discrepancy between the PDO indicators in the Results Framework and the monitoring table in the PAD, which caused confusion on the ground. On balance, the Bank’s performance at entry of the Project was moderately unsatisfactory. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision On the positive side, the Bank provided adequate supervision on safeguard aspects, by deploying financial and environmental specialists in the country. They helped the Project to successfully address the grievance received in January 2018 that triggered OP 4.12 on Involuntary Resettlement (See Section 10.a. on Safeguards) On the negative side, the Project team failed to provide strong supervision initially. For example, the Bank did not conduct supervision missions to meet the implementing agency’s expectation. There was a space of 8 months between the first and the second supervision missions by the Bank, even though the Task Team Leader recognized the PCU’s strong demand for support during the initial period after the Project became effective in February 2010 (ICR, page 25). Over the course of eight years of Project implementation, there was an average of 2 supervision missions per year (ICR, page25). On balance, the Bank’s supervision was moderately satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design There were three notable shortcomings in the M&E design: (1) a discrepancy between the results framework and the monitoring table in the PAD, (2) insufficient outcome-level indicators, and (3) unspecified sources for baseline data. First, there was a contradiction in the PAD in which the PDO-levels indicators in the results framework differed from those in the monitoring table (ICR, page 23). Second, the PDO indicators in Component 1 were keyed to capturing evidence of achievements of outputs, not outcomes. Third, the data sources for baseline data were not specified, which made monitoring of data difficult and raised questions about the appropriateness of the levels of targets. The PDO indicators and the intermediate outcome indicators did not have gender-disaggregated targets. Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) b. M&E Implementation During Project implementation, the PDO indicators for PDO (iii) and (iv) were revised at the restructuring in 2013. The revised PDO indicators weakened their relevance for collecting evidence on the achievement of outcomes. Revisions to PDO indicators were summarized in below:  Indicator. The ratio of total revenue to total operating expenses reaches 70 percent for CORAAPPLATA, including electricity costs - Dropped. As a result, there was no PDO indicator to collect evidence for the financial performances of CORAAPPLATA. The ICR stated that the indicator was dropped because the bank funds were insufficient to make impactful contribution to the achievement of the indicator's target.  Indicator. CORAAPPLATA has completed and made operational a cadastre of consumers covering all its operational areas. – Revised as follows. o Indicator. CORAAPPLATA has identified 100 percent of its potential consumers and has made operational an active registry of at least 40,000 users, covering at least 60 percent of its operational areas. The indicator was revised to include specific quantitative targets on measuring operational performance of CORAAPPLATA.  Indicator. Number of regional utilities signing “Performance and Financial Support Agreements” and meeting operational and commercial performance targets (e.g. targets for non-revenue water, revenue collection rates, and cadastre coverage) outlined in these agreements to participate in the APL II, the target is to have two utilities, other than CORAAPPLATA, meeting these standards. - Revised as follows. o Indicator. Compliance agreements were signed on improving of cadastre, commercial systems, administrative and financial systems of CORAAPPLATA, CORAAMOCA and CORAAROM, as well as a sewerage systems O&M strengthening program for CORAAPPLATA. The indicator was revised to exclude the preparation for the second phase of the program due to the delay in implementation of this Project. Moreover, the portion on meeting operational and commercial performance targets were excluded in the revised indicator, causing the indicator to measure output-level achievement instead of outcome-level achievement. Discontinuation of the Adaptable Program Loan (APL) was apparent in 2015, but the revision of PDO (iv) was not completed until 2017. On the other hand, PDO indicators for PDO (ii) were not revised to align with the reduced scope of the activities from the Puerto Plata region to the City of Puerto Plata, causing discrepancies between targets and completed interventions. c. M&E Utilization The M&E data was utilized to provide evidence for the July 2013 restructuring, which included various adjustments to the results framework given that several indicators were considered overly ambitious and not aligned with the actual interventions (ICR, page 23). The evidence collected from M&E activities Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) were utilized among the PCU and the Task Team, and not widely communicated with stakeholders. Possibilities for utilizing the collected evidence in subsequent Projects were not indicated. M&E Quality Rating Modest 10. Other Issues a. Safeguards The Project was classified as Environmental Category A and triggered the Bank’s safeguard policies on Environmental Assessment (OP/BP/GP 4.01), Natural Habitats (OP/BP 4.04), Physical Cultural Resources (OP 4.11), Involuntary Resettlement (OP/BP 4.12). A comprehensive Environmental and Social Management Framework (ESMF) including environmental impact assessment (EIA) of two identified investments were developed and disclosed during preparation. The ICR noted that the ESMF was implemented satisfactorily with Bank supervision; however, the ICR did not include quantitative indicators on the extent to which the safeguard objectives were achieved (ICR, page 24). OP 4.12 on Involuntary Resettlement was triggered and adhered to. At Project closing, compliance with this OP was rated as Satisfactory and was based on the efforts taken by the PCU to ensure adequate implementation of the Project’s Resettlement Policy Framework regarding the affected community before works began for the submarine outfall and wastewater pre-treatment plant (ICR, page 66). Along CORAAPPLATA’s property line of the pre-treatment plant were two vendors, whose small structures are used as convenience stands and encroach on CORAAPPLATA’s property. The Project built a perimeter wall around the property to ensure safety of the plant and local population. The wall was constructed around the small structures to ensure they remained on the outside of the perimeter. Since the location of the stands did not pose any risk to the vendors themselves nor did it interfere with the wastewater pre-treatment plant’s normal operation, both vendors remained operating their businesses normally until they voluntarily decided to move. The Bank also received a grievance in January 2018 from a complainant, through legal representation, alleging that CORAAPPLATA had seized land for a pumping station. A formal response to the complainant was issued by the PCU. The response was prepared in close coordination with the Project’s Social Safeguards Specialist and the Bank’s Grievance Redress Service (GRS) to clarify that the land in question and related infrastructure were not part of the Bank-financed Project or any previous Bank investment and as such, this matter needed to be resolved between the complainant and CORAAPPLATA. The PCU facilitated a meeting between the two parties and the issue has been satisfactorily addressed (ICR, page 24). b. Fiduciary Compliance Financial Management: Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) At the time of Project closure, the rating was Moderately Satisfactory. Financial Management arrangements were satisfactorily maintained and Interim Financial Reports (IFRs) were submitted in a timely manner. Overall, the Project complied with fiduciary requirements. During Project implementation challenges were faced with counterpart funding that was earmarked to conclude works and supervision of the submarine outfall. This was subsequently addressed by the Government and allocation of the counterpart funds to the Ministry of Economy, Planning and Development (MEPyD) was consigned in the Government’s Fiscal Budget for calendar year 2018. The final audit report for the 15-month period from January 1, 2017 to March 30, 2018 concluded that the PCU complied with the terms of the loan agreement and the applicable laws and regulations and that PCU’s internal controls were overall adequate (ICR, page 25). The ICR team reported that the Project had no issues regarding the timeliness of the external audit and it received a satisfactory rating by the FM specialist at the project closing (TTL meeting, August 21, 2019). The Closing Audit by the Independent Auditors, BDO Audit Firm, SRL, stated: "In our opinion, the States of Disbursement Requests reasonably present the Certificates of Expenses (SOEs) submitted to the World Bank from December 2016 to March 31, 2018. Likewise: (a) these expenditures are eligible to be financed under loan agreement No.7680-DO; (b) the accounting and internal control procedures used in the preparation of SOEs are adequate, and MEPyD has maintained adequate documentation to support requests for reimbursement of expenses incurred; and (c) the loan funds have been used solely for the purposes of the project” (ICR, page 52). Procurement: Procurement was rated as Moderately Satisfactory by the ICR team (ICR, page 25). Low procurement capacity in the PCU caused delay in the preparation and launching of bidding processes for technical and environmental studies during the initial years of Project implementation. c. Unintended impacts (Positive or Negative) There was a positive unintended impact on enhancement of installation of micrometers. The Project provided financial assistance to CORAAPPLATA and CORAAMOCA to install 4,000 and 3,000 micrometers, respectively. Building upon the successful installation of micrometers, the regional utilities entrusted private operators with the expansion of metering and the commercial aspects of their distribution system (ICR, page 14; footnote 22, page 16). CORAAPPLATA’s activities contributed to an increase in the volume of metered water sold from 36 to 54 percent along with a reduction in non-revenue water from 57 to 45 percent between 2015-2017 (ICR, page 14). The coverage of metering increased from 4 percent to 42 percent between 2012-2017 for CORAAMOCA (ICR, para 41). d. Other --- 11. Ratings Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons IEG summarizes below the lessons identified by the ICR team with modifications: 1. Adequate feasibility analyses at the design stage need to be conducted, as they can critically affect project outcomes. When the submarine outfall was designed, the feasibility study and the geotechnical survey were not adequately conducted. It caused the design to be re-done after the construction work has started. As a result, the Project needed to reduce its scope of activities due to lack of time and resources to implement them. 2. The realism of achieving development objectives within the project duration needs to be rigorously assessed, as lack of realism can deter efficient implementation of project activities. It was essential to adequately access institutional capacity and realism to design the Project’s activities, so that they could be implemented, and their outcomes measured, within the original project duration of three and a half years. 3. PDO indicators should be directly relevant to measuring outcomes that are attributable to project interventions, and need to be monitored accurately to generate adequate and robust evidence on efficacy. The Project’s PDO indicators, which lacked relevance or attribution to the Project, could not provide sufficient evidence to prove the Project’s achievements in the outcome objectives. 4. Failing to address capacity constraints at the early stages of project implementation can critically affect project outcomes. This Project did not implement remedies to improve stakeholders’ capacities in the beginning of the Project, which became one of the main reasons for delays in project implementation. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was concise and candid in presenting the Project’s achievements. The ICR appropriately pointed out the shortcomings of the revised PDO indicators to collect evidence on the achievement of outcomes. Moreover, the ICR made a clear effort to support the efficacy of the project by providing evidence that were not collected Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DR Water & Sanit in Tourist Areas (P054221) by the PDO indicators (ICR, page 37-38). The efficiency section was clearly argued, yet the financial analysis had limited consistency with the appraisal. The ICR used a different method from the PAD to conduct a financial analysis; thus, the results of financial analysis were not comparable. Other parts of the document provided a logical and clear structure. a. Quality of ICR Rating Substantial Page 21 of 21