Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) Report Number : ICRR0020312 1. Project Data Project ID Project Name P094311 EG INTEGRATED SANITATION & SEWERAGE INFR Country Practice Area(Lead) Egypt, Arab Republic of Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-75120,TF-95516 30-Jun-2014 201,500,000.00 Bank Approval Date Closing Date (Actual) 20-Mar-2008 31-Dec-2015 IBRD/IDA (USD) Grants (USD) Original Commitment 120,000,000.00 2,911,175.00 Revised Commitment 120,000,000.00 2,379,458.46 Actual 98,413,718.70 2,379,458.46 Prepared by Reviewed by ICR Review Coordinator Group Ebru Karamete Fernando Manibog Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives This Review covers both Integrated Sanitation and Sewerage Infrastructure Project (P094311) and the Trust Fund portion (P119805). The same project development objectives were used in the Loan Agreement (p. 5) and the Project Appraisal Document (PAD) (p. 4): “to contribute to the sustainable improvement in: (a) sanitation and environmental conditions for the resident communities; and (b) the water quality in the selected drainage basins within the served areas”. Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? --- d. Components The project had three components: 1. Provision of sanitation systems within selected drainage sub-basins (Appraisal Estimate: US$172.4 million , Actual: US$140.3 million) Prior to the restructuring, this component supported the construction of centralized sanitation systems to serve 222 large villages and the development of decentralization sanitation systems for around 120 small villages. It also supported a German-financed pilot to introduce natural wastewater treatment technology. The coverage of this component was significantly reduced in July 2012 to cover only 106 large villages and 30 small villages. The restructuring also removed the pilot activity. 2. Establishment of a local result-based monitoring and evaluation system. (Appraisal Estimate: US$1.0 million, Actual: US$0.5 million) This component supported the establishment of a results-based monitoring system to measure the impact of the improvements in sanitation coverage on environmental and water quality. This included support for the design of the system as well as support for the necessary infrastructure. Due to the lack of grant funding, the monitoring and evaluation activities under this component were scaled back significantly. 3. Institutional development and capacity building (Appraisal Estimate: US$18.5 million, Actual: US$8.5 million) This component supported a variety of capacity-building activities, including support for project management, capacity building, and training. Due to the lack of anticipated levels of grant funding, the scope of work for this component was reduced significantly. Studies planned under the component, including water quality modeling, identification of solid waste sources, and septage management were dropped. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Total project cost at appraisal was estimated at US$ 201.5 million (including physical and price contingencies). However, the actual costs were US$ 149.3 million, or 74 % of the appraisal estimate. The underspending was because of slow implementation due to design, capacity and commitment issues. Financing. At the time of appraisal, the project included a US$120 million loan from the World Bank (IBRD-75120). The World Bank support amounted to US$98.4 million (82 percent of planned amount). The appraisal also included US$20.5 million from the governments of Germany and the Netherlands although this financing was not provided in the end. The ICR did not provide the reasons for the Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) withdrawal of these funds. These funds were exclusively funding capacity building activities under Component 2 and 3, and there were serious bottlenecks in implementing these activities impacting overall project outcomes. Post -appraisal the project added US$11.9 million in parallel financing from the governments of Japan and Switzerland; at closing this was $9.0 million (75 percent of original amount). Borrower Contribution: The Borrower contribution was estimated at US$ 61 million at the time of appraisal. At the time of closing, the government contribution was US$16.2 (27 percent of planned amount). Dates: The closing date of the operation was extended for 18 months to Dec 31, 2015, through a Level II restructuring in July 2012. This extension was necessary to enable the completion of the delayed activities. Restructuring: The project went through three Level-II restructurings on July 2, 2012, September 2014 and June 2015. The first restructuring in 2012 aimed to adjust components and extend the closing date, while the second restructuring increased Bank funding percentage to 100 percent in order to hasten the disbursements, and the final restructuring revised some of the indicators. 3. Relevance of Objectives & Design a. Relevance of Objectives The project's objectives were substantially relevant both to the government's strategy and the World Bank's support for Egypt. Specifically, the World Bank's 2009 CAS included specific outcomes related to the access and quality of service delivery and improved water management. This was reaffirmed in the interim CAS approved in 2011. The current Country Partnership Strategy (2015-2019) continues to focus on higher quality and greater access in rural sanitation, as demonstrated by a follow on project with a similar set of objectives. However, the PDO was over-ambitious, which is described in the next section. Rating Substantial b. Relevance of Design Although this was the first sanitation intervention of the Bank in Egypt, it had quite a complex design with the use of innovative technologies, involvement of two implementing agencies, and an ambitious objective. The project design logic was weak, as the project activities, their design and budgeting of funds were not aligned sufficiently to achieve the over-ambitious project development objective,” to contribute to the Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) sustainable improvement in: (a) sanitation and environmental conditions for the resident communities; and (b) the water quality in the selected drainage basins within the served areas”. The shortcomings were: (i) The focus on water quality in selected drainage basins was less clear. This required close coordination between multiple ministries, and agencies at national and local levels, which was missing in project design. Furthermore, the M&E systems to measure attributable achievements was not established from the start. (ii) The project included interventions to improve the sanitation and environmental conditions for communities, through increased coverage of centralized and decentralized sanitation infrastructure systems and wastewater treatment plans. This was supported by efforts to improve the technical capacity of the different implementing agencies. However, there was little focus on how households would connect to the system or how much they would pay. Social assessment and citizen engagement was non-existent, therefore no consultation took place before construction. This led to significant problems with convincing households to connect to infrastructure built by the project, as well as land acquisition. (iii) Certain key components on institutional development central to the success of the project were exclusively dependent on external trust funds. This severely impacted the project when the initial donors withdrew. (iv) Technical design included use of untested innovative technologies, which were not compatible with the local laws, i.e., the engineering code of practice; therefore designs had to be revised later on. Also, no detailed designs were prepared during the planning stage. Both of these issues resulted in underestimating the real cost of works. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improvement in sanitation and environmental conditions for the resident communities, rated Modest. Rationale Outputs • The project established two new wastewater plants, compared to a target of four new plants. • Total length of all networks (gravity and house connection) constructed (installed and approved) under the project including sewage collection networks (main lines and branches) exceeded targets (440 km actual compared to a target of 140 km). Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) • Out of the planned 19 centralized wastewater collections systems in Phase I, only nine projects was completed by the time the project got closed. Under Phase 2 two new wastewater treatment plants and 58 network systems was planned to be constructed but at the project closing date no projects in Phase 2 were completed and no new house connections were established. • Decentralized sanitation systems were included under the second phase, with construction starting only in early 2015. The PAD estimated that 120 villages would be served, but this was scaled reduced to 30 villages during restructuring. At the closing date of the loan, no decentralized systems were completed and no new house connections have been established for these systems. By the time the project was closed, 14 decentralized sanitation projects were under construction and scheduled to be completed by late 2016. • The intermediate outcome indicator of 25 decentralized systems with at least 70% of households contributing to O&M costs, was not achieved, as no decentralized systems were operational by the time the project was closed. • 130 employees from WSCs, NOPWASD and HCWW received training for at least five days on different aspects (on financial, organizational, social, and technical aspects). This exceeded the target of 25 people, however considering the involvement of central and decentralized institutions and communities, the target and allocated time is very limited. The ICR (p.17) reported that capacity building for the Regional Sanitation Units (RSU) and Community Development Associations (CDA)-responsible for operating and maintaining the decentralized sanitation systems- could also not be completed. Although some 28 CDA members received training, training programs had to stop when the project was closed. It is critical that continued support be provided for the sustainable operation and maintenance of these decentralized systems by the local communities. Outcome: Connecting villages and households to the water systems is an important necessary step to improve the sanitary conditions in communities. The project significantly under-delivered in terms of connections. Under the project's initial design, 69,000 households were expected to be connected to centralized systems while only 13,300 households were actually connected. Likewise, the project expected to connect 6,500 households to decentralized systems, while none were actually connected by the end of project. Overall, the project provided 66,500 people with improved sanitation facilities compared to a target of 379,500 people. There were two original indicators that were included in the PAD to measure achievements under this objective, but the ICR reported limited results on these based on a beneficiary survey: (i)percentage of target population expressing satisfaction with local environmental conditions associated with improved wastewater management; and (ii) percentage of households practicing improved hygiene behavior as measured through qualitative research. The ICR (p. 20-21) reported that based on a beneficiary survey that was conducted with beneficiaries of 7 centralized systems (also 30 villages for decentralized systems but none of the works were completed during that time), 89% of the households connected are satisfied with the level of service they received. 63% of interviewees reported they were satisfied with the consultation process before and during construction. However, other anecdotal data contradicts these figures -- informal discussions in some villages indicated significant dissatisfaction with the quality and speed of construction. 76% of survey respondents recall receiving hygiene messages during the last three months from different sources, including directly from the project but also from community health workers and CDAs. 61% of respondents were aware that washing hands with soap and water will prevent diarrhea. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improvements in the water quality in the selected drainage basins within the served areas, rated Modest. Rationale Output: The outputs were the same with the outputs of the first objective. Outcome: The interventions to improve sanitary conditions at the village and household level certainly contribute to the overall water quality in the drainage basins. The outcome indicator “the reduction in the pollution load in local water bodies (tons of biological oxygen demand-BOD per annum) targeted a reduction of 986 metric tons per year of BOD; in reality the reduction was only 539 tons. While this is an important reduction, this was not estimated through direct measurement of receiving water bodies, but calculated based on number of households connected. Therefore, this one measurement of pollution does not adequately capture the extent of water quality improvement. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Efficiency Economic and Financial Efficiency: The ex-ante analysis included a cost-benefit analysis by using capital plus operating unit costs, and private and public unit benefits that were estimated. On this basis, the project’s ERR was found to vary between 10-15 percent using low-case and high case assumptions. Also, the cost per beneficiary was estimated at US$146. Based on the ex-post analysis the project did not provide good value for money to the borrower or to the World Bank. The unit cost was extremely high at US$705 per beneficiary (page 37 of the ICR). For comparison, the estimated average for rural MENA is US$ 108/capita. The cost over-runs were mainly due Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) to change in engineering designs when detailed designs were made, as well as weak capacity of contractors and weak supervision. Most other planned outputs were also substantially more expensive than originally estimated or simply not produced. For example, the results-based M&E system was not established. The ICR estimated the rate of return of the project, based on a set of assumptions and methodology similar to the appraisal analysis. The estimates ranged from 0.9 percent to 9.5 percent; with the lower estimate presenting a more realistic figure. These figures clearly shows that the project was economically inefficient. Operational Efficiency: There were significant operational and administrative inefficiencies mainly due to complex implementation arrangements involving two implementing agencies resulting in lack of coordination, loss of accountability and thereby serious delays in implementation. The closing date was extended for 18 months to allow time to complete sub-projects that faced delays. However, the 18-month extension did not deliver expected results and 17 percent of the Loan was cancelled. In addition, several centralized systems and none of the decentralized systems were completed at the time that the project closed. It was expected that these systems would be completed with government funding. Project efficiency is rated Negligible, due to the significant economic, administrative and operational inefficiencies. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  10.00 Not Applicable 0 ICR Estimate  .90 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project is rated unsatisfactory. This reflects a substantial rating for the relevance of objectives and a modest rating for the relevance of design as well as two modest ratings for the two sub-objectives for efficacy. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) Finally, the efficiency rating is negligible, reflecting the high cost relative to the modest outcome. a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating The risk to the sustainability of project investments is high. Sustainability of Investments. There were quality issues with several works finished under Phase I, thus putting the sustainability of these investments at risk. Also, unfinished works were left to be completed by the government's funds, which is likely to face delays. The sustainability of the decentralized systems is even less clear. None of these systems were finished by the time the project closed in December 2015. This was done under the assumption that the government would finish the construction of these systems. At the time that the ICR was submitted (June 2016), it was not clear if the government would provide the necessary financing for these systems (ICR p. 7-8, 13). In addition, the capacity development support provided by the project to the NGOs who are responsible for O&M of these decentralized systems, was left unfinished. This is likely to risk sustainability of these investments if continued support is not provided to the NGOs. Sustainability of WSCs. The current tariff levels do not cover costs, and the WSCs will require further support to effectively manage wastewater services. The ongoing Program for Results is supporting efforts to address these challenges, and the GOE remains committed to the rural sanitation agenda, with plans to increase tariffs at cost recovery levels in 5 years, but if these issues are not addressed within the near future, the sustainability of ISSIP 1 outcomes will continue to be at risk. (ICR p. 13, 20). a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance a. Quality-at-Entry a . Quality-at-Entry There were significant shortcomings at Quality at Entry, which are as follows: • The Bank developed the project building upon background analysis and a number of technical Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) missions. While the World Bank did attempt to follow the government's agenda, the design was overly ambitious. This included a complex design that included two implementing agencies. • The risk matrix downplayed many of these risks as "moderate" without significant mitigation measures (pages of 12-13 of the PAD). The project design exclusively depended upon contributions from other development partners for essential capacity building activities; many of these contributions were not forthcoming. Yet this was identified as a moderate risk in the PAD (page 13). • Citizen engagement was missing. The project design did not anticipate complicated issues associated with land acquisition nor the difficulties in having households pay connection fees and water tariffs (page 23 of the ICR). While the PAD did include a land resettlement (acquisition) plans, this does not appear to include an estimate of the extent of the needs. These issues occur in similar projects and could have received greater attention. • Detailed designs for works were not completed before implementation. In addition, technical design included use of untested innovative technologies, which were not compatible with the local laws, i.e., the engineering code of practice, therefore they had to be changed, and conventional systems had to be used. This made the systems costly, and also considerably delayed implementation. • Lack of capacity of contractors was not anticipated during design. This also brought serious implementation delays. • M&E framework had many shortcomings, including insufficient indicators, lack of baselines, and monitoring frameworks. Quality-at-Entry Rating Unsatisfactory b. Quality of supervision There were serious shortcomings in the World Bank's supervision of the project: • The project had a number of changes in its task team leaders, including soon after effectiveness. The World Bank's supervision was not proactive to address implementation issues and was slow in responding to delays, particularly in the first years of the project. This also contributed to a low level of candor in reports, with the project receiving a positive internal rating from 2008 to 2012, despite a number of clear delays. • While the government and the World Bank agreed to a restructuring in 2012 (one year after the Revolution), this restructuring did not address many of the underlying issues, particularly bottlenecks created by the dual implementing agencies. Likewise, it did not change the PDO although it was clear that the project could not adequately implement all of the elements contained in the PDO, particularly those related to sub-objective 2. • One key issue was that the project closing date was not extended further to allow for completion of the activities, many of which at critical stages, in spite of the project gaining momentum in the last year of its Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) implementation and considering the country’s difficulties with the revolution, which at that time was slowing down most of the activities. During a case study mission by IEG in November 2016, report by the country unit revealed that the decision of not extending ISSIP I was done based on a review of the overall portfolio of projects in Egypt, and projects that did not disburse as planned were cancelled/not extended in order to encourage the counterpart toward rapid implementation. While it is understood that the Bank does not aim to create stretched implementation periods and constant expectation by stakeholders for extensions, this decision failed to consider the fact that infrastructure projects take time to plan and design and once construction starts, disbursements increase considerably. Many parties interviewed during the case study mission mentioned that had the project been extended, more works would have been completed and probably the project outcome could have been more favorable. The decentralized schemes particularly were negatively impacted as the capacity building of the NGOs were left unfinished. • There was also a problem with timing of the follow-up operation. Two years into the implementation of ISSIP 1, ISSIP 2 was approved with similar issues in design, requiring a single PIU to manage both projects, thus stretching its capacity, despite the fact that ISSIP 1 had yet to achieve any results. The project may have seen stronger results if effectiveness had been delayed until the approach had been tested, and lessons learnt were drawn fully. Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance At the highest levels, the government had strong commitment to achieving the development objectives. This reflected the project's close alignment to government strategy. However, the government supported inclusion of the implementing agency NOPWASD, in addition to the Holding Company, thus creating serious bottlenecks in implementation. The government did not borrow for technical assistance activities, despite the importance of this "soft" support in a complex project and this was particularly important after several of the donors dropped out of the project. As the project closed in 2015, it was unclear if the government would provide sufficient counterpart funds to finalize many of the investments that were supported by the project (page 25 of the ICR). Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance There were serious shortcomings in the performance of the different implementing agencies. There was Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) limited coordination between the two main implementation agencies. This led to overlapping processes as well as delays with procurement and compliance with safeguards policies. While the Steering Committee operated, it did not appear to be effective in coordinating between the implementation agencies (page 125 of the ICR). While the project management teams were established in a timely fashion, there were delays in preparing draft bidding documents; according to the PAD they should have been prepared prior to effectiveness (January 2009), but the documents were only finalized by April 2009 (page 8 of the ICR). In 2013, an internal review rated the performance of the two project management teams (one for each primary implementing agency) as unsatisfactory. This reflected the lack of coordination between the two teams and subsequently between the two agencies. The restructuring in 2012 produced a set of more streamlined processes that led to faster disbursement. Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design Due to lack of grant funding, there was no work done in intially developing the M&E system. During restructuring, the PDO indicators were revised, and a basic M&E system was designed to track the number of households connected. Rather than directly measuring treatment plant effluent, water quality monitoring in the selected drains was calculated based on the estimated reduction per person in the effluent pollution loads entering the water bodies, multiplied by the number of connections. However, BOD reduction calculated from the number of household connections is not a suitable indicator for the water quality of the receiving water body. The water quality is affected by multiple factors. b. M&E Implementation No baselines or targets were established for the PDO indicators, and no results were tracked until the M&E indicators were redefined in July 2012. Following restructuring, the consultants funded through Swiss State Secretariat for Economic Affairs and Japan Social Development Fund developed a more comprehensive monitoring system that tracked and reported on results on a regular basis. c. M&E Utilization Once established, the M&E system was primarily used for project management of the works, since the primary focus of the reporting was on disbursements and construction progress. Once treatment plants are completed and households connected, the M&E system is unlikely to be able to monitor progress on the Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) broader objectives of improved environmental conditions and water quality. In addition, monitoring the number of households connected was incomplete. M&E Quality Rating Modest 11. Other Issues a. Safeguards Environmental Safeguards. The project was classified as a Category B project, having triggered Operational Policy (OP) 4.01, Environmental Assessment. Due to the nature of the project design, both environmental and social assessments followed a framework approach. At appraisal Environmental and Social Impact Assessment Framework (ESIAF) report was prepared to present an Environmental and Social Management and Monitoring Framework (ESMMF) which included mitigation measures, monitoring plan, and institutional aspects. The ESAIF found the project to have major positive environmental and social impacts, resulting from the overall improvement of the water quality in a number of drains and canals in the project command area (PAD p. 21). The ICR (p.12) reported that during implementation, project staff responsible for environmental management lacked the appropriate training, resulting in significant delays for the Environmental and Social Impact Assessment (ESIA), thus postponing the first phase of construction. Although the midterm review emphasized the importance of bringing in a full-time expert to address these issues, an environmental specialist was hired only in late 2013. The ICR did not report on environmental risk mitigation activities. Social Safeguards. The possibility of resettlement during land acquisition for construction of the treatment plants triggered OP 4.12, Involuntary Resettlement. A Resettlement Policy Framework (RPF) establishing overall resettlement objectives and principles was prepared, including criteria for the screening of subprojects, guidelines for mitigation and compensation, implementation arrangements for the RPF, and a capacity assessment. The ICR (p. 12) reported that the PIU initially struggled to apply social safeguards. Numerous supervision missions highlighted the need to pay closer attention to potential resettlement issues. Despite this, a social safeguards specialist was not hired until late 2013. No specific resettlement action plans were developed during the project, as land acquired under ISSIP 1 was donated by the beneficiary communities or purchased through a willing buyer, a willing seller process. b. Fiduciary Compliance Financial management (FM). The ICR (p. 13) reported that GOE was in compliance with all fiduciary covenants. Most reports were submitted on time, but there was a delay of over a year in launching a computerized accounting and financial management system, which was necessary to streamline processes and improve data quality. As a result, the PIU was unable to provide bidding documents in a timely manner until the system was in place in July 2013. A subsequent FM review noted that institutional arrangements for the project slowed disbursements, due to an excessively lengthy payments approval and processing cycle. Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) Procurement. Due to NOPWASD’s and HCWW’s unfamiliarity with World Bank guidelines, combined with the complex institutional arrangements for ISSIP 1, procurement was a critical bottleneck to implementation. Over the first several years of the project, procurement was significantly delayed due to a lack of clear responsibilities, disagreements over bidding approaches, processing delays and overlapping audit systems. The use of pre-qualification (against World Bank recommendations) also significantly delayed procurement. Following efforts to simplify procurement procedures and eliminate redundant approvals and reviews, an independent procurement review in 2012 found procurement processes to be satisfactory. c. Unintended impacts (Positive or Negative) Unintended impacts that were reported were not relevant. d. Other Poverty: The project was not specifically targeted at poor households. If the works are completed as planned, poor households are likely to benefit through reduced expenditures on cesspit emptying. In some of the villages that are already connected, there was initial evidence of housing values improving due to access to sewerage. There is also a risk of negative poverty impacts under ISSIP 1 if the remaining works are not constructed, households are not connected, or the works are not adequately maintained. Gender. There is some evidence of positive gender impacts, particularly in communities where decentralized systems are being built. Women were actively engaged as CDA members and consulted during the construction process. A beneficiary survey conducted in 2015, found women surveyed had the same level of satisfaction as men about the information they obtained on the decentralized systems and the way they were consulted in the course of the project. The women surveyed report that they were actively engaged during the construction process. 36% of respondents reported a problem to the CDA or WSC during construction in the decentralized communities using the official complaint process. The number of complaints filed were equally divided between men and women. 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Institutional and project risks Risk to Development make the risk to the Substantial High Outcome sustainability of the project outcomes high. Bank Performance Unsatisfactory Unsatisfactory --- Moderately The unsatisfactory Borrower Performance Unsatisfactory Unsatisfactory performance of the Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) Implementing Agency aligned with the overall Borrower Performnace rating with the overall Outcome rating (both unsatisfactory). Quality of ICR High --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons This Review finds the following lessons: • Infrastructure projects may result in more positive outcomes if sub-projects are given sufficient time to finish the works that have already started. The ISSIP 1 experience demonstrated that had the project been given an extension, the works that started under Phase 2 as well as the capacity development activities that were still ongoing, could have been finished. The Bank’s decision to close the project based on an overall portfolio review did not consider the fact that many bottlenecks were resolved and the project started to disburse rapidly during the last year. Although the unfinished works were left to be completed through government funding, the availability of this funding was not certain. • Linked with the earlier lesson, follow-on projects need to be designed after the earlier project design is tested sufficiently, so that its lessons can be brought into the next Project. Before completing ISSIP I, ISSIP II project came into effect with similar design bottlenecks. Thus, two problematic projects were created back to back without any opportunity for improvement. The ICR provided comprehensive lessons. The most important follow with some modification of language: • Complex project design jeopardizes project outcomes. ISSIP 1 addressed rural sanitation as both a means to improved water quality and an objective in its own right. Additionally, the project design included multiple innovations, and required high levels of coordination across government agencies and funding from multiple stakeholders. Institutional arrangements were weak involving two institutions without clear split of responsibility and ownership arrangements. A borrower unfamiliar with World Bank procedures working in a challenging sector is unlikely to be able to effectively carry out such a complex project. A simpler project design, and better institutional arrangements adapted to the prevailing context would have had a greater Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) likelihood of success. • Technical assistance and policy reform are critical for sustainability of infrastructure projects and require stable funding. ISSIP 1 provides additional evidence that infrastructure cannot be leveraged effectively outside of a supportive policy framework and institutional environment. Although the PAD identified critical capacity and policy constraints, funding issues and delays in launching Components 2 and 3 pushed TA to the final years of the project. Given the long time frame required for policy reform and capacity building, this TA would have been significantly more effective if it began well before construction started. Had the TA been funded through the IBRD loan or a global trust fund, this delay may have been avoided. • Citizen engagement is a necessary step that needs to be integrated in infrastructure projects. By engaging with communities earlier in the project, several problems could have been avoided, including the delays in land acquisition, villages blocking construction, and unwillingness to pay connection fees. Difficulties in identifying ownership, negotiating land prices, and navigating clearances from multiple ministries resulted in long delays in land acquisition. Addressing land acquisition prior to effectiveness could have reduced the risk of implementation delays. Until the JSDF-funded consultant began work, there was minimal outreach to communities impacted by the project. More community consultation could have helped build greater support for the project, which was initially opposed by some villages. • Effective procurement requires an understanding of contractor capacity, as well as sufficient contract supervision. Many of the contractors eligible to bid on the sub-projects lacked the financial resources and technical ability to effectively carry out the complex technologies specified in the bidding documents. This resulted in multiple rounds of bidding, major cost overruns, and delays in beginning construction. Moreover, awarding two or three contracts to the same small contractors constrained their financial capacity to complete the respective contracts, which resulted in significant delay in completion of their contracts. Prequalification introduced to exclude low-capacity contractors for Phase II schemes resulted in bid prices significantly higher than the cost estimate, leading to rebidding, which significantly delayed the procurement process. Also poor or lack of contract supervision during Phase I of the project, also caused delays as well as poor quality works in some cases. 14. Assessment Recommended? No Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG INTEGRATED SANITATION & SEWERAGE INFR (P094311) 15. Comments on Quality of ICR The ICR was well written, quite comprehensive and clear with candid articulation of implementation challenges as well as good formulation of lessons. It also provided adequate evidence to support its ratings. One missing point was that the ICR did not include any risk mitigation activities for the environmental safeguards. a. Quality of ICR Rating High Page 16 of 16