Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) Report Number: ICRR0021780 1. Project Data Project ID Project Name P115695 CN-Bayannaoer Water & Env. Comp. Manage Country Practice Area(Lead) China Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-80470 31-Dec-2017 66,012,411.52 Bank Approval Date Closing Date (Actual) 24-May-2011 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 80,000,000.00 0.00 Revised Commitment 72,080,000.00 0.00 Actual 66,488,226.43 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Dileep M. Wagle Ramachandra Jammi IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The original Project Development Objective (PDO) as stated in the Loan Agreement (Schedule 1, page 5) and the Project Appraisal Document (PAD, page 2): To support Bayannaoer: (a) to better utilize water resources to develop higher income activities for local residents without compromising the water use for irrigation, which would adversely affect the Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) rural poor; and (b) improve the water environment of Wuliangsuhai Lake by reducing water pollution entering such lake. The revised PDO as stated in the restructuring paper approved on May 24, 2014 (paragraph 14): (a) to better utilize water resources without compromising the water use for irrigation; and (b) to improve the water environment of Wuliangsuhai Lake by reducing water pollution entering such lake. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 24-May-2014 c. Will a split evaluation be undertaken? Yes d. Components There were four components (PAD, page 3). 1. Reclaimed Water Supply. The estimated cost at appraisal was US$72.3 million. The actual cost was US$36.8 million. This component planned to finance construction of four new Water Reclamation Plants (WRPs), raw water transmission mains and distribution networks in four industrial estates (Wuliangsuhai, Ganqimaodu, Hanjinhouqi and Wuyuan) for sourcing water reclamation from the tertiary treated wastewater at the industrial estates and from nearby agricultural drainage canals. The scope of this activity was reduced to two WRPs and the construction of WRPs at Wuyuan and Hanjinhouqi industrial estates was dropped during implementation (discussed in section 2e). 2. Wastewater Management. The estimated cost at appraisal was US$52.1 million. The actual cost was US$44.6 million. This component financed construction of three Waste Water Treatment Plants (WWTP) at Wulateqianqi, Ganqimaodu, and Wulatehouqi industrial estates) and secondary and tertiary treatment and distribution networks for recycled water in Ganqimaodu and Wulatehouqi industrial estates. 3. Constructed Wetlands, Non-point Pollution Control Pilot, and Canalization of the Wuliangsuhai Lake. (While point sources of pollution refer to pollutants from discharge pipes such as from factories and sewerage treatment plants, non-point pollution sources refer to pollutants from a large area, such as water emptied into streams or rivers after accumulating contaminants from sources like gardens, parking lots or construction sites). The estimated cost at appraisal was US$33.4 million. The actual cost was US$24.1 million. There were three sub-components: (i) construction of wetlands at the inlets of Wuliangsuhai Lake for reducing pollution coming into the lake, by intercepting agricultural runoff and industrial wastewater: (ii) Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) construction of grids/canals within the lake to improve water circulation: and (iii) implementing a pilot program for upgrading agricultural non-point pollution control measures. 4. Technical Assistance (TA). The estimated cost at appraisal was US$2.7 million. The actual cost was US$1.4 million. This component provided TA for: (i) assisting the Bayannaoer Hetao Water Affairs Group Company Limited (BWAG) in finalizing designs for water/wastewater processes and related bidding documents, managing contracts and preparing monitoring reports: (ii) developing an industrial clean production mechanism for pollution control: (iii) preparing a macro-level strategic business development plan for BWAG: and (iv) training BWAG staff in financial management and operation and maintenance of facilities and wetlands. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated cost at appraisal was US$164.5 million. The estimated cost was revised downwards, due to the reduction in project scope during restructuring (discussed below). The revised estimate US$119.0 million. The actual cost was US$114.0 million. Project financing. The project was financed by an IBRD loan of US$80.0 million. US$8.0 million of the loan was cancelled during the second project restructuring (discussed below). The revised estimate of the loan was US$72.0 million. The amount disbursed was US$66.4 million. There was parallel financing from the United Nations Development Program (UNDP) for complementary activities of improving water quality and introducing a green development model for local residents around the Wuliangsuhai Lake. Borrower contribution. The borrower contribution was estimated at US$84.5 million at appraisal. The revised estimate was US$47.0 million. Their actual contribution was US$47.6 million. Dates. This project approved on May 24, 2011, became effective on October 27, 2011 and was scheduled to close on December 31, 2017. The project closed a year behind schedule on December 31, 2018. Other changes. There were two Level 2 restructurings. These changes were made with the first restructuring on June 2015.  The project scope was reduced. Two of the four WRPs were dropped in response to the low industrial water demand, resulting from the fall in the global and Chinese demand for coal. As a consequence, the WRP facilities to be created under the project were no longer needed. The capacity of the Reverse Osmosis (RO) systems at the remaining two WRPs (Wulatehouqi and Ganqimaodu) were downsized in response to the low demand for reclaimed water at the industrial estates.  Two new outcome indicators - reduction in the volume of Chemical Oxygen Compound and Nutrient load reduction - were added, as the original indicators were deemed to be not sufficient to measure the project's contribution to reducing pollution entering the Wuliangsuhai lake.  The original target value for the outcome indicator "reduction rate of groundwater abstraction from self-wells" was reduced from 90% to 50%.  The project cost was reduced for a combination of factors, including dropping of the two WRPs and savings from competitive bidding on some contracts. The allowable disbursement percentage for the Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) Works expenditure category from the Bank loan was increased from 49% to 90%. This led to the reduced counterpart funding from US$84.5 million to US$47.0 million. These changes were made through the second restructuring on December 2017.  The two clauses from the original PDO, "to develop higher income activities for local residents" and " adversely affect the rural poor" were dropped. Developing alternative water resources through treatment and recycling was meant to facilitate industrial growth and support higher income activities. This was no longer possible in the wake of the economic downturn of the industrial sector in the project area. The second clause was dropped as the actual industrial water demand did not compete with water use of the rural poor.  The three WWTPs were to be retrofitted with low-flow (Reverse Osmosis) systems in view of the lower inflow conditions under the changed macroeconomic context.  US$7.9 million of the Bank Loan was cancelled due to the downsizing of the capacity at the remaining two WRPs (at Wulatehouqi and Ganqimaodu), given the low demand for reclaimed water at the industrial estates.  The percentage of counterpart funding was reduced by over half (56%), and the share of Bank funding was increased from 49% to 90%. (ICR, paragraph 22).  The closing date was extended by a year from December 31, 2017 to December 31, 2018, to facilitate completion of the installation of the low flow systems. Split rating. Given the change in PDO, this assessment is based on a split rating of objectives, with 63% of the Bank loan having been disbursed before restructuring and the balance 47% after restructuring. 3. Relevance of Objectives Rationale Country context. Despite rapid economic development and poverty reduction in China before appraisal, there were vast regional disparities and environmental challenges relating to water pollution. Bayannaoer Municipality is a lagging municipality in the Inland Mongolia region, with an economy dependent on the agricultural sector. It is also rich in mineral resources, especially coal. The municipality was eager to diversify and promote local industries, by developing the coal to chemicals, metallurgical and food- processing industries. These industries however would require significant water resources and also generate wastewater pollution. The objective of securing "alternative water supply" through reclaiming water from agricultural drainage and treated domestic/industrial wastewater for developing local industries and arresting pollution discharges into Wuliangsuhai lake was important for the municipality. Government strategy. The PDOs were aligned with the national objectives, articulated in China's 12th Five Year Plan for 2011-2015 at appraisal and the 13th Five Year Plan for 2016-2020. The 12th Plan identified green development, resource conservation and environmental protection as national priorities. The 13th Plan identified the need for balancing growth with environmental issues and specifically highlighted the need for comprehensive water quality management and protection of Wuliangsuhai Lake. In Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) 2012, China issued the "Most Stringent Water Resources Management System", aimed at adopting strict measures to control total water use, improve water use efficiency and control water pollution. Alignment with the Bank strategy. The PDOs continue to be well-aligned with the Bank strategy. At appraisal, the project was consistent with two pillars of the Country Partnership Strategy (CPS) for 2013- 2016: (i) reducing poverty and inequality and promoting social inclusion: and (ii) managing scarce resources and environmental protection (CPS, page 13). The World Bank had implemented several water supply and waste water projects in China in the past, including one project in the Bayannaoer municipality (Northern Irrigation Project), which assisted the government in rehabilitating irrigation areas. With rapid economic development in the project area, the regional priorities had switched from agricultural water management to a more comprehensive water and environment management. The Bank responded to the changing regional priorities, given its global experience with integrated water and environmental management issues. The PDOs were relevant to the Country Partnership Framework (CPF) issued in 2019, for the 2020-2025 period. The second engagement area of the CPF underscored the need for promoting greener growth through among other things, reducing water pollution and strengthening sustainable management of natural resources (CPF, page ii). Despite the relevance of the PDO to the government and Bank strategy, one of the original PDOs, "developing higher income activities for local residents" turned out to be unrealistic, given the project time frame. It was also inappropriate, given that none of the project activities specifically aimed at developing higher income activities. Further, the clause "without adversely affecting the rural poor" was unclear, given that the industrial demand for water did not compete with water use of the rural poor in the project area. Unlike the original PDO, the revised PDO was focused on outcomes that were achievable with the scope of the project. The revised PDO is rated as substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To better utilize water resources to develop higher income activities for local residents without compromising the water use for irrigation, which would adversely affect the rural poor Rationale Theory of Change. The causal links between project activities, outputs and outcomes were logical. Construction of Water Reclamation Plants (WRPs) and facilities to reuse wastewater from agriculture drainage canals and new Waste Water Treatment Plants (WWTPs) at the industrial estates, was aimed Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) at increasing alternative water sources for developing local industries. Activities such as constructing wastewater treatment pipelines, wastewater collection networks, wetlands at Wuliangsuhai Lake inlets, implementing non-point pollution control pilot and constructing canals/grids, were aimed at reducing the amount of industrial and non-point agricultural wastewater pollution in the lake. The outcomes of these activities were likely to have long-term beneficial environmental impacts. The project activities were for the most part appropriate and were of adequate scale to generate a critical mass of support to be able to able to achieve the PDO. However, while the outcome of creating alternative sources of water supply was clear, the intended outcome of " developing high income activities" was ambitious and probably ambiguous, given that none of the project interventions specifically aimed at achieving this outcome. Outputs (ICR, pages 31-33).  Of the originally envisioned four WRPs, two were completed in 2017 at Wulatehouqi and Ganqimaodu industrial estates with associated facilities (transmission mains and distribution network pipelines), at project closure. TA was provided for finalizing the design of WRP systems and related bidding documents.  Industrial water demand was much lower than projected at appraisal (at 27% of the forecast), due to the economic downturn. Few industries moved into the estates and industrial wastewater generation was much less than originally forecast and the facilities originally designed under the project were either oversized or no longer needed. Hence, smaller Reverse Osmosis units were installed at the two WRPs.  The Project Management Office, contractors, construction supervision engineers and other relevant agencies were trained on topics such as Bank policies and procedures for resettlement, environment management, project management and fiduciary management.  A long-term business development plan was developed for Bayannaoer Hetao Water Affairs Group Company Limited, for helping it to transform from a state-owned institution to a more market-oriented company.  1,228 jobs were created at the Wulateqianqi, Ganqimaodu and Wulatehouqi industrial estates. Outcomes.  The two WRPs that were constructed with smaller Reverse Osmosis Units were not operational, as the Bayannaoer municipality in 2018 began to locally implement strict new environmental measures to protect Wuliangsuhai Lake. A new regulation issued by the municipality after the project started, prohibited any point-source pollution discharge into the lake (including the brine wastes generated from the project WRPs). Hence, the two WRPs were not operational when the project closed. As a result, the following outcomes were not realized.  No water supply was reclaimed from the WRPs, as compared to the target of 55%.  No water tariffs were collected, as compared to the original and revised targets of 80% and 70% respectively. The project had envisioned the WRP operations to be fully self-financing through tariffs. This objective was not realized.  There was 100% reduction of groundwater abstraction from self-wells at closure, exceeding the original and revised targets of 90% and 50% respectively. This outcome however was not attributable to the project for two reasons: (1) Following the slowdown in industrial development in Bayannaoer, there was a reduction in demand for industrial water and this limited the need for groundwater supply at the industrial estates: and (2) the enactment and enforcement of new municipal government regulations in 2016, required the shutdown of all private/self-owned wells in the industrial estates. Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695)  According to the ICR (paragraph 44), the industrial water use did not compromise the water use for irrigation in the municipality (with the average demand for water for irrigation remaining stable at 4.39 billion cubic meters between 2009 to 2017), due to the overall slowdown in industrial activities in the Bayannaoer municipality. Rating Negligible OBJECTIVE 1 REVISION 1 Revised Objective To better utilize water resources without compromising the water use for irrigation Revised Rationale The outputs and outcomes discussed above are relevant to the revised objective. Given that the revised objectives were either not met or not attributable to the project, the achievements of this PDO are rated as negligible. Revised Rating Negligible OBJECTIVE 2 Objective To improve the water environment of Wuliangsuhai Lake by reducing water pollution entering such lake. Rationale Output (ICR, pages 16-19).  Three WWTPs were constructed in the industrial estates of Ganqimaodu, Wulateqianqi and Wulatehouqi, as targeted. However, due to the slow development of the industrial estates, the actual inflows to the WWTPs were too low to meet the minimum operating requirements for the WWTPs. Hence following the second restructuring, the three WWTPs were to be retrofitted with low-flow systems. The low-flow WWTP systems were completed for the two WWTP at Wulatehouqi and Ganqimaodu industrial estates as targeted, for treating small volumes of wastewater at the industrial estates. Due to delays in the government review process, the low-flow system at Wulateqianqi was not completed when the project closed. Subsequently, the operation of this facility was transferred from the Bayannaoer Hetao Water Affairs Group, Company Limited (BWAG) to a new company (Inner Mongolia Bayannaoer Capital Water Company Limited), which financed the low-flow system. This system was completed after project closure. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695)  The pilot to reduce non-point source pollution through optimizing fertilizer application and reducing non-point source pollution from entering Wuliangsuhai lake was completed, as targeted.  Over 3,000 farmers were trained in the new technology and over 20,000 training materials were distributed.  The wetlands at the inlet to Wuliangsuhai and canals/grids were completed to improve water circulation. The ICR (paragraph 55) notes that these wetlands were only completed in September 2017 and put into operation on May 10, 2018. However, wetland operation was disrupted from June 18 to August 19, 2018 , due to a large diversion of floodwater and this suspended the monitoring operations. As such, pollution monitoring took place for less than one year and hence missed the peak pollution removal period (July to August, since according to the ICR (paragraph 55), constructed wetlands usually function better in summer with higher temperatures and flourishing plants).  The reduction in wetland pollution was as follows: (i) The total pollution load reduced to 815 tons a year as compared to the target of 4,625 tons: (ii) The Chemical Oxygen Demand reduced to 787.6 tons/a year (target 3,850 tons/a year: (iii) The Total Nitrogen load reduced to 19.6 tons/a year (target 680 tons/a year: (iv) The Total Phosphorous load reduced to 7.3 tons/a year (target 95 tons/a year): and (v) the Annual Reed Production reduced to 22,000 tons/a year (target 33,150 tons/a year).. Outcomes.  According to the ICR (page 31), the volume of Chemical Oxygen Compound (COD) load reduction achieved under the project was 794 tons a year at project closure, representing 20% of the target of 4068.0 tons a year.  The nutrient load (nitrogen) reduced to 19.70 tons a year, well short of the target of 784.20 tons a year.  The COD of water quality at Wuliangsuhai Lake's outlet decreased to 30 metric tons on December 2018, short of the target of 60 metric tons. The total nitrogen quality of Wuliangsuhai Lake's outlet decreased to two metric tons, exceeding the target of three metric tons. The total phosphorous quality of Wuliangsuhai Lake's outlet decreased to .07 metric tons, exceeding the target of 0.20.  Given that external factors (such as, limited amount of generation of wastewater and the Bayannaoer municipality’s decision to divert water from the Yellow River to recharge the Wuliangsuhai Lake for environmental restoration) could have influenced the outcome, it is difficult to ascertain the extent to which the project contributed to the limited outcomes. The project completed all the proposed activities, but the outcome indicators were only partly realized. Pollutant loading into Wuliangsuhai Lake was reduced and the quality of the water environment at the lake showed an improvement, with contributions from the project and from other external factors. Rating Modest OBJECTIVE 2 REVISION 1 Revised Objective This objective was not revised. Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) Revised Rationale --- Revised Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Achievement of the revised PDO - to better utilize water resources without compromising the water use for irrigation - was negligible, given that the revised objectives were for the most part, not realized. Efficacy of the second objective - - to improve the water environment of Wuliangsuhai Lake by reducing water pollution entering such lake - is rated as modest, as the outcome indicators were partly realized. While it is difficult to ascertain the extent to which the project contributed to the outcome given the external factors, it is reasonable to conclude that the project made a modest contribution to realizing the outcomes, Overall efficacy is rated as modest, given that the project outcomes were to an extent impacted by external factors, beyond the project’s control. Overall Efficacy Rating Primary Reason Modest Low achievement 5. Efficiency Economic analysis: An economic analysis was conducted for project activities associated with construction of Water Reclamation Plants (WRPs) and the Wastewater Treatment Plants (WWTPs). These components accounted for 98% of the appraisal estimate. The methodology entailed calculation using the tariff rate times the quantity of water supplied/treated and based on the assumption that collection rates for water supply and sewerage would grow from 60% to 85-90% during the first three years. The ex ante Economic Internal Rate of Return (EIRR) for WRPs was 8% and 4.8 to 5.4% for the WWTPs.The ICR (page 41) notes that the three WWTPs constructed by the project were in low flow operation and the two WRPs were not in operation. Therefore, they were not generating economic benefits. The direct benefits for Wuliangsuhai lake were assumed to come from increased aquaculture, reed harvesting and tourism. The ex ante EIRR for this activity was estimated at 11.6%. The ICR (page 41) does not provide details on the ex post EIRR but notes that, the economic benefits were not realized. Due to the newly enforced environmental regulations for Wuliangsuhai Protection, aquaculture in the Lake was forbidden. Many small paper factories were closed and hence the expected benefits from the harvested reeds were not realized and tourism in and around Bayannaoer did not increase significantly during implementation. Financial analysis: The project included a financial covenant for ensuring financial sustainability. This covenant required that from 2016, the total revenue of the water and wastewater facilities should be at least Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) equal to the sum of total operating expenses, depreciation and interest, minus the accumulated profit/loss from the previous year of operation. This covenant was not met as the WRP and WWTP facilities did not operate as planned, and hence did not generate the expected revenues. Administrative and Operational Efficiency: There were cost savings due to competitive bidding processes. Coordination and administration barriers between the Bayannaoer Hetao Water Affairs Group Company Limited and various municipal and autonomous region government agencies impacted on project performance. Many project activities were delayed due to delays in the review and approval of project documents. For instance, the approval of the low-flow wastewater treatment system took one year. The project closed a year later than planned for completing the low-flow treatment system at the Wulateqianqi industrial estate. However, this system was not complete when the project closed. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the original and the revised PDO was substantial. Efficacy of first PDO - to better utilize water resources without compromising the water use for irrigation - was negligible, both before and after restructuring. Efficacy of the second objective - to improve the water environment of Wuliangsuhai Lake by reducing water pollution entering such lake - was modest before and after restructuring. Efficiency was negligible. These ratings were considered and weighted by the shares of disbursement before restructuring when 63% was disbursed and after restructuring when 47% was disbursed after restructuring (0.64*1+0.46*3 = 2.2). This rating scale corresponds to an overall unsatisfactory outcome. a. Outcome Rating Unsatisfactory 7. Risk to Development Outcome Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) Macroeconomic risk. Given the continued economic downturn in the global and Chinese demand for coal, there is risk to the potential benefits from the WRPs and WWTPs constructed under the project. While the Wulatehouqi WWTP is expected to act at full capacity, as a large factory is expected to open in the industrial estate, the remaining WWTPs are expected to operate at their low-flow conditions in the near future and there is uncertainty regarding the operationalization of the WRPs. Technical risk. Given that the WRPs and WWTPs are either currently idle or operating at low capacity, there is the risk that the equipment and facilities' durability could be compromised for lack of maintenance. 8. Assessment of Bank Performance a. Quality-at-Entry This project was prepared based on the experience from several water supply and waste water projects in China. Even though treatment of industrial facilities was not mandatory in this region of China at preparation, this project proactively invested on WWTPs to capture and treat the wastewater from rapidly growing industrial estates. This proved to be useful when new regulations in 2017 made WWTPs mandatory at all industrial estates. The implementation arrangements were appropriate, with the Project Management Office, established in the Bayannaoer Hetao Water Affairs Group (BWAG), the agency in charge of implementing the project (PAD, page 5). Several risks were identified at appraisal, including substantial risks associated with underutilization of project investments due to fewer than expected industries moving into the industrial estates and limited fiduciary capacity of the Bayannaoer municipality. Mitigation measures incorporated at design, included having conservative estimates of near-term demand for water in the industrial estates and close supervision during implementation. With mitigation measures, project risk was rated as moderate at appraisal (PAD. page 8). The arrangements made at appraisal for safeguards and fiduciary compliance were appropriate (discussed in section 10). There were moderate shortcomings at Quality-at-Entry. First, as indicated in section 3, one of the original PDOs - developing higher income activities for local residents - was ambitious (given the project time frame) and unrealistic (given that there were no specific project activities aimed at achieving this outcome. and second, there were shortcomings in M&E design (discussed in section 9a). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Fifteen Implementation Status Results Reports (ISRs) were filed over the project lifetime of seven years (twice a year supervision missions). There was continuity of leadership, with the same Task Team Leader throughout supervision, and limited turnover through the project (ICR, paragraph 88). The team proactively identified issues and made recommendations for appropriate resolution. For instance, when it became Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) clear in 2013 that development in some industrial estates was significantly lower than expected, the Bank team proposed cancelling two of the WRPs (formally reflected through the first project restructuring). Likewise, in 2014, the team proposed an analysis and design of retrofitted systems for low-flow WWTPs (reflected through the second restructuring).The support provided by the team aided in safeguards and fiduciary compliance (discussed in section 10). There were moderate shortcomings in supervision. For one, the second restructuring did not fix the mismatch between the PDO, the results framework and the project activities (discussed in section 9b). Two, although the team identified the need to consider the low-flow solution during the Mid-Term Review in December 2014, the low flow solution was ultimately requested by the government only in October 2017. Closer engagement with the government could have helped in planning for the installation of the low-flow system earlier during implementation. And three, given the change in environmental regulations that restricted the operation of the project’s WRPs, the Bank team could have been more proactive in working with counterparts to address the problem of discharge of brine (the main reason as to why the operation permit had not been provided to the WRPs). Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design There were two key outcome indicators: Reduction in groundwater extraction from self-wells and improvement in the quality of water due to pollution reduction during implementation. There was a mismatch between parts of the PDO and the indicators in the results framework. For instance, the only indicator relating to the PDO of "better utilization of water resources" was about reducing groundwater pumping, while there were indicators relating to the maintenance of irrigation supplies stated in the PDO. There were no indicators relating to "higher income activities" or "adverse impacts on the rural poor" as stated in the PDO. Further, many of the targets for pollution reduction were defined in absolute values of pollutant loads reduced (such as coming from the WWTPs or going to the lake). This definition was highly sensitive to the slowdown in economic activity in Bayannaoer, which negatively impacted the volume of wastewater generated at the industrial estates. Given that inflow volumes were low, the absolute targets could not be met. It would have been more appropriate to use relative target values, such as “reduction in the percentage of pollutant loads". b. M&E Implementation The ICR (paragraph 80) notes that the Project Management Office regularly collected monitoring data and presented the information as required. However, some activities were not conducted as required. For Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) instance, the water quality from main agricultural drainage canal and from domestic and industrial wastewater that was to be continuously monitored from January 1, 2011, was only implemented in 2016. c. M&E Utilization The mismatch between the PDOs and indicators made the M&E system less effective in monitoring performance and informing project management. The ICR (paragraph 81) notes that when the impacts of external macroeconomic conditions and environmental regulations were noticed, the M&E system and targets were not adjusted to reflect the significant impacts of external factors. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as a Category A project under the World Bank safeguard policies. Four safeguard policies were triggered at appraisal; Environmental Assessment (OP/BP 4.01): Natural Habitats (OP/BP 4.04): Involuntary Resettlement (OP/BP4.14): and Safety of Dams (OP/BP 4.37). (PAD, page 13). Environmental Assessment. The environmental impacts identified at appraisal, included adverse wastewater discharge from construction camps, airborne dust, noise, disruption and disposal of spoiled materials (PAD, page 7). An Environmental Impact Assessment (EIA) was conducted and an Environmental Management Plan (EMP) was prepared and publicly-disclosed at appraisal (PAD, page 58). The ICR (paragraph 83) notes that regular environmental monitoring confirmed environmental compliance and that there were no public complaints or occupational health and safety incidents during execution. The ICR (paragraph 83) notes that along with the more stringent environmental regulations, following the initiation of the Wuliangsuhai Wetland Program from 2016, the project contributed to significant improvements in water and ecosystem quality. According to the 2018 monitoring data, the quality of Wuliangsuhai Lake effluent had improved from below Class V to nearly class IV, which exceeded the project’s expectations. Natural Habitats. This safeguard policy was triggered, as the project supported construction of artificial wetlands at the inlet of Wuliangsuhai lake. The ICR (paragraph 83) notes that there was compliance with the policies on Natural Habitats. The project interventions were implemented away from the core area of the provincial level natural reserve, in accordance with applicable national policies and requirements. Involuntary Resettlement. The PAD (page 59) notes that four towns or townships were expected to be affected by the project and that construction of wastewater treatment plants would require 50.5 hectares of land (including 2.5 hectares of collective farming land, 15.5 hectares of collective unused land and 32.5 hectares of state-owned land. A total of five rural households including that of one ethnic Mongolian woman were to be affected by permanent land acquisition. A Resettlement Action Plan was prepared at appraisal to address land acquisition and resettlement issues (PAD, page 59). The ICR (paragraph 84) notes that there was compliance with the social safeguards on involuntary resettlement during implementation, 53 hectares of permanent land were acquired during implementation and 22 hectares of land were used Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) temporarily. The ICR also notes that the resettlement policies were carried out consistent with the resettlement plan and compensation was paid to the affected families before the land was acquired. Safety of Dams. Although the project did not construct dams, this safeguard policy was triggered, as some of the water supply systems to be constructed under the project would draw directly from reservoirs controlled by existing dams (Yongming Dam and Wangba Dam). The Operation and Maintenance arrangements for dams were reviewed and found to be acceptable and emergency preparedness plans for the dams were prepared at appraisal (PAD, page 60). The Bank and the Bayannaoer Hetao Water Affairs Group Company, Limited, were to monitor the safety of dams, operations and maintenance procedures, and emergency preparedness and supervise implementation of remedial works during implementation (PAD, page 61). The ICR (paragraph 83) notes that there was compliance with the safeguards on safety of dams. b. Fiduciary Compliance Financial management. A financial management assessment of the Bayannaoer Hetao Water Affairs Group, Limited (BWAG) was conducted at appraisal. The assessment concluded that BWAG's financial management arrangements were satisfactory and the financial management risk was rated as modest at appraisal (PAD, page 38). The ICR notes that there was compliance with financial management issues. The ICR (paragraph 85) notes that project audits were carried out by independent auditors and audit reports were submitted in a timely fashion (with only two cases of slight delay). Overall project FM performance was assessed as satisfactory. Procurement management. An assessment of the procurement management assessment of BWAG was conducted at appraisal. The assessment concluded that the procurement risk was substantial, for factors like insufficient experience of the staff in Bank procurement guidelines and weak technical capacity in preparing technical specifications and bidding documents (PAD, pages 44 and 45). Mitigation measures incorporated at design, included training procurement staff and providing financing for a qualified consulting firm for reviewing design and bidding documents. An overall procurement plan was prepared at appraisal and this plan was to be updated to reflect project implementation needs (PAD, page 46). The ICR (paragraph 86) notes that there were no deviations from the Bank’s procurement policies and procedures and procurement performance was satisfactory. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) Outcome Unsatisfactory Unsatisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The ICR draws the following main lessons with some adaptation of language. 1. Projects should consider flexible designs and implementation approaches to better mitigate macroeconomic risks. This project was designed to mitigate water and environmental problems from coal-led industrial development and assumed the continued growth of heavy industries in the municipality. Although macroeconomic risks were recognized at design, the project was unable to overcome the risks due to the changes in macroeconomic conditions. It would have been useful to have considered more flexible technical designs (such as phasing of investments and scalable facilities) or more flexible financing instruments (such as no-interest credit lines or competitive grants to allow the industrial estates to take risks). 2. Projects should have realistic PDOs that are consistent with the results framework. This project had weakness in the design and realism of the PDOs. The original and revised PDOs in this project were not clearly defined and were overly ambitious. Future projects should carefully follow Bank guidelines to define realistic and clearly structured PDOs, with clear and logical connection to the project components, the outcome indicators and the indicator target values. The experience of this project shows that for projects with similar flaws, the PDOs and the results framework should be proactively corrected through early restructuring. 3. Hard investments in physical infrastructure need to be better aligned with “soft” investments in policy and institutional development to achieve better water and environment management. This project invested “hard” wastewater treatment and water reclamation infrastructure and encountered risks associated with macroeconomic uncertainty. Future projects could more holistically analyze a portfolio of demand management and support political/institutional/managerial investments in addition to the hard treatment and reclamation infrastructure options to support better water and environment management. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Bayannaoer Water & Env. Comp. Manage (P115695) The ICR is well-written and candid. It candidly discusses the issues encountered in the wake of the external macroeconomic conditions, which necessitated revised technical solutions. The lessons drawn from the experience of implementing this project are thoughtful. The ICR is consistent with the guidelines and appropriately conducts a split rating of objectives. a. Quality of ICR Rating Substantial Page 16 of 16