ICRR 13782 Report Number : ICRR13782 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 11/18/2013 Country : China Project ID : P042109 Appraisal Actual Project Name : Second Beijing US$M ): Project Costs (US$M): 1,251.51 1,238.33 Environment Project L/C Number : L4566 Loan/ US$M ): Loan /Credit (US$M): 349.00 319.03 Sector Board : Environment US$M): Cofinancing (US$M ): 25.00 24.80 Cofinanciers : Board Approval Date : 06/20/2000 Closing Date : 12/31/2006 01/31/2011 Sector (s): Sewerage (49%); Energy efficiency in power sector (48%); Other industry (2%); Sub-national government administration (1%) Theme (s): Water resource management (20% - P); Pollution management and environmental health (20% - P); Environmental policies and institutions (20% - P); Climate change (20% - P); Other urban development (20% - P) Prepared by : Reviewed by : ICR Review Group : Coordinator : John R. Eriksson Alain A. Barbu Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: Schedule 2 of the Loan Agreement (LA) and of the GET Trust Fund Grant Agreement (GA) state the Project Development Objective (PDO) as: "to assist Beijing in alleviating air and water pollution ..." These statements imply two project objectives: (1) alleviation of air pollution and (2) alleviation of water pollution. The Project Appraisal Document (PAD, p.2), states the PDO as "to bring about a visible and sustained alleviation of air and water pollution in Beijing ..." Since they are legal instruments, IEG will use the statement of project development objectives contained in the LA and the GA. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The PAD (p.2) identifies four groups of activities intended to help achieve the two project development objectives : (a) convert scattered coal-fired boilers to natural gas boilers; (b) promote energy conservation in heating systems; (c) construct key wastewater trunk interceptors and associated treatment facilities; and (d) strengthen environmental management institutions of Beijing . (These groups parallel the project descriptions contained in Schedule 2 of the LA and GA, which state that the PDO is to be achieved "through (a) promotion of natural gas as an alternative to coal as a fuel source for boilers; (b) promotion of energy conservation in district heating systems; (c) building of sewage collection networks and treatment facilities in the Liangshu River and Qing River basins; and (d) strengthening of Beijing’s environmental management policies and institutions ." These are means to achieve the objectives, not objectives in themselves .) These activities are further divided into nine components (p.51). The first four components (1-4) summarized below relate to air quality and energy conservation . The next three components (5-7) relate to water quality and financed the construction of sewers and wastewater treatment plants in two river systems (Liangshui and Qing). The eighth component supported institutional development of the Beijing Drainage Company . A further seven wastewater components were added in November 2006 at the request of the Government after the boiler conversion component was scaled back in late 2003 (see nos. 1 and 9-15 below; ICR, p.24)). The last component (no.16), "Land Acquisition and Settlement, " was not financed by the Bank but is shown in view of its significance for project efficacy and since it incurred a significant portion of total project costs . The total project costs estimated in the PAD come to US$1,251.51 million, including contingency costs of US$ 208.97 million. In addition. the GEF Grant of US$25.00 million was to finance studies and some of the training, technical assistance (TA) and technology development and demonstration in components nos . 2 and 3 below (PAD, p.55). ( appraisal : US$417 1. Boiler Conversion : (appraisal 417 .37 million; actual : US$37 US$ 417. 37 .36 million ) The Beijing Municipal Government US$ 37. was to assist in converting all small and medium -sized coal-fired boilers to gas boilers within the Smoke -control Area of Beijing by 2005. Conversion from coal to gas would require regulatory and pricing measures to internalize the environmental cost of coal and gas distribution infrastructure, as well as easing the financial and technical barriers to conversion. The Shihuan Jietian Energy Technology Corporation (Shihuan Corporation) was to import gas boilers and associated equipment and sell them to boiler owners . The Bank loan would finance the equipment needed for converting 2000-2500 scattered coal-fired heating boilers of medium capacity . These represented about one -third of medium-sized boilers that had access to natural gas within the Smoke -control area, and about two-thirds of those believed to require and afford installment financing . The PAD states that the market and the program were "too novel to design implementation with great certainty " (p.10) and therefore the major elements of the component would be evaluated and if necessary adjusted after about two full years of operation . Shortly after the Midterm Review in November 2003, the total cost of this component was scaled back by US$380.01 million to US$37.36 million (the loan allocation was reduced from US$ 165 million to US$16.79 million) for two reasons: (1) the designation in September 2000 of Beijing as the site for the 2008 Olympics that led to a much larger Government -led and supported program of coal-to-gas conversion, following a different business model from that promoted by the Bank project, and (2) the lagging performance of the new entity, the Shihuan Corporation . The Board approved amendments in November 2006 that added wastewater components (Components 9-15), replacing a major portion of the first Component. 2. Gas Boiler Market and Technology Development, including Energy and Environment Study (appraisal : US$27 27 .14 million; actual : US$17 US$ 27. 17 .68 million plus approx . US$16 US$ 17. US$ 16 million from GEF Grant P 064924 ) Technology Development and Capacity Building Activities (using about $7.4 million of the GEF Grant) Introduce and develop more efficient models of boiler configuration, installation, and operation . Introduce advanced technologies (e.g. condensing boiler sections, zone and radiant heating ). Monitor and evaluate converted boilers, to improve technical efficiency and information . Training/documentation to raise technical levels of boiler assemblers and operators and service firms . Advisory services to assist with design, installation and operation of boiler conversion . Market Development and Demonstration Activities (using about $9.1 million of the GEF Grant) Arrange cooperative procurement to lower costs . Disseminate information to owners, operators and general public to develop technical and marketing capacity . Develop standard configurations and piloting installation of dispersed heating supply systems . Install and monitor model gas boilers replacing at least 45 coal-fired boilers. 3. Heating Energy Conservation, including Beijing Low -Carbon Development Study (appraisal US$ 11. 11 .17 million; US$ 9 million from GEF Grant ) Newly established Heating Energy Conservation actual US$ 7.12 million plus approx . US$9 Center (HECC) would carry out the following efficiency improvements : Development of a network of specialists and contractors for heating energy efficiency promotion . Dissemination of best practice models for heat delivery systems, insulation and other conservation measures . Energy auditing of heating system chains and providing advice on energy conservation options . Pilot installation and evaluation of heat metering and control, for policies to motivate end -user conservation and to retrofit selected buildings. Studies and training to improve data base and policy /institutional frameworks of incentives and regulations to facilitate use of cleaner energy and conservation, and environmental management . (appraisal 4. Air Quality Monitoring and Decision Support : ( US$3 appraisal US$ 3.19 million; actual : US$ 3.85 million ) Assist ability of Beijing Municipal Government to acquire, analyze, and use information on air quality and pollution sources for timely policy and planning purposes . Support Beijing Environmental Protection Bureau's procurement of various types of equipment and software for monitoring, analysis, simulation, and evaluation, as well as training of policy-makers and its own staff and that of related agencies (transportation, health, and planning departments ). ( appraisal US$ 88. 5. Liangshui River Sewers : (appraisal 142 .60 million ) 88 .20 million; actual US$ 142. Construction of 48 km of trunk interceptor sewers along the Liangshui River . ( appraisal US$ 185. 6. Liangshui River Wastewater Treatment Plants : (appraisal 231 .00 million ) 185 .00 million; actual US$ 231. Construction of three secondary wastewater treatment plants along the Liangshui River . US$ 53. 7. Qing River System Sewers (appraisal : US$53 40 ; actual : US$86 53 .40; 86 .60 million ) Construction of 26 km of trunk US$ 86. interceptor sewers along the Qing River .. 8. Beijing Drainage Company Development (appraisal : US$6 US$ 6.00 million ) The Beijing US$ 6.14 million; actual : US$6 Drainage Company was to undertake an institutional development program through policy actions, technical assistance and training to improve its technical and managerial capacity works, including procurement of a set of equipment for improved maintenance operations . These were to enable the company to become an autonomous, self-financing entity responsible for planning, construction, financing and operation of the drainage, sewerage, and sewage treatment systems of Beijing . Additional Components Following the scaling back of the Boiler (first) Component, the Government requested project financing in 2006 for additional wastewater and related environmental components . The cost data below are for actual costs (according to the Team, appraisal costs are not available ). 11 . River Rehabilitation (US$164 9-11. 164 .50 million; no cost figure is given for each of three river rehabilitation US$ 164. components ) There were three water environment rehabilitation components (at times treated as subcomponents in the ICR) involving civil works, automatic water quality monitoring, studies and project management . 9. North Moat and the northern reaches of the Liangma River 10. 10 . Lower reaches of the Qing River 11. 11 . Liangshui River section (Dahongmen Sluice Gate-BDA No. 1 Dam). 12 . Chaoyang District Water Environment Rehabilitation (US$139 12. 139 .91 million ) Two-lake Connection Canal; US$ 139. rehabilitation of Ba River, Beixiao River and Xiaotaihou River; and institutional strengthening, 13. US$ 23. 23 .79 million ) Rehabilitation of Xinfeng River, including flow 13 . Xinfeng River Rehabilitation in Daxing District : (US$23 regulation, automatic water quality monitoring, associated works and project management support . 14. Water Re -use Project (US$17 14. 17 .00 million ) A reclaimed water treatment plant of 20,000 m3/d capacity to supply US$17. water to industries in the Beijing Economic -Technological Development Area. 15. 15 . Reconstruction of a Solid Waste Transfer Station US$ 20. 20 .86 million ) Reconstruction of the Datun solid waste (US$20 transfer station with capacity of 1,800 tons/day. Original Component 16 . Land Acquisition and Resettlement (appraisal : US$250 16. 250 .82 million; actual : US$303 US$ 250. 303 .00 million ) While no Bank US$ 303. or GEF financing was allocated to land acquisition and resettlement, this was a substantial and original element of the wastewater component. A Resettlement Action Plan was prepared by the Borrower in accordance with national standards and Bank policies . The final report was approved by the Bank in 2000. The Plan states that the project will affect about 1840 households, 6,586 persons, 345 enterprises and 5,475 staff. The total land area to be acquired permanently was about 1,255 mu (83.7 hectares) and the total number of people to be moved was 2,555. (see 10a below for further discussion) d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The Loan and GEF Grant were approved on 06/20/2000 with effectiveness on 05/11/2001 and projected closing 12/31/2006. The loan was restructured on 11/02/2006. Both the Loan and Grant were extended at the time by 27 months from 12/31/2006 to 03/31/2009. The ICR does not explain the reason for this extension, which applied both to the Loan and GEF Grant but the project team indicates that it was necessary because of the time required following the Midterm Review to prepare the seven new wastewater operations . The GEF Grant was subsequently extended twice, first to permit completion of the combined cooling, heating and power plant demonstration project, and second to 01/31/2011 to utilize Grant savings (of US$ 2.77 million) to prepare a project based on low -carbon economy that was to be financed subsequently (ICR, p.45). The ICR asserts that the Project Development Objective remained unchanged during implementation . The Board did not approve a change in the Objective and was not asked to do so . However, as a result of restructuring, seven new water-related components were added, existing water components were expanded and several air pollution -related components were sharply cut back (pp.2-4). In fact, the air-related components were reduced by over 90%, from 59% of total project costs to 32%, and the water components were increased by 150%, from 41% of project costs to 68%. (These figures do not include the costs of land acquisition and resettlement, which were not covered by the loan but are properly viewed as part of the total costs of the water pollution objective . When these costs are taken into account, the differences before and after restructuring are not quite as sharp but still significant, with air -related components dropping from 45% to 26% and water-related components increasing from 55% to 74% after restructuring.) The borrower's contribution to the project at appraisal was US$ 877.51 million and its actual contribution was US$894.50 million. Its share of total costs increased slightly from appraisal to closing, from 70.1% to 72.2%. Conversely, the Bank's loan contribution decreased from US$ 349.00 million at appraisal to US$319.03 million actual (the GEF grant declined from US$25.00 million to US$24.80 million), implying a slight relative decline in the Bank-GEF share from 29.9% to 27.8%. (ICR, p.25). The total cost of the project declined from US$ 1,251.51 million at appraisal to US$1,238.33 actual. US$28.97 million of the loan was canceled at the Government's request . 3. Relevance of Objectives & Design: a. Relevance of Objectives: High. High . Improving water quality and flows in the major rivers and streams, improving air quality, and reducing greenhouse gas emissions in Beijing continue to be consistent with national and municipal priorities . These priorities, especially to improve air quality, were reinforced when Beijing was selected in September 2000, three months after Board approval of the project, to host the 2008 Olympics. The Project Development Objective was also consistent with the World Bank ’s Country Assistance Strategy of 2000 with respect to safeguarding the environment and reducing infrastructure bottlenecks . Six years later, the objective remained relevant to the 2006 Country Partnership Strategy with respect to pillar three, on managing environmental challenges. During project implementation, the relative focus within the Project Development Objective shifted from alleviation of air pollution to alleviation of water pollution when the boiler conversion component was down -sized after the Midterm Review in November 2003. The GEF grant-financed technical assistance was relevant to enhance the Project Development Objective through : (i) support for boiler conversions, and transfer of technology for boiler conversion and efficient heating conservation; (ii) demonstration of efficient and rational use of natural gas in combined cooling, heating and power generation; and (iii) development of policy, strategies and action plans to address climate change, and low carbon economy development . b. Relevance of Design: Modest The components of the project were relevant to the Project Development Objective along with the associated results framework (ICR, pp.iv-vii). While coal-fired heating and power systems were a major source of air pollution in Beijing, the project underestimated the difficulty and length of time required to switch from coal to gas -fired systems. This situation could have been taken into account in project design . In particular, the selected institutional model for the boiler conversion component was not well aligned with the Project's alleviation of air pollution targets . 4. Achievement of Objectives (Efficacy): The Project Development Objective was to "assist Beijing in alleviating air and water pollution ". Alleviation of Air Pollution . Efficacy of this element of the Project Development Objective is rated modest . Outputs The project helped facilitate private sector leadership of the emerging market for delivery of small and medium size natural gas boiler conversions by financing the conversion of 700 coal-to-gas boilers. However, this result compares with a planned conversion at appraisal of 2,000 boilers. Institutional capacity strengthening included the following : The air quality monitoring, data analysis, projections, and health warning functions of the Beijing Environmental Protection Bureau were strengthened . An "Air Quality Monitoring and Decision Support System " was established and reportedly enabled monitoring of major air pollutants during the Beijing Olympic Games in 2008. Five design institutes with expertise in gas boiler conversion design were strengthened . An international consultant was hired at project inception to help develop the Shihuan Jietian Energy Technology Corporation's management capacity and business planning . According to the ICR, "this calculated risk did not pay off" and the Corporation was "very slow to develop the necessary skills and the requisite competitive mentality and management capacity ." Therefore, its performance in marketing natural gas boilers had little impact on project outcomes (p.17). The ICR reports the following studies supported by project technical assistance : Options for managing long-term municipal energy demand and supply, and for addressing the environmental impact of high energy consumption Policy and strategies for low carbon economy development for urban infrastructure sectors, demonstrations and specific actions. The GEF grant focused on boiler conversion and air quality . The results are limited to outputs . The grant is reported to have had some catalytic effect in the conversion to natural gas and in promoting energy efficiency . The ICR cites: (i) demonstrations for boiler market development; (ii) introduction of viable models for the boiler conversion market; (iii) introduction of best practices in gas technology; (iv) introduction of heating energy conservation; (v) development and financing of an efficient combined cooling, heating and power plant; and (vi) establishment of a training center for gas boiler technology and design, energy audits, and heating conservation . Several of these achievements were accomplished through the Heating Energy Conservation Center, supported by the project and the GEF Grant . The Center assumed some functions that the Shihuan Corporation was intended to fill . Outcomes The following achievements are reported at the outcome level . A 70% reduction in SO2 emissions to 36 ug/m3 by 12/31/2009 (target of 60 by 3/31/2008 from a baseline of 120 as of 12/31/1998). Recorded 80 days by 3/31/2009 when air quality was worse than Chinese Class II ("good") (target of 91 by 3/31/2008 from a baseline of 265 days as of 12/31/1998) The extent to which the conversion of coal to gas and the parallel improvements in air quality can be attributed to the project is not clear. The ICR argues that the project had a catalytic effect . This may be, but the extent of boilers converted to natural gas supported by the project was only 35% of the number planned at appraisal . In its discussion of efficiency, the ICR acknowledges that "the overall emission reduction and air quality improvement directly attributable to the project is well below original estimates " (p.14).There were two main reasons for this shortfall : first, weak performance by Shihuan Corporation, charged with overseeing conversion and promoting private sector involvement; and second, the fact that the Government initiated an aggressive program to convert from coal to natural gas when the decision was reached in the latter part of 2000 for the Olympic Games to be held in Beijing in 2008. These developments make it difficult to attribute any proportion of the improvement in Beijing's air quality from 2000 to 2009 to the project. Alleviation of Water Pollution . Efficacy of this element of the Project Development Objective is rated substantial . Outputs - original subcomponents The main wastewater treatment in the Liangshui River system achieved 100% by 12/31/2009 (non-specified target from a baseline of no treatment as of 12/31/1998) Construction of 45 km of interceptor sewers along Liangshui River by 3/31/2009 (target of 45 km by 3/31/2008 from a baseline of zero in 6/30/2000). Construction of 26 km of interceptor sewers along Qing River by 6/30/2008 (target of 26 km by 3/30/2008 from a zero baseline in 6/30/2000) Strengthened Beijing Drainage Group for project design and management, operation and maintenance of the entire wastewater system. Outputs - new subcomponents When it became evident at the Midterm Review in November 2003 that the boiler conversion component would not achieve its target, the US$417 million that had been allocated to boiler conversion was scaled back by US$ 380 million and at the request of the Government in 2006, US$366 million of this amount was allocated to additional wastewater investments. Baselines at the output level are not indicated for the seven new post -restructuring wastewater components. Quantitative targets ("planned at appraisal") and results are shown for some elements of some components but not for others . Where there are matching appraisal and achievement figures, outputs were either achieved or exceeded (ICR Annex 3, pp.26-30). Outcomes - original subcomponents Major pollutants in the Liangshui River downstream from waste water treatment plants : Chemical Oxygen Demand (COD) declined to 68.1 mg/litre in 12/31/2009 (target of 100 by unspecified date from baseline of 236.7 as of 12/31/1998) Biological Oxygen Demand (BOD) declined to 19.5 mg/litre in 12/31/2009 (target of 20 by unspecified date from baseline of 88.9 as of 12/31/1998) Surface water quality of the upper reaches of the Laingshui and Qing Rivers improved from worse than Class V to Class III-IV in most sections. Outcomes - new subcomponents The ICR makes the following qualitative observations (pp. 28-32): Outcomes of the added subcomponents for water environment improvements (in Liangma River, lower reaches of the Liangshui and Qing Rivers, Ba River, Beixiao, Xiaotaihou and Xinfeng River ) included: reduced pollution discharges to the rivers, improvement of river water quality from Class V or worse than Class V, to Class III in some sections, and to Class IV in others; and reduced flood damage through river channel improvements; and environment improvements resulting in improved amenities and enhanced land prices . Outcomes for the Ba and Beixiao Rivers in Chaoyang District were : reduction of wastewater pollution from about three million m3 per year of wastewater that would have otherwise entered the rivers . Avoided flood damage to nearly 270,000 people through increase of the flood protection standard from 1:10 years to 1:20 years and 1:50 years in some sections of the rivers . Additional achievements in water resources : increased supply of treated wastewater to industry by construction of a 20,000 m3/day water re-use plant protection of groundwater through improvements in solid waste management . 5. Efficiency: Efficiency is rated modest . Cost-benefit analyses were conducted at appraisal for both the boiler conversion and wastewater elements . However, the ICR indicates that it was not possible to compare the results at appraisal with an end of project assessment because the major restructuring in 2006 altered project composition and impact and because of the difficulty of attributing the contributions of the project as distinct from those of other environmental improvements undertaken by the Municipal Government (pp.14-15). The ICR does make several observations about cost-effectiveness and efficiency but the information provided refers solely to benefits and is not sufficient for a cost-effectiveness analysis because comparative cost -effectiveness data are lacking for similar investments in Beijing or elsewhere. Efficiency in terms of timeliness is relatively satisfactory given the complexity of the project and the restructuring that became necessary at the time of the Midterm Review . The ICR cites "external reasons" for the restructuring (p.14). The Government's response to the Olympics decision was at least in part external but the continued weakness of the Shihuan Corporation cannot be characterized as "external." Stronger capacity and leadership by the Corporation might have reduced the need for a strong Government role in promoting boiler conversion . Original wastewater components were completed by the Midterm Review at the end of 2003, but the seven new components were not added until 2006 and were completed by late 2009. The Project Team explains that the requirements of design and appraisal for each new wastewater component as well as an orderly scaling back of the boiler conversion component accounted for the three-year interval between 2003 and 2006. The project closed on 01/31/2011 owing to two subsequent extensions of the GEF grant for reasons explained in section 2d, above. The Beijing Municipal Government increased wastewater tariffs several times during project implementation, most recently in December 2009. At project appraisal, the tariff for residential consumers in Beijing was RMB 0.30/m3 and by 2009 it was RMB 1.04/m3. The Municipal Government intended to increase tariffs to meet the full cost of wastewater collection and treatment in a phased manner . The Cost Recovery Ratio (Including subsidies) varied between 1.16 in 2005 to 0.77 in 2006 and rose to 1.07 in 2009. It therefore appears that progress was made in the collection of wastewater tariffs (as suggested by the tariff rate increase for residential consumers ), albeit with rather sharp fluctuations between adequate and inadequate cost recovery from one year to the next . However, there was a long lag (including a 27-month project extension) between the substantial reduction in the largest (boiler) component of the project and the identification and approval of new wastewater components . Insufficient information was provided for a cost-effectiveness analysis of the energy efficiency and conservation components . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: There were significant shortcomings in aspects of the operation's performance . While the rating of Relevance of Objective is High, the rating of Relevance of Design is Modest owing primarily to the weakness in the design of the air pollution alleviation element. Efficacy of the air pollution alleviation element is rated Modest in view of the fact that only 35% of the planned conversion of coal to gas -fired boilers was accomplished and the resulting conclusion that the overall emission reduction and air quality improvement directly attributable to the project was well below original estimates. The Efficacy of the water pollution alleviation element was substantial but somewhat attenuated by the partial absence of results framework data that would permit a more rigorous assessment of the results obtained by the new, post-restructuring water pollution alleviation sub -components. Efficiency is rated Modest owing to the poor performance of the Shihuan Corporation, the entity relied upon for the coal -fired boiler conversion component as well as a result of the lengthy time required for restructuring and deficiencies in economic analysis . This results in an overall Moderately Unsatisfactory rating for Outcome. a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Risk to development outcome is rated as negligible to low . The National Government made a national commitment to improve the environment, as reflected in its five year plans . Efforts by the central and municipal governments to improve air quality were expected to continue through programs to reduce pollution emissions from industries; expand the use of natural gas in Beijing; improve efficiency in the remaining coal -fired heating units; introduce measures to control automobile emissions; and mainstream heating energy conservation demonstration pilots. Improvement in water quality was given high priority and financial support by the Municipal Government, resulting in a low risk that water quality would deteriorate . The Government is also expected to continue cost recovery for wastewater services in a phased manner to ensure sustainability . There is no evidence in the ICR to suggest changes that might be detrimental to achievement of development outcome . a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry: For air pollution reduction, the project design envisaged a manageable segment of the potential market for boiler conversions; financing to address a constraint facing boiler houses at the time; and grant -financed technology transfer and demonstrations to support the conversion program; a mechanism to procure and market natural gas boilers; and capacity enhancement for air quality monitoring and projection . To respond to this ambitious plan, the Bank included significant technical assistance from a GEF grant to provide the expertise . The Borrower finds the Bank to have marshalled timely expertise of high technical quality for project preparation However, the chosen institutional model for the boiler conversion component, utilizing a new and untested parastatal turned out to be a poor choice and the boiler conversion component was scaled back to about 10% of the original component cost. Among a total of 15 indicators, five key indicators were deleted (with Board approval in November 2006) because of lack of data or methodology . These were: (1) carbon dioxide (CO2) released from heating systems; (2) ambient concentration of major air pollutants (one of three outcome indicators ); (3) coal consumption for heating (one of 11 intermediate outcome indicators); (4) costs of gas boilers, services and operations {intermediate outcome indicator); (5) emissions from heating boilers (intermediate outcome indicator). Two air quality outcome indicators were added to replace the deletion of the CO 2 indicator: (i) annual average concentration of sulphur dioxide (SO2) and (ii) days per year with air quality worse in Beijing than Class II in China . While the Board approved deletion of the five above-mentioned indicators, as noted above, approval for a change in the Project Development Objective was not requested since the team concluded that even with the changes in components, the overall objective did not change . The ICR (p.18) acknowledges that the design of the key performance indicators was weak, but that they were amended during project implementation (by Management at the Midterm Review in 2003 and approved by the Board in 2006). For the wastewater sector, project design incorporated expansion of wastewater collection and treatment in Beijing, continuation of institutional reforms to improve the autonomy of Beijing Drainage Company, improvements in utility management, and long -term sustainability of the wastewater sector . These design elements were in place at entry and with the exception of the institutional model for boiler conversion, appear to have been necessary and sufficient . Taking these considerations into account, including the strong endorsement by the Borrower of the Bank's performance at entry, Quality at Entry is rated Moderately Satisfactory. at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Supervision missions identified and highlighted weaknesses in project management, contract management and institutional development. It advocated improvements in detailed designs, construction management and implementation, utility management and compliance with financial performance covenants in the wastewater sector. The Bank identified the need to restructure the project at the 2003 mid-term review and responded favorably to the Borrower’s request in mid-2006 to restructure the project when large loan savings accumulated, and agreed to add seven new components consistent with the project wastewater subobjective . But the evidence of preparation for the new components is very thin . The results orientation of the restructuring Project Paper of July 25, 2006, is very poor. It contains no Results Framework Analysis, baselines or targets . It is lacking significant detailed cost estimates for the new components and does not include any ex -ante economic analysis for these components . . The slow progress in the implementation of GEF technical assistance resulted from the diminished pace of boiler conversions and gas technology development and from the complex design and procurement for the combined cooling/heating/power demonstration plant. Implementation progress was nonetheless rated satisfactory until 2007, when it was downgraded to Moderately Satisfactory for a year. Given the problems encountered in implementation of the boiler conversion unit, which ultimately led to restructuring, it is surprising that earlier supervision ratings were Satisfactory . The task team leader was based in Washington until early 2008 when leadership was transferred to a Beijing based staff. The location of several key members (procurement, financial management, environment, and social ) in Beijing throughout facilitated frequent contacts between the Bank and the Beijing Project Management Office, and resolution of issues in a timely manner . However, the Borrower noted that there were three Bank task team leaders during the course of the project, "which affected the project progress in certain stages to some extent " (ICR, p.53). The ICR does not indicate explicitly whether the project complied with Bank policy on environmental safeguards . Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Beijing Municipal Government’s performance was key to achieving air quality targets before the start of the Beijing Olympic Games. This achievement occurred with limited project funding but was supported by the GEF-funded technical assistance for gas technology and heating energy conservation . The Municipal Government supported the establishment of the Heating Energy Conservation Center, which developed into a resource center for the promotion of gas technology and heating energy conservation . A permanent training facility was established to provide continuing training . The Municipal Government carried out a review of the institutional arrangements for the wastewater sector and established the current institutional arrangements after considerable discussion with the Bank . With the Government’s commitment and financial support, these arrangements are expected to be sustainable in the medium-term. Government Performance Rating Satisfactory b. Implementing Agency Performance: The Bank worked with five implementing agencies . If it were not for the major shortcomings in the performance of the Shihuan Jietian Energy Technology Corporation, the overall implementing agency performance would be satisfactory, but Shihuan's poor performance brings the overall performance to Moderately Satisfactory. The overall performance is higher than it might have been because following project restructuring in 2003, the role of the Shihuan Corporation was reduced and the Heating Energy Conservation Center assumed the Shihuan technical assistance functions . The Beijing Project Management Office provided effective coordination and, according to the ICR, performed well in project monitoring and in resolving difficulties as they arose . . Beijing Drainage Group . The Group exercised leadership in its role as a lead wastewater sector agency, All activities, including procurement of works, goods and services, are reported to have been executed in a satisfactory manner. Beijing Environmental Protection Bureau . The Bureau’s capacity to monitor air quality, analyze data and make projections was enhanced with project support . The Bureau managed a number of monitoring stations covering the entire city, and developed a state -of-the-art air quality monitoring and decision support system to enhance its monitoring and analytical capacity . The Bureau provided daily reports of air quality and health advisories, as necessary . Shihuan Jietian Energy Technology Corporation did not possess the attributes necessary to lead a major commercial activity. Shihuan staff, drawn mainly from the Beijing Gas Development Company (which was formerly a department of the Municipal Government ) was not prepared for a commercially oriented operation, and proactive marketing and competition . In a gradual growth scenario, which was originally envisioned by the Bank and the Gas Development Company, the Shihuan Corporation might have had a chance to become better at selling gas boilers over time . But the acceleration of boiler conversions soon after the launch of the project quickly stretched the Shihuan Corporation ’s capability. The Shihuan role in the project diminished due to the reduction in the boiler conversion component, and the transfer of the grant technical assistance to the Heating Energy Conservation Center . Heating Energy Conservation Center . The Center assumed a greater role in the project when implementation of the grant technical assistance was transferred from the Shihuan Corporation . The Center implemented all the technical assistance components satisfactorily, albeit with some delays . It developed into a repository of information on boiler conversions, heating energy conservation and new technology for efficient natural gas use for cooling, heating and power generation . The ICR does not indicate explicitly whether the project complied with Bank policy on environmental safeguards . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E framework was designed to capture the project ’s impact on air and water quality, incorporating pertinent indicators, such as: SO2 concentration; number of days the Beijing population was exposed to air quality worse than Class II standard; COD discharges to the Liangshui River; progress in physical and financial expenditures by component; and resettlement implementation . But some monitoring indicators selected at appraisal were deleted because either (a) data for three indicators were not available: (i) CO2 from heating, the only Global Environmental Indicator; (ii) coal consumption from heating, one of 12 performance indicators; and (iii) costs of gas boilers, services and operations . a performance indicator; or (b) there was no methodology available for collecting the data for two indicators (i) the ambient concentration of major pollutants. one of two outcome indicators; and (ii) emissions from heating boilers, a performance indicator . Two indicators were added: annual average concentration of SO 2, a major pollutant (an outcome indicator) and number of days a year with air quality worse than Class II in Beijing (an outcome indicator). Three other performance indicators were amended. No indicator was substituted for the deleted Global Environmental indicator . The ICR does not explain why key indicators were chosen for which there were no data or methodology for data collection . The Project Team indicates that when project preparation began in 1998, there was not as much emphasis on results frameworks and indicators. After the Midterm Review in 2003, indicators were informally deleted, added, and /or amended and the deletions were approved by the Board in 2006. b. M&E Implementation: Monitoring and evaluation were to be carried out primarily through monitoring the agreed outcome and output performance indicators that were reported in the semi -annual progress reports and prepared by the Beijing Project Management Office in collaboration with the implementing agencies and the Beijing Environmental Protection Bureau. The air quality monitoring system developed under the project helped to monitor the impact of the air pollution control interventions and provide the Municipal Government and the public daily air quality information and health advisories, as necessary . c. M&E Utilization: Monitoring information was utilized to: support policy and actions to enhance air pollution reduction measures; make investment plans to further reduce pollution discharges to the river system; and accelerate project implementation and management. The Beijing Environmental Protection Bureau made daily air quality reports and weekly water quality reports, drawing on information generated by project support . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: Specialists of the task team from the Beijing office supervised implementation of safeguard issues . The team carried out field visits to review progress and ensure the submission of the external monitor ’s report on safeguard implementation. Resettlement reporting was reviewed, and commented on as necessary . (i) Environmental Safeguards. The project was judged a Category “A� project, and a comprehensive environmental assessment was carried out in accordance with the policies and procedures of China and the World Bank . All the implementing agencies established independent environmental management teams that were responsible for implementation, supervision and monitoring of the Environmental Management Plans . Emergency response plans for environmental pollution were also prepared to guarantee environment protection at the construction sites. Specific environmental protection measures included staffing and training, dust control, noise control, and disposal of sludge . During supervision, the Bank specialists recommended further improvements to environmental standards on a case -by-case basis, particularly, the management of sediment from river dredging . Internal and external environmental monitoring results have confirmed that through implementation of the Plans, the adverse environmental impacts resulting from construction were mitigated and controlled to acceptable levels . There were no complaints on environment issues throughout the project implementation period . However, the ICR does not state whether the project complied with the Bank's environmental safeguards policies . (ii) Social Safeguards. Resettlement activities were carried out satisfactorily in accordance with Chinese regulations and World Bank policies, including provisions of the Resettlement Action Plan, which included the types and levels of compensation to be paid. Measures were also taken during project preparation to reduce resettlement as much as possible. No minority issues were identified. The comprehensive Resettlement Completion Report, submitted at the end of 2008, concluded that resettlement implementation was successfully completed with the full participation of the affected persons; their standards of living had improved; and there were no outstanding resettlement issues at project closure. b. Fiduciary Compliance: Appropriate financial management arrangements were put in place to ensure proper use and accounting of project funds. Financial management was carried out satisfactorily, and no significant issues arose during implementation . All audit reports were unqualified. Procurement activities were carried out satisfactorily by all agencies . The Beijing Project Management Office and Project Implementation Units improved their procurement capacity progressively during project implementation . Some useful lessons emerged, e .g., the need for improved quality of detailed designs and bidding documents, and the need for closer communication among the Implementation Units, the Ministry of Commerce and the Bank task team, to avoid delays in bid evaluations for procurement of equipment . c. Unintended Impacts (positive or negative): None. d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately There were significant shortcomings in Unsatisfactory aspects of the operation's performance . The rating of Relevance of Objective is High but the ratings of Relevance of Design and of Efficiency are Modest . The water pollution element of Efficacy is rated Substantial but the Efficacy of the air pollution element is rated Modest. Development Outcome rating is therefore rated Moderately Unsatisfactory. Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Moderately Quality at Entry had moderate to Unsatisfactory significant shortcomings that were somewhat offset by strong positive comments from the Borrower, resulting in a Moderately Satisfactory rating for Quality at Entry. Supervision had significant shortcomings, which translates into a Moderately Unsatisfactory rating. Per the Harmonized Criteria agreed by IEG and OPCS, overall Bank Performance is rated Moderately Unsatisfactory . Borrower Performance : Satisfactory Moderately Given that Government Performance Satisfactory was rated Satisfactory and Implementing Agency Performance was rated Moderately Satisfactory, per the Harmonized Criteria, Overall Borrower Performance is rated Moderately Satisfactory. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons were selected from the "Lessons Learned" section of the ICR (pp.20-21) and condensed and edited. 1 .Wastewater Sector Institutional Reform . Sustainable wastewater service provision requires a combination of (i) politically feasible tariffs; (ii) ensuring service provision at low cost; (iii) municipal government assumption of responsibility for servicing debt; and (iv) provision of timely cash transfers to meet shortfalls, especially in relation to new investments. use Adequate water re-use in water scarce areas (at least 50%) requires addressing the following : (i) 2. Water Re -use. adequate quality and reliability of re -claimed water; (ii) market-related pricing of reclaimed water; and (iii) analysis of the costs and benefits of using water supplied through the system vs . reclaimed water. 3. Participation of International Experts and Sharing of Knowledge . The utilization of international expertise is important for the effective adoption of a new technology, such as natural gas boiler technology transfer and heating conservation. 4. Intervention in Commercial Operations . In areas involving significant commercial operations (such as boiler conversions), newly established municipal entities with no commercial expertise are not likely to succeed . It might be more effective in such circumstances for a relevant government entity to partner with a commercial company . Alternatively, conversion to the new technology might be left to market forces with some incentives provided to end-users. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The outcome, intermediate outcome and performance indicators are for the most part appropriately specific and quantitative. However, findings and conclusions in the ICR are at times based on assertions rather than backed up by specific evidence. The ICR does not explain the reason for the extension of the loan and GEF grant by 27 months, from 12/31/2006 to 3/31/2009. The ICR lacks an explicit statement as to whether the project complied with the Bank's environmental safeguards . There are many technical terms and government entities cited in the ICR text . While covered quite well in the list at the beginning, they should have been spelled out before an abbreviation or acronym is first used in the text (e.g. COD, BOD). The ICR is generally well-written and comprehensive. But there is some repetition, especially of achievements, in different sections . a.Quality of ICR Rating : Satisfactory