Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) Report Number : ICRR0020644 1. Project Data Project ID Project Name P098915 CN-GEF Sino-Singapore Tianjin Eco-City Country Practice Area(Lead) China Social, Urban, Rural and Resilience Global Practice L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-97018 30-Jun-2016 64,063,636.00 Bank Approval Date Closing Date (Actual) 22-Jul-2010 30-Jun-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 6,163,636.00 6,163,636.00 Revised Commitment 6,163,636.00 6,163,636.00 Actual 6,163,636.00 6,163,636.00 Prepared by Reviewed by ICR Review Coordinator Group Victor M. Vergara Victoria Alexeeva Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives Grant agreement (p.6) To assist Sino Singapore Tianjin Eco-City Administrative Committee (SSTECAC) to develop Sino Singapore Tianjin Eco-City (SSTEC) as an energy and resource efficient and low Green House Gas (GHG) emission city. Project Appraisal Document (p.4) To assist SSTECAC to develop SSTEC as an energy and resource efficient and low GHG emission city. Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components Component 1: TA, Software, and Equipment for Implementation Framework of the SSTEC Master Plan and Dissemination Activities (Appraisal US$1, 582,000 – Actual US$2,598,000). Subcomponent 1A: Eco- city advisory panel for providing advisory service on (a) implementation of the SSTEC master plan; (b) management and coordination of the project; (c) monitoring and evaluation (M&E) of master plan implementation; and (d) replication and dissemination of the experience of master plan implementation. Subcomponent 1B: TA, software and equipment for creating an implementation framework for the SSTEC master plan, which included assisting the SSTECAC to (a) develop policy, regulations, incentives, and institutional frameworks; for example, promotion of efficient energy/resource use and achieving key performance indicators (KPIs) in water, solid waste, and energy; (b) develop financial and economic analysis models for investment decisions on projects, and public infrastructure facilities; (c) review KPIs, calculate GHG emissions, and (d) provide on-the-job training on selected topics. Component 2: TA for Public Transport System (Appraisal US$788,000 – Actual US$789,800) Technical advisory services on Transport Oriented Development (TOD) , including (a) carrying out a review and gap analysis across all stages of planning; (b) preparing a detailed public transport network and incremental services delivery plan; (c) feasibility studies on public transport; and (d) institutional arrangements. Component 3: Green Building Pilot Investment and TA (Appraisal US$61,350,000 – Actual US$60,270,000). Pilot investment in two green buildings and technical advisory services for Green Building Evaluation Standard (GBES) implementation. The two buildings are public housing for low-income segment of SSTEC households and a middle school, including the provision of incremental construction costs of energy/water efficiency and renewable energy use. Revised Components Subcomponent 1A (establishing eco-city advisory panel) was canceled because of lack of response from the short-listed consultants when the procurement for consultants to serve the committee was carried out in 2011. The failure to recruit experts to serve on the proposed panel was largely because of (a) the purpose, scope of work, and deliverables of the proposed services were unclear; and (b) the budget appeared to be too small to gain traction from external experts, particularly considering the 60-month duration of the contract. In addition to the cancellation of Subcomponent 1A, the actual contract price for Subcomponents 1B (framework for master plan implementation) and 3A (investment in two green buildings) was less than the estimated price owing to competitive bidding. All these changes resulted in a total savings of US$1.9 Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) million. In December 2014, as part of the conclusion of the midterm review, the following new activities were added to the existing components (second restructuring package) to utilize the total savings of US$1.9 million. Component 1 - Enabling Framework (i) Integrated water technical assistance (TA) (US$500,000); (ii) M&E equipment and software for energy utilization (US$740,000) Component 2 - Green Transport Updating transport planning and strategy (US$500,000) Component 3 - Green Buildings Promoting and strengthening green building awareness (US$160,000) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Source Type of Financing Appraisal Actual % of Appraisal Estimate Borrower Counterpart contribution 57,900.0 58,360.7 100.01% GEF Grant 6,160 6,160 100.00% Total 64,060 64,520.8 Notes: Costs are in 1000 USD Dates The project was approved July 22, 2010 and closed on time without extensions on July 30, 2016. The project was restructured twice: the first time in 2012, to replace the Project Implementation Agency for the middle school energy initiative; and the second time in 2014, to include new components, as a result of midterm review findings. The new components are detailed in the project objective and components section. 3. Relevance of Objectives & Design a. Relevance of Objectives Tianjin, China’s third largest city (population 11.8 million in 2008), was developing Sino Singapore Tianjin Eco-City (SSTEC) in collaboration with Singapore. The Framework Agreement was signed between the Chinese and the Singaporean Premiers in November 2007. The site of the SSTEC is non-arable salt land, located in the Tianjin Binhai New Area (TBNA), the power house of Tianjin’s economic and demographic growth, which was experiencing one of the fastest growth rates in China. It was planned that 350,000 people would live in the 34.2 km2 SSTEC area by 2020. SSTEC adopted a mixed land use plan to accommodate not only housing but also service-oriented and high technology/environment-related industries which were expected to create 190,000 jobs (about 80 percent of the working population). Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) The development objective of the project was relevant to the priorities of the national and the municipal governments and was aligned with the World Bank strategies as well as the GEF strategic objectives. The project was initiated and prepared in response to the Government’s initiative for ‘clean growth’ set out in the 11th Five-Year Plan (2006 to 2010) and consistent with the World Bank’s CPS for 2006– 2010. The PDO was relevant to the GOC’s and the World Bank’s strategies to pursue resource-efficient and low carbon development pathways during the entire project implementation period. The project development objective was also relevant to China’s 13th Five-Year Plan (2016–2020) which set emission intensity targets for 2020 and incorporated green development as a priority. In 2012, the National Development Reform Commission launched a low-carbon city program, which drew participation from over 42 cities and provinces. As China committed to its GHG emissions under the Paris Climate Agreement, efforts to reduce emissions are being intensified. Cities will play a central role in China’s climate change mitigation strategy. The project objectives remained relevant to the World Bank’s FY2013–2016 China CPS, particularly to the pillar of ‘supporting greener growth’. The PDO was consistent with the World Bank Group’s Climate Change Action Plan (issued in April 2016), which aims to increase its climate investment from the current US$2.2 billion a year to a goal of US$3.5 billion a year. Rating High b. Relevance of Design The project design was relevant and linked to achieving the PDO. The design contributed to SSTECAC’s ability to implement its low-carbon and resource-efficient strategy by addressing transport, energy efficient buildings, and city management. The first two sectors account for over 85 percent of the emissions and energy use of the eco-city. The fulfillment of the eco-city’s objectives depended on SSTECAC’s ability to manage these two sectors. Second, a combination of infrastructure investments (in green buildings) together with the TA component to improve green building management was designed to demonstrate economic benefits for energy efficiency while strengthening the capacity of implementing green building regulations. However, the project design had weaknesses. The coverage of the activities in the first component, enabling framework, was broad and overly ambitious. Even though the support covered key policy areas relevant to promoting energy and resource efficiency and low carbon emissions, the range of activities from policy, regulatory and institutional frameworks to financing models and monitoring mechanisms, posed challenges in achieving intended outcomes. The second weakness is related to the results framework, which did not capture the impact or intermediary outcomes of some of the project activities, especially those related to the ‘enabling framework’, which made it difficult to assess the actual outcome. The concept of “transit oriented development” (TOD) was highlighted in PAD as one of the key objectives of the Eco City. However, the results framework included very few TOD metrics. Most important, there is no basis to substantiate the implementation of the TOD concept under the project, undermining Tianjin’s effectiveness as an Eco-city. The most important weakness in the design is the absence of measures to address challenges in land use both in terms of spatial location relative to the built up area as well as the urban texture, which promotes efficient land use from a smart growth perspective. Finally, the project design did not asses or appropriately mitigate the Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) core risk inherent in any greenfield urban development: fostering economic development by attracting businesses. Linked to this risk the project did not identify the risk of having a slower population growth than predicted and its implication for the viability of the EcoCity. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To develop Sino Singapore Tianjin Eco-City (SSTEC) as an energy efficient city. Rationale Outputs • The project provided pilot investment in two green buildings—a middle school and a public housing for low-income households; and TA for enhancement and implementation of the GBES of SSTEC for achieving energy savings and GHG reduction. The total of US$2.4 million was used for ‘enhancement measures’ including energy efficient windows, insulation walls and roof, and solar heating system. • Technical advisory services were carried out to develop policy, regulations, incentives, and institutional frameworks for the promotion of efficient energy/resource use and achieving KPIs in the sectors of water, solid waste, and energy. • Technical advisory services were carried out to promote green transport trips and develop an integrated public transport system, focusing on the mass transport system, including bus rapid-transit, to be introduced as the first step of the project’s public transport system and strengthen institutions to support the planning, development, and regulation of public transport. Outcome The pilot housing achieved its outcome targets for energy savings in 2014, ahead of the schedule. Through in-house lighting, cooking, and other electrical devices the savings reached 4,902 MWh, exceeding the target by 48 percent. Similar package of enhance measures was also made for the middle school, which resulted in energy savings of 1,627 MWh, which was about 10 percent lower than the target. The slight shortfall was largely because (a) the energy used for cooking was higher than estimated because the middle school serves food for people living close to the school as there were no restaurants or canteens in the area; and (b) the heating system is powered by natural gas instead of electricity as originally designed. The piloting on the use of renewable energy was not satisfactory in achieving its target of 60 percent of renewable energy use for public housing and 20 percent for the middle school set out in the IOIs. In 2015, Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) the former reached 9.9 percent and the latter 9.45 percent. One of the main reasons was related to the difficulties of solar panels in driving full operation of heating/cooling systems to keep the level of ‘comfort’ in cold winters and hot summers. As such, the installed solar system has not been in full operation. Nevertheless, the two pilot green buildings have both exceeded the minimum requirements of the current GBES. Taken together, the total incremental annual energy savings is 6,529 MW compared to the GEO Indicator 3 target of 5,453 MW. The SSTEC GBES is more stringent than those prescribed under the national standard. The project produced implementation manuals for the GBES, providing guidelines for the evaluation and verification process for compliance with green building standards. The PDO set a target of 25 percent buildings in SSTEC to exceed the energy-efficiency standards under the current SSTEC GBES. In 2015, all the buildings in the eco-city have complied with the GBES and about 93 percent of the buildings have exceeded the GBES requirements, among which 42.7 percent of the buildings have reached the highest national standard, three stars. The implementation manuals supported by the project were adopted by the SSTECAC’s construction bureau to enforce GBES compliance. The examples set by the investment in the two green buildings were promoted by the SSTECAC, through trainings, workshops, and videos, and played a positive role in disseminating good practice. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To develop Sino-Singapore Tianjin Eco-City (SSTEC) as a low Green House Gas (GHG) emission city. Rationale Outputs • The same as under Objective 1. Outcomes The pilot housing achieved its outcome targets for GHG emissions in 2014, ahead of the schedule. Through in-house lighting, cooking, and other electrical devices, there were GHG emission reductions of 2,928 tCO2. Similar package of measures was also made for the middle school, which resulted in equivalent to 707 tCO2, which was about 10 percent lower than the target. Taken together, the resulting in emission reductions of 3,551 tCO2, against the Global Environmental Objective Indicator target of 3,132 tCO2e. The eco-city utilized the completed public transport guidebook to support the establishment of its first public transportation company in the context of stop selection, bus routing, bus selection, and O&M. Share of public transport mode within SSTEC has gone from zero at project start-up to 40–50%. In addition, the updated Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) master plan would now focus on demand analysis, road traffic, public transportation, slow and static traffic, intelligent transport, transportation management, and preparation of an implementation plan. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To develop Sino-Singapore Tianjin Eco-City (SSTEC) as a resource efficient city. Rationale Outputs • Technical advisory services were carried out (I) to develop finance and economic analysis models based on lifetime cost-benefit analysis concept and application of this model to key public infrastructure investments; (ii) develop a finance mechanism and plan for capital investment and revenues/expenditures projections for key public infrastructure and public utilities; (iii) develop a SSTEC municipal finance model based on capital investment finance plan and revenues/expenditures projections of the key public infrastructures and facilities; and (iv) provide on-the-job training to the SSTECAC staff on the finance mechanism. • Four reports were completed: Tianjin EcoCity River System Status Investigation and Analysis Report, River System Interconnection and Circulation Report, Research Report on Optimization of Drainage System, and Research Report on Rainwater System and Resource Utilization Based on Low Impact Development. Guidance has been provided to the staff and management of the environment bureau with regard to water resource management and drainage systems. • Geographic Information System and management information system were installed to support the daily project management and M&E. Outcomes The technical advisory services enabled policy, regulatory and institutional frameworks for materializing the vision and objectives of the SSTEC master plan; the framework is now in use. The resource efficiency outcomes need to be put into context on the overall performance of the city. Having a resource efficient city requires that a critical mass of city dwellers benefit from services – a disproportionally large investment in city services for a reduced population is inherently inefficient. In addition, in the context of SSTEC - an important aspect of efficiency deals with mitigating strategic and fiscal risks associated with a supply-driven urban expansion. The component developed strategic fiscal and risk management models but it is unclear if these were consequential. Evidence of this is that no apparent measures were taken given the fact that the core business investment -- the animation studio - did not take Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) place and the overall population growth in SSTEC was slow. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Efficiency As a requirement by Global Environmental Facility the project undertook incremental cost analysis of direct and indirect emission reductions at appraisal stage and completion rather than a benefit cost analysis and rate of return. The efficiency analysis presented in the ICR covers 91% of total grant costs and includes two aspects: incremental cost analysis for green buildings (39% of total grant) and assessment on TA components related to enabling framework and transport (52% of total Grant). Two underlying factors limited the efficiency of the project. The undervalued nature of carbon emissions in Tianjin and the overestimate on the population of the EcoCity which exaggerated the efficiency of investments. Green Buildings. The incremental cost analysis was largely based on the assumptions of CO2 emission reductions from direct and indirect sources. Direct emission reductions are attributable to the pilot investment in two buildings, estimated at annual energy savings of 5453 MWh and emission reductions at 3,132 tCO2e. The incremental cost to be financed by the GEF was projected at US$3.66 million at the time of the project appraisal. The payback period was calculated for 8 years for the public housing and 12 years for the middle school. TA Components - Transport and Enabling Framework. The project estimated total emission reductions from Transport at 392,782 tCO2 based on the assumption that the eco-city fully implements its KPIs in 2020 and has a population of 350,000. However, the project also recognized that quantifying carbon emission reductions as a result of implementing TA components would be difficult because there was no established methodology. Instead, the project opted for a qualitative incremental cost assessment: (a) achieving a higher green transport mode share; and (b) increased likelihood of successful implementation of the master plan and, thus, reduced citywide CO2 emissions; and (c) demonstration impact that the model for resource efficiency could be replicated by other cities in China. The cost analysis for investment carried out in the preparation of the PAD did not take into account the cost of carbon as it was not available in China. The ICR conducted insightful analyses presented below: In 2013, Tianjin, as one of seven pilots for emissions trading selected by the GOC, officially launched its emissions trading scheme (ETS), covering enterprises from key industrial sectors, whereby the companies would have to pay for each ton of emissions in excess of their emissions quotas. A ton of CO2 traded at the Tianjin Environment Exchange was priced at US$2.88 in June 2016 and the GOC has announced that a national ETS will be launched in 2017. It would be difficult to project the price range of emission for the next 20 years. However, the World Bank has, since 2015, recommended the “use of social value of carbon in economic analysis of investment projects” with the base price set at US$30 per ton to increase to US$80 per ton by 2050. Using the World Bank’s shadow carbon price, the annual cost savings from direct emissions reductions of the pilot investment alone could be US$106,680; and for indirect emissions US$12.8 million Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) annually. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project provided a technical insight for the SSTEAC and this has the potential of improving the overall quality of the SSTEC. However, given the design and approach of the monitoring and evaluation system it is not possible to ascertain it with high level of confidence. As the ICR points out, while the project-proposed methodologies for comprehensive monitoring and tracking of the city’s KPIs, they were difficult to apply as they were based on a much larger population base. More importantly, except for real time monitoring of energy use, given the nature of the KPIs - the M&E focused largely on outputs rather than outcomes which limited its utility. The project provided technical assistance resulting in strategically valuable input in relation to planning, building and operating the SSTEC. However, the strategic and global nature of the resources sought to add value and assist Sino Singapore Tianjin Eco-City Administrative Committee (SSTECAC) to develop Sino Singapore Tianjin Eco-City (SSTEC) as an energy and resource efficient and low Green House Gas (GHG) emission city. In this context, during supervision the project failed to flag the evolution of critical risks inherent in the supply driven urban expansion. As appraised, the project identified but underrated the risk of SSTEC not attracting business investment with the consequence of fostering a satellite residential town or eco-enclave. The visible slow growth of employment and population – as well as the fact the key mitigation measures did not materialize – were not flagged in the ISRs. The outcome of the project as designed was not sensitive to the overall success and viability of the Eco city. For example, even though two of the development objectives were substantial and one modest - the eco-city is far behind its own development and marketing targets. The Eco City was planned to accommodate 350,000 residents and 190,000 jobs by 2020. While people are gradually moving into the city, the pace of population growth and job creation is much slower than originally envisaged. By 2016, the city has about 40,000 residents. The project is set in a context where the risk to development outcome is high. Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) IEG would like to note that the project outcome rating is barely moderately satisfactory given that the relevance of design is rated modest, the achievement of one sub-objective out of three is rated modest, and the efficiency is modest. a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating The Risk to Development Outcome rating has two dimensions: (i) the likelihood that some changes may occur that are detrimental to the ultimate achievement of the operation’s development outcome; and (ii) the impact on the operation’s development outcomes of some or all of these changes materializing. From an operational, sector, and country context the integration of the two adjacent areas to the eco-city was intended to overcome fragmentation and push economic and urban development at scale, which could play a positive role in attracting investment, job creation, and residents. The implementation of KPI and monitoring of some of the performance standards is likely to be challenging in a much bigger area. An important factor considering the risk to development outcome is the size of the population of the eco-city. While the actual implementation of the project was not directly affected by the slow population growth, the underlying assumptions were that the urban facilities would be used by a much larger population than the current base. A much smaller population base means the project benefits would not be as large as originally estimated. The estimate of reaching 350,000 people by 2020 was too optimistic. The overall population in 2017 is about 40,000 and is unlikely to reach the target of 350,000 by 2020. As such, some recommendations made by the project are unlikely to materialize and the much smaller population raises the question on the risks to development outcome. There is also uncertainty over the types of industry that the eco-city would host, which would have significant implications on the core value of the development outcome. Recognizing all these risks, the SSTECAC issued its ‘Development Plan’ (2014–2020) to reaffirm the objectives of the eco-city and the extension of the KPI indicator system to all the new areas. However, given the track record of supply driven urban expansion around the world, there is a high level of uncertainty on the impact of these initiatives and a risk that the isolation of the eco-city will contribute to a fragmented urban region with negative consequences on environmental impact and economic development. a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) a. Quality-at-Entry Quality at Entry refers to the extent to which the Bank identified, facilitated preparation of, and appraised the operation such that it was most likely to achieve planned development outcomes and was consistent with the Bank’s fiduciary role. Given the pilot nature of the Sino-Singapore Tianjin Eco-City Project quality at entry is of particular importance. In this context, the project team had to carry out due diligence with rigor on several fronts and approach the design of the project accordingly. In this context, the ICR recognizes preparatory work but also shortcomings in identification, preparation, and appraisal. An important factor considering quality at entry is the accuracy of population estimates to justify the project and develop a viable design. The underlying assumptions were that the urban facilities would be used by a much larger population than the current base. A much smaller population base means the project benefits would not be as large as originally estimated. Despite the substantive analytical work for project preparation, the risks faced by the Eco city in attracting business investment were underestimated and thus the mitigation measures presented at the time of appraisal were inadequate. However, the other two major risks were correctly rated as substantial after mitigation. Finally, as the ICR highlights that the KPIs for the project were vague. This low level of granularity limited the ability of the project team and the implementing agency to supervise the operation with sufficient rigor. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Quality of supervision refers to the extent to which the Bank proactively identified and resolved threats to the achievement of relevant development outcomes and the Bank’s fiduciary role. Bank performance is rated, as applicable, against the following criteria, (i) Focus on Development Impact; (ii) Adequacy of Supervision Inputs and Processes; (iii) Candor and Quality of Performance Reporting. The ICR details a series of challenges faced by the project during appraisal as well as measures that were taken to address major issues during supervision. In addition, it recognizes that in 2012 no supervision missions were carried out. Against this background the rating of project performance in ISRs are moderately satisfactory for 2012 and satisfactory thereafter. Even with implementation challenges and design flaws in overestimating the growth of the Eco city – the ratings were satisfactory for the next seven missions held 2013 – 2016. This puts into question the quality and candor of performance reporting. Indeed, the ICR points to the fact that “Project implementation progress was reported, and legal covenants were monitored and enforced. However, given that the KPIs for the project were vague, it would have greatly facilitated the tracking of progress and monitoring of impact had the Implementation Status and Results Reports (ISRs) provided more specific descriptions of the outcome and impact of the individual items listed in the results framework”. Another important consideration raised as a lesson by the ICR points to the fact that supervision missions for a TA type of project could be enhanced by expert meetings with policy makers in the relevant areas. The ICR emphasizes that the project was relatively high-risk and during implementation and the World Bank monitored the risk closely and recognized the possibility that the risk would materialize. However, four consecutive ISRs from April 2014 to April 2016 marked the risk as low – even as actual population of the Eco- City lagged far beyond projections. Finally, it is contradictory to have the ICR state that the World Bank made an informed decision not to cancel or restructure the project because the rewards would be extremely high if Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) the project succeeded – even though the ISRs were satisfactory. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance Government remained committed partners in the project, working closely with both the World Bank and other implementing agencies to ensure successful preparation and implementation. According to the ICR the counterpart funds were sufficient—which was about US$58 million —and allocated on time. No major issues with requesting funds or processing reimbursements were encountered. Government Performance Rating Satisfactory b. Implementing Agency Performance To facilitate the project implementation, the Tianjin Municipal Government conformed a PMO, which included representatives from the Tianjin Municipal Government, Tianjin Urban and Rural Construction and Communication Commission, and the SSTECAC. The SSTECAC is the Chinese local authority established to meet the objective of carrying out all the government administrative functions for Sino-Singapore Tianjin Eco-City with an integrated mandate to oversee the eco-city planning and implementation process. The PMO proactively addressed institutional issues such as proposing to replace the PIU—Tianjin Eco-City Construction and Investment Company. Implementing Agency Performance Rating Satisfactory Overall Borrower Performance Rating Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The M&E framework added value by monitoring and evaluating the increase in the share of public transport within SSTEC as well as the incremental annual energy savings and GHG reduction in the two GEF grant– funded pilot green buildings. However, there were weaknesses in the design of M&E. Most of the project Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) indicators under Components 1 and 2 were output-based, the project progress reports simply recorded the completion of the activities. Thus, the project’s progress reporting did not fully measure whether these activities result in the outcomes required to achieve the goals of these components and the PDOs. b. M&E Implementation Data collection and reporting for M&E was undertaken by the PMO. Indicators were added to new components, but similar to existing components, all of them were output-based, such as ‘reported completed and accepted by SSTECAC’. The component on pilot investment in green buildings had a more robust M&E design and implementation. The implementing agency collected data and monitored energy savings, emission reductions, and the use of renewable energy on an annual basis after the completion of building construction. The information was incorporated into semiannual progress reports as well as the borrower’s Implementation Completion and Results Report. c. M&E Utilization To a limited extent the M&E focused on assessing whether the theory of change within the project causality logic was sound but in cases where it did it lead to reframing of strategy – for example on issues regarding the definition of bus stops and pedestrian access. The project-proposed methodologies for a comprehensive monitoring and tracking of the city’s KPIs were difficult to apply as they were based on a much larger population base than the actual population at completion of the project. Except for real time monitoring of energy use, given the nature of the KPIs - the M&E focused largely on outputs rather than outcomes which limited it utility as well as its utilization. In this context there were limited shifts in the project’s direction and outcome attributable to M&E activities M&E Quality Rating Modest 11. Other Issues a. Safeguards The project was classified as a Category B, with the Environmental Assessment (OP 4.01) safeguard triggered. Environmental impacts from pilot building construction were moderate, confined, and of temporary nature, as detailed in the SSTEC Green Construction Code. The TA activities would not create any direct adverse environmental impact; however, the studies supported the overall implementation of the SSTEC master plan and could lead to downstream development that might potentially have environmental impacts. The Guidelines of the Environmental Impact Management, acceptable to the World Bank, were prepared for the TA. The most serious potential negative effect -- which is addressed in the project only in passing -- is the effect Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) of land use within and outside the Eco city boundaries. The 400x400 meter blocks have the potential of locking in an urban land use which is unfriendly to transport and pedestrian. In addition, the discontinued urban growth can generate urban sprawl as well as excessively long commuting trips. According to the ICR, there were adverse impacts related to land acquisition and involuntary resettlement as a small part of the whole SSTEC project, but this took place before the project began. Specifically, in total, 16km2 land was acquired by the government from three villages, two state-owned enterprises, and a military enterprise within the project area. Two of the villages had collectively owned land, while the third village was already transformed into an urban neighborhood community which became state-owned. The resettlement process began in 2007 and all the land was acquired in 2008. All resettlement was completed in late 2008 or early 2009 by TMG with the exception of the relocation of one military factory, the Bayi Salt Plant (the legal agreement was signed for relocation in 2008 and SSTEC has agreed to allow the office of the Plant to stay on until their new office building is ready around 2011). The PMO reported that no additional land would be required for the project and the task team foresees that the project implementation will be within the current project land. In summary, the resettlement process started before the Bank project identification and was completed by early 2009. (PAD p. 13). According to the ICR, implementation of the safeguard instrument Resettlement Policy Framework was satisfactory. All the project pilot investment on buildings were on existing public land. Throughout implementation, no additional land acquisition or resettlement occurred and no social risks or negative impacts emerged (ICR, p.10). b. Fiduciary Compliance Financial Management. Appropriate FM arrangements were put in place to ensure proper use and accounting of project funds. Project audit reports were provided for the entire implementation period with unqualified (clean) audit opinions, and no FM-related issues were raised in the audit reports (ICR, p.11). Procurement. The project included some high-value consulting services contracts and small-value contractors through national competitive bidding and shopping methods. The overall procurement progress was satisfactory, according to the ICR (p.11), and no substantial deviations from the World Bank’s policies and requirements were identified in the procurement prior and post reviews throughout the project implementation period. The only complaint received from one consultant was about payment delay. At the request of the World Bank, the PMO rectified the problem by actively communicating with the consultant in writing and making payment on time. c. Unintended impacts (Positive or Negative) A positive impact can be the demonstration effect that the city will have as a learning laboratoty of urban innovation. The principal unintended negative impact is the fragmented urban expansion which has the potential of resulting in environmental and economic costs compared to contiguous urban expansion. This Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) problem is exacerbated by the slow demographic growth of the EcoCity. d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory The very significant discrepancy of the actual and Risk to Development projected population growth of Substantial High Outcome the Eco-City puts the development outcome at high risk. Moderately Moderately Bank Performance --- Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory --- Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons From IEG • Fostering efficient urban expansion. Adhering to fundamental urban planning principles, it is necessary to avoid fostering supply driven discontinuous urban expansion which promotes urban sprawl and inherent resource inefficiencies. SSTEC design principles of ecological design would be enhanced by taking into account land use and spatial considerations. • Population estimates need to be realistic. Population growth estimates of newly established urban settlements require careful and independent verification which can rely on objective methodologies. Overly optimistic growth scenarios which rely on supply driven projections of employment require appropriate risk Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) mitigation to ensure the flexible implementation of urban expansion. From the ICR with some adaptation of the language: • Timely upstream technical assistance to cities can have a significant impact on urban planning and strategy. Despite the relatively small amount of funding, especially in comparison with the massive investment made in the eco-city by the GOC, the impact of TA on the strategic direction of the eco-city’s low carbon development and resource management was substantial. The TA activities were initiated during the planning and development phase of the SSTEC, thereby ensuring inputs for shaping the modality of the city’s resource management and transport mode. • Understanding connectivity of a city. City planning in China often places an overt thrust on the physical infrastructure within cities— such as roads, buildings. However, the livability of a city is not only dependent on physical conditions, but also on its people, their social networks, community, and culture. The factors that drive people’s decision to move from a familiar social and environmental setting to a completely new city are manifold. Providing physically better living conditions is not be sufficient condition to foster urban expansion, as the experience of the eco-city. • Engagement and dialogue are critical for maximizing the impact of technical assistance. The project produced numerous reports and analytical products. Utilizing the recommendations of the reports requires putting in place a process and making considerable efforts for dialogue with the relevant agencies. For example, after the submission of the reports by the consultants, workshops and work meetings could have been organized by the World Bank to facilitate the policy dialogue and support the clients for the implementation of the recommendations. Also, the supervision missions for a TA type of project could be enhanced by expert meetings with policy makers in the relevant areas. 14. Assessment Recommended? Yes Please explain The project represents an important pilot Eco-City experience with significant experimental components and approaches that merit field assessment after 2020 when the plan is expected to be mature. From an accountability perspective, it would be important to document the full evolution of the project. Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-GEF Sino-Singapore Tianjin Eco-City (P098915) 15. Comments on Quality of ICR The ICR provided a sufficient and adequate quality of evidence describing the project in each stage of the project cycle as well as analysis to explain the outcome, risk to development outcome as well as the performance of the Bank and the borrower. The ICR also presented, to the extent possible, an outcome- driven account and it is not an implementation narrative. The ICR has internal consistency across the different sections and the lessons presented are based on the evidence and analysis. a. Quality of ICR Rating Substantial Page 17 of 17