Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) Report Number : ICRR0020958 1. Project Data Project ID Project Name P092631 CN-Xi'an Sustainable Urban Transport Country Practice Area(Lead) China Transport & Digital Development L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-75580 30-Jun-2015 514,405,000.00 Bank Approval Date Closing Date (Actual) 24-Jun-2008 31-Mar-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 150,000,000.00 0.00 Revised Commitment 150,000,000.00 0.00 Actual 150,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Victoria Alexeeva Fang Xu Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives Original Objective: "To improve transport accessibility and mobility in Xi’an Municipality while protecting its cultural heritage and reducing the environmental impact of the urban transport system" (Loan Agreement dated September 29, 2008, page 5). The statement of objective is identical in the Project Appraisal Document (PAD, page 7). It further specified that the priority area for achieving this objective was the Ming Walled City (MWC). Revised Objective: "To improve transport accessibility and mobility in Xi’an Municipality and enhance air- quality monitoring of the urban transport system" (Amendment to Loan Agreement dated January 6, 2015, Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) Annex 1, page 5). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 22-Dec-2014 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components Original Components 1. Road Network (estimated total cost: US$232.1 million).This component supported (i) improvements at selected sites to enhance the functionality of the First and Second Ring Roads and Taibai Nan Road as major traffic routes in Xi’an Municipality; and (ii) upgrading of the road network by improvement of the existing Xincheng and Dongcheng primary roads and of the existing Meibei secondary road, and construction of the new Lvgong primary road, including a rail underpass, to improve accessibility in the urban area of Hu Xian within Xi’an Municipality. 2. Public Transport (estimated total cost: US$67.3 million). This component supported (i) construction of two new passenger transport terminals to be located in the eastern part of Xi’an Municipality; and southern part of Xi’an Municipality; (ii) development and implementation of integrated public transport priority and traffic management packages in about thirteen selected bus corridors in Xi’an Municipality with a total length of about 130 kilometers; (iii) construction of a bus depot at Xinzhu to enable the expansion of services in the north east sector of Xi’an Municipality between the Second and Third ring roads. 3.Traffic Management (estimated total cost: US$39.1 million). This component financed development and implementation of selected traffic management and road safety measures to enhance the operation of the existing road network in Xi’an Municipality, including enforcement of road user education programs, and provision of: (i) an area traffic control (ATC) system; (ii) road safety equipment and software: (iii) vehicle parking equipment; (iv) traffic facilities for pedestrian and cyclists; and (v) technical assistance. 4. Air Quality Management (estimated total cost: US$17.5 million). This component supported implementation of Xi’an Municipality Environmental Protection Bureau’s comprehensive plans for air quality improvement and reduction of vehicular emissions, including: (i) construction of Xi'an Municipality Ambient Air Supervising and Monitoring Center; (ii) provision of equipment for motor vehicle emission inspections; (iii) construction of two ambient air quality monitoring sub-stations and two traffic air pollution monitoring sub-stations and provision of equipment for those sub-stations; (iv) provision of equipment and software for air quality assessment and information publication; (v) development of a motor vehicle emission control plan; and (vi) provision of technical assistance. 5. Cultural Heritage (estimated total cost: US$54.2 million). This component supported (i) preservation of Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) the cultural heritage of the Han Chang’an site by recreating about 8.4 kilometres of the old Han Dynasty road network in the area of the Weiyang Palace; and (ii) construction of about 27 kilometers of bicycle routes around the main tourist sites inside the Ming Wall City. 6. Institutional Development (estimated total cost: US$4.2 million). This component supported strengthening of the institutional capacity of Xi’an Municipality in integrated public transport planning and development through, inter alia: carrying out studies on: (a) strategic urban transport planning; and (b) financial planning for investments in transport infrastructure in order to develop capacity to formulate investment plans; provision of technical assistance to improve capacity in the areas of traffic management and road safety; and provision of domestic and international staff training. Revised Components The components remained the same but with the following revisions to the activities: 1. Road Network (actual total cost US$284.9 million): Upgrading of the Dongcheng Road in Huxian was cancelled. 2. Public Transport (actual total cost US$53.6 million): Development and implementation of integrated public transport priority and traffic management packages in about thirteen (13) selected bus corridors was reduced to two (2). The construction of the new Chengchen bus depot with equipment was added in western Xi’an. 3.Traffic Management (actual total cost US$13.6 million): Intersection channelization, accident remedial measures, and road-user education were cancelled. Engineering project supervision was added. 4. Air Quality Management (actual total cost US$26.9 million): Ventilation and water purification equipment for laboratory construction and equipment was added for the air-quality monitoring center. 5. Cultural Heritage (actual total cost US$17.3 million): A 27-km bicycle route was cancelled in the Ming Wall City. 6. Institutional Development (actual total cost US$1.7 million): Strategic urban transport studies were replaced by the analysis of Xi’an’s sustainable transport project experiences and achievements. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost: At the design stage, total project cost was estimated at US$458.14 million including physical and price contingencies. The actual cost was U$398 million due to significant changes in the design and cancellation of multiple activities. Financing: The Bank's commitment for this project was US$150 million which was fully disbursed. Borrower contribution: Borrower contribution was US$308 million at design stage, the actual contribution was US$ 247.98 million. Dates: The original closing date of June 30, 2015 was extended twice to March 31, 2017 to allow the completion of project activities and resettlement. The project was restructured in 2015 to drop several activities, revise the objective and the results framework. The closing date was extended by one year to June 30, 2016 in order to enable the restructured project to be completed. A further extension of nine months was approved to complete the resettlement of 76 households under the project. Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) 3. Relevance of Objectives & Design a. Relevance of Objectives Original Objectives. The Government of China identified urban infrastructure development as a priority area. The project supported the implementation of State Council Opinion #46 on "Priority to Public Transport", the National Road Traffic Safety Law 2004, and stricter emission regulations issued by the State Environmental Protection Agency (SEPA) in July 2005, designed to regulate in-use vehicle emissions in a loaded condition (PAD, page 7). At project design stage, Xi’an municipality also identified strengthening public transport performance and improving local air quality as priorities, in addition to enhancing transport planning capacity, protection of MWC and facilitating economic development in Huxian. The WBG Country Partnership Strategy for China (2006-2010) set the objectives of (i) promote balanced urbanization, and expanding access to basic social and infrastructure services; and (ii) manage resource scarcity and environmental challenges, through reducing air pollution, optimizing energy use, improving land administration and management, protecting cultural and natural heritage, promoting cultural tourism, and observing international environmental conventions. The WBG CPS for China (2011-2015) again emphasized investments in urban public transport, cultural heritage, traffic safety, and emission regulation. The original objectives were broadly relevant to the Xi’an municipality priorities, however these priorities evolved regards specific investments in public transport, bus lanes, non-motorized transport, and cultural heritage in Xi’an leading to changes in targeted interventions (ICR, page 7, 16). Revised Objectives. The revised PDO of “improving transport accessibility and mobility in Xi’an Municipality and enhance air- quality monitoring of the urban transport system” are relevant to the priorities of the government and the latest WBG country partnership strategy FY13-16 with the strategic theme of supporting greener growth, which focused on promoting low-carbon urban transport, among other. Rating Revised Rating Substantial High b. Relevance of Design Original Design. The original design was not sufficient for the achievement of a broad PDO. The linkages between the project outputs, expected outcomes and the objectives were weak. The implementation of urban road network improvement, public transport development and traffic management activities would contribute to mobility and accessibility improvement in the area within the influence of the project. But the broad objectives related to cultural heritage and environmental protection were not supported by adequate activities.The cultural heritage component relied on the implementation of a bike improvement program and road rehabilitation in the tourist area. The project’s environmental interventions were limited to equipment provisions and the development of Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) a motor vehicle emission control plan. The objective of “protecting cultural heritage and reducing the environmental impact of the urban transport system” could not be achieved unless the project had included a set of more comprehensive interventions. At the same time, the design was complex with 47 sub-components involving multiple activities in road infrastructure, traffic management, public transport, and preservation of cultural heritage (with enabling components for transport and environmental management) in disparate areas in and around Xi’an. The scope had to be significantly reduced during implementation. Revised Design. The revised statement of the objectives retained its focus on accessibility and mobility, and substituted two objectives related to cultural heritage and environmental protection with enhancing air-quality monitoring of the urban transport system. This change enhanced the relevance of the design to the revised PDO. The design was simplified, with substantial reduction of the number of activities to speed up the implementation. The newly added objective of enhancing air-quality monitoring of the urban transport system was, however, more output-oriented. Rating Revised Rating Modest Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve transport mobility in Xi’an Municipality. Rationale Main outputs: • Six selected road corridors (about 12 km with interchanges) were improved as planned at appraisal stage; • Three roads (about 12km) were upgraded. The original plan was to upgrade four roads but was reduced to three roads at the time of project restructuring. • Four new bus terminals and depots were constructed as against the original plan of constructing three. • One bus priority lane was built under the project as against the original plan of 13; the remaining 12 were financed by the local government fund. • An area traffic control (ATC) system covering 200 junctions, road safety equipment and software, vehicle parking equipment, traffic facilities for pedestrians and bicycles were installed as planned at project design stage. • Institutional development activities including studies on traffic management, road safety, financial planning and domestic and overseas training. A Xi'an Urban Transport Strategy study was completed with local funds. 15 training and study tours in urban management were completed against the targeted 18. Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) Outcomes under originally- defined and revised targets. High. The improved mobility could be evidenced by the faster traffic flows in the project area or on the improved road infrastructure. Under this project, the project’s support for mobility enhancement was mainly through infrastructure improvement and traffic management. At the project closure, the evidence of improved mobility was the increased motor vehicle speed from baseline of 12km/hr to 45km/hr on the key corridors under the project (East Gate, South First Ring, Yuxiang Gate), exceeding the original target of 18-25 km/hr and revised target of 20km/hr. Rating High PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Same as above. Revised Rationale Details as outlined above. Revised Rating High PHEFFICACYTBL Objective 2 Objective Improve transport accessibility in Xi’an Municipality. Rationale Outputs as described above under Objective 1. Outcomes under both the original and revised indicators. Modest. The improved accessibility could be evidenced by improved ability of transport service users to reach goods, services and activities, the accessibility is affected by multiple factors including mobility, quality and affordability of transport service, land use plan, etc. Under this project, the project’s support for accessibility enhancement was mainly through infrastructure improvement and traffic management. However, the project did not provide evidence on improved accessibility, e.g., within one hour travel time, the road users now have access to more facilities and/or service centers than before the project. The achievement of “improved accessibility ” is rated modest for lack of evidence. Rating Modest Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Revised Objective Same as above. Revised Rationale Details as outlined above. Revised Rating Modest PHEFFICACYTBL Objective 3 Objective Protecting cultural heritage in Xi’an Municipality. Rationale Main outputs • 8.4 km of five Han Dynasty roads were recreated as planned. • The construction of a 27- km bicycle route was cancelled in the Ming Wall City. Outcome The project identified a number of indicators to measure the achievement of this objective but these indicators did not fully capture the nature of protection of cultural heritage, as they focused on reduction in travel time and bicycle use to cultural heritage sites within the MWC. At the time of project restructuring, the construction of bicycle route was cancelled and the objective was dropped. At project closure, there was no evidence of the achievement of this objective. Rating Negligible PHREVDELTBL PHINNERREVISEDTBL Objective 3 Revision 1 Revised Objective Objective was dropped. Revised Rationale Not applicable Revised Rating Not Rated/Not Applicable Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) PHEFFICACYTBL Objective 4 Objective Reducing the environmental impact of the urban transport system in Xi’an Municipality. Rationale The achievement of this objective would be evidenced by the reduced GHG emission and/or pollutants from the urban transport system in the city of Xi’an. The project did not have a clear indicator for this objective. The main interventions to reduce motor vehicle emissions were the development, adoption and implementation of a Motor Vehicle Emission Control Plan (PAD, page 38), and enhanced air quality supervision and monitoring (see outputs under Revised Objective below). At the time of project restructuring, the objective and the adoption of the Motor Vehicle Emission Control Plan were dropped. At project closure, there was no evidence of the achievement of this objectives. Rating Negligible PHREVDELTBL PHINNERREVISEDTBL Objective 4 Revision 1 Revised Objective Enhance air-quality monitoring of the urban transport system in Xi’an Municipality. Revised Rationale Outputs • An area traffic control system was installed as planned (there are stationary and mobile traffic air-quality monitoring facilities). • Environmental management information system was established and is operational. • A Motor Vehicle Emission Control Plan to reduce vehicle emissions was developed and submitted to Xi'am Municipality. Outcomes The number of vehicles receiving emissions testing increased from 40 in 2008 to 2500 in 2017, significantly above the target of 500. Revised Rating Substantial PHREVISEDTBL Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) 5. Efficiency Economic analysis. At appraisal, the cost benefit analysis was conducted only for the road network component that estimated an economic internal rate of return (EIRR) of 14 percent, with an NPV of RMB296 million (at a 12 percent discount rate). Transport user benefits (i.e., savings in vehicle operating costs [VOCs], accidents and public transport passenger travel time) were estimated for each road scheme. Reduction in travel times for automobile users was not quantified because of the lack of reliable data. At closure, the cost-benefit analysis was expanded to the traffic management component as this component contributed to travel benefits estimated under the road network. No public transport benefits were quantified because only one bus lane was implemented in the project. The ICR (page 24) points to an extreme paucity of data, both at appraisal and at completion. Necessary and crosschecked data used in the ex-post CBA was imported from several sources. Benefits from VOC savings were assumed to be very small. Reductions in accident costs were not calculated due to the lack of data. The ex-post EIRR was estimated at 33%, with an NPV of US$ 595.5 million (at a 12 percent discount rate). Benefits were estimated based on the selected assumptions for travel demand and costs with reference to different studies (ICR, page 23-24). Cost effectiveness. The ICR does not provide a cost analysis or unit cost comparison for similar road works. Operational and administrative efficiency. The project's efficiency was weakened by considerable delays during implementation and significant changes to the design. The project scope was significantly reduced, while the closing date was extended by 21 months. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 56.00 Appraisal  14.00 Not Applicable 71.00 ICR Estimate  33.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Original Objective. Relevance of objectives is rated substantial, and that of design is modest. The achievement of the objective to improve transport mobility in Xi’an Municipality is assessed as high; improvement in accessibility was achieved only modestly for lack of evidence in relation to the outcome measure for accessibility. Two other objectives of protecting its cultural heritage and reducing the environmental Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) impact of the urban transport system are rated negligible, as the project did not include related activities in support of these objectives and no evidence is available. Efficiency is assessed as modest for the weak operational efficiency of the project, despite a satisfactory economic rate of return. Based on the above ratings, the overall project rating is moderately unsatisfactory. Revised Objective. Relevance of objectives is rated high, and that of design is substantial. The achievement of the objective to improve transport mobility in Xi’an Municipality is assessed as high; improvement in accessibility was achieved only modestly for lack of evidence in relation to outcome measure for accessibility. The project also helped enhance air-quality monitoring of the urban transport system in Xi’an Municipality to a substantial extent. Efficiency is assessed as modest for the overall weak operational efficiency of the project, despite a satisfactory economic rate of return. Based on the above ratings, the overall project rating is moderately satisfactory. A split rating is applied based on the disbursement shares before and after the project restructuring in 2015, which were 40% and 60% accordingly. The weighted value is 3.6 (equivalent to moderately satisfactory). a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating O&M costs. The ICR does not discuss what agency would be responsible for maintenance but notes (page 11) that for the roads, there is historical precedent with proven competence in road maintenance and its financial support; for public transport, the continued expansion of services indicate sustainability. It however also points to the uncertainties related to improving public transport and expanding traffic management to cover a wider area. Institutional. Government ownership is evident. XMG applied to and was approved by the Ministry of Transport to be one of the thirty pilot cities for a Transit City in China. The City developed and put in operation 20 bus priority corridors with a total length of 325 km. The Municipality also implemented a city-wide bicycle program with a fleet of 52,000 public bicycles with annual 70 million users. The implementation of the motor vehicle emission plan continues and over 31,000 vehicles were tested in 2017. Plans are being developed for further development of the historic Weiyang Palace area. Technical. No difficulties were experienced in the transition to expanded post-project operations with the new environmental management information system and equipment, indicating that technical capacity is strong to maintain the modest equipment. a. Risk to Development Outcome Rating Modest Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) 8. Assessment of Bank Performance a. Quality-at-Entry Project preparation began in 2003, with concept review in 2005, and final approval in 2008. The comprehensive project vision was supported by public consultations, studies and official inputs during preparation. Despite a long preparation time, more attention could have been paid to the project design, resettlement scope, and possible changes in policy. As the ICR (page 7) notes, the client and the Bank held different opinions regarding the most effective remedies for issues faced by Xi’an and the project content evolved up to approval. The Bank emphasized improvement of public transport and non-motorized transport, while the client focused improvement of private motorized vehicle travel speed. The original design had six components which included 47 sub-components (see Section 3). The numerous implementing entities added to the difficulties. Numerous implementing agencies and procurement plan for 114 contracts stretched the capacity of the Project Management Office (PMO). While risk assessment was comprehensive, mitigation measures, in particular in relation to procurement, proved to be insufficient. Other shortcomings included: the public transport and cultural heritage components were not technically ready; the magnitude of resettlement required was significantly underestimated (4,770 people estimated at appraisal versus 11,433 resettled); and M&E was deficient due to the broad PDO and some indicators had no baseline data at approval. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision There was a slow progress in the first period of implementation, largely reflecting the weaknesses of design and the lack of readiness. The ICR (page 16) also points to weak supervision before project restructuring that contributed to implementation and disbursements delays. After project restructuring, the Bank team, supported by management, was proactive to resolved implementation issues including delays in procurement, some shortcomings in financial reporting, and local supervision oversights. The measures taken by the team sped up the project implementation. The PDO and results framework were revised at restructuring, however no outcome indicator was defined for accessibility. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) a. Government Performance There was an overall government commitment to the urban infrastructure development, however as the ICR (page 16) describes there were disagreements and changes in governance concepts, particularly on public transport (cost recovery and privatization), in implementation priorities (e.g., bus lanes and bike routes). Key government officials changed during implementation. Decision making in the project steering committee was sometimes slow and there were occasional delays or changes in approvals, counterpart funding for contractors, and grievance redress for affected interests. Coordination among government agencies was insufficient. There were notable delays in restructuring and resettlement compensation, which required the second nine-month extension. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance There were several implementing agencies (IAs), and the ICR described their performance as follows: The road network component's IAs: There were no issues in Xi’an, while in Huxian a learning curve in financial and project management was aided by training. 10 of the 12 road contracts were completed before restructuring. The public transport component's IAs: This component was affected by changes in government priorities. There were issues with repeated design and site selection delays for the bus depots. Traffic management IA : This component was affected by the lack of design readiness and some parts were cancelled. The procurement of ATC, software, hardware, and the inter-agency coordination among four bureaus proved to be challenging. After restructuring and engineering support, IA delivered the results. Air Quality Management component IA: This expanded component had the most diverse procurement portfolio (22 contracts), which caused delays due to coordination and the volume of work. The Motor Vehicle Emission Control Plan was implemented. The Air Quality Monitoring building with modern laboratories was executed well with benefits that exceeded project targets. Cultural heritage IA: This component was reduced by 60 percent. The Municipality used its funds to implement a citywide bicycle rental program and build a cultural heritage display center at the site, which received the United Nations Educational, Scientific and Cultural Organization (UNESCO) World Heritage status in 2014. The PMO of the project as a whole: PMO executed and managed the component well and on time. Study tours, training and technical assistance studies, consultancy and training in traffic management, and three useful financing studies were completed. On balance, the overall rating for IA Performance is moderately satisfactory. Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) 10. M&E Design, Implementation, & Utilization a. M&E Design In the original results framework, there were two main outcome indicators: (i) a trip modal split (including walking) to and from the MWC, and (ii) bicycle use and travel times to cultural heritage sites within the MWC. A trip modal split (including walking) into and out of MWC reflected the target for increasing the share of walking from 21.7% to 22%, public transport from 25% to 28%, motor vehicle from 18% to 21%, and the decrease in bicycle usage from 35.2% to 29% (PAD, page 39). It is not clear why the share of bicycle usage was to be reduced in the original results framework, in particular if the project supported the construction of a bicycle route. The original M&E design had significant shortcomings: i) flawed and complex results chain reflected by some components that did not contribute to the achievement of the PDO, ii) lack of baseline for many indicators; and (iii) choice of outcome indicators that did not fully reflect the objectives and many were not attributable to the project and hard to measure. The revised M&E design at the time of project restructuring was improved, and the outcome and results indicators established a logical results chain for the PDO. There were however some shortcomings too, i) there was no indicator for improved accessibility (such as number of service centers within an hour travel); (ii) bus terminals, which are important for dispatching, on-time service and passenger information, were built but not counted; (iii) ambiguity of some indicators remained (e.g., “maintenance days”); and (iv) there were no indicators for safeguards (ICR, page 9). b. M&E Implementation Originally, the implementing entities, advised by the PMO, were to carry out the measurement of the M&E indicators but this did not work out as intended. Measurement tasks were subsequently outsourced to a university, and the M&E system was implemented in an acceptable manner. c. M&E Utilization The revised M&E was overall useful in monitoring the progress of implementation and achievement of the revised PDO but the objective of "improved transport accessibility" was not measured by any indicators. The M&E results were used to calculate benefits in the project CBA. M&E Quality Rating Modest 11. Other Issues Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) a. Safeguards The project was classified as Category A under OP 4.01 Environmental Assessment. Two other safeguard policies were triggered by the project: Physical Cultural Resources OP/BP 4.11, and Involuntary Resettlement OP/BP4.12 (PAD, page 21-22). The policies did not change during project restructuring. Environmental safeguards. According to the ICR (page 10), Environmental Impact Assessments (EIAs) and Environmental Management Plans (EMPs) were adhered to. Bank supervision missions rated compliance with environmental guidelines and EMP implementation as satisfactory. Xi’an has stringent site management regulations for planning, construction, supervision and public communication. Comprehensive mitigation measures for noise and dust, worker camp management, handling of construction waste, and disturbance to local traffic were consistently implemented. The road component of the Weiyang Palace historic site adopted planning, design, construction and supervision practices that were consistent with the project’s EMP. Chance find procedures for the protection of unidentified cultural heritage resources were followed during construction (ICR, page 10). Social safeguards. The ICR (page 10) reports that Bank social safeguard instruments, including the Resettlement Action Plan (RAP) and the Resettlement Policy Framework (RPF), were complied with and implementation performance was rated moderately satisfactory. In all, 918,700 m2 (1,378 mu7) of land was acquired, 152,000 m2 (228 mu) less than originally planned, and 99,945 meter2 of houses and buildings were demolished. A total of 2,371 households (11,433 people) were resettled under this project. This is significantly more than the 527 households (4,770 people) estimated at appraisal due to changes in the project design. Payments to 39 households in Xinjiamiao village and to 37 households at the Weiyang. Palace construction site were delayed and resulted in the second extension of the loan closing date. 39 households in Xinjimiao chose monetary compensation and received full compensations by December 31, 2016; however, 37 households at the Weiyang Palace site chose to receive new housing instead of monetary compensation and elected to wait for the new resettlement site. These households will continue receiving transition subsidies until they move to their permanent houses b. Fiduciary Compliance Financial management. The PMO provided timely project audit reports to the Bank in accordance with the legal agreement. Management Letter comments were duly addressed by the PMO. The Bank’s implementation review noted minor shortcomings: (i) late submission of some Interim Financial Reports (IFRs); (ii) minor delays in disbursement before restructuring; and (iii) weaknesses in Huxian PMO’s financial management system. These issues were resolved through joint efforts by the Provincial Finance Bureau and the implementing entities; training was provided to the Huxian PMO to improve its financial reporting. Procurement. The Bank’s procurement policies and procedures were complied with, as were China’s procurement laws and regulations. Professional procurement agents and experienced with international financial institutions provided support when necessary. There were procurement delays due to the time taken for government approvals, site availability for bus depots, the complexity of technical specifications for issuing the bidding documents, and an occasional lack of coordination between participating City Bureaus and the Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) PMO. The initial total of 114 contracts (the final number was 87) in multiple agencies was a management challenge in terms of specifications, approvals and implementation. c. Unintended impacts (Positive or Negative) --- d. Other The ICR (page 15) reports the following benefits: The new building and equipment enabled the Environment Bureau to begin monitoring not only air quality but also noise, soil, radiation, acid rain, and sandstorms. Another impact was the heightened awareness of cultural heritage and the image of the city in urban transport planning and land use. 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory Risk to Development Modest Modest --- Outcome Moderately Moderately Bank Performance --- Satisfactory Satisfactory Moderately Moderately Borrower Performance --- Satisfactory Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons IEG has selected three lessons from the ICR: • Cost uncertainty in resettlement must be addressed during project preparation. It is difficult to estimate the exact resettlement cost during project preparation, which depends on identification of affected households, permits for construction and land acquisition, and the agreed compensation rates that are known Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) after the engineering designs are approved. Delays in engineering design approval or changes in designs lead to delays in reaching agreement with affected households on the resettlement terms. Therefore, it is important that this potential uncertainty is explicitly recognized during preparation, and that timely coordination is instituted between the different agencies tasked with engineering design and resettlement compensation. • Disjointed procurement leads to delays. The public transport, traffic management and air quality monitoring components included disjointed component designs (e.g., separate procurements and disbursement percentages for buildings, furniture and ventilation, and separate procurements for CCTV cameras, enforcement cameras, wireless networks, variable message signs), which caused delays and difficulties in coordination. Aggregation of component elements saves procurement resources and costs and helps avoid delays. • Maintain flexibility during implementation to address any potential issue not foreseen during preparation. Xi’an had undergone a rapid economic and social change since the start of this project’s preparation. The project had witnessed political crosswinds and resulting policy changes. For political and policy reasons, some diligently prepared sub-components were canceled at appraisal and during implementation. Generally, if the project city, province, and country circumstances are complex and the necessary remedies are comprehensive, the best approach would be to develop the project in one year or less (and not in three or more years as in the present project) to reach understanding and agreement on the principal issues and immediate implementable remedies. A more complete and effective set of responses and remedies to the principal issues can then be developed during implementation. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR is concise for this complex project with multiple activities. It describes candidly the issues that affected project implementation and provides an insightful analysis of project design, results framework, and the quality at entry. The lessons are evidence-based. The efficacy section could have been expanded with a more comprehensive analysis of attributable outcomes achieved under the project, including the accessibility. While the description of safeguards is adequate, the ICR does not mention Involuntary Resettlement OP, among others, on page 7. The ICR refers to the CPS 2011-2015, while the latest one is CSP FY13-16. Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN-Xi'an Sustainable Urban Transport (P092631) a. Quality of ICR Rating Substantial Page 17 of 17