Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) Report Number: ICRR0022446 1. Project Data Project ID Project Name P132154 Nanchang Urban Rail Project Country Practice Area(Lead) China Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-82620 31-Oct-2018 189,492,846.23 Bank Approval Date Closing Date (Actual) 20-Jun-2013 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 250,000,000.00 0.00 Revised Commitment 189,492,846.23 0.00 Actual 189,492,846.23 0.00 Prepared by Reviewed by ICR Review Coordinator Group Elisabeth Goller Peter Nigel Freeman Victoria Alexeeva IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Loan Agreement (LA, page 5), the project development objective (PDO) was to provide an effective urban mass rapid transit system of appropriate quality along the Line 2 corridor from ZhanQianNanDaDao Station to XinJiaAn Station. In this review, the achievement of the PDO is assessed separately for the subobjectives to "provide an effective urban mass rapid transit system along the Line 2 corridor from ZhanQianNanDaDao Station to XinJiaAn Station" and "provide a mass rapid transit system Page 1 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) of appropriate quality along the Line 2 corridor from ZhanQianNanDaDao Station to XinJiaAn Station". This is in line with the assessment in the ICR. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Component 1: Construction of Line 2 (estimated costs US$1,139.04 million, without contingencies; actual cost US$957.13 million): This included all construction activities for 23.8 km of track and 21 stations of the urban rail Line 2, which connects western and eastern Nanchang. The Bank was expected to finance civil works contracts no. 5, 6, and 7, which included eight stations and tunnel sections. The stations included three interchange stations with the planned Line 3 and a future Line 4, and an interchange with Nanchang East railway station. The project was to partially contribute to the cost of a command-and-control center for the urban rail network. Component 2: Equipment for Line 2 (estimated costs US$624.99 million, without contingencies; actual cost US$347.53 million): This included all the equipment necessary for the successful operation of Line 2, such as rolling stock, power supply, control system, signaling system, communication system, monitoring system, fare collection system, safety and security system, ventilation and air conditioning system, water supply, sewerage and fire protection system, and station auxiliary equipment. Component 3: Design, Construction Management, and Technical Assistance (estimated costs US$185.48 million, without contingencies; actual cost US$62.27 million): It was to finance all activities for project design and preparation, construction management and quality assurance, and technical assistance and capacity building for relevant staff in the Nanchang Municipality and the Urban Rail Company (URC). The technical assistance included ridership modeling, scenario testing, fare integration, land value capture around stations, and financial management and internal audit functions for the Nanchang URC. Component 4: Safeguards and Other Construction Costs (estimated costs US$370.65 million, without contingencies; actual cost US$674.97 million): It was to finance land acquisition and resettlement costs, construction site preparation, including environmental mitigation measures, and other project related construction costs, such as engineering insurance, work safety assurance, inspection and acceptance, and project cost estimation. The project components were not revised during implementation. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Page 2 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) The total project cost was US$2,189.96 million, which is 85.17 percent of the appraisal cost estimate of US$2,571.23 million. Financing: The project was expected to be financed through an IBRD loan of US$250 million. Only US$189.49 million was disbursed, which is 75.80 percent of the loan amount. Borrower Contribution: The expected borrower contribution at appraisal was US$2,321.23 million (PAD datasheet). The ICR does not explicitly mention the borrower contribution, but it can be calculated as the difference between total actual project cost and the IBRD financing, i.e. US$2,000.47 million. This is 86.18 percent of the appraisal estimate. Dates and Project Restructuring: The project was approved on June 20, 2013, became effective on October 21, 2013, and was expected to close on October 31, 2018. It was extended by 14 months to December 31, 2019. The project extension on August 17, 2018 was necessary because of delays caused by ineffective coordination for utility relocation and traffic diversion, resettlement of commercial activities along Line 2, and the parallel construction of Lines 3 and 4. The restructuring to extend the closing date also included a minor revision of the results framework for the project implementation delays. 3. Relevance of Objectives Rationale Context at Appraisal. Like many other Chinese cities, at appraisal, Nanchang, with a population of 3.3 million in 2010, suffered from rapid urban expansion. This led to the doubling of the motorized travel distance from 2002 and 2010. Nanchang's old city center in the east was routinely congested. Car traffic in the western part of the city, with large avenues and ample parking, was expected to grow rapidly. With 13.5 percent, the public transport modal share was low compared to other Chinese cities. However, Nanchang's share of non-motorized transport (NMT) was higher in than in those cities. The western part of the city was still under development. Nanchang planned to implement Line 2 in anticipation of this future urban development to guide its urban structure and optimize its urban space layout by applying transit- oriented development (TOD) concepts to the new areas. When the project was prepared, public transport integration, TOD, and land value capture (LVC), powerful tools to enhance the effectiveness, sustainability, and quality of urban rail systems, were new to the Nanchang Municipality. At project completion, Nanchang's population had grown to 5.36 million (2019). It had an urbanization rate of 75 percent. This rate was projected to reach 83 percent by 2030. The average annual private vehicle growth was 13 percent from 2013 to 2019. The ICR, para 26, mentions that "Nanchang’s social and economic development, coupled with the expansion of the city’s scale and increase in travel distances, would continue to put pressure on the urban area," including on its transport system. Page 3 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) Previous Sector Experience. The project design followed the Bank's urban transport experience in other Chinese cities, especially the one of the Kunming Urban Rail Project, which also included TOD. In the latter, the need for an early focus on modal integration, effective coordination between stakeholders, and the integration between urban rail investment and the surrounding real estate features surfaced as important lessons. Relevance to Government Strategies. At appraisal, the PDO was in line with the 2012 State Council Directive on the Prioritization of Urban Public Transport Development and China’s 12th 2011-2015 Five Year Plan, which mandated increasing public transport modal share to 40 percent for large cities and aimed at creating integrated transport systems. The construction of Line 2 was part of Nanchang's 2009-2016 Urban Rail Transit Construction Plan. At completion, the PDO remained in line with China’s 13th 2016- 2020 Five Year Plan, which aimed at “developing better modern comprehensive transportation systems” by promoting efficient, integrated, low carbon, smart and safe transportation services. It was also in line with China’s 2014-2020 New Urbanization Plan, which guided local governments to develop more compact, green, smart, human, and transit-oriented cities and highlighted the need to accelerate the development of mass transit, such as urban rail and bus rapid transit, to build an integrated low-carbon urban transport system. Finally, it was in line with the 2015 Ministry of Housing and Urban-Rural Development guidelines on the planning and design of areas along urban rail transit, which are essentially TOD guidelines. The construction of Line 2 remained highly relevant for Nanchang, which in its second phase of urban rail development from 2015-2021 includes extensions of Lines 1, 2, 3 and 4. Relevance to Bank Assistance Strategies. At appraisal, the PDO was in line with Bank’s FY11-FY15 Country Partnership Strategy for China, which focused on (i) supporting greener growth and (ii) promoting more inclusive development. At completion, the PDO remained in line with Bank's FY20-FY25 Country Partnership Framework for China. Engagement area 2 “Supporting greener growth” envisages low-carbon transport and cities with the aim to mitigate greenhouse gas emissions and environmental impacts from transport through integrated planning of transport and urbanization. The PDO statement is outcome-oriented and clear, but lacks a definition of "appropriate quality". The PDO is somewhat timid in its ambitions for a country like China because key determinates of the urban mass rapid transit system's effectiveness, such as the success of TOD and LVC were not measured. Although a lower carbon footprint was the key focus of the Bank assistance strategies and increased public transport use reduces this footprint in cities, the project did not explicitly measure this reduction from the modal shift from private vehicles to public transport (for details see section 9). Overall, relevance of objectives is rated substantial. This rating reflects the PDO's high relevance in the light of Government and Bank priorities, and the somewhat timid ambitions of the PDO because of its measurement shortcomings. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) Page 4 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) EFFICACY_TBL OBJECTIVE 1 Objective To provide an effective urban mass rapid transit system along the Line 2 corridor from ZhanQianNanDaDao Station to XinJiaAn Station. Rationale The theory of change for subobjective 1 was that the activities related to (i) designing, constructing, and providing systems and equipment for Line 2, (ii) providing technical assistance on ridership modelling, LVC, and financial management, (iii) carrying out study tours and training, and (iv) supporting safeguards implementation and construction site preparation would have as outputs (i) a completed Line 2, (ii) completed technical assistances, studies and reports, and (iii) trained public officials. In terms of outcomes, this was to lead to an effective urban mass transport system because of (i) the new Line 2 itself, (ii) its physical and tariff integration with the rest of the public transport system and NMT, (iii) TOD development along Line 2, and (iv) the use of LVC mechanism. The effectiveness of the system was to be measured through (i) ridership level compared to the forecast, (ii) time savings for users on Line 2, and (iii) the increase in the proportion of urban rail commuters along Line 2 with access to a car. As a result, the effective urban mass transport system was to provide access to economic opportunities for Nanchang's residents and reduce negative externalities. In the long run, it was expected to lead to a more equitable and inclusive development in the city and a low carbon urban transport system (PAD, paras 17, 18, 19, and annex 1). Outputs:  Urban Rail Line 2 operational, consisting of 23.8 km of rail track, 21 stations, tunnels, equipment, systems, and rolling stock, and maintenance sections.  Completed integration technical assistance, consisting of ridership modeling and fare setting study, physical integration study, and a bus-rail integration plan.  Accessibility assessment of all stations along Lines 1 and 2 to improve seamless transfers and safe crossings around and movement within stations.  Completed financial management and internal audit functions technical assistance, which according to the Bank task team was to support project implementation capacity.  Completed TOD and LVC technical assistance.  Trained government officials on TOD and LVC.  Completed study tours, e.g. TOD property Inspection visit to Japan Rail Transit, joint seminar in Urumqi on smart urban transportation and professional housing development, China Rail Transit Property Development Seminar in Chengdu, and annual workshops on metro development, including TOD in Shanghai. Outcomes: The Bank task team pointed out that the ridership modeling and fare setting study looked at scenarios of bus and rail integration, among others with separate and very low fares for each mode and somewhat higher fares, but discounted for multiple trips. Nanchang Municipality chose the separate low fares scenario instead of a discounted integrated fare. The Bank task team confirmed that fares are very low because they are subsidized and affordable even if passengers have to pay for multiple trips. Fares are mostly paid through smart cards, which ensures integration. The smart card was introduced before the project. The Bank team Page 5 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) pointed out that the success of its contribution to integration was mainly in physical terms, including station design, bus stop and terminal location, street lighting, junction channelization to improve intersections near the rail system, NMT facilities, and a park and ride station for Line 1, which was the result of the integration study. The bus-rail integration plan led to restructured bus routes, which according to the Bank task team eliminated competing bus services along Line 2 and ensured the integration of bus and train timetables for the first and last trains of the day. The integration between Line 2, the other rail lines, the bus system, and NMT and the reorganization of bus routes along Line 2 enhanced Line 2's effectiveness because buses do not compete with Line 2 and public transport users can conveniently switch between different public transport and other transport modes. The positive users' experience is reflected in the high users' satisfaction mentioned under subobjective 2 below and the increased public transport ridership. According to the ICR, para 34, with the full opening of Line 2 in June 2019, its average daily ridership reached 189,500 passengers. By October 2019, the average daily passenger ridership reached 206,800 passengers, exceeding the target of 200,000 passengers. In addition, the average daily passenger volume of Line 1 increased by about 6.6 percent because of Line 2 transfers. By the same date, the total passenger volume of Lines 1 and 2 reached 508 million passengers. The ICR, para 34, also highlights that there is potential for further ridership increase for Line 2, all urban rail lines are being extended, and ridership on buses also increased due to better integration. According to the ICR, para 36, Line 2 caused a modal shift from cars to public transit. A survey showed that 39.9 percent of commuters along Line 2 had access to a car in the household in 2019. This is significantly more than the target 15 percent. Survey participants indicated that they gave up their private vehicles and switched to Line 2 because of a more affordable, faster, and more comfortable travel. As pointed out in section 9, having access to a car, however, does not necessarily mean that Line 2 commuters switch from the car to public transport. They could also have switched from buses or NMT, which was very high compared to similar cities in China. The Bank task team confirmed that the share of NMT in Nanchang is decreasing while motorization is increasing, but this would have happened also without Line 2, which offers an attractive transport alternative to the car. For three origin destination pairs monitored along Line 2, the travel time on public transport modes went down on average by 23 minutes, slightly exceeding the target of 22 minutes. A small additional travel time decrease took place through the reduction of the travel interval of Line 2 from 8 minutes to 6 minutes and 30 seconds (ICR para 35, and annex 1). The TOD and LVC technical assistance and the enhanced capacity and interest of government officials in these topics, according to the Bank task team, led to (i) improved street design around rail stations to enhance walkability, (ii) the identification of areas for development along for the whole rail system and Line 2, and (iii) the necessary zoning changes for mixed and more compact land use with higher floor area ratios. It also led to the development of two TOD sites along Lines 1 and 2. One of these sites, the interchange station of Lines 2 and 3, was the direct result of the collaboration between Nanchang Municipality and the Bank right from the beginning of the project. The site is still under construction and includes a hotel, office space, and a shopping mall. Once completed, it will provide revenues for urban rail development. The Bank task team clarified that for the other TOD site, i.e. the interchange station between Lines 1 and 2, they had no direct involvement. However, part of its success can be ascribed to the capacity building and continuous Bank engagement on these topics. The latter also led to the issuing of a regulation that provides the Nanchang Railway Transit Group with on earmarked portion of the revenues from the auction of land along the rail lines. Page 6 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) This progress made on TOD and the use of development-based LVC financing methods also enhanced the effectiveness of the urban mass rapid transit system. Improved accessibility to rail stations and a compacter city development with dense, mixed land use are likely to encourage walking and attract more public transport ridership. However, a compacter city development takes time to materialize, hence is not measurable by project end. The LVC financing mechanisms, which according to the task team are already partially in use, are generating revenues from property to sustain rail development. The progress made in Nanchang in terms of TOD and LVC is not trivial because most of the world’s urban mass transit systems cannot cover operating costs through fare revenues, let alone capital expenses (ICR, para 40), and the examples of successful LVC schemes implemented by Bank client countries are still rare. The ICR, para 36, mentions that the shift from public transport to cars and greater public transport use in the city are evidence that the effective urban mass transport system developed under the project also likely contributed to enhanced access to economic opportunities for Nanchang's residents and reduced negative externalities. Indeed, according to the latest survey in 2018 (Line 2 was partially opened in 2017), the public transport modal share in the city rose from 13.5 percent at project preparation to 16.4 percent of total urban trips, including 13.2 percent by bus and 3.2 percent by metro. Reductions to greenhouse gas emissions are also expected from the use of TOD and further development of the rail system through LVC revenues (ICR, para 40). The project enhanced the effectiveness of the urban mass rapid transit through (i) the new Line 2 itself, (ii) its integration with public transport and other modes, and (iii) TOD and LVC. It has also likely contributed (and still will contribute) to enhanced access to economic opportunities and reduced negative externalities. Therefore, the efficacy of subobjective 1 is rated substantial. Rating Substantial OBJECTIVE 2 Objective To provide a mass rapid transit system of appropriate quality along the Line 2 corridor from ZhanQianNanDaDao Station to XinJiaAn Station. Rationale The theory of change for subobjective 2 was that the activities related to (i) designing, constructing, and providing systems and equipment for Line 2, (ii) providing technical assistance on ridership modelling, LVC, and financial management, (iii) carrying out study tours and training, and (iv) supporting safeguards implementation and construction site preparation would have as outputs (i) a completed Line 2, (ii) completed technical assistances, studies and reports, and (iii) trained public officials. In terms of outcomes, this was to lead to an urban mass transport system of adequate quality, evidenced by universal accessibility, reduced travel times, affordable fares, and other quality features. Adequate quality was to be measured through the user satisfaction rate. Quality urban mass transit was to increase ridership, with a modal shift from private vehicle. This in turn was to enhance access to economic opportunities for Nanchang's residents and reduce negative externalities. In the long run, it was expected to lead to more equitable and inclusive development in the city and a low carbon urban transport system (PAD, paras 17, 18, 19, and annex 1). Page 7 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) Outputs:  The outputs were the same as for subobjective 1. Outcomes: According to para 38 of the ICR, all stations of Line 2 are air conditioned and provide universal access for disabled and elderly people. The design of Line 2 is gender-informed, with some large stations providing baby changing and feeding rooms and a bathroom facility ratio for men to women of 1:2. The station decoration incorporates regional culture, and the surrounding environment enhances the pedestrian experience. Line 2 used energy-saving and environmentally-friendly measures, such as natural ventilation shafts, acoustical dampers, LED lights, energy saving escalators, and enhanced vibration reduction measures. As mentioned under subobjective 1 above, Line 2 is fully integrated with the rest of the public transport system and with NMT and applied TOD concepts to its stations. Finally, Line 2 reduced the travel time along the corridor. As a consequence, the overall users' satisfaction for Line 2 by project closure in 2020 was 92.45 percent for males and 92.98 percent for females, exceeding the original target of 80 percent. The user survey covered the comfort level in the train, service quality, transfer convenience, travel information service, waiting environment, riding comfort, and waiting time. The ICR, para 37, argues that the high-quality standard of Line 2 is evidenced by the modal switch from cars to public transport discussed under subobjective 1 above. The satisfaction rate for universal access facilities increased from 82.97 percent in 2019 to 91.30 percent in 2020, and the satisfaction rate for integration/ease of transfer increased from 77.84 percent in 2019 to 89.47 percent in 2020. The project provided a high-quality urban mass transit system as reflected in the user satisfaction, hence the efficacy of subobjective 1 is rated substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Both, subobjectives 1 and 2 are rated substantial, hence the overall efficacy is also rated substantial. Overall Efficacy Rating Substantial Page 8 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) 5. Efficiency Economic Analysis: At appraisal, a cost benefit analysis was carried out based on without and with project scenarios. The analysis used a 30-year time horizon for the operation of Line 2, from 2018 to 2047, and an eight percent discount rate. The benefits considered included (i) travel time savings for passengers, (ii) operating cost savings for urban transport, comparing the rail operating cost per passenger with the car and bus operating costs per passenger, and (iii) reduced accidents, congestion, pollution, noise, and greenhouse gases (PAD, para 44 and 45). The main assumption spelled out in ICR, annex 4, page 40, are reasonable. The costs covered construction, equipment, safeguards, and other construction costs of the project. These costs accounted for 90.2 percent of the proposed total project cost at appraisal. The analysis estimated an economic internal rate of return (EIRR) for the project of 9.6 percent and a net present value (NPV) of RMB 3,34 million. The benefit-cost ratio was 1.2 (ICR, para 45). The sensitivity analysis assumed a 50 percent increase of investment costs, the reduction of traffic growth rates by half from 2018 to 2030, and the exclusion of external benefits. The worst-case scenario resulted in an EIRR of 7.1 percent (ICR para 42). By referring to the Urban Rail Development Handbook (Pulido Daniel; Darido, Georges; Munoz- Raskin, Ramon; Moody, Joanna. 2018, Washington, DC) the ICRR makes the case for lower discount rates than 12 percent for urban rail projects. This is because such projects involve high investments at the beginning, with all expected returns materializing over a long system lifetime, hence a higher discount rate would penalize this type of project. By project close, the ex-post cost benefit analysis covered 85.8 percent of total project cost. The following adjustments were made to the analysis: (i) the project's operational period shifted to 2020 to 2049; (ii) its cost was about 15 percent lower than at appraisal, and (iii) data on the economic and social context, such as population, employment, and GDP projections was updated. The analysis showed an EIRR of 11.93 percent and a NPV of 8.00 billion RMB. The benefit-cost ratio was 1.6 (ICR, para 44 and 45). Administrative and Operational Efficiency: The project's efficiency suffered from delays in implementation pointed out in section 2. However, the original implementation period of five years was tight (and flagged as a potential risk in the PAD). The project cost was about 15 percent lower than estimated at appraisal. This was due to (i) lower contract prices than estimated obtained through competitive bidding; (ii) construction activities at Qingshan Road Intersection Station and Fu- Ba Section financed by counterpart funds for a total of about US$14 million; and (iii) the exchange rate from RMB to US$ increased from 6.2 at project appraisal to 7.0 at project completion (ICR, pages 39 and. 40). The ICR, page 40 points out that the exchange rate increase influences the NPV in US$ but does not affect the EIRR. Although project implementation took longer than planned, the investment cost was lower than estimated. The project also has a higher rate of return than estimated at appraisal. Therefore, on balance, the efficiency of project implementation is rated substantial. Efficiency Rating Page 9 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 90.20 Appraisal  9.60  Not Applicable 85.80 ICR Estimate  11.93  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project's relevance of objectives, efficacy, and efficiency are rated substantial. Therefore, the overall outcome is rated satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR (paras 88 to 99) identifies the following risks to development outcomes, with which the IEG evaluator agrees:  Financial sustainability. The ICR considers this risk as low because the project's financial sustainability was tested under multiple scenarios, including a pessimistic scenario of low ridership and high operations and maintenance costs, and the required contributions to support the project are not expected to exceed 1.2 percent of Nanchang Municipality’s disposable income. In addition, UCR is qualified to use the system's financial model and is expected to ensure that Nanchang Municipality is fully aware of the resources necessary to financially sustain the system.  Reliance on land finance. The ICR highlights, on the one hand, that Nanchang Municipality has the necessary conditions to use LVC. Nevertheless, there is a potential risk that overreliance on land financing might overheat real estate markets and make housing unaffordable for poorer people. This would be because of gentrification of transit station areas and lack of public-private experience in jointly delivering property development projects using complex LVC procedures. On the other hand, rising land prices for mixed use present a favorable condition for successful development-based LVC. The risk of market overheating is mitigated by the work done under the ongoing Global Environment Page 10 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) Fund grant to support TOD planning at the municipality level and along Line 2, which is expected to feed into the next version of Nanchang's Masterplan.  System maintenance. The ICR points out that (i) UCR's fleet management follows industry standards that include daily, monthly, quarterly, and annually inspections, (ii) the Bank team created awareness on the importance of public transport asset management, and (iii) UCR was developing a comprehensive information system for asset management during the ICR mission. Therefore, this risk is relatively low. Based on discussions with the Bank task team, IEG highlights the following additional risks:  Reduction in public transport demand. The Bank task team pointed out that motorization has been increasing quickly and with raising incomes, it is expected to increase even more. The Bank task team explained that Nanchang is applying parking charges, which are low. The Bank task team mentioned that they consistently insisted on travel demand management policies, but any type of charging is politically sensitive and no Chinese city to date has managed to introduce congestion pricing. Therefore, in the absence of effective travel demand management policies, there is a risk that some of the public transport ridership gains could be reversed.  Bus and rail integration. The Bank task team pointed out that this integration has been successfully completed and there is no foreseeable risk that it could be undone.  Covid-19 pandemic. The ICR, para 30, mentions that the demand of Line 2 went back to normal quickly after the outbreak of COVID-19 pandemic. Therefore, the Covid-19 pandemic does not seem to pose a serious risk to the sustainability of the development outcome. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank task team intervened in a highly relevant sector for China. Although this was the first urban transport project in Nanchang, the task team benefited from an extensive knowledge of urban transport issues in China and decades of successful collaboration with Chinese cities. The project design was relatively simple, which is justified given the complexity of urban rail projects. The task team incorporated in the project design lessons from the Bank's urban transport portfolio and other urban rail projects in China, including paying early attention to bus and rail integration, ensuring stakeholder coordination, especially for integration, focusing on the complementarity of urban growth and transport strategies for the effectiveness of the rail system, and being realistic in the demand forecast. The task team ensured that Nanchang Municipality carried out a walkability survey at each station and reviewed the land use plans along the line. The task team also ensured that the Nanchang Municipality carried out a refined demand forecast analysis with critical variables that support urban rail ridership, such as bus and rail level of integration and changes in fares. This was not yet common for Chinese cities (PAD, para 29). Page 11 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) According to the ICR, para 83, the task team provided inputs on engineering, policy, and institutional aspects. During preparation, the Nanchang Municipality, URC, and the task team discussed international and domestic practices to improve track alignment, station design, urban planning, and the need for municipal financial contributions for Line 2. The URC developed a financial model to determine the project's financial sustainability needs. The Bank task team pointed out that during project preparation they engaged the government on the broader issues related to urban rail implementation through several workshops with experts, and study tours, among others to New York and Seoul. The Bank task team ensured the diligent preparation of the social, environmental, and fiduciary aspect of the project. The ICR, para 82, points out that project preparation included extensive consultation, which is remarkable given the client's unfamiliarity with Bank policies and shows the Bank's strong support to project preparation. The task team identified most implementation-related risks, including the unfamiliarity of the project implementation office with Bank rules and procedures, the technical complexity of urban rail projects, the need to focus on integration and institutional coordination, the tight project implementation schedule, and the need for large amounts of counterpart funds (PAD, para 43). The overall risk was correctly rated substantial. The mitigation measures were largely adequate, but the project had shortcomings in terms of institutional coordination, especially for utility relocation and traffic diversion, which contributed to the project implementation delays (ICR, para 59). The Bank team did, however, not identify any risk related to TOD and LVC even though this was new for Nanchang Municipality and an important aspect of the rail system's effectiveness. As mentioned in section 9, the project had shortcomings in M&E. Because these shortcomings were largely accounted under the relevance of objectives and M&E ratings and overall are minor, Bank performance in ensuring quality at entry is rated satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank task team regularly monitored the project implementation progress and ensured compliance with financial management, procurement, and safeguards procedures. The ICR, para 84, points out that mission findings were candidly documented in aide memoires and the project ratings in the ISRs were appropriate. The task team provided technical advisory support to the project, which according to the ICR, para 85, was valued by the client. The task team helped with the technical aspects of project implementation, for instance, by hiring an experienced engineer to assist URC in construction management and sequencing, and identifying shortcomings in accessibility and integration in the first phase of the project, which were subsequently corrected. The Bank task team strongly focused multimodal integration, TOD and LVC, and managed to help make progress on these essential topics. The Bank task team pointed out that they also pushed the agenda of other urban transport topics, such as travel demand management and provided support, for instance, in parking pricing. Page 12 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) In summary, the Bank task team used this urban rail infrastructure project as a platform to provide broader support to urban transport in Nanchang and carried out an adequate supervision effort under the project. Consequently, Bank performance in supervision is rated satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project management office was to coordinate the relevant agencies in collecting the data and information required for project monitoring and evaluation. The office was expected to reflect the progress and results in the regular progress reports, and take appropriate corrective actions as needed (PAD, para 38). The results framework included four PDO and seven intermediate indicators. Three of the PDO indicators were listed for subobjective 1 and one for subobjective 2. These indicators were largely adequate to measure the project's achievement, but there is room for improvement. The ICR, para 63, points out that the PDO indicator "urban rail passengers with access to cars" measures both subobjectives, i.e., the effectiveness and quality of urban rail. Although shifts from private vehicles to public transport enhance the effectiveness of urban rail and are a likely indication that the system is of high quality, being a rail passenger with access to a car does not necessarily mean a shift from the car to public transport because of Line 2. Rail passengers with access to cars could have used the bus system previously. Therefore, a better indicator could have been "previous car users who shifted to Line 2". The Bank task team clarified that the decision to survey "urban rail passengers with access to cars" instead of "urban rail passengers who switched from the car to Line 2" was taken because the concept of shifting is not always clear, i.e., when passengers use both the rail system and the car. Still, modal shift surveys have been successfully carried out in other cities, and the effort would have been worthwhile. Instead of measuring the ridership on Line 2 and comparing it with the demand forecast, the project could have compared the number of public transport users on the corridor without and with Line 2 to understand the effectiveness of rail versus bus based public transport. The Bank task team explained that they used the pre-project ridership information in the corridor in the economic analysis. They decided to compare the actual ridership with the demand forecast for Line 2 because it was simpler. The project could also have included an indicator to measure the contribution of integration, TOD, and LVC to the effectiveness of the rail system, such as number of people integrating from different public transport modes and their cost with and without the new system, initial signs of more compact urban development along the Line 2, and amount of LVC-related resources collected. The ICR, para 64, points out that the lack of evidence supporting the incorporation of TOD-related activities collected as part of the project's M&E Page 13 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) made it difficult to assess the Bank's contribution in providing guidance on developing TOD in Nanchang and to capture the projects achievements on this important aspect. The Bank task team clarified that they decided not to include an indicator on TOD or LVC because they feared that this would have obliged Nanchang Municipality to use the Bank's safeguard policies for future real estate developments using TOD or LVC schemes. The ICR, also in para 64, points out that an indicator on the number of people benefiting from improved accessibility, such as percentage of low-income people within 500 m of metro stations or number of jobs accessible to them within 45 or 60 minutes could have been used to evaluate the project's impact on inclusive development. Finally, the project could have captured the reduction in the carbon footprint because of a model shift from private vehicles to public transport as measure of the system's effectiveness. The intermediate indicators monitored essential elements that contribute to the effectiveness and quality of the system: level of physical integration, bus lines reorganized, availability of single card for rail and bus, and percentage of station with barrier-free accessibility for people with reduced mobility. Again, intermediate indicators to measure the impact of TOD and LVC could have been useful if Nanchang Municipality would not have been so interested and committed on these issues. All indicators had baselines and targets. Most of these targets were reasonably ambitious. For instance, an 80 percent overall user satisfaction for Line 2 seems rather ambitious, considering that the Singapore metro rail had a satisfaction score of 7.8 (out of 10) in 2019. Its satisfaction was 7.7 in 2017, and 7.9 in 2018. The overall service satisfaction of the New York was 53 percent in 2020, 46 percent in 2019, and 34 percent in 2018. The passenger number target of 80 percent of the demand forecast by project start is reasonable. The travel time savings targets for rail users are substantial. For modal shift, a 15 percent target would have been overly ambitious based on international experience, but as the indicator only measured the access to a car, the target was greatly overachieved. The PAD, annex 1, indicates the data collection sources and gives some definitions (the adequate quality was not defined), but provides limited information on data collection methodologies. The M&E framework was updated during the 2018 restructuring to reflect changes in the project implementation schedule by (i) adding a indicator on travel time savings along the Stage I section of Line 2, (ii) adjusting the ridership target for Stage I, and (iii) updating the indicators' timeline. The ICR, para 66, points out that the Bank's task team missed the opportunity to include an indicator to measure the impact of TOD, which may have affected the dialogue with the municipality on the importance of transport and land use integration. b. M&E Implementation The project management office regularly provided the Bank's project implementation support missions with data and information to monitor progress in project implementation and towards achieving the PDO. The Bank task team consistently reported this information in aide-memoires and Implementation Status and Results Reports (ISRs). Page 14 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) c. M&E Utilization The information collected was useful in assessing the project implementation progress. Data collection did not pose an undue burden on Nanchang Municipality, and most data will likely be collected also after project closure. URC carried out the passengers' satisfaction survey on an ongoing basis to evaluate the rail system's operations and inform improvement proposals. The ICR, para 57 mentions that the technical assistance on ridership modelling helped URC prepare independent, updated ridership forecasts that enabled a more refined analysis of critical variables that support metro ridership, including the level of bus and rail integration, changes in fares, pace of land development, and the impact of restrictions on car use. This was considered crucial for the future monitoring of project outcomes. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as category A for environmental assessment purposes. The main environmental concerns identified included impacts on physical cultural heritage, vibrations, spoils, and safety issues during construction and operation. The project was also expected to have large resettlement implications. The following safeguards policies were triggered: Environmental Assessment OP/BP4.01, Physical Cultural Resources OP/BP 4.11, and Involuntary Resettlement OP/BP4.12. The project management office prepared an environment impact assessment, an environment assessment, an environmental and social management plan, a comprehensive assessment of project-affected people, and a resettlement action plan. They also carried out the first cumulative impact assessment for an urban rail project in China and assessed the significance of valued ecosystem components. This led to enhanced mitigation measures during design, which would have been too expensive or disruptive to adopt during operations (ICR, paras 69 to 71, and 73). According to the ICR, para 70, the potential environmental impacts of main concern, including on physical cultural heritage, were carefully addressed through mitigation measures in project design, alignment optimization, and construction management. The ICR, para 72, points out that the project's environmental management was highly satisfactory, and the Bank recognized it as a best practice for urban rail projects. In terms of social safeguards, the project affected 1,344 urban families, 130 enterprises, and 41 rural families, and required the acquisition of 170,468.13m2 of land. The project provided multiple mitigation measures, such as cash compensation and housing relocation to displaced families. The ICR points out that 27 percent of affected families gained an improvement in their living environment. The total resettlement action plan implementation cost was US$ 514 million, more than twice the amount estimated at appraisal, funded entirely by the client. The World Bank Urban Rail Handbook included the relocation of the No. 28 High School as best practice due to the mitigation of the disruption to commuting habits and classroom schedules. The Bank task team and the external monitoring reports confirmed the satisfactory Page 15 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) implementation of the resettlement action plan. The project's social safeguard performance was satisfactory. The project implementation office also successfully applied the Bank’s compensation and resettlement concepts to Lines 3 and 4 (ICR, paras 74 and 75). b. Fiduciary Compliance As for procurement, the three project-financed civil works contracts were procured in a timely manner, using advance procurement. Only two months after project effectiveness, all contracts were signed. URC was slow in reviewing, responding, and handling contractual claims. The Bank asked for the establishment of a Dispute Resolution Board and the hiring of an experienced contract expert to assist URC in handling the claims and contract variations. The Bank task team pointed out that the Dispute Resolution Board was not established. Because the task team made the timely settling of claims a top priority, URC developed a timeline for claim settling, streamlined its internal procedures, and hired an experienced fiduciary expert. At the time of writing the ICR, some claims were not resolved (ICR, para 60). Because of these claim handling issues, in 2018 the ISR rated procurement as moderately unsatisfactory. It was upgraded to moderately satisfactory in the subsequent ISR. In terms of financial management, even if URC had no previous experience with the Bank's financial management rules and procedures, the project's fiduciary compliance was adequate. The project had minor delays in reviewing and processing civil work progress certificates and interim payment request. Except for this, the project's financial management arrangements were adequate throughout its implementation (ICR, para 80). URC submitted the project's interim financial reports on time. The annual financial reports had “unqualified” opinions. The ISR rated the financial management performance of the project as satisfactory. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Moderate shortcomings in the selection of indicators and Quality of M&E Substantial Modest limited information on data collection methodologies. Page 16 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) Quality of ICR --- Substantial 12. Lessons The following lessons have mostly been derived and summarized from the ICR, with minor additions by IEG: Infrastructure financing can be a strong lever for institutional strengthening and sustainable urban development in high-capacity countries with limited need for Bank financing. By financing a small portion of an urban rail infrastructure project, in Nanchang the Bank got a seat at the table to influence the broader urban transport and development agenda and ensure sound resettlement and environment management practices. The Bank used this investment project to engage on integration, TOD, LVC, and universal accessibility, and help build the capacity in these areas. This led to important progress on all aspects. Even if the Bank only financed a small portion of the project, its safeguards policies applied to Line 2 as a whole, ensuring strict resettlement and environmental practices, especially in terms of consultation. Financing a small part of a complex infrastructure might become a model for the Bank to provide technical assistance, advice, and capacity in high-capacity countries with limited need for Bank financing. Early and intensive coordination is critical for an urban rail system to achieve its potential as a mass transit mode. The success of a mass transit mode is dependent on integration and this requires simultaneous (i) physical integration (interconnection between different transport infrastructure), (ii) operational integration (multimodal service planning), and (iii) fare integration (interoperable fare technology as well as comprehensive fare and subsidy policy across the entire transit system). With this in mind and being aware of the necessity to timely coordinate with many different actors, the Bank asked the client to set up a project leadership group. This group included the vice-mayor, deputy municipal secretary general, URC's chairman and key municipal line agencies, such as Finance Bureau, Development and Reform Commission, Urban Management Commission, Planning Bureau, Construction Commission, Land Resources Bureau, Environmental Protection Bureau, Housing Management Bureau, Price Bureau, Traffic Management Bureau and Bus Company. The group prepared, agreed, and implemented the integration plan jointly, and this was the key for the successful public transport integration. The project did not have such coordination group to support the construction of Line 2. This caused significant delays in the re- location of utilities and the implementation of the temporary traffic deviation, which both depend on coordination. The lack of joint urban rail and broader transport and land use planning made the urban rail project implementation more cumbersome. Compact urban development requires deregulated land use towards higher density and mixed land use along the entire alignment of the urban rail system. Such system will only deliver premium in property values in its influence area if it is integrated with the stations and surrounding space, forming a station-centric transportation, housing, catering, shopping, entertainment, and culture space. In this project, the land use aspects to ensure the necessary density and mixed land use for the rail system had to be retrofitted on a case-by-case basis. The project identified the potential areas for development along the urban rail system and ensured the necessary zoning changes to achieve the higher densities and mixed use. This required Page 17 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) a lot of discussions and was cumbersome. A more efficient way of proceeding could have been to carry out the urban rail planning jointly with a revision of the land use and zoning planning. In a project with multiple stakeholders, an implementation arrangement with a single implementation agency might jeopardize the complete use of project resources. The Nanchang project was implemented by URC, which is responsible for the reimbursement of the loan. Therefore, it was not possible for other project stakeholders to use the unused loan proceeds to enhance the integration experience, and these aspects had to be cancelled. For instance, if the bus company had been an implementation agency, they could have used some of the loan proceeds to provide additional feeder services or other elements to enhance the integration experience. However, a higher implementation complexity due to multiple implementation agencies needs to be weighed against its potential benefits. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is very well written, clear, consistent, and complete. It includes useful graphs, designs, and photos. It follows the guidelines both with respect to ratings and narrative. However, it is a bit long, and certain sections, such as the one on relevance of objectives, that could easily have been reduced in size without losing essential information. The quality of the analysis is good. There is adequate focus on evidence both through the indicators and by citing additional facts and literature. In terms of evidence, since the achievements with respect to the indicators depended on passenger counts, travel time measurements, and survey, it would have been useful to include a methodology annex in the ICR. However, while this would have helped evaluate the quality of evidence, it is not requested or common in ICRs. The theory of change in graphical form is well done. The efficacy section is outcome oriented, but it lack details on how technical assistance and capacity strengthening activities contributed to the success of TOD and LVC. The ICR cites a few outcomes that are not sufficiently explained. For instance, the ICR, para 37, mentions that Line 2 enhanced reliability, but it is not explained how this reliability was achieved. Similarly, in para 33, it lists the means through which integration was achieved, but does not describe how it contributed to effectiveness. With respect to the cost benefit analysis, more details on the calculation of the externality costs would have been desirable instead of simply referring to a reference document. This is especially true because externalities account for a quarter of all benefits. In addition, the ICRR does not justify the 8 percent discount rate in light of Nanchang's per capita growth rates in line with the 2016 note on "Discounting Costs and Benefits in Economic Analysis of World Bank Projects." The lessons are well thought through and based on the specific experience and findings for the project, but it required a discussion with the task team to fully appreciate them. Page 18 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Nanchang Urban Rail Project (P132154) On balance, given the laudable aspects and shortcomings, the quality of the ICR is rated substantial. a. Quality of ICR Rating Substantial Page 19 of 19