Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) Report Number: ICRR0022915 1. Project Data Project ID Project Name P123384 VN-Danang Sustainable City Development Country Practice Area(Lead) Vietnam Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-52330,IDA-60320 30-Jun-2019 141,676,035.08 Bank Approval Date Closing Date (Actual) 26-Apr-2013 30-Jun-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 202,500,000.00 0.00 Revised Commitment 151,881,588.97 0.00 Actual 141,676,035.08 0.00 Prepared by Reviewed by ICR Review Coordinator Group Cynthia Nunez-Ollero Kavita Mathur Victoria Alexeeva IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (FA, p.4) and the Project Appraisal Document (PAD, paragraph 11), the Project Development Objective (PDO) was to expand access of Da Nang City residents to improved drainage, wastewater collection and treatment services, the arterial road network, and public transport in selected areas. This review will assess the performance of the project against this singular PDO. Page 1 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 31-May-2017 c. Will a split evaluation be undertaken? No d. Components 1. Drainage and Wastewater Improvement (US$75.10 million in IDA, plus US16.90 million in counterpart financing for a total of US$92.0 million at appraisal; revised to US$129.71 million in IDA, and US$24.33 million in counterpart financing for a total of US$154.04 million in the 2017 AF; US$73.47 million in IDA, and US$33.96 million in counterpart financing for a total of US$107.43 million, actual). This component was to finance stormwater management and drainage improvements, including the rehabilitation of two retention lakes; rehabilitation and construction of about 11 km of box culverts and about 2 km of open canal; and outlets to existing lakes and offshore outlets in the areas of My Khe and My An. This component was also to finance wastewater collection and treatment including: (i) development of about 40,000 house connections; (ii) construction of approximately 216 km of wastewater collection and transmission pipelines; (iii) construction of Lien Chieu wastewater treatment plant (WWTP) with capacity of 40,000 m3 /day; (iv) expansion of Hoa Xuan WWTP from 20,000 to 40,000 m3 /day; and (v) upgrading of the existing Son Tra, Hoa Cuong, Phu Loc and Ngu Hanh Son WWTPs. Revised Component: The May 2017 Additional Financing (AF, see below, Dates) was to add the development of a separate house-to-sewer connection system in the coastal tourist area of My An – My Khe and additional drainage works to minimize the risk of polluting the beach and residential areas. 2. Bus Rapid Transit Development (US$41.40 million in IDA plus US$8.80 million in counterpart financing, for a total of US$50.20 million at appraisal; revised to US$53.47 million in IDA, plus US$8.78 million in counterpart financing for a total of US$62.25 million at the 2017 AF; US$9.24 million in IDA plus US$5.41 million in counterpart financing for a total of US$14.65 million actual). This component was to finance the establishment of an integrated fare collection and Intelligent Transport Systems (ITS) to develop a Bus Rapid Transit (BRT) system. This component was to finance all aspects of a complete BRT system with a core BRT route (from the emerging University District focused on Viet Han College in the South to the Hoa Khanh Industrial and Technology area in the North) and three additional BRT branded services from Hoi An, partially via the Coastal Road tourist area, to the central interchange at March 29 Park and the Airport; from the central interchange at March 29 Park and the Airport to Thoquang (Sontra Peninsula); and from Airport/Central Bus Station/Hoa Khanh Terminal to the Ba Na Hills tourist area). This component was to include: (i) 13.0 km of dedicated median transitways and queue jumpers at congested intersections; (ii) 216 stations or stops (50 on the 23.8 km BRT trunk route) and two terminals, one each in the North and the South; (iii) ITS applications, such as smart-card ticketing, real-time passenger information, and signal Page 2 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) priority; (iv) a depot, including bus storage or parking; and (v) 81 low emission BRT buses for the basic BRT trunk route. Revised Component: The May 2017 AF was to establish an integrated fare collection, and an ITS for public transport to consolidate the BRT system scheduling service, fleet management, and traffic control. The October 2019 restructuring (see below, Dates) updated the scope of BRT to include high-quality bus service (HQBS), without the BRT features of dedicated lanes, ground-based ticket checking, and low-floor buses and stations. The acquisition of BRT buses was dropped because the city government preferred private operators to independently procure their buses. The June 2021 restructuring shifted the focus of this component to developing ordinary bus services only as the government’s revised its priority and adopted a COVID-19 response strategy. 3. Urban Strategic Roads (US$47.30 million in IDA funds plus US$30.60 million in counterpart financing for a total of US$77.90 million at appraisal; revised to US$58.19 million in IDA, plus US$36.57 million in counterpart financing for a total of US$94.76 million at the 2017 AF; US$35.28 million in IDA, plus US$49.80 million in counterpart financing for a total of US$85.08 million, actual). This component was to finance the construction of two new east-west connecting roads to the north-south bypass of Da Nang and the national expressway network, and five resettlement sites adjacent to the proposed roads. The 30.2 ha resettlement sites were to include basic infrastructure consisting of roads, sidewalks, streetlights, electric utilities, water connections, and wastewater and sewerage systems. Revised Component: The May 2017 AF was to finance the improvement of DH2 (Hoa Nhon - Hoa Son) Road to divert local urban traffic from the Da Nang – Quang Ngai and Da Nang Bypass Expressways. 4. Technical Assistance and Capacity Building (US$8.70 million in IDA plus US$6.60 million in counterpart financing for a total of US$15.30 million at appraisal; revised to US$8.65 million in IDA, plus US$8.29 million in counterpart financing for a total of US$16.94 million at the 2017 AF; US$1.25 million in IDA, and US$13.07 million in counterpart financing, for a total of US$14.32 million, actual). This component was to finance technical assistance (TA) in urban infrastructure management directed at the People’s Committee and other relevant departments of Da Nang. The TA was to include: (i) planning and management of drainage and wastewater systems; (ii) oversight of public transport management and operations, management of road safety, urban traffic, and parking; (iii) management and monitoring of public service performance contracts; (iv) development of a green development and sustainability index for Da Nang; and (v) project implementation support. Revised Component: The AF did not revise this component but in the October 2019 restructuring, changes to the BRT, and limits in the use of the AF for capacity building, reduced the scope of this component (ICR, paragraph 60, see also Section 4 Efficacy below). 5. Activities Transferred from the Da Nang Priority Infrastructure Investment Project (PIIP) (US$30.0 million in IDA plus US$6.80 million in counterpart financing for a total of US$36.8 million at appraisal; revised to US$24.93 million in IDA and US$ 5.40 million in counterpart financing, for a total of US$30.33 million at the 2017 AF; US$22.43 million in IDA and US$4.76 million in counterpart financing for a total of US$27.19 million, actual). This component was to finance the completion of the construction of the following under the PIIP including supervision, fiduciary, and safeguards compliance: (i) the southern road link connecting to the Hoa Phuoc – Hoa Khuong Road; (ii) the Hoa Xuan and the Son Tra wastewater treatment plants, (iii) the Phu Loc Bridge, and (iv) the embankment at the Phu Loc River. Page 3 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) Revised Component: This component was not revised. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The actual total project cost was US$248.67 million, slightly lower than the original total project cost of US$272.20 million. Financing: The project was financed by an original and AF credit from the International Development Association (IDA). An AF of US$72.5 million brought the total cost to US$272.5 million. The credit disbursed US$141.7 million. US$110 million was cancelled consisting of US$70 million from the original IDA credit and US$40 million from the 2017 AF. These funds were cancelled because of the reduced scope of the public transport investments from developing the BRT to developing an HQBS instead (ICR, paragraph 20). Another US$10.21 was unutilized (ICR, Annex 3, footnote). Borrower Contribution: The city government committed US$69.70 million and disbursed US$107.0 million. The Bank Team confirmed in their June 5, 2022 email to IEG that the city government has implemented its commitment to finance the operations and maintenance (O&M) and subsidy needs of the BRT (PAD, paragraph 36). At the 2021 restructuring (see Dates below), the government increased its commitment to US$83.37 million. The Bank Team clarified in their June 5, 2022 email to IEG that the published ICR will be updated to provide the correct amount of borrower contributions. Dates and Restructuring: The project was approved on April 26, 2013 and made effective on July 29, 2013. The Mid Term Review (MTR) was conducted on August 21, 2015. The original closing date was on June 30, 2019. The project was extended by 24 months to close on June 30, 2021. The project underwent three Level II restructurings and an Additional Financing (AF, Loan No. 6032).  On May 31, 2016 to move the project’s results framework from the FA to the relevant project manual.  On May 31, 2017 AF of US$72.52 million was approved to scale up the scope of the first three components (see above and Section 4 Efficacy below) accompanied by changes in costs (see above), and disbursement estimates; reflect these changes in the results framework; and procurement.  On October 7, 2019 to introduce changes to the indicators in the results framework and extend the closing date by 24 months, from June 30, 2019 to June 30, 2021.  On June 4, 2021 to introduce additional revisions to costs and components. This reallocated costs between disbursement categories and revised disbursement estimates. This restructuring cancelled US$110 million in IDA credit, from the original and AF credit and revised risk ratings due to the impact of the pandemic and the natural disasters that occurred between February and September 2020 (ICR, paragraph 62). This restructuring reduced the scope of PDO Outcome 3 as the allocation for procuring BRT buses was reduced. The public transport outcome shifted away from investing in BRT to developing HQBS. According to the restructuring paper (RES33824), the amount proposed for cancellation was the result of three factors: (i) savings and procurement efficiencies, (ii) the government’s decision to not fully implement the investments under Component 2, i.e., development Page 4 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) of the BRT system, and (iii) the government’s decision to use its own funds for activities that will not be completed by the project closing date. Split Rating: No split rating of the outcome of the project was undertaken. The PDO remained unchanged throughout project implementation. Wording of the PDO outcome indicators were clarified without making any substantial change to the PDO. The AF provided additional scope and beneficiaries. Revised target values were commensurate to these additional scope and were generally achieved or exceeded. The 2019 restructuring replaced the scope of the Bus Rapid Transit (BRT) component with a high-quality bus service (HQBS) (see Section 4 Efficacy below). With the expanded scope of the project brought by the AF and the realignment of BRT-related activities, a split rating is not deemed necessary. The project is to be assessed against the revised target values. 3. Relevance of Objectives Rationale Country Context. By 2013, Vietnam’s urban population was 25.4 million (ICR, paragraph 1) increasing to 36.6 million people by 2021 (Statista, online). Urban areas continue to grow and are widely acknowledged to contribute significantly to its economic growth. However, inadequate urban infrastructure and unreliable drainage and transport services in urban areas proved to constrain continuing growth. City Context: Da Nang, a tourist-dependent coastal city, is Vietnam’s fourth largest city with 880,000 people according to the 2009 census and a little over 1 million 10 years later. By 2011, the city begun to invest in improving its drainage system and piloted household connections to the drainage and sewerage system. Other areas of the city were poised for similar improvements. The city’s public transport system was supported by 91 public buses to serve the public with fewer than 3 percent daily trips. As a result, residents use private vehicles instead. This consisted of 500,000 motorcycles and 30,000 cars and escalating to 20-30 percent a year. Traffic congestions affect safety and productivity, and these gave the city government an incentive to innovate on improving the public transport service (ICR, paragraph 3). The city’s Master Plan for 2030–2045 approved in June 2021 aimed to achieve “a green and sustainable Da Nang.” The PDO supported the government’s priority to improve the provision of sanitation and wastewater services. The city government committed its own funds to improve the outcomes of the PIIP with additional activities. Works of about US$311 million are underway and focused on better managing and improving wastewater management plants, renovating the city’s drainage system in select areas, and improving drainage canals. Another US$260 million are underway to construct city roads and renovated bridges (ICR, paragraph 27). Country and City Plans: Vietnam’s 2009 Adjustment of the Master Plan for Urban Development to 2025 and Vision to 2030 articulated its urbanization strategy. The government aimed to achieve a well resources, green, and efficiently managed urban centers. Growing demand for urban infrastructure investments in the area around Da Nang, led to establishing a third economic zone in that central region. The Da Nang 2010 Master Plan and Sectoral Strategy to 2030, with a vision toward 2045 prioritized wastewater, sanitation, and transport infrastructure investments. The Da Nang People's Committee approved a wastewater management strategy in 2010 calling for a gradual increase of treatment capacity as connections and transmission expanded. The city transport development agenda prioritized the public transport system Page 5 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) upgrade, evident in the People’s Council plan for 11 bus routes and acquisition of 160 buses in September 2012. In addition, the strategy called for building the capacity of its transport agencies (ICR, paragraph 4). The PDO was highly relevant to the city’s plans and supported the country’s urbanization strategy. World Bank Partnership Framework: The PDO remained highly relevant to the World Bank Group’s Country Partnership Framework (CPF) for FY2018–22 for Vietnam. The project directly contributed to all three pillars of the CPF. The project contributed to the first pillar, “Enabling inclusive growth” by scaling up essential public services delivered to Da Nang residents, including the poorer residents in flood-prone areas. The project also contributed to the second pillar, “Investing in people and knowledge” by strengthening the city’s capacity to plan, design, and operate these services. Finally, the project contributed to the third pillar, “Ensuring environmental sustainability and resilience” by investing in drainage and sewerage to reduce environmental pollution and flooding. This third aspect of the project also reflected an adaptation response to climate change. The project also contributed to the cross-cutting area of “Governance” by building capacity for investment planning; introducing a new green development and sustainability index for the city to measure and monitor progress across a set of indicators; and introducing management systems for wastewater, public transport monitoring, and coordination of management and training activities. Previous World Bank Experience in the Sector and in the city: Beginning in the late 1990s, the World Bank and Da Nang have implemented the Three Cities Sanitation Project, and the Priority Infrastructure Investment Project (PIIP). Da Nang became the first Vietnam city to have a comprehensive wastewater management strategy. Da Nang emerged to lead the country in good governance in the sector. PIIP focused on roads, bridges, and upgraded certain areas. This project moved from area-based infrastructure, and exploited the connectivity offered by the city’s trunk infrastructure and how to manage this effectively. In addition, the project would complete infrastructure initiated under PIIP that lacked sufficient funding to complete during the project period. The PDO of this project was specific to Da Nang and responded to its growing need for investments in drainage and public transport. However, the objective appeared ambitious, covering more than one sector even as the city was more familiar with less complex, single sector Bank- financed projects. At implementation, the challenges of a multi-sector design were borne out. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To expand access of city residents to improved drainage, wastewater collection and treatment services, the arterial road network, and public transport in selected areas of Da Nang City. Rationale Page 6 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) Theory of Change: At appraisal, the Theory of Change (ToC) was represented by the indicators in the Results Framework showing the causal relationship between inputs and outputs. Inputs included the construction and rehabilitation of urban drainage and wastewater treatment plants; completion of urban infrastructure works (such as the construction of the southern road link; upgrading of wastewater treatment plants; and embankments and associated construction supervision and safeguards monitoring services initiated under the Priority Infrastructure Investment Project (PIIP); construction of urban arterial roads; development of a bus rapid transit (BRT) system; and capacity building for government staff for improved planning and management of basic services and public transport. The outputs were improved drainage and wastewater treatment infrastructure, arterial roads, BRT infrastructure, improved bus transport system. The May 2017 AF and October 2019 restructuring updated the target values of the output indicators to capture the added activities and revised the description of one output indicator to improve how it is measured. Outcome was improved access to the drainage and transport services to city residents by connecting households to sewer systems, linking key areas of the city, and reducing travel time between areas. The BRT infrastructure was to improve the public bus transport system and expand access by city residents to a reliable alternative to cars and motorbikes. Investments in urban infrastructure would address infrastructure deficiencies and service gaps, improving infrastructure and delivery of both basic services and public transport to residents. The project interventions would build capacity to manage a sustainable green city. The May 2017 AF and October 2019 restructuring added a new outcome indicator to capture the impact of the additional activities. The ToC assumed investments would have sufficient funding for operation and maintenance (O&M); the investments would respond to the needs of the residents; and that the government would implement the BRT. This last assumption was borne out at implementation. The government replaced the BRT with the High Quality Bus Service (HQBS) instead. Outputs: Under drainage and wastewater collection and treatment services:  From the preceding PIIP, the following Wastewater Treatment Plants (WWTPs) were constructed and upgraded as targeted: o Upgrading of 2 units of the Hoa Xuan WWTP; o Construction of Lien Chieu WWTP; and o Upgrading of Son Tra WWTP. This Plant was commissioned in July 2017.  The 2017 AF added the following output indicators: o 11,179 households (original target 10,000, target exceeded) were provided with individual household connections to sewer systems in the My An and My Khe areas. Sewer lines and house service connections (no data provided) were also provided in the resettlement areas. o 64.8 km (original target 60 km, target exceeded) of wastewater collection drains in Da Nang were completed. The infrastructure was installed with an auto-monitoring system that had flow meters and rain sensors that transmit real time data to the wastewater treatment stations and pump stations. o 64,860 m (target 60,000 m; target exceeded) of new drains were constructed in Lien Chieu. Page 7 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384)  New sewer lines were laid, and rainwater drainage routes were renovated in Thanh Khe, Lien Chieu, Hai Chau, Son Tra, and Nhu Hanh Son districts as targeted. New sewer lines were laid in Lieu Chieu, Huyem Tranh, and Cong Chua Road.  Wastewater collection systems and pumping stations were built at Phan Lang Lake in Son Tran and Ngu Hanh Son districts. Sewer pumping stations were constructed in Son Tra, Hai Chau, and Lien Chieu districts.  25.8 million m3 (original target 20 million m3, target exceeded) of wastewater were treated in the wastewater treatment plants. In the 2017 AF, the method of measuring this indicator was revised.  330 tons per year (original target 4,000 tons, target not achieved) of biochemical oxygen demand (BOD) pollution load was removed by the treatment plants. The measurement of wastewater treated was changed from “Volume (mass) of BOD pollution load” to “Volume of wastewater treated” to reflect capacity of the treatment plants. BOD represents the amount of oxygen consumed by bacteria and other microorganisms while they decompose organic matter under aerobic (oxygen is present) conditions at a specified temperature. Under the arterial road network  From the preceding PIIP, the following were completed as targeted: o southern road link connecting to the Hoa Phuoc – Hoa Khuong Road; o Phu Loc Bridge; and o embankment of Phu Loc River.  27.12 km (original target 17 km, revised to 28 km to reflect the added DH2 road, target substantially achieved) of new arterial and rehabilitated roads were constructed consisting of the following: o 19.89 km of the arterial road network, which included:  7.8 km long road DH2 Road from intersection NH 148 to DT 602;  3.2 km of the Extended Nguyen Tat Thanh Road from the Golden Hills urban area to Da Nang High Tech Park; and  8.89 km road of Hoa Phuoc-Hoa Khuong and Hoa Phuoc-Co Co bridge. o Other Roads (7.23 km) including the 545 km Southern Ring Road. and smaller roads sections constructed in various resettlement sites. o Two bridges, 134.5 m of Qua Giang and 213 m of Song Yen.  The following seven traffic intersections were completed as targeted: o Nam Tran; o Ton Duc Thang-Hoang Van Thai; o University of Education -Nguyen Khuyen; o Polytechnic University – Nguyen Luong Bang; o Intersection at the beginning of No. 5 Road; o Nguyen Luong Bang – Phan Van Dinh; and o No. 5 Street- Vo Duy Khuon.g Under public transport:  No high-quality buses were procured (original target 81, revised target 60, target not met).  The following were completed, as targeted: 300 bus stops, 9 bus terminals in Xuan Dieu, Pham Hung, Viet Han, Hoa Vang, Bau Tram, Non Nuoc, Nguyen Luong Bang, Hoa Tien High-tech zone, 29- 3 park, an Interchange Facility at Da Nang Airport, the Bau Tram depot and terminal, a vehicle Page 8 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) underpass at a major intersection of Dien Bien Phu and Nguyen Tri Phuong, pedestrian bridges at 29- 3 Park and Dien Bien Phu street, and other associated infrastructure for bus operation. This was an original outcome indicator with target values revised in the 2019 restructuring.  Technical assistance was provided to the Da Nang Traffic Light and Public Transport Management Center (DATRAMAC) for public transport management such as systems management, planning, and operations, as targeted.  35 consultations were held to encourage the participation of vulnerable groups. 70 percent (original target 70 percent, target achieved) participated. Outcomes:  Outcome 1 - under expanded access by Da Nang city residents to improved drainage, wastewater collection and treatment services - an area of 655.85 ha (original target 400 ha, revised to 500 ha, target exceeded) benefited from drainage coverage and flood protection. This is a corporate result indicator that is more intermediate rather than an outcome indicator. About 506,800 (original target 400,000, revised to 500,000, target exceeded) people in urban areas were provided with access to improved sanitation service. Of these 255,900 (original target 205,000, revised to 255,000, target exceeded) were women; and 250,900 (original target 195,000, revised to 245,000, target exceeded) were men. The drainage and sewerage infrastructure reduced the incidence of uncontrolled run-off and flooding in several city areas –events that have intensified because of climate change. Prior to the project, areas of Hai Ho, Tho Quang – Bien Dong, Yen The Bac Son, and Da Co canals experienced flooding during the rainy season. After the completion of the Hai Ho drainage sewer, the Tho Quang – Bien Dong drainage sewer line, the Yen-The Bac Son line and the wastewater collection pipes along Da Co, canal flooding has not occurred in these areas. the Ho Xuan and Son Tra plants increased the wastewater treatment capacity.  Outcome 2 – under expanded access by city residents to arterial road network – The May 2017 AF revised the target level of two PDO outcome-level indicators and added a new outcome indicator to reflect the added activities. Restructuring also corrected the wording of the original PDO-level Indicator from “reduction in travel time” to “travel time” (in minutes, by car on weekdays during peak hours): The Bank Team confirmed this definition of the outcome indicator. o travel time from highway 14B to coastal road - 14.5 minutes (baseline 25 minutes, original target 14 minutes, target substantially achieved). o travel time from High Tech Park to City Center” - 15.6 minutes (baseline 30 minutes, original target 15 minutes, target substantially achieved) o travel time from highway 14B to DT602 - 17.9 minutes (original target 14 minutes, target partially achieved). The 2017 AF added this indicator. At closing, the road was to be completed and expected to be fully operational by June 30, 2022. The 17.9 minutes refer to the time reduced along the partially completed road.  Outcome 3 – under expanded access by residents to public transport, on an average weekday 46,540 people used public transport, including those served by inter-provincial routes (baseline 22,000, original target 132,000 revised to 62,000, target partially achieved). Of the total, 24,665 were women (baseline 12,000, original target 701,000, revised target 31, 620, target partially achieved). In the case of men, there were 21,875 (baseline, 10,000, original target 62,000,000, revised target 30,380, target partially achieved). The ridership data was taken from a 2019 survey. COVID-19 prevented the conduct of another survey at closing. Although at closing the lanes and new buses were not procured using project funds, the city expanded the regular inner-city bus services using its own funds and a private bus operator provided buses (ICR, paragraph 37). The investments and capacity built Page 9 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) and ongoing expansion by the city post-closing, provide a level of confidence that increased use of and expanded access to public transport will be achieved in the near future. Overall, the efficacy of the project to achieve this objective is rated Substantial with moderate shortcomings. Most of the outcome indicators were achieved or exceeded. Target indicators for outcomes associated with the public transport interventions were partially achieved. A shortcoming was in the use of beneficiaries as an outcome indicator to achieve the expansion of the drainage services (see also Sections Quality at Entry and M&E). This was more at an output rather than an outcome level indicator. Some outcomes were described but without supporting data. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall, the efficacy of the project to achieve its objective is rated substantial with moderate shortcomings. Five components were to achieve this objective. Of the five, three components substantially achieved outcome targets, with moderate shortcomings because not all the outcome indicators sufficiently addressed the outcome of the interventions. The remaining two components only partially achieved outcome targets. In sum, overall target outcome indicators were partially achieved. The government and the World Bank agreed in the June 2021 Post-Closure Action Plan that the government would complete unfinished activities – 4 wastewater works, the DH2 Road, and resettlement works. The completion of the works is expected to reasonably achieve the expected outcomes of the PDO. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency: At appraisal and during the AF, the cost benefit analysis used a 12 percent discount rate and a “with” and “without” appreciation of land value. The works transferred from PIIP (component 5) were not included in the efficiency analysis. The following were the estimated economic internal rates of return (EIRRs) for each infrastructure component with land appreciating in value: 15.9 percent for drainage and wastewater improvements;19.3 percent for bus rapid transit; 25.1 percent for the Nguyen Tat Thanh extension and 19.7 percent for the Hoa Phuoc – Hoa Khuong road. Sensitivity analyses applied a combined 10 percent increase in costs and a 10 percent decrease in benefit (PAD, Table 3) and resulted in the following EIRRs: 12.1 percent for drainage and wastewater improvements; 15.5 percent for bus rapid transit; and strategic urban roads at 14.4 percent for the Nguyen Tat Thanh extension and 13.2 percent for the Hoa Phuoc – Hoa Khuong Road. Page 10 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) All works under this component were completed and the Son Tra Wastewater Treatment Plant commissioned by July 2017. The PAD did not provide an overall EIRR for the project. At closing, the same methodology at appraisal was used except for a 10 rather than the 12 percent discount rate, following the World Bank's guidelines. Actual costs were used. Benefits of the drainage and wastewater investments included improved health outcomes from reduced health-related expenditures; increased productivity due to time savings; reduced flood damage; and increased land values. Urban roads benefits included annual savings in vehicle operating cost (VOC) and travel time using “with” and “without” scenarios. Another benefit was the appreciation of land values within the first 10 years of the opening of the roads (ICR, Annex 4). A separate economic efficiency analysis was applied to the bus rapid transit. The EIRRs, with appreciation of land value, were 24.20 percent for drainage and wastewater improvements; 27.7 percent for Nguyen Tat Thanh Extension Road; 30.4 percent for the Hoa Phuoc – Hoa Khuong Road; and 32.1 percent for the DH2 road. The overall EIRR for all three components at closing was 25 percent with land value appreciating. There was no comparable EIRR for the combined three components at appraisal. According to the ICR, the rates of return at closing were adversely affected by the extended implementation period, which reduced the benefit stream, and lower actual costs because of procurement savings (ICR, Annex 4, paragraph 19). Administrative and Operational Efficiency: Several factors affected implementation efficiency. Delays were caused by procurement, reduced government commitment, and land acquisition challenges. AF and three level 2 restructurings were undertaken. The project was extended by 24 months. At closing, works remained to be completed because of the combined impact of the pandemic, and remaining land acquisition challenges. The government expects to complete remaining works by December 2022 (ICR, paragraph 35). Procurement (see Section 10 Other Issues, (b) Procurement below) started later than planned. The multi sector implementation structure caused protracted coordination. Subprojects were revised late in the procurement stages. The AF was to scale up the first three components but had more delays. The BRT bus design was changed. Technical reviews were delayed. Procurement issues surrounded the BRT systems, further delaying subsequent civil works. After receiving negative publicity, the government replaced the BRT with a High-Quality Bus Service (HQBS, see Section 4 Efficacy above). The government did not use AF funds for capacity building (ICR, paragraph 60). Land acquisition, resettlement, and compensation of Project Affected Persons (PAPs) experienced delays. The scope and designs of investments were periodically revised to minimize the rehabilitation and resettlement activities. Not all lands could readily be acquired. In June 2021 the government requested to cancel funds not expected to be used and provided assurances that remaining works will be completed by December 2022. The ICR reported unconvincing operational efficiencies. One was a 29 percent (Annex 3 showed 20 percent) utilization rate of the estimated project management costs despite a two-year extension. It was unclear if that budget was overestimated in the first place. The Bank Team clarified that estimated project management cost was 29 percent lower than envisaged at appraisal. and not utilization rate. Competitive bidding resulted in lower investment costs. But the use of less resources due to procurement and contingencies savings, and the reduced scope of the public transport investments due to limited work variations, led to US$110 million or 40 percent of the credit to be cancelled (ICR, paragraphs 20 and 61). Overall, the efficiency of the project is rated Modest. The positive rates of return of the project investments were overshadowed by the operational inefficiencies, procurement delays, and cancellation of IDA credit. There is no comparative ERR to show all 3 components at appraisal and at closing. Page 11 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate  25.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the objective is rated High. Efficacy of the project to achieve its objective is rated Substantial with moderate shortcomings. There is a reasonable expectation that the remaining works will be completed and will result in achieving outcome targets. Efficiency is rated Modest because of operational inefficiencies. The overall outcome is rated Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The following pose risks to the project’s development outcomes:  Covid 19 pandemic: The pandemic had a negative impact on overall implementation and on the public transport outcome. At closing, national and city government policies designed to limit the impact of COVID-19 are likely to continue to pose substantial risks to the transport-related outcome of the project.  Impact of Natural Disasters. Da Nang faced seven storms and eight floods between February and September 2020. These delayed progress on several construction works and necessitated extensive remedial work on others. This led to delays in the completion of these works and subsequent payment to contractors. The recurrence of natural disasters is a substantial risk.  Operations and Maintenance (O&M) Needs: The Da Nang Drainage and Wastewater Management Company, which managed the PIIP subprojects, also manage the sanitation and wastewater investments. The city has implemented the full cost recovery of the sanitation and wastewater Page 12 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) investments under its wastewater strategy (2010). The city expects to continue this full cost recovery policy by increasing charges, if required, to cover O&M needs of the infrastructure. The Roads Maintenance Company of the Department of Transportation financed by the city budget (ICR, footnote 27) was to undertake the O&M of the arterial roads. The project team confirmed that this is being done. The city also prepared a detailed plan assigning a dedicated transport management center or DATRAMAC for the O&M of public transport assets for the period 2022– 2025 until private companies are selected by competitive tender after 2025. Training in procurement and contract management under the project was to benefit bus operators in effective management. After closing, the city has provided its commitment to complete works that were unfinished by the time the project close but unclear if there was commitment to continue to allocate O&M funds. This is a moderate risk. 8. Assessment of Bank Performance a. Quality-at-Entry This third engagement of the World Bank with Da Nang followed the PIIP. The findings of a pre-feasibility study informed the design of the BRT as part of an integrated, multi-sector approach to solve traffic congestion. Preparation highlighted the risks from insufficient data on transport demand and political commitment, and readiness to implement the BRT. To mitigate these risks, design focused on establishing a transport authority, build its capacity, and procure private buses prior to the BRT roll out. Procurement delays pervading the World Bank Vietnam portfolio was to be addressed by having bidding documents ready for 30 percent of total financing needs before the Board approved the credit. The multi-sector approach corresponded to the city’s vision to be a green and sustainable urban center and matched the World Bank’s priorities for the country (see Section 3 Relevance of Objectives above). A multi-sector implementation structure was to be carried out using staff experienced with Bank-financed projects and whose capacity was strengthened under the PIIP. Design appeared to have disregarded the PIIP experience in land acquisition and issues negotiating resettlement and rehabilitation of Project Affected Persons (PAPs, see Section 10 Other Issues below). Design proved to be ambitious and miscalculated the complexities and required additional technical capacity to implement the BRT. The inadequate assessment of risks and lack of readiness to implement were borne out at implementation (see Quality of Supervision below). Overall, the quality at entry is rated Moderately Satisfactory. Readiness to implement was hampered by: (i) an overly ambitious multi-sector design; (ii) inadequate assessment of implementation capacity and arrangement; and (iii) disregard for the lessons offered by PIIP, particularly in land acquisition. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Page 13 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) The World Bank team conducted 17 supervision missions supplemented by technical and fiduciary missions over the 8-year project period. The fiduciary and safeguards supervision missions included site- visits and field consultations. The Task Team Leader stayed on for the first half of the project period. Key technical design staff who were involved in preparing the project remained for the first year of implementation. Most team members were based in the country and maintained close communication with the government to address evolving priorities in a timely manner. At project closure, some works remained to be completed (see Section 4 Efficacy above). The Bank and the government agreed on a plan for the government to complete the remaining works and extend Bank supervision of fiduciary compliance until December 2022 (ICR, footnote 26). The following caused implementation delays: (i) a multi sector procurement structure led to protracted coordination; (ii) the procurement process started later than planned; and (iii) land acquisition posed challenges (see Section 10 Other Issues below). The Mid Term Review (MTR) was conducted in August 2015 shortly after the credit became effective (the Data Sheet showed this as July 2013 but July 2014 in paragraph 58) but did not adequately assess the impact of the implementation delays. The Bank team prepared AF to scale up the first three components and reduce the other two (ICR, footnote 7) counting on continuing government commitment to improve public transport using the BRT. Allocating AF for the BRT proved premature as public opinion against the BRT increased and the government replaced it with the HQBS instead. The terms of the AF discouraged the government from accessing funds for capacity building. According to the Bank Team the Government was reluctant to utilize the IDA and AF for capacity building activities because of its higher interest rate, In addition, the structure of the AF and original credit did not allow fungibility of monies for activities even though there may have been funds left in the original credit at lower cost. The government requested a restructuring in June 2021 to cancel funds not expected to be used and to use its own resources to complete the remaining works by December 2022. The Bank team would continue to supervise compliance with fiduciary safeguards until then (ICR, paragraph 61). The quality of Bank supervision is rated Moderately Satisfactory. The Bank team provided adequate support to ensure that the project remained relevant to the government’s evolving priorities. At supervision, the Bank team addressed implementation challenges through AF, restructurings, and timely support to the government. The AF proved to be less effective than envisioned. Overall, the quality of Bank performance is rated Moderately Satisfactory. Quality at entry and at supervision are both Moderately Satisfactory. Shortcomings at entry were related to the identification of risks and the lack of indicators to adequately reflect the outcome of the project interventions, particularly in the expansion of drainage services in selected areas of the city. Mitigation measures were not sufficient to overcome the ambitious multi sector design of addressing traffic congestion to improve the public transport system. Shortcoming at supervision was derived from the inadequate assessment of implementation delays to inform the components of the AF and missed opportunity at restructuring to strengthen indicators to better capture the outcome of improved services. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory Page 14 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s M&E system was designed to assess progress toward the achievement of the PDO. The theory of change was sound. The causal relationship between five key components or activities and outputs reflected in 13 intermediate results indicators (IRIs) that led to 5 outcomes. Targets were relevant, realistic, measurable, and achievable. Some of the outcome indicators, however, were not sufficient to cover all the outcomes of the project interventions. This was true for the outcome indicator of the interventions surrounding the drainage, wastewater collection, and treatment services. Baselines and targets were provided for all identified indicators. M&E was designed to be implemented by an outside consultant rather than as part of the function of the PMU. b. M&E Implementation The M&E system was implemented as designed. The Project Management Unit hired an independent consultant to monitor progress against the indicators of the results framework and the quality of physical assets built. The arrangement for completing the remaining works after closing noted that the PMU would continue to monitor compliance with Bank policies (ICR, paragraph 72). Initial coordination difficulties between the PMU and local authorities affected the Project Affected Peoples data but resolved by 2018. Periodic reports monitored progress on resettlement works. Beneficiaries, such as vulnerable groups were reported to have been consulted in coming up with the drainage investments and the O&M requirements of the completed assets (ICR, paragraph 45). The 2017 AF and 2019 restructurings updated output and outcome target values to highlight the relevance of results indicators and reflect changes in scope and design of activities. An output indicator was revised from “Volume (mass) of BOD pollution load removed” to “Volume of wastewater treated” to measure the volume of treated water (output) rather than the quality of the input. However, no outcome indicator captured the impact of the reduced BOD load. Other outcome indicators, beyond the number of beneficiaries, such as improved health of beneficiaries or reduced property losses, were not added during implementation. Beneficiary satisfaction surveys were conducted for completed activities. COVID-19 restrictions in 2020 and 2021 did not allow for planned comparable surveys at project close. However, the PMU M&E reports added sewerage connections and public consultations as part of evaluation at closing. Impact studies and beneficiary surveys were to be conducted after COVID-19 and after the post-project close in December 2022. c. M&E Utilization M&E data were used to assess the implementation status of the project, raise issues with the city government and the World Bank management. However, the 2017 AF did not use M&E data to inform resource allocation or rationalize added activities. The 2019 restructuring used M&E data to revise targets and activities and extend the project closing date. After 2019, resettlement and relocation data Page 15 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) led to the adoption of corrective land acquisition measures. By 2020 and at closing, COVID-19 prevented further assessments and surveys to measure results. Overall, the M&E system was reasonably designed. Data was regularly collected. Inconsistencies in data were resolved. Reports were timely. The shortcoming in the use of data was reflected in the premature allocation of resources during the AF that eventually led to substantial cancellation of resources. M&E is rated Modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards Environmental and Social Safeguards: The project was classified at appraisal as Environmental category “B” (requiring a Partial Assessment) because of the type, scale, nature, and magnitude of its potential environmental impacts (PAD, paragraph 55). The project triggered the World Bank’s safeguards policies on Environmental Assessment (OP 4.01), Physical Cultural Resources (OP 4.11) and Involuntary Resettlement (OP 4.12). A Resettlement Policy Framework and six Resettlement Action Plans (RAPs) were prepared and implemented (ICR, footnote 23). A grievance redress mechanism (GRM) was established. The project complied with the environmental safeguards policies (ICR, paragraph 69). According to the Bank Team, the project complied with the Bank safeguard policy OP/BP 4.11 on Physical Cultural Resources. The project involved the relocation of 1,200 graves and one pagoda. The relocation process was carried out in line the procedures and requirements as specified in the Resettlement Plans. No other physical cultural resource was identified during project excavation work. In 2013, new legislation required that land acquisition and related compensation functions were to be handled by local district authorities. Cities established Land Fund Development Centers (LFDCs) to collect data on persons eligible for resettlement. These LFDCs approved documentation. The preceding PIIP highlighted complexities in negotiating with Project Affected Persons, but design appeared to have underestimated the scale of land acquisition and the subsequent resettlement and rehabilitation needs of the project. The project experienced delays due to the following: (i) the PMU initially managed beneficiary data that led to multiple documentation of PAPs, with incomplete information on which to base compensation. The PMU and the District Compensation, Assistance, and Resettlement Board (DCARB) and local authorities took time coordinating data, processing approvals, and documenting PAPs; (ii) discrepancies between the World Bank guidelines and local laws required time to reconcile data; (iii) compensation negotiations with affected households took time; and (iv) the 2017 AF led to design and scope changes (ICR, paragraph 55) that added acquisition of land, resettlement, and rehabilitation requirements. After the 2017 AF, the PMU and local authorities rationalized data management and coordination (ICR, paragraph 57) to address the delays associated with resettlement needs. The project acquired 143.72 ha of land, affecting 4,730 PAPs almost twice the appraisal estimate (2,855) due to works added in the AF. Of the 4,730 PAPs,1,076 were relocated and 4,566 were compensated based on replacement cost surveys undertaken by an independent appraiser. The approved Entitlement Matrix of site-specific RAPs defined compensation and assistance packages for the affected households. Page 16 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) Eighty-three percent of households surveyed indicated satisfaction with the compensation price and resettlement support, but 17 percent were not because they expected a higher price and/or resettlement land lots (ICR, footnote 24). Questions and complaints from the affected households were classified and assigned for resolution according to the GRM in place. Between July and December 2021, 94 additional households received compensation and agreed to transfer their land. At closing, 164 cases remained pending. The Post-Closure Action Plan included actions to resolve them (ICR, paragraph 72). According to the ICR, the project complied with OP 4.12 (ICR, paragraph 70). Partial non-compliance cases for certain activities were identified in 2020 that required land acquisition and three resettlement sites that were added and implemented in contracts without the World Bank’s prior “no objection.” The World Bank review in June 2021 confirmed that these issues were resolved (ICR, paragraph 70). b. Fiduciary Compliance Financial Management: The project complied with the World Bank Financial Management (FM) policies and procedures (ICR, paragraph 73). The PMU had an adequate FM team. Independent audits were conducted. Eight audit reports from fiscal years 2013 to 2020 were submitted on time, were found acceptable, with unmodified opinion. Interim Financial Reports (IFRs) were submitted on time. However, a State Audit of Vietnam from April 2013 to June 30, 2019, found ineligible expenditures: (i) VND1,905 billion (US$82,382) paid to several contractors and consultants and (ii) VND3,958 billion (US$173,034) in excess payment to another contractor. The government fully refunded the World Bank in October 2021 (ICR, paragraph 73). Procurement: The project complied with World Bank procurement policies and procedures (ICR, paragraph 74). The PMU had designated procurement staff. Detailed procurement plans were regularly updated. Initial procurement was delayed because the government installed complex approval requirements and delegated procurement to technical departments, which led to lengthy approvals. Contrary to the mitigating measure identified at preparation, procurement for the works, except for PIIP carry over activities under component 5 (ICR, paragraph 51) were started only after the credit became effective. Complex, multi-sector packages required interdepartmental coordination. Design changes were made at late stages of the procurement processes to the scope of work or to reduce resettlement requirements, required additional planning and approvals, and further implementation delays. The PMU and DATRAMAC staff received contract management training, adopted additional due diligence, and subsequently improved the quality and efficiency of the procurement processes. The World Bank received and investigated several procurement-related complaints, Sanctions were imposed on several international and national companies. The World Bank recommended that Da Nang: (i) sign contract amendments to extend nine outstanding works contracts; (ii) expedite selection of consultancy services; and (iii) assess the adequacy and adjust as needed the budget allocated for contract extension. c. Unintended impacts (Positive or Negative) --- d. Other Page 17 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Weaknesses in outcome Quality of M&E Substantial Modest indicators; M&E data was not used to inform AF. Quality of ICR --- Substantial 12. Lessons The ICR presented five lessons from the operations, with some modifications below:  Complex public transport investments may require standalone project financing. In this project, the pioneering BRT-related transport investments was a novel, complex solution to traffic congestion and was highly dependent on consumer behavior. This component allocated with significant investments would require highly technical expertise, realistic assessment of implementation capacity, and government commitment to support the implementation of an ambitious design to address traffic congestion. The innovation was dependent on swaying consumer behavior and would have required more customized consultations with beneficiaries to gain some confidence and a generous comfort level from end users to support innovative solutions.  BRT investments may fail without long-term political commitment, positive public demand and perception, and strong implementation capacity. This project and the Hanoi Urban Transport Development Project (P083581) both sought to transform a commuter’s core mode of conveyance—from private motorbike to public system. In both cases, the project proved highly ambitious and targeted consumer behavior and preferences. In this project, consumer preference swayed government commitment to move ahead with planned investments. Corrective measures or design adjustments were introduced based on end user feedback. A strong communication strategy may include aspects of influencing public behavior to gain support for a new system.  Projects that may require land acquisition in Vietnam would benefit from early local coordination. In this project, land acquisition and related compensation functions were legislated to local district authorities. Cities established Land Fund Development Centers to collect data on persons eligible for resettlement and rehabilitation, approve documentation. Inadequate coordination led to delays. Land acquisition processes and compensation decisions took more time. Local authorities need to be consulted early to streamline procedures or incorporate design features in investments that would minimize land acquisition or resettlement and rehabilitation of Project Affected Persons. These early Page 18 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) consultations may foresee choke points in implementation and design measures to overcome delays.  Additional Financing may be served by a realistic assessment of physical and financial progress of additional components to be financed. In this project, the May 2017 AF funds were committed for the Components 1, 2, and 3. However, activities under Component 2 were ongoing. A detailed assessment of the progress of all the activities under Component 2 may have identified those activities that could proceed without waiting for actions precedent. In this way, premature additional commitments may be avoided.  Experienced Project Management Units may continue to benefit from support when implementing transformative projects. In this project, the city and the PMU were familiar with World Bank policies after previously implementing two Bank-financed projects. However, this project was more complex, involved more than one sector, and was dependent on consumer behavior to achieve the PDO. The PMU had to implement activities in sectors they had no capacity or experience, resulting in delays. Continuous capacity building, such as hiring consultants, highly technical advisors familiar with local policies and conditions may supplement initial capacity shortcomings. Ongoing capacity support to even an experience PMU would help achieve a project's objectives. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR followed the guidelines. The report was results oriented. A clear narrative of the operation summarized salient points. The evidence was based on identified surveys and reports. Footnotes provided explanatory information. Analysis was concise. The evidence was clearly linked to findings, internally consistent and results mutually reinforcing. Annexes provided additional information to strengthen the arguments supporting project outcomes. Annex 4 extensively discussed the economic efficiency of the project. Annex 1 was particularly helpful and highlighted the links between outputs and outcomes. Lessons were derived from the project experience, pointing to the need to take advantage of the MTR to closely assess performance to craft a responsive AF strategy and resource allocation. They were, however, largely stated in the form of recommendations or findings. Minor inconsistencies did not affect the substance of the narrative. For example, the project became effective in July 2013 in the data sheet but July 2014 in paragraph 58. In another, project management used 29 percent of appraisal estimate but 20 percent in Annex 3. Note: the project team informed IEG that the client had provided an incorrect costing table for the ICR. It did not include counterpart funding on the land acquisition cost and project management cost. The project team had requested for and received updated figures from the Government on counterpart funds. The ICR will be revised with updated figures and resubmitted to SECPO to replace the current ICR. Page 19 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN-Danang Sustainable City Development (P123384) a. Quality of ICR Rating Substantial Page 20 of 20