Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) Report Number: ICRR0022585 1. Project Data Project ID Project Name P123201 MZ- Cities & Climate Change Country Practice Area(Lead) Mozambique Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-50660,TF-16486,TF-16580 15-Dec-2017 122,448,604.69 Bank Approval Date Closing Date (Actual) 03-Apr-2012 30-Sep-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 120,000,000.00 15,750,000.00 Revised Commitment 131,560,796.54 15,561,840.66 Actual 122,517,360.69 15,561,840.66 Prepared by Reviewed by ICR Review Coordinator Group Maria Shkaratan Kavita Mathur Victoria Alexeeva IEGSD (Unit 4) P146059_TBL Project ID Project Name P146059 Mz Cities and Climate Change PPCR AF ( P146059 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 0 Bank Approval Date Closing Date (Actual) 23-Oct-2014 Page 1 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 0.00 Revised Commitment 0.00 0.00 Actual 0.00 0.00 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The original project development objective (PDO) was to strengthen municipal capacity for sustainable urban infrastructure provision and environmental management which enhance resiliency to climate related risks. The revised PDO was to strengthen institutional capacity for local revenue enhancement and land use management in targeted municipalities, and to enhance climate resilience in the coastal cities of Beira and Maputo. For the purposes of this ICR review, the objective will be assessed as follows: PDO1: to strengthen institutional capacity for local revenue enhancement and land use management in targeted municipalities. PDO2: to enhance climate resilience in the coastal cities of Beira and Maputo. A split rating is not justified for this evaluation because the ambition of the project remained the same after the revision, and the revised PDO is better worded and more specific in defining the two-fold objective of the project and the intended project results; it does not change the focus of the project or its expected outcomes. The revised PDO also reflects the transfer of funds from erosion works in the city of Nacala to the priority drainage works in the city of Maputo at restructuring. Again, this transfer did not change the ambition of the project. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 03-Nov-2017 Page 2 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) c. Will a split evaluation be undertaken? No d. Components 1. Original components Component 1: Strengthening the municipal sector (at approval: US$35 million/actual: US$32.13 million). This component aimed at supporting municipal capacity to plan, manage, and finance climate- resilient urban development, including through improved decentralized financing and management of urban environment and infrastructure. Sub-component 1A: Local level support for improved municipal governance (at approval: US$27.5 million/actual: Not available (NA)). Support to 20 municipalities through: i. A Municipal Performance Grant (MPG) pilot program to demonstrate improved institutional performance through municipal investments. Grants would complement the intergovernmental transfers to municipalities and were to be used for capital investments only, e.g., for improved resiliency of roads, bridges, drainage infrastructure, or for urban service provision equipment (e.g., refuse trucks). Successful pilots would be later scaled up at national level. ii. Technical assistance (TA) to improve urban planning and land use management: development of climate vulnerability assessments, flood and erosion management instruments, urban planning and land use methodologies, and related training. iii. TA to support municipal financial sustainability: development of methodologies for revenue administration, and related training and equipment; design of financial management systems. Sub-component 1B: National Level Support for Improved Municipal Governance (at approval: US$ 7.5 million/actual: NA). Support to: (i) Ministry of State Administration and Public Service (MAEFP) and the Ministry of Finance (MEF) to monitor performance of municipalities and develop improved intergovernmental and municipal policies and systems; (ii) National Association of Municipalities in Mozambique (ANAMM) to provide capacity building services to municipalities. Component 2: Enhancing resilience of strategic coastal cities (at approval: US$85 million/actual: US$90.4 million). This component would support improved resilience to weather-related threats in the coastal cities of Beira, Nacala and Maputo. This involved: development of a strategic master plan and drainage rehabilitation works in Beira (US$61.9 million); erosion control and drainage works in Nacala (US$6.4 million); and development of the Master drainage and sanitation plan in Maputo (US$7.8 million) (inclusion of drainage works in Maputo depended on the availability of financial resources). 2. Changes in components at additional financing and at restructurings: (i) Additional financing (AF) of US$15.75 million from the Strategic Climate Fund under the PPCR (SCF- PPCR) was approved in 2014 and became effective in April 2015. It financed additional investments in green urban infrastructure in Beira under Component 2. The scope of the project was increased in line with the added funding. Specifically, green urban infrastructure developed in Beira included a 17-hectare park and 4 km of improved natural drainage, providing additional flood protection, improved water retention, preservation of mangroves, and improved ecological condition of the Chiveve river. Page 3 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) (ii) On the basis of the mid-term review (MTR) recommendations, the project first restructuring in November 2017 included the following: erosion control studies and works in Nacala were canceled due to insufficient funds remaining after exchange rate fluctuations (devaluation of XDR-USD reduced the original IDA credit by US$9 million); these funds were reallocated for priority drainage works in Maputo. The ambition of the project remained the same. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The actual project cost was US$122.52 million, compared to the appraisal estimate of US$120 million. Project Financing: The only financing source in the original project was an International Development Association (IDA) credit. The original project financing was US$120 million. In 2014, a loan of US$6.50 million and a grant of US$9.25 million were provided by the SCF-PPCR. After the AF, project financing amounted to US$135.8 million. After restructuring, the total financing was US$125.75 million. Actual financing at closing totaled US$122.5 million. Borrower/Recipient Contribution: The Recipient did not make any contribution. Project Dates The project was approved on April 3, 2012 and became effective on July 18, 2012. The mid-term review was on April 4, 2016. The original closing date was December 15, 2017. The IDA credit was extended twice, totaling two years, and the final closing date was December 31, 2019. The SCF-PPCR funding was also extended twice and closed on September 30, 2020. Overall, the project extensions totaled two years and nine months. The project went through two level one restructurings: The first restructuring was approved in November 2017 for the following reasons: 1. The PDO was revised to express more clearly the two-fold objective of the project: (i) to strengthen institutional and financial management capacity of small municipalities to support climate resiliency and (ii) to implement drainage works in Beira and Maputo. The revised PDO also reflects the transfer of funds from erosion works in Nacala to the priority drainage works in Maputo at restructuring. 2. The results framework (RF) was aligned with the revised PDO. 3. Within Component 2, the erosion control studies and works in Nacala were cancelled and the funds were transferred to drainage works in Maputo. Related changes in the RF involved: (i) removal of the indicators of the Nacala activities; (ii) adding indicators of the Maputo drainage works. 4. The project closing date was extended. The second restructuring, approved in December 2019, was due to the cancelation of a TA on urban planning instruments and risk mapping for selected beneficiary municipalities; and two related intermediate indicators were dropped. Page 4 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) 3. Relevance of Objectives Rationale The country is characterized by high exposure and vulnerability to natural hazards, but damage from the extreme events could be reduced. Average annual losses from floods and cyclones were estimated around US$440 million (based on the catastrophe modelling study conducted during project implementation in 2018). Urban flooding and erosion are increasing with climate change, and measures to prevent the damage and manage the response to natural disasters need to be put in place. The country, especially the coastal cities, which suffer the most, need climate resilient infrastructure and increased capacity of the governments (central and local) to maintain it, to manage urban land use, and, importantly, to mobilize and manage resources needed for sustainable service provision and disaster response. The emphasis is on the decentralized financing and management of the urban environment and infrastructure. At the time of approval, the project was highly relevant to the country conditions, well-aligned with national priorities, and consistent with the World Bank development strategy. Cities suffered the most from the increasingly severe natural hazards, while sufficient investment in urban flood protection and drainage was lacking, and municipal capacity to manage damage prevention and disaster response was inadequate. National climate resilience priorities were defined in the National Adaptation Program of Action (NAPA) (2007) and further developed in the Strategic Program for Climate Resilience (SPCR) (endorsed in 2011). The objectives and technical standards for urban sanitation and drainage systems were formulated in the National Sanitation Sector Policy. Decentralization and consolidation of municipal administrations was prioritized in the government Five-Year Program for 2010-2014. The project contributed to two pillars of the World Bank’s FY12-15 Country Partnership Strategy (CPS): (i) Vulnerability and Resilience; and (ii) Governance and Public Sector Capacity. The project was aligned with the World Bank’s Strategy for Making Development Climate Resilient in Sub-Saharan Africa (2009). The PDO remained highly relevant at closing. The government Disaster Risk Management (DRM) laws and regulations, adopted in 2020, push for mainstreaming the DRM and climate adaptation at national and local levels. Reducing natural disaster risks is a key strategic objective of the GoM’s Five-Year Development Program 2020-2024. The Bank has been increasing its support to climate resilience and municipal governance in the country during the project life, which demonstrates the continued relevance of the PDO. While the project was the first Bank’s urban resilience operation in the country, at project’s closure, there were several active or pipeline Bank operations built on it including the Cyclone Idai and Kenneth Emergency Recovery and Resilience Project (CERRP, P171040), which finances the second phase of Beira's drainage system, and the Urban Development and Decentralization Project (P163989), which continues support to urban development and municipal revenue generation, including an improved version of the MPG program. The relevance of objective is rated high. Rating Relevance TBL Rating High Page 5 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen institutional capacity for local revenue enhancement and land use management in targeted municipalities. Rationale The theory of change (TOC) for this objective is as follows. Input 1 consisted of: (i) the MPG pilots to develop municipal capital investments for a future scale-up; (ii) assessments, methodologies, and training for municipal urban planning and land use management; and (iii) methodologies, tools and training for municipal financial management. These inputs, if successfully developed, were expected to causally lead to the strengthening of the institutional capacity of the selected municipalities to plan, manage, and finance climate- resilient urban development with the focus on land use management. Input 2 consisted of support to key central level agencies in creating an efficient intergovernmental system with increased decentralization and improved decision-making at the municipal level. These inputs were expected to lead to better capacity of the government overall, including at the municipal level, to support urban development, land use, and DRM in a decentralized system, as well as local revenue enhancement. This would support the achievement of Objective 1. The long-term outcome would be that municipalities become more capable to manage land and have stronger financial sustainability. The TOC in the ICR clearly indicates inputs, outcomes, and long-term outcomes. However, the ICR does not discuss the underlying assumptions necessary for the TOC to work. The results chain is logical and relevant to the country’s needs. TA to municipalities, which equips them with approaches and tools to manage urban planning, land use, and finances, coupled with capital investment pilots and training, constitutes a package of capacity building activities proven to be effective in Bank projects. Including activities to improve the intergovernmental relations will increase the sustainability of the outcomes under Objective 1. Outputs The outputs related to Objective 1 mainly reflect improved financial management capacity of the municipalities and their MPG performance, the targets were mainly reached: 1. Eleven municipalities achieved at least 20 percent increase in revenues from municipal property tax (IPRA). The target of 14 municipalities was not reached. 2. The increase in the aggregate number of properties in municipal IPRA cadasters was 555 percent, significantly exceeding the target of 340 percent. This target was established at restructuring of November 2017. The original target was 40 percent. 3. Twenty-two municipalities achieved 70 percent or more of the financial execution of their annual MPG by December 31, 2018, exceeding the target of 15 municipalities. Page 6 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) 4. Thirty-six improved community water points were constructed or rehabilitated as part of the MPG program, significantly exceeding the target of 21. This high achievement is explained by the choice made by many municipalities participating in the MPG to invest in small water systems in peri-urban areas. 5. Training provided to the staff of the participating municipalities was found excellent by 78 percent of the trainees surveyed, and 45.5 percent reported being prepared to transfer the acquired skills to other staff members (ICR, page 12). Outcomes 1. The PDO targets under Objective 1 were achieved overall: (i) The project improved urban living conditions for more than 667 thousand people, while the target was 418 thousand people. This includes more than 336 thousand female beneficiaries, compared to the target of 213 thousand. (ii) Thirteen municipalities achieved their planned increases in own-source revenues, as compared to the fourteen municipalities targeted. (iii) Annual increase in aggregate municipal own-source revenue was 209 percent, significantly above the targeted 100 percent. (iv) Twenty municipalities put in place an operational land register, as compared to the target of 16. 2. Under Objective 1, the project’s key success was in developing municipal institutional capacity for improved municipal management of finances and of urban land use. First, the project strengthened financial independence of the participating municipalities. Annual increase in municipal own-source revenue was 209 percent, exceeding the target of 100 percent. The share of municipal income from own-source revenue increased from 19 to 22 percent between 2013-2018, and 11 of the 20 participating municipalities increased their IPRA revenues by over 20 percent (ICR, page 12). Second, the project improved municipal capacity for land use management. All 20 participating municipalities implemented operational land registries, allowing to register land and property and systematically linking it to IPRA, the most important municipal tax in Mozambique (ICR, page 12). The project contributed to a significant increase in the issuance of land and property related permits and authorizations. Between 2014 and 2018, in the participating municipalities, the number of allocated plots increased eight-fold, the number of construction permits rose almost four-fold, and the number of utilization licenses grew more than seven-fold. In the same period, the participating municipalities added 133,265 plots to the municipal land registries created or strengthened by the Project, increasing the number of properties in the municipal IPRA taxation cadaster by 555 percent and overall own- source revenue by 209 percent. 3. The project also implemented, as planned, support to key central level agencies, where the aim was the creation of an efficient decentralized system to support municipalities with their new mandate in the decentralized intergovernmental system. This is expected to lead to increased sustainability of the projects key achievements described in point 2 above. Support to the Directorate for Local Governance Development (DNDA) at MAEFP focused on strategies to help municipalities to make own decisions. In MEF, training was provided, and a municipal finance monitoring system was designed. Support to ANAMM in Page 7 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) preparing its strategic plan, financing assemblies, and training its members helped to strengthen their capacity to promote municipal interests at the national level and provide better services to the municipalities. 4. The project’s missed opportunity under Objective 1 is related to the task of strengthening urban resilience. The TA on risk-informed urban planning for prioritized municipalities was cancelled. After a long period of procurement delays, the GoM canceled this TA in 2018, explaining it by the upcoming municipal elections and the need for better coordination with the Ministry of Land, Environment and Rural Development (MITADER). This TA was planned as synergetic with the TA on municipal land use management and urban planning (see point 2 above) and focused on resilience aspects in urban spatial planning. 5. Overall (under both objectives), the project provided 667 thousand people with improved urban living conditions, significantly above the target. This indicator was calculated as follows: the infrastructure built by the project (both under the MPGs and in Component 2) was associated with the population living in close proximity to it (and therefore having received access to it, e.g., roads, bridges, drainage), or number of users based on specific norms from utilities and municipal authorities (e.g., water supply points, public toilets). The total of the population who received the benefit of access to the infrastructure financed by the project are “people provided with improved urban living conditions”. There was a small difference in calculating the target and the result: the target used the population data from the 2007 census, projected into 2017, while the result used the population data from the 2017 census, which was not yet available when the target was established. Since the projections underestimated the population growth, there is a corresponding overestimation of the actual achievement of the PDO indicator 1. However, the TTL informed the IEG that this overestimate is limited, and the conclusion that this target was overachieved stays (based on the infrastructure construction outcomes, which exceeded the target due to savings under Component 2). Rating Substantial OBJECTIVE 2 Objective To enhance climate resilience in the coastal cities of Beira and Maputo. Rationale The theory of change (TOC) for this objective is as follows. Input 1 consisted of developing drainage master plans for Beira and Maputo. Input 2 comprised rehabilitating drainage canals in Beira and Maputo and implementing natural drainage improvements in Beira. These inputs would lead to improved drainage, less flooding, and avoided land erosion, and therefore improved climate resilience of Beira and Maputo, achieving Objective 2. The results chain is logical and relevant to the country’s needs. Well-functioning drainage infrastructure is a basic necessity in cities where flooding is a regular event and a baseline action in building climate resiliency. Outputs Page 8 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) Main outputs related to Objective 2 reflect two achievements: design of drainage master plans for major coastal cities (Beira and Maputo) and rehabilitation of drainage canals (Beira and Maputo) and natural drainage courses (Beira). All targets were reached. 1. Two drainage master plans for major coastal cities were completed, reaching the target of two. 2. The length of the Beira’s drainage canals rehabilitated was 11.67 km, exceeding the target of 9.5 km. 3. The length of Maputo’s rehabilitated drainage canals was 2.16 km, exceeding the target of 1.6 km. 4. Four km of Beira’s natural drainage courses were improved and protected, reaching the target of four km. Outcomes 1. The PDO indicators for Objective 2 were mostly achieved: (i) Area in Beira and Maputo totaling 1,140 hectares benefited from reduced flooding or erosion, exceeding the target of 1,100 hectares. (ii) The project improved urban living conditions for more than 667 thousand people, while the target was 418 thousand people. This includes more than 336 thousand female beneficiaries, compared to the target of 213 thousand. (note: this PDO indicator is used for both objectives) 2. Under objective 2, the project’s key value added was in creating well-functioning drainage systems in Beira and Maputo, thus protecting significant parts of the cities from flooding and erosion. Specifically, in Beira, in addition to rehabilitating drainage canals (see output 2 above), the project supported construction of a retention basin and a flood gate control system, delivering flood protection to 234,100 people. In Maputo, in addition to drainage canals (see output 3 above), a retention basin was financed by the project, protecting 71,154 people from floods. The Beira’s flood protection system has proven to be effective in reducing flooding and resilient to extreme events before project closure, during two extreme rainfalls, in January 2019 and in March 2019. There are two related outcomes:  By creating well-functioning drainage systems under Objective 2 and providing infrastructure improvements under the MPGs under Objective 1, the project improved urban living conditions to 667 thousand people, exceeding the target (please see discussion under Objective 1, Outcomes).  By creating well-functioning drainage systems under Objective 2, the project protected an area of 1,140 hectares from flooding, exceeding the target. These estimates were made using hydrological- hydraulic modeling with and without drainage infrastructure provided by the project. Two similar but not identical models were used at design and at completion; the modeling at completion was configured with updated hydrological-hydraulic parameters and drainage infrastructure as-built (thanks to savings, additional drainage works were completed than originally planned). Therefore, the comparison between the target and the outcome is fair but not absolutely precise. 3. An additional successful outcome was climate resilient urban green development in Beira. Specifically, the project added value to the improvement of natural drainage in Beira by developing a large urban park, thus providing a co-benefit of better living conditions to the population. Also, in Beira, the project invested in green infrastructure around the Chiveve River drainage basin, strengthening flood protection of Page 9 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) that area of the city. In this case, the co-benefits comprised improved living conditions and biodiversity protection: preservation of mangroves and improved ecological heath of the Chiveve River. 4. However, the originally planned investment and studies for erosion control in Nacala was canceled. While the reason (insufficient funds due to exchange rate fluctuations) was beyond the Bank or the government control and the funds were efficiently used for drainage investments in Maputo, still this is a missed opportunity. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale For Objective 1, the efficacy is Substantial. The project achieved the key outcomes of municipal revenue enhancement and improved urban land use management; the RF, as well as additional evidence in the ICR, provide sufficient proof. Coupled with capital investment pilots, municipal capacity building constituted an effective package supporting outcomes sustainability. Project activities aimed at providing the central government with tools to continue municipal capacity building and functioning within a decentralized system are expected to further increase the sustainability of project outcomes. There was a minor shortcoming: the TA on risk-informed urban planning was cancelled by the government. The TA was designed to focus on resilience aspects in urban spatial planning, in synergy with the financial and urban land use management aspects of municipal development. This shortcoming had an insignificant negative impact on the overall outcome. For Objective 2, the efficacy is Substantial. The project enhanced climate resiliency in the major coastal cities of Beira and Maputo by financing drainage master plans, rehabilitation of drainage canals, and construction of retention basins and a flood gate control system. The ICR provides evidence of reduced flooding and concludes that the results of this investment are as expected. In Beira, the resiliency of the project’s physical investments was proven during two extreme rain events which occurred before project closure. There were minor shortcomings: activities in Nacala were canceled due to exchange rate fluctuation and funds’ insufficiency, and there were procurement delays. The overall efficacy is substantial. Overall Efficacy Rating Substantial 5. Efficiency Page 10 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) The economic analysis was done at project appraisal and at closure. 1. Economic analysis at appraisal was done for Component 2 only (physical investments in flooding and erosion control measures), for Beira and Nacala. Since the project was restructured in November 2017 to cancel the Nacala investments and use the financing for new investments in Maputo, the IEG is only reviewing the analysis done for Beira. The analysis concluded that the Beira investment was economically feasible with the net present value (NPV) of US$49.5 million and the internal rate of return (IRR) of 18.1 percent. The Beira investment constituted 51 percent of the total project cost at appraisal. Methodology of the economic analysis at appraisal. The methodology used was cost benefit analysis. The costs comprise the project drainage investments and the related training and equipment for operations and maintenance (O&M), as well as annual O&M spending at 0.25 percent of the investment cost. The benefits comprise the newly established drainage fee (15 percent of the property tax) and avoided flood damage costs. The latter were estimated based on historical data and a hydrologic mathematical model for storm simulation, separately for: (i) industrial sector, (ii) formal housing; (iii) informal housing; (iv) agriculture; (v) roads; (vi) electricity infrastructure; (vii) transport; (viii) tourism; and (ix) loss of life. The costs included losses to capital infrastructure; loss of profits or earnings due to the impact on economic activity during flooding; and welfare impacts such as increased illness and diseases (malaria, cholera, tuberculosis, etc.) The present value of costs and benefits was calculated for the lifetime of the project investments (30 years) with the discount rate of 8 percent. The sensitivity analysis was conducted. 2. Economic analysis at closure. The analysis was done for both components. For Component 1 (capacity building), the project was estimated to be economically feasible, with the NPV of US$10.6 million and the IRR of 12.1 percent. For component 2 (physical investments), the estimates are available for Beira and not for Maputo. The NPV for Beira is US$70.9 million and the IRR is 15.9 percent. Overall, economic analysis covered approximately 77 percent of the total project costs at closure. The ICR notes that while the analysis for Maputo was not done, the Component 2 investments in drainage in Maputo are expected to have a similar return on investment to that in Beira. Methodology of the economic analysis at closure, Component 1. The economic analysis for Component 1 covers the infrastructure investments made through the MPG and the related TA in increasing municipal own- source revenue. The benefits comprise: (i) increased municipal own-source revenue due to the project, estimated by comparing participating and non-participating municipalities, and (ii) an estimated income increase for the beneficiaries of the infrastructure investments at three percent annually (a conservative estimate based on literature listed and quoted in the ICR), applied to the current urban average annual wage of US$1,000 of the employed population, which constitutes 45 percent of the total population (source for both numbers: Ulrich Lachler and Ian Walker. 2018. “Mozambique Jobs Diagnosticc: Volume 1 - Analytics.” World Bank, Washington, DC). The resulting number of beneficiaries is 312 thousand people. Methodology of the economic analysis at closure, Component 2. The economic analysis was conducted by analyzing the flood risk reduction benefits of the Beira drainage works and quantifying the benefits of improved mobility. The flood reduction is estimated using the dedicated flood simulation model FLORES based on a high- resolution data and rainfall statistics. The expected economic damage from flood and the number of affected people with and without the rehabilitated drainage was computed and compared. The improved mobility was assessed by quantifying the impact of road investments along the drainage on travel time. The commute to the city center where most of the jobs are was reduced by 3 percent due to the project. The time saved was monetized using average monthly wage and the number of beneficiaries. Page 11 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) 2. Operational/ Administrative efficiency. Several factors including exchange rate fluctuations, project delays, and high resettlement costs, negatively affected project efficiency. A devaluation of XDR to US$ during implementation reduced funds available by US$9 million. Because of the MZN to US$ devaluation, the net loss was US$6.6 million. Also, the project closing date was extended by two years and nine months. Partially, it is explained by additional activities introduced at the AF, there were also delays with design studies, procurement, and resettlement. Disruption was caused by Cyclon Idai and by COVID-19. Overall, the economic analysis for Component 2 (physical investments constituting 71 percent of total project costs at approval and 74 percent at closure) was sound, and the methodologies it used at approval and at closure comparable. Component 1 (capacity building constituting 29 percent of the total project cost at appraisal and 26 percent at closure) analysis was done at closure only. This analysis is also sound, thorough, and based on assumptions supported by literature (the TTL provided some clarifications in this respect at the IEG request, as some of the details were missing from the ICR). The benefits were calculated on the basis of project results, making the attribution to the project clear. This analysis can be used by other projects as a best practice example. Operationally, the project was able to adjust to exchange rate fluctuations, efficiently transferring the diminished (due to the exchange rate) funds from Nacala to Maputo, where the funds were sufficient for drainage investment (note that drainage works in Maputo were a project contingency at design). While project inefficiency was limited and the main factors for the closing extension of almost three years were outside of the Bank control, some were internal and comprised delays with procurement of works, resettlement, and the design-related studies. The project efficiency is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 51.00 Appraisal  18.10  Not Applicable 77.00 ICR Estimate  15.90  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Based on the high relevance of objectives, substantial efficacy, and substantial efficiency, the overall outcome rating is Satisfactory. a. Outcome Rating Satisfactory Page 12 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) 7. Risk to Development Outcome Institutional Risk. The project strengthened the capacity for local revenue enhancement, but there is moderate risk that this capacity may not be retained. The number of properties in municipal cadasters increased by over 500 percent due to the project, however, continued collection of property tax depends on the actions of the technical staff and can be affected by their high turnover. To mitigate that risk, the project invested in related training, and 45.5 percent of trained municipal staff indicated their readiness to transfer acquired skills to colleagues. Institutional outcome sustainability is also supported by active projects including the Urban Development and Decentralization Project (P163989), which builds on the reviewed project and is investing in the institutional capacity of the municipalities. Financial Risk. The Beira drainage system requires continuous operation of the flood gates and the cleaning of the canals from solid waste. Beira’s Autonomous Sanitation Services (SASB), the agency responsible for O&M, faces institutional and financial challenges. To mitigate this risk, the project invested in strengthening of the SASB’s capacity to manage Beira’s drainage system through targeted TA. However, the SASB is not able to finance O&M of the system sustainably, creating a substantial risk to project sustainability. The drainage investments in Maputo face a similarly risk. The project was designed to include activities that would mitigate financial risks. This was further supported by the ongoing US$120 million second phase project for drainage works and coastal protection (P171040). The activities included capacity building on O&M (such as workshops and training); support to a low-cost O&M design; support to the national and local government in designing a joint roadmap for O&M; policy dialogue regarding transferring the earmarked sanitation fees to municipalities; a possible introduction of a drainage fee as a share of municipal property tax; and support to explore private sector potential contribution to drainage. Government ownership. While the Bank has included extensive risk mitigation measures in the reviewed project, as well as in the second phase project, an important external factor outside of the Bank control is government commitment, which has varied in respect to the O&M, with the necessary decisions being delayed. Full government commitment to O&M will remain key to secure development outcomes in the long term. Exposure to natural disasters. The country is highly vulnerable to natural disasters, and considering their increasing severity, the resiliency of the infrastructure built by the project might become insufficient in the future. 8. Assessment of Bank Performance a. Quality-at-Entry The Project was aligned with the government’s strategy for strengthening climate resilience and supporting institutional strengthening and fiscal capacity of municipalities. It was also well-aligned with activities supported by the Bank and the development partners and already anticipated the inclusion of a likely Additional Financing (AF) from the Strategic Climate Fund under the Pilot Program for Climate Page 13 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) Resilience (SCF-PPCR) to scale up climate resilience investments in Beira. The lessons learned from previous urban operations in Mozambique and globally were taken into account at design stage. Component 1 incorporated lessons on importance of systemic institutional strengthening for urban management based on growing international evidence. Component 2 used the experience from previous projects with large engineering works by maximizing economies of scale through the procurement of a small number of large contracts. The PDO, as originally defined did not adequately capture the two-fold project activities. The PDOs were revised during restructuring. The implementation arrangements were adequate overall, although the ownership by the central level ministries that were to support the implementation of Component 1 could have been better, especially in facilitating more progress on resilient urban development. Key risks the project would face were adequately assessed, including capacity risks related to project management and implementation at national (MAEFP) and municipal levels, and largely adequate mitigation measures were identified. However, some risks associated with the decentralized implementation of small works by municipalities under the MPG were underestimated and required adjustments during implementation (see safeguard compliance). The Bank was closely engaged with the client and other development partners (in particular KfW) during the preparation. This was critical for mobilizing climate finance and co-financing for the AF for green infrastructure investments to improve natural drainage in urban areas. These were innovative activities for Africa at the time. Quality-at-Entry Rating Satisfactory b. Quality of supervision During supervision, the Bank reported issues with project implementation and reflected them in performance ratings. The Bank supported the MAEFP and AIAS in improving their compliance with financial management (FM), procurement management (PM), and safeguards. Supervision of safeguards, procurement, urban development, civil engineering, and financial management was supported by specialists. Implementation missions with technical visits of construction sites were conducted regularly for both components. At the same time, the frequent changes in task team leaders (TTLs) were considered to be somewhat disruptive by the PIU staff, as different TTLs expressed different priorities, which especially affected Component 1. The works under the project experienced recurring issues of non-compliance with occupational health and safety (OHS) and community health and safety plans. There were four fatalities: a worker was hit by a third-party vehicle while moving from one site to another; a 9-year old child drowned in the retention basin construction site during off-work hours; and two pedestrian died in traffic accidents involving contractor’s vehicles. These accidents were assessed as caused by contractor’s non-compliance with community and worker safety practices and with national employment and labor law (such as weekly rest and maximum number of work hours). Both the client and the Bank adequately reported the accidents and managed response using the Bank’s Environmental and Social Incident Response Toolkit (ESIRT). The Bank Page 14 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) required an OHS audit, and the recommended corrective actions were incorporated in the OHS, Labor Management and Traffic Safety Plans. In relation to COVID-19, sanitary precaution measures were adopted early on, producing satisfactory results. Since the implementation challenges were addressed by the team, the quality of supervision is rated Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project's RF after the 2017 restructuring was adequately linked to the project TOC, as described in the ICR, as well as to the PDO and to the logic of project interventions in the PAD. The indicators reflected all outcomes of the PDO. The RF indicators were at the right level and presented a mix of output and outcome indicators. The intermediate indicators reflected all project activities in the context of project expected outcomes. All indicators were quantitative and time-bound, and all of them had baselines and targets. However, the RF underwent changes over the project life, and it is clear that the original RF design was far from optimal. Also, only basic sex-disaggregated indicators were applied. Better integration of gender aspects in the monitoring and evaluation framework, especially data collection, could have strengthened the project’s ability to address gender inequality. b. M&E Implementation The baseline data collection was fully carried out. All of the indicators in the RF were quantitative, and actual achievement was reported in the ICR. However, most of the indicators defined at approval were adjusted or replaced during implementation to reflect changes in project activities, improve the adequacy of the RF, and incorporate new corporate results indicators. These adjustments improved the alignment between project activities and expected results. The data in the RF are reliable and good quality. However, there are a few data point requiring more explanation that the RF provides. In particular, in one case, a significant increase in targets at restructuring lacks an explanation; in two cases, the methodology for calculating the target differs from the one for estimating the actual result; and in one case it was not clear how the outcome was calculated. The ICR notes that the two PIUs closely monitored and timely reported to the WB the achievement of the targets; and that the M&E mid-term and final reports were transparent and of sufficient quality. Page 15 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) c. M&E Utilization M&E reports and feedback were continuously shared with the World Bank team and incorporated in the Implementation Status and Results reports (ISRs). Also, M&E was mainstreamed into the institutional architecture of the implementing partners, which shows its sufficient quality. In addition, M&E was used to change the implementation direction at the MTR. Specifically, at the MTR, Component 1 was downgraded to moderately satisfactory based on the RF results, which lead to the restructuring of this component in November 2017. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The implementation of the Safeguards Policies included the preparation of Environmental and Social Management Frameworks (ESMFs), Resettlement Policy Frameworks (RPFs), specific Environmental and Social Impact Assessments (ESIAs), Environmental and Social Management Plans (ESMPs), and Resettlement Action Plans (RAPs). The project was classified as Environmental Category “B” at appraisal. There was no change in environmental category during restructuring. The initial assessment triggered OP 4.01 (Environmental Assessment) and OP 4.12 (Involuntary Resettlement). The 2017 Restructuring triggered OP 4.04 (Natural Habitats) and OP 4.36 (Forests), for the green infrastructure works in Beira that included interventions in mangrove areas. Minor social and environmental issues from small infrastructure works under Municipal Performance Grants (MPGs) were addressed. The need to improve the screening and supervision procedures was identified and established during implementation. The problems were addressed through: hiring of a full-time safeguard specialist by the PIU, improvement of the MPG operations manual, and more frequent monitoring and supervision visits. As a result, safeguards rating improved from Moderately Unsatisfactory to Moderately Satisfactory. Minor resettlement impact was addressed by screening for potential resettlement gaps in all MPG subprojects and creating RAPs for two cases with gaps. Environmental issues were managed effectively. There were no significant adverse environmental impacts from the drainage works in Maputo, and their implementation had only minor deviation from the ESMP. The Beira drainage works posed a series of environmental challenges, which were effectively managed. They had impact on protected mangrove ecosystems, requiring the amendment of Component 2 ESMF (QPGAS) to include OP 4.04 on Natural Habitats and OP 4.36 on Forests. Serious issues of non-compliance with Operational Health and Safety (OHS) and community health and safety plans were experienced during construction works in Beira, leading to four fatalities: one worker and three community members. The reason for these tragic accidents was contractor’s non- compliance with community and worker safety practices and with national employment and labor law (such as weekly rest and maximum number of work hours). The accidents were reported, and the response Page 16 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) managed based on the Bank’s Environmental and Social Incident Response Toolkit (ESIRT). An OHS audit produced recommendations, and corrective actions were incorporated in the OHS and Labor Management and Traffic Safety Plans. Resettlement, temporary relocation, and compensation of Project Affected People (PAPs) were managed well in Beira. The resettlement instruments were prepared. The RAP for Beira drainage works was implemented in accordance with OP 4.12. The project funded construction of 21 new houses and 81 latrines (the PAD mentioned that between 50 and 80 existing structures might be affected by project financed works (page 46)). The loss of 498 trees was compensated. Approximately 610 PAPs received compensation for the loss of crops. By project closing, the grievance redress mechanism had no complaint regarding unpaid compensation or compensation value. Resettlement under the Maputo drainage works was problematic and faced challenges in its implementation. At the time of the RAP preparation, it turned out that the PAPs (the total of 18 people) had already received compensation from a private developer and resettled. The task team had problems with obtaining data on the status of the land and the resettlement procedures. The audit found that the monetary compensation was not in compliance with the Bank's safeguards policies or the applicable national legislation. On the basis of the audit, remedial actions were taken to ensure compliance with OP 4.12 before the start of works. After project closure, the Bank followed a post-closing action plan to address safeguards issues in Beira where the works co-financed by KfW went beyond the closing date of the project. The issues were: temporary relocation and reintegration of vendors into a new market, a labor audit (including OHS aspects), and an environmental audit related to the restoration of the mangroves. At the time of the ICR completion, the implementation of corrective measures was advanced but not complete. The TTL informed IEG that after project closure, the safeguards were addressed as planned albeit with some delay. The current situation is as follows: the vendors are due to be fully reintegrated in August - September 2021 (the infrastructure was handed over to the municipality in July 2021); the labor audit has been completed, with some minor outstanding questions on salaries; and the environmental audit has been completed. b. Fiduciary Compliance Financial management (FM): The FM requirements for audits and financial reporting were adequate, particularly after adjustments to supervision following the MTR. The MTR identified weaknesses in financial planning and analysis of Component 1. Increased Bank support and supervision, including the introduction of quarterly reporting on financial management in 2017, improved compliance, which was reflected in the ISRs by the end of November 2017. No major issues were identified for Component 2. Procurement: Challenges with procurement caused major delays in both components. For Component 1, the Independent Procurement Audit of the MPG found that most municipalities, despite guidance and support from the PIU, did not comply with the procurement procedures in the MPG Operations Manual. Shortcomings were also identified in procurement processes of the PIU. Compliance improved after the Page 17 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) PIU developed an action plan to address the procurement issues. For Component 2, AIAS complied with the Bank’s procurement requisites but had delays and cost overruns in the procurement and contract management of engineering services caused by engineering design deficiencies and lengthy contract price adjustments. After the AIAS team received training and TA on contract management, the situation improved. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons 1. When local capacity needs are assessed and reflected in the project schedule at start, both efficacy and efficiency are positively affected. In relation to the MPGs which were successfully utilized in the project as mechanisms to incentivize own-source revenue generation, sufficient implementation capacity at local level was critical, and creating it required more time than expected. The time was needed for the preparation of technical design and safeguards instruments, procurement of contractors, and supervision of works for the MPG subprojects. 2. On-the job training can be critical for project longer-term results and inclusion of technical experts in teams to provide continuous on-the-job training and advice can be a solution. Project’s results from municipal institutional strengthening were uneven and slow until on-the-job support was extended in response to the request of the beneficiaries. This led to a significant improvement. As a result, 78 percent of the participating staff assessed training as excellent and 45.5 percent said they were ready to transfer the acquired skills to other staff members. 3. Support to national agencies needs to be clearly aligned with project outcomes, and sufficient incentives need to be provided to foster necessary reforms. There are three lessons learned in this respect. First, there needs to be a mechanism at national level to incentivize the implementation of key reforms. In the project, the absence of such mechanism led to delays and cancellation of some activities, including the development of a municipal financial management Page 18 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) system. Second, disbursement-linked indicators for Investment Project Financing may be useful for similar projects to incentivize reforms that are difficult to implement. In the project, expected results from the national level support for policy and institutional reform were not fully defined and not reflected in the RF and therefore could not be monitored. Third, a PIU within one of the counterpart line ministries might not be the best option because it can jeopardize inter-ministry coordination. In the project, the PIU was within the MAEFP, but an independent PIU could have supported urban development and decentralization objectives, which are by definition cross-sectoral, and better at balancing the interests of all sectors involved. 4. It is important that infrastructure construction works in densely populated areas are performed under special OHS and community protection measures including traffic safety plans. OHS-related capacity building to implementing agencies and contractors is needed to ensure safety. This would include training and certification of drivers and operators, remote speed control systems, and detailed monitoring, in addition to specific contract covenants (for engineers and contractors) in case of non-compliance with the community safety protection measures. In Beira, four fatalities in the community were caused by accidents involving project’s construction vehicles. 5. Post-construction technical support is critical for sustainability of project outcomes in countries with low institutional capacity. The project supported client’s capacity to operate and maintain improved drainage system in Beira during its first year of operation (commissioning phase). This included the allocation of technical consultants to oversee and guide Beira’s Autonomous Sanitation Services (SASB) in the daily operation of the system over a period of 12 months, as well as extensive training of the SASB’s personnel, preparation and validation of operation manuals and protocols, acquisition of additional spare parts, and simulation exercises for emergency response during critical events. It significantly contributed to adequate performance of the drainage system to Cyclone Idai and other extreme events, minimizing flooding in the areas drained by the system. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is of a good quality and delivers relevant and valid information, substantial evidence, and a thorough analysis. The ratings are fully supported by the narrative. There are minor shortcomings in the ICR’s description of the project’s economic analysis as relates to Component 1: it is not clear how some of the assumptions were made. Another minor shortcoming is that some of the results framework indicators are not clearly described and two of them are presented as providing data which are not comparable between the target and the actual outcome. This unclarity, however, does not affect the validity of the ICR's analysis or the ratings. The TTL clarified both the assumptions in Component 1 efficiency analysis and the unclarities related to the results framework, showing that targets and result are, in fact, comparable. To note, the length of the ICR is 29 pages, which is almost twice the recommended 15-page length of the main text. Page 19 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ- Cities & Climate Change (P123201) a. Quality of ICR Rating Substantial Page 20 of 20