Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005657 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-82720 ON A LOAN IN THE AMOUNT OF BRL 596.95 MILLION (US$ 300 MILLION EQUIVALENT) TO THE State of Sao Paulo FOR THE Sao Paulo State Sustainable Transport Project 30 September 2021 Transport Global Practice Latin America And Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 1, 2013) Currency Unit = Brazilian Real BRL 1.98985 = US$1 US$0.50255 = BRL 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Paloma Anos Casero Regional Director: Franz R. Drees-Gross Practice Manager: Nicolas Peltier-Thiberge Task Team Leader(s): Georges Bianco Darido ICR Main Contributor: Nicolas De Leon De Maria ABBREVIATIONS AND ACRONYMS ETESB Environmental Agency (Companhia Ambiental do Estado de São Paulo) CdL Comissão de Licitação CONSEMA State of Sao Paulo Environment Council CPF Contry Partnership Framework CPS Country Partnership Strategy CREMA Performance Based contracts for rehabilitation and road maintenance DER Road Agency, Sao Paulo State Government (Departamento de Estradas de Rodagem) DRM Disaster Risk Management EA Environmental Assessment EEZ - ZEE Economic and Ecological Zoning ESIA Environmental and Social Impact Assessment FM Financial Management IG Geologic Institute (Instituto Geológico) IPPF Indigenous Peoples Planning Framework ISDS Investor State Dispute Settlement MIGA Multilateral Investment Guarantee Agency NHSFO Non-Honoring of Sovereign Financial Obligations Product OP/BP Operational Policies/Bank Policies PDLT Plano de Desenvolvimento da Logística e do Transporte PDN Programa de Prevenção de Desastres Naturais e Redução de Riscos Geológicos PPA Plano Pluri Annual - multi annual plan PPP Private Public Partnership RAP Relatorio Ambiental Preliminar - Preliminary Environmental Assessment RPF Ressettlement Policy Framework SLT Secretariat of Logistics and Transport SMA Environment Secretariat SoSP State of Sao Paulo SP Sao Paulo SPDR Planning and Regional Development Secretariat UBA Unidad Basica de Atendimento UCPR Unidade de Coordenação de Projetos WBG World Bank Group TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 10 II. OUTCOME .................................................................................................................... 12 A. RELEVANCE OF PDOs ............................................................................................................ 12 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 20 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25 C. BANK PERFORMANCE ........................................................................................................... 27 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 42 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 43 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 44 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 46 The World Bank Sao Paulo State Sustainable Transport Project (P127723) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P127723 Sao Paulo State Sustainable Transport Project Country Financing Instrument Brazil Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Secretariat of Transport and Logistics, Road Agency (DER- State of Sao Paulo SP) Project Development Objective (PDO) Original PDO Contribute to the improvement of the State’s transport and logistics efficiency and safety while enhancing the Borrower’s capacity in environmental and disaster risk management. Page 1 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 300,000,000 265,092,779 265,092,779 IBRD-82720 Total 300,000,000 265,092,779 265,092,779 Non-World Bank Financing 0 0 0 Borrower/Recipient 129,000,000 129,000,000 46,500,000 MIGA Guarantee 300,000,000 300,000,000 300,000,000 Total 429,000,000 429,000,000 346,500,000 Total Project Cost 729,000,000 694,092,779 611,592,779 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 14-Jun-2013 18-Dec-2013 28-Nov-2016 31-Mar-2019 30-Mar-2021 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 24-Feb-2015 162.73 Change in Results Framework Change in Components and Cost Change in Financing Plan 05-Nov-2018 215.37 Change in Results Framework Change in Loan Closing Date(s) Change in Disbursements Arrangements Change in Implementation Schedule 16-Dec-2020 256.67 Change in Results Framework Change in Loan Closing Date(s) Change in Implementation Schedule 18-Dec-2020 256.67 Page 2 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 03-Sep-2013 Satisfactory Satisfactory 0 02 22-Mar-2014 Satisfactory Satisfactory 38.35 03 27-Oct-2014 Satisfactory Satisfactory 114.02 04 28-May-2015 Satisfactory Satisfactory 178.68 05 22-Jan-2016 Moderately Satisfactory Satisfactory 189.06 06 22-Jun-2016 Moderately Satisfactory Satisfactory 198.95 Moderately 07 22-Feb-2017 Moderately Unsatisfactory 199.22 Unsatisfactory Moderately 08 22-Nov-2017 Moderately Unsatisfactory 199.74 Unsatisfactory Moderately 09 23-May-2018 Moderately Unsatisfactory 199.78 Unsatisfactory 10 13-Jun-2018 Moderately Satisfactory Moderately Satisfactory 199.78 11 04-Mar-2019 Moderately Satisfactory Moderately Satisfactory 215.42 12 17-Sep-2019 Moderately Satisfactory Moderately Satisfactory 247.63 Moderately 13 21-May-2020 Moderately Unsatisfactory 247.70 Unsatisfactory 14 23-Dec-2020 Moderately Satisfactory Moderately Satisfactory 256.67 15 13-Apr-2021 Moderately Satisfactory Moderately Satisfactory 256.67 Page 3 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 8 Sub-National Government 8 Other Public Administration Transportation 91 Rural and Inter-Urban Roads 91 Other Transportation Water, Sanitation and Waste Management 1 Other Water Supply, Sanitation and Waste 1 Management Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 46 Trade 46 Trade Facilitation 46 Private Sector Development 25 Jobs 15 Job Creation 15 Public Private Partnerships 10 Finance 1 Finance for Development 1 Disaster Risk Finance 1 Page 4 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Urban and Rural Development 34 Urban Development 15 Urban Infrastructure and Service Delivery 15 Rural Development 16 Rural Infrastructure and service delivery 15 Land Administration and Management 1 Disaster Risk Management 3 Disaster Response and Recovery 1 Disaster Risk Reduction 1 Disaster Preparedness 1 Environment and Natural Resource Management 3 Environmental policies and institutions 3 ADM STAFF Role At Approval At ICR Regional Vice President: Hasan A. Tuluy Carlos Felipe Jaramillo Country Director: Deborah L. Wetzel Paloma Anos Casero Director: Ede Jorge Ijjasz-Vasquez Franz R. Drees-Gross Practice Manager: Aurelio Menendez Nicolas Peltier-Thiberge Task Team Leader(s): Eric R. Lancelot Georges Bianco Darido ICR Contributing Author: Nicolas De Leon De Maria Page 5 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At the time of the appraisal, the State of Sao Paulo (SoSP) was one of Brazil’s most advanced States: with a GDP of US$500 billion (2009)1, it accounted for 33% of Brazil’s total GDP; its population of 41 million (20% of the Brazilian population) had one of Brazil’s highest GDP per capita (US$16,200 in 2010, more than 50% above the national average), a wealth which was growing steadily (5% per year in average over the past years); its Human Development Index (HDI), at 0.833, was the third highest among the States of Brazil. Yet, the transport system still faced a number of challenges to its development agenda, some of which were inherited from its past development patterns: unbalanced towards the road mode and concentrated on a limited number of radial road corridors to the largest economic poles, resulting in capacity and road safety issues and rendering the State’s transport system vulnerable to disruptions due to natural disasters with potential impacts on the economy. 2. The State had the goal to position itself as a leader among the most dynamic emerging regions, and this ambition was embedded in the 4-year development plan (2012-2015 Plano Pluri Anual, PPA). The State’s strategy to support the implementation of the PPA and meeting its commitments related to the transport and climate change agendas relied on three important pillars: • Enhancing competitiveness by improving transport efficiency and safety, in particular, supporting the design and implementation of the State’s Logistics and Transport Development Plan (PDLT, Plano de Desenvolvimento da Logística e do Transporte), an updated version of the first long-term transport development strategy in Brazil, formulated by the State in 2000. The PDLT aimed to foster quality service in transport, mobility of people and goods and improved logistics and competitiveness of the regional and national economies, while promoting sustainable development. • Improving sustainability through better environmental management, in particular, supporting implementation of the State’s strategic environmental assessment (Avaliação Ambiental Estratégica, AAE) and the economic and ecologic zoning (EEZ) respectively to evaluate environmental and socio-economic impacts of economic activities and regulate productive investments, so that growth will remain sustainable. • Creating a multi-sectoral foundation for resilience, in particular, supporting the implementation of the Disasters Risk Management Program for Prevention of Natural Disasters and Reduction of Geological Risks (Programa de Prevenção de Desastres Naturais e Redução de Riscos Geológicos, PDN). This innovative policy aimed to increase the State’s resilience to natural hazard risks in response to increasing damages and costs resulting from natural disasters. 3. On the competitiveness side, while the SoSP transport system was one of the country’s most modern and diverse, its transport and logistics capacity remained largely insufficient to match current and future needs and was recognized as a major bottleneck for regional and national logistics. With an estimated average yearly death toll of 2,500 in the State, road safety was a matter of growing concern. Given the State’s weight in the national economy and the concentration of regional logistics flows on its transport networks, these challenges in transport were 1 The State economy mostly relied on services and industry with respectively 44% and 54% of the GDP (2009) Page 6 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) challenges for the country as a whole2, and thus improving the efficiency and safety of the transport sector was critical to ensuring that the sector adequately advances the State’s and the country’s development agendas. 4. To address the above challenges, the SoSP sought the Bank’s support for its large investment programs in transport and to develop cutting edge practices under the above key sector policies . Given the size of the investment program and resulting financing needs, the SoSP was also proactively looking for ways to increase the participation of the private sector, including through concessions and PPP, as well as financing from international private lenders. In that context, the SoSP and the Bank explored the possibility of using MIGA’s guarantee as a credit enhancement instrument which would allow additional private funding in the amount of US$300 million to be leveraged at favorable conditions. This additional financing from international private lenders allowed the scope of activities under the Project’s physical component to be scaled-up and accommodated additional road rehabilitation and upgrading works. 5. Based on these objectives, the proposed Project was designed to contribute to two of the four strategic pillars of the World Bank Group’s Country Partnership Strategy (CPS) for 2012-2015 discussed by the Executive Directors on November 1, 2011 (Report No. 63731-BR), specifically under its 3rd and 4th pillars: (i) promote regional economic development through improved policies, strategic infrastructure investments, and support for the private sector in frontier areas; and (ii) further improve sustainable natural resource management and enhance climatic resilience while contributing to local economic development and helping to meet rising global food demand. 2 Logistics costs in Brazil were estimated between 15 and 18% of GDP, higher than other BRICS and twice as much as OECD countries; death toll on Brazilian road networks exceeds 40,000 yearly, an extremely high figure compared to OECD countries. Page 7 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Theory of Change (Results Chain) Project Development Objectives (PDOs) 6. As per the Legal Agreement, “the objective of the Project is to contribute to the improvement of the Borrower’s transport and logistics efficiency and safety while enhancing the Borrower’s capacity in environmental and disaster risk management. Key Expected Outcomes and Outcome Indicators 7. The key outcome indicators for the PDOs where the following: • PDO 1 - Improve the State’s transport and logistics efficiency and safety: o Indicator a: Roads in good and fair conditions as a share of total classified roads with IRI <3; o Indicator b: Increased intermodality measured by increased proportion of exported biofuel transported by waterway on the Tiete river; o Indicator c: Improved transport management measured by the increased proportion of the State road network managed by performance; o Indicator d: Improved road safety measured by the reduction in fatalities on the 100 most critical spots of the State road network. • PDO 2 - Enhance the Borrower’s capacity in environmental management • PDO 3 - Enhance the Borrower’s capacity in disaster risk management: Page 8 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) o Indicator e: Improved transport management proactively coping with disaster through the establishment of contingency plans in the DER’s local representations covered by the Project (UBA) Components 8. Component 1: Improving transport and logistics efficiency and safety (estimated costs US$ 394.25 million, of which US$ 275.45 million from IBRD financing). Component 1.1: Rehabilitating and upgrading the State’s transport networks through support to the Road Agency (DER) road rehabilitation and upgrading program. Restore and upgrade the State’s transport networks to improve connectivity, reduce logistics costs, and improve road transport safety through the rehabilitation and improvement of the existing transport infrastructures. Component 1.2: Sustainable transport planning and management. Improvement of the State capacity to plan and manage the sector through the carrying out of studies and the acquisition of goods to the benefit of the Secretariat of Logistics and Transport’s (SLT) 9. Component 2: Strengthening sustainable environmental and land use planning and territorial management capacity (estimated costs US$ 18 million, of which US$ 12.6 million from IBRD financing). Component 2.1: Supporting sustainable land use planning and territorial management. Improvement of the State’s capacity to plan and manage sustainable land use and territorial development in an integrated fashion through the carrying out of studies and the acquisition of goods. Component 2.2: Improving environmental enforcement and environment quality monitoring. Improvement of the State’s capacity to manage and monitor the environment through the carrying out of studies and small works and the acquisition of goods. Component 2.3: Supporting the modernization of the Environmental Licensing System. Improvement of the State’s capacity to efficiently process environmental licenses and environmental strategic assessment services through the carrying out of studies and the acquisition of goods 10. Component 3: Increasing State’s resilience to natural disasters (estimated costs US$ 16 million, of which $US 11.2 million from IBRD financing). Component 3.1: Mainstreaming disaster risk management in the transport sector. Mainstream disaster risk management in transport planning and work execution through carrying out studies, small works, and the acquisition of goods to the benefit of the Geologic Institute (IG). Component 3.2: Enhancing disaster risk management policy and institutional capacity). Strengthening the State’s capacity for disaster risk management through studies, capacity building activities, and the acquisition of goods to upgrade and support the implementation of the State’s disaster risk management program (Programa Estadual de Prevenção de Desastres Naturais e de Redução de Riscos Geológicos - PDN) to the benefit of the IG. Page 9 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 11. The PDOs and the target for the PDO 1 indicator measuring the condition of the roads and for the PDO 3 indicator measuring contingency plans in a sample of the road agency’s local representations remained unchanged through the Project’s lifetime. The end-target completion dates for the PDO indicators were extended according to the new loan closing dates proposed during the second and third Project restructurings of December 31, 2020 and March 30, 2021 respectively. Revised PDO Indicators 12. The PDO level results indicators were adjusted during the second and third restructurings of the Project approved on November 5, 2018 and December 16, 2020, respectively. Table 1: Changes of PDO Indicators during implementation Outcome 1: improve the State’s transport and logistics efficiency and safety PDO Indicators at 2nd Restructuring PDO Indicators at Appraisal (11/2018) PDO Indicators at 3rd Restructuring (12/2020) Roads in good and fair condition as a share of Same Same total classified roads (from 34% to 48%) Share of Project roads in good and fair condition (%) - - Comment: This indicator was added as it was directly attributable to the project. Increased intermodality measured by Percentage increase of export products Change in the average Vehicle Operating increased proportion of exported biofuel transported on the Tiete river (from 0% to Costs on Project roads (from 0% to 5%) transported by waterway in the State on the 12%) Comment: The original indicator could not be Tiete river (from 0% to 30%) Comments: The original indicator became directly reported (only estimated) until the obsolete as focus on biofuel (mainly new bridge was open to larger vessels and transported by waterway) was no longer vehicles and was affected by many external relevant since the federal government cut factors in export volumes that could not be subsidies on ethanol and the production and attributable to the project. The revised transportation of biofuel dropped to almost indicator represented a better indicator of zero. The revised indicator measured any the transport and logistics efficiency export products. Another important external outcomes of the project. cause was the drought in 2016 that resulted in a drastic reduction of Tiete waterflow and the SoSP banning waterborne transport on the river to prioritize energy production. Improved transport management measured Dropped. by the increased proportion of State road Comments: Dropped for attribution issues. network managed by performance (from 32% Except for the SP-463 pilot performance- to 50%) based road rehabilitation contract, the - Project had no activities that could result in such outcome. The outcome could have been relevant, had the Project had a strong focus on PPP and concessions. Improved road safety measured by the Number of fatalities on the 100 most critical Same reduction in fatalities on the 100 most critical spots of the State road network (from 124 to spots of the State road network (from 124 to 62) 62) Comments: Refined as a more specific indicator. Page 10 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Outcome 2: enhance the Borrower’s capacity in environmental management PDO Indicators at Appraisal PDO Indicators at 2nd Restructuring PDO Indicators at Closing Number of days required for SMA/CETESB to Same approve Environmental Preliminary Reports (Relatório Ambiental Preliminar – RAP) for roadworks (from 156 to 117 days or 39 days).ments: The intermediate result indicator “Improved environmental control - measured by the reduction of average delay for licensing transport works” (from 360 to 330 days or 30 days) for Component 2 (Environment, institutional strengthening) was refined as a more specific indicator and moved up as a PDO 3 indicator with a slightly increased target. Outcome 3: enhance the Borrower’s capacity in disaster risk management PDO Indicators at Appraisal PDO Indicators at 2nd Restructuring PDO Indicators at Closing Improved transport management proactively Number of contingency plans established for Same coping with disaster through the selected UBAs (3). establishment of contingency plans in a Comments: Refined as a more specific sample of DER’s local representations for indicator. traffic management (pilot UBA) (target: Contingency plans adopted in 3 UBAs (local representation of road agencies) which territories have been mapped, DER, Road Agency) Revised Components 13. No changes were made to Project components during implementation. Other Changes 14. All other changes made during the three restructurings are summarized in Table 2 below. More information about the changes in the Results Framework can be found in Table 9 in Annex 6. Table 2: Other changes during implementation First Restructuring - 24 February 2015 Proposed Change Explanation Change to Financing The Project financing increased from US$429 million to US$729 million with the availability of co-financing Plan from a private loan of US$300 million from Banco Santander covered by MIGA under its Non-Honoring of Sovereign Financial Obligations (NHSFO) guarantee. It was allocated to subcomponent 1.1 (Rehabilitation and Upgrading the State’s transport networks) to finance additional selected roads from the State Program. The IBRD financing for the subcomponent remained unchanged. Change in Disbursement The IBRD loan’s disbursement schedule was revised to take into account the Government’s plans to use first Estimates the proceeds from the international private banks to finance the Project’s works with an aim at optimizing the Project financing costs. Change to Components Project activities were not changed, but the total cost of Component 1.1 was revised from US$394.25 and Cost million to US$694.25 million due to the availability of co-financing from Banco Santander (see the change to Financing Plan). It increased the scope and targets of the Component 1.1 (see the changes in Result Framework). In addition, one of the bridge reconstruction on the Tiete river was replaced by another similar bridge reconstruction on the same Tiete river but in a different location, following the same wider clearance standards upstream. The cost of the other two components remained unchanged. Page 11 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Appraisal Summary Based on the calculation for the revised Component 1.1, over 20 years, the net present value (NPV), at a Change in Economic and 12% discount rate, and the related internal rate of return (IRR) of the investments in this road rehabilitation Financial Analysis and maintenance subcomponent were respectively estimated as R$1,229 million and 38.7%. Appraisal Summary The same rehabilitation and construction standards were to be applied to the additional road sections under Change in Technical subcomponent 1.1. The reconstruction of the bridges involved the deconstruction of actual infrastructures Analysis and their replacement by bridges with larger main spans. Designs of the new bridges were reviewed and agreed with the Bank. Appraisal Summary The Environmental and Social Impact Assessment of the Project was updated by the State of Sao Paulo to Change in Social Analysis take into account the enlarged scope of the works under subcomponent 1.1. No new safeguard policy was triggered, and no major adverse impact was expected. Appraisal Summary The Environmental and Social Impact Assessment of the Project was updated by the State of Sao Paulo to Change in take into account the increased scope of the works under subcomponent 1.1. No new safeguard policy was Environmental Analysis triggered, and no major adverse impact was expected. Appraisal Summary Because the US$300 million co-financing from a private loan from Banco Santander S.A. was added to the Change in Risk Analysis project, it exposed the Project to the risk that the co-financier could abandon the project. The risk to the PDO was rated Moderate. Second and Third Restructurings - 5 November 2018 and 16 December 2020 Proposed Change Explanation Change to Loan Closing The loan closing date was extended by 21 months, from March 31, 2019 to December 31, 2020, and then Date from December 31, 2020 to March 30, 2021. Change in Disbursement The IBRD loan’s disbursement estimates were made until the extended closing dates. Estimates Change to Implementation schedule was revised to reflect the loan closing date extension. Implementation Schedule Rationale for Changes and Their Implication on the Original Theory of Change 15. The primary reasons for most changes were to (i) accommodate delays in implementation, both on the physical and institutional strengthening components, (ii) improve the suitability of the Result Framework to fully capture the Project’s outcomes and ambitions, and (iii) expand the list of highways eligible to Project financing including new roads due to the introduction of the US$300 million private co-financing during the first restructuring and the State of Sao Paulo’s priorities changing during the second restructuring3. However, all these changes did not have any impact on the intended Theory of Change since the PDO objectives and their ambition remained the same. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 16. The PDO remains highly relevant to the World Bank’s Brazil Country Partnership Framework (CPF) for FY18-23 (Report No. 113259-BR). The CPF objectives to provide more inclusive and sustainable urban services, and to support the achievement of Brazil’s Nationally Determined Contributions (NDC) with a particular focus on land use tie in with 3In September 2016, the State of São Paulo sought new roads to enter within the scope of the financing, while others would exit. See No. 675/2016 / GS-ACR on 9/20/2016 by the Secretary of Finance in the Sao Paulo State Government. Page 12 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) CPF Focus Area 3 “Inclusive and sustainable development”. By this CPF Focus Area 3, the Bank envisions to support government at all levels to improve the quality, efficiency, and sustainability of urban and inter-urban transport services, as well as the implementation of strategies to build resilience in urban areas, especially in poor high-risk areas. The PDOs are directly relevant for these objectives. Additionally, by the CPF objective to mobilize greater investment in infrastructure to improve services, including through PPPs tied to Focus Area 2 “Private sector investment and productivity growth” states that the WBG will invest and leverage private sector financing to improve transport infrastructure to reduce logistics costs and improve national level connectivity. The objectives of inclusive and sustainable development, and of improved infrastructure services elements are directly present as Project activities (rehabilitation of infrastructure paired with enhanced environmental management) and/or directly linked to the Project’s Results Framework (Annex 1). This strongly supports the relevance of the Project’s PDO. 17. The Project’s PDO also remains directly relevant to the State’s Strategic Objectives of “Improving urban quality of life with adequate mobility” and “Sustainable development preserving the environment and protecting the population from natural disasters” spelled out in the 2020-2023 Plano Pluri Anual (PPA)4. By these objectives, the State aims to expand access to urban services and to a quality transport network, as well as to achieve more integrated environmental management promoting sustainable development and expanding investments in social protection against natural disasters. These strategic objectives are perfectly encompassed in the PDO, the three components of the Project and expected outcomes. In addition, the Project also remains relevant to the PPA’s strategic objective of economic development promoting investments and innovation, which aims to create jobs and income opportunities by attracting investments in productivity, expanding and integrating the logistic infrastructure according to the demands of the economy, and expanding actions aimed at regional development that reduce inequalities and provide greater access to public goods and services throughout the territory. 18. Since there are no shortcomings in the relevance to the current Bank CPF, and the operation provides clear evidence of the alignment of the PDO to the CPF, the overall relevance of the PDO is therefore rated “High”. 4 http://www.ppa.sp.gov.br/ppa2023/ProjetoLei Page 13 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome PDO objective 1: Improve the State’s transport and logistics efficiency and safety 19. The Project was expected to improve the State’s transport and logistics efficiency and safety by rehabilitating 690km of roads and reconstructing one bridge on Tiete river under Component 1. The Project improved the State’s transport and logistics efficiency and safety to a substantial extent, as explained below. At the time of closing, the Project had delivered 680.13km of rehabilitated roads with improved designs, construction of third lanes, duplications, slope protection and improvement of intersections, as well as improved road safety features. This also included the construction of the new bridge over the Tiete river with a larger main span to allow passage of larger vessels and vehicles. 20. Reduction in average Vehicle Operating Costs on Project Roads. This is an optimal indicator for measuring the transport and logistics efficiency outcomes resulting from the Project investments affecting road conditions (pavement roughness), volume of traffic, type of vehicles, and the types of improvements on the Project roads. A baseline of zero was considered, since the goal was to measure the full impact of the cost reduction in Project roads before and after the execution of the works, and data was collected yearly following the HDM-4 RUC model from the World Bank’s Collaboration for Development, and the target was to achieve a 5% reduction. At Project closing, an aggregated 5.54% Vehicle Operating Cost reduction was achieved, largely due to the improved pavement and design conditions of the Project road network. In several Project roads, the value achieved about and well over 10%, reaching a maximum of 22% cost reduction in one of the segments. 21. Share of Project roads in good and fair condition. For the purpose of the ICR, a sub-indicator was added for the share of Project roads in good and fair condition considering only the 690 km network of Project roads, of which 680.13 km were completed. This indicator was calculated in similar fashion to the parent indicator but is directly attributable to the Project because it includes only the Project roads. A thorough analysis of the IRI results was carried out considering a 10% decrease in utility from year to year and the results overachieved the target significantly. The share of Project roads with IRI < 3 (=good) in the 2015-2021 period went from 42% to 74%. This supports the proposed substantial contribution of the Project to improving road transport conditions. Page 14 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) 22. Experience with Performance-Based road maintenance and rehabilitation contracts (also known as CREMA). The Project included one pilot CREMA contract, covering about 100km of roads, which was a new modality for Sao Paulo. The contract had a total value of BRL 120.8M and was implemented for 60 months (May 2015 to December 2020). The values for the total reductions and retentions registered in the last month of measurement were of BRL 81,895 and BRL 7 million respectively. The pilot has resulted in an improved sustainability of the roads under the contract, with an enhanced quality of the road maintenance and at a lower cost. It has also promoted a better use of public resources through the responsibility acquired by the contractor for positive results under the pressure of receiving penalties, and it has facilitated obtaining financial resources for the maintenance of highways5. Some pictures of the most recent maintenance work in 2019 and 2020 can be found in Annex 6 (images 6 and 7). 23. Roads in good and fair condition as a share of total classified roads in the State (previously a mandatory Core Sector Indicator). At appraisal, the baseline of the road network in good or fair condition was 34 percent of total State classified road network of 13,000 km (including originally 380km of Project roads), with a target to increase it to 48 percent. Improvements in road standards under State financing included rehabilitation, construction of third lanes, duplications, slope protection and improvement of intersections, as well as improved road safety features based on the recommendations of the International Road Assessment program (iRAP) pilot survey. Additionally, for the roads to be considered as successfully improved, an International Roughness Index (IRI) of less than 3 (=good) had to have been achieved. Before the Project, the selected roads had an average IRI of about 4.3 (=fair), with some segments nearing a value of 8 (=poor). At Project closing in March 2021, the share of classified roads in the State in good and fair condition for the year 2020 was 40%, slightly underachieving the expected final target. At the end of 2019, this value was 49%, which exceeded the target, but a great deal of road maintenance was unexpectedly deferred in 2020 due to the COVID-19 pandemic. Public health restrictions and protocols greatly limited maintenance and rehabilitation activities of the State road network for most of 2020, thus explaining the unexpected decrease in road pavement condition since the previous measurement. In addition to directly rehabilitating part of the above State roads, the Project’s contributions to improving State’s roads standards included knowledge gained from iRAP survey and maintenance experience from CREMA contract. 24. Number of fatalities on the 100 most critical spots of the State road network. At appraisal, the baseline of 124 fatalities was established through surveys made by the State on the 100 most critical spots identified as the 1km sections of the road network with highest number of accidents with death in 2012. The end target by Project completion was to halve this number to 62 fatalities a year, which was achieved. Road improvements by the Project that contributed to this achievement include adding railings, improved pavement conditions, divided highways with paved shoulders, improved signage and pavement markings, improved access management and crossings, and other safety interventions. It is also important to note that the State of Sao Paulo established in February 2015 a Road Safety Steering Group made up of representatives from secretariats of education, health, public safety, transport, metropolitan transport, planning, and civil society including groups representing people with disabilities. At Project closing, the target had been achieved for the year 2020, with 58 fatalities that year. Yearly data showed consistent decreases in road fatalities since 2016 and a supplemental sub-indicator (rolling average) was added for the 2016- 2020 period to mitigate fluctuation and better represent the actual reduction in road safety risk during that period due to improvements in the State's road network. This positive trend continued in 2020 during the pandemic when mobility reductions due to COVID restrictions were observed. 5 CREMA Relatorio Sintese, Desenvolvido pela Planservi Engenharia Ltda – Gerenciamento PTLMA Page 15 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Table 3: Summary table of PDO 1 and Intermediate Results Indicators by Component PDO 1: Improve the State’s transport and logistics efficiency and safety Achievement at March Indicator Name Baseline 2013 End Target 2021 2021 PDO Indicators Reduction in average Vehicle Operating Cost on 0 5 5.54 Project Roads Roads in good and fair condition as a share of total 34 48 40 roads Share of Project roads in good and fair condition 34 48 74 (supplement) Number of fatalities on the 100 most critical spots of 124 62 58 the State road network Number of fatalities on the 100 most critical spots, 124 62 62 2016-2020 average (supplement) Intermediate Results Indicators by Component Component 1: Improving transport and logistics efficiency and safety Roads rehabilitated, non-rural (km) 0 690 680.13 Number of bridges constructed 0 1 1 PDO objective 2: Enhance the Borrower’s capacity in environmental management 25. Enhancing the Borrower’s capacity in environmental management was to be achieved through (i) improving the Client’s capacity to review and approve environmental reports for roadworks, (ii) designing an Ecological and Economic Zoning (ZEE) for the State, and (iii) the installation of underwater monitoring stations. These activities were envisioned to be completed under Component 2: Strengthening sustainable environmental and land use planning and territorial management capacity: capacity building in land use planning and territorial management and regulation focusing on addressing environmental impacts in support of a greener and more inclusive growth. 26. Number of days required for SMA/CETESB to approve Environmental Preliminary Reports . The Project supported the modernization of the State’s environmental licensing system and improved its capacity to efficiently process environmental licenses and environmental strategic assessment services through the carrying out of studies and the acquisition of goods aimed at: (i) upgrading and updating the management systems for processing environmental licenses and simulating potential environmental impacts, and (ii) enhancing the capacity of the CETESB ensuring efficient and reliable data collection in monitoring water and air quality. With a baseline of 156 days before the Project, the end target of 117 days was achieved with data as of December 2020, and it will continue to be monitored. This indicator varied a lot over the years, since it depends on the number of projects that go through SMA and other factors. However, significant progress attributable to the Project was made through the years: the value for the year 2018 was 110 days; the value for 2019 was 129 days, and the last available yearly value for 2020 was 116.5 days. This was achieved through modernization of hardware and software of the environmental agencies, the development of a computer-based application for licensing procedures and simulations on environmental impacts and through direct training to staff and carrying out education and communication activities. 27. In addition to meeting the PDO indicator target, this institutional strengthening activity has resulted in a significantly improved capacity of the State in land use and territorial management, environmental monitoring and licensing. Environmental licensing processes have been streamlined and modernized without lowering the ambition of environmental standards. Environmental impact analysis systems were improved through geo-referenced information systems, the creation of a scenario room for integrated assessment with training of personnel to use the Page 16 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) systems, in addition to the recovery, systematization and availability of information related to licensing carried out in the last 10 years. Additionally, in terms of outputs, unmanned aerial vehicles for monitoring were acquired, an expansion and modernization of the air quality monitoring network, with the acquisition and installation of six more automatic stations and modernization of the network was undertaken, and an expansion and modernization of the surface water quality monitoring network, with the expansion of points monitored by the network by 10% and modernization of 13 automatic stations. 28. Through the Project’s intermediate indicators, an Ecological and Economic Zoning (ZEE) proposal was developed and will be presented to CONSEMA in November 2021. The ZEE is a technical and political planning instrument that establishes territorial planning and management guidelines, considering the environmental and socioeconomic characteristics of the different regions of the State. Its purpose is to support the formulation of public policies in line with strategic guidelines for sustainable development, as well as guiding the licensing of productive activities. The ZEE delimits portions of the territory that have common natural and socioeconomic vulnerabilities and potentials, for which social, economic, and environmental goals are established. It also aims to provide integrated and georeferenced information on the state of Sao Paulo, enabling a wide availability of data to support public discussions around the goals of regulation and appropriation of the territory. Additionally, 57 underwater monitoring stations were installed in Aquíferos Bauru and Guarani Aquifersarea, which modernized and tripled pre-project capacity in automatic water quality analysis. Monitoring systems for environmental impact analysis were expanded and modernized, and technical teams received trainings in the country and abroad on environmental analysis, conflict mediation and public management. Table 4: Summary table PDO 2 and Intermediate Results Indicators by Component PDO 2: Enhance the Borrower’s capacity in environmental management Baseline End Target Achievement at Indicator Name 2013 2021 March 2021 PDO Indicators Number of days required for SMA/CETESB to approve Environmental 156 117 116.5 Preliminary Reports (Relatório Ambiental Preliminar – RAP) for roadworks Intermediate Results Indicators by Component Component 2: Strengthening sustainable environmental and land use planning and territorial management capacity Presentation of the Ecological and Economic Zoning (ZEE) to CONSEMA No Yes No Number of underwater monitoring stations 21 57 57 PDO objective 3: Enhance the Borrower’s capacity in disaster risk management 29. Enhancing the Borrower’s capacity in disaster risk management was to be achieved through (i) developing DRM contingency plans in selected UBAs, (ii) developing a platform connecting the State’s DRM systems, and (iii) carrying out disaster risk mapping assessments in selected municipalities. These activities were envisioned to be completed under Component 3: Increasing State’s resilience to natural disasters: improvement of the State’s capacity to manage disaster risk, particularly risks linked to climate change in the transport sector. Page 17 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) 30. Through the PDO indicator “Contingency plans established for selected UBAs”, the Project significantly improved the State’s transport management, proactively coping with disaster through the establishment of contingency plans in a sample of DER’s local representations for traffic management (UBAs). At appraisal, the baseline was zero contingency plans, with the end target to achieve three. Despite administrative and procurement delays, the target was met, and three plans were delivered to the Bank by the Geological Institute within the State Secretariat of Infrastructure and Environment for the UBAs of Caraguatatuba, Mogi das Cruzes and Sao Vicente. This objective was supported with an intermediate indicator of “municipalities in the Sao Paulo metropolitan region with a concluded Disaster Risk mapping”, with a pre-project baseline of 12 that was almost quadrupled to 45, surpassing the established target of 38. These studies provided municipalities with prevention plans for natural disasters for the protection of around 9 million people living in the selected municipalities. The plans also included suggestions for works and interventions in commercial and residential areas such as planting vegetation, slope reconstruction, interventions in storm sewers and draining channels etc. Another result from the studies was the registration of natural disasters and accidents in the municipalities from 1994 to 2018 and the use of satellite imagery to identify and count all building structures in the municipalities. As an illustrative example, in April of 2018, a landslide near a tunnel in Rodovia Anchieta caused the closing of the highway. In events like this, technicians are deployed to the site to assess the damage and the possibility of a new landslide and traffic will be able to be restored much faster thanks to the contingency plans put in place at the nearby UBAs. 31. Lastly, Component 3 also envisioned the development and operationalization of a platform connecting the State Disaster Risk monitoring systems, which at Project closing was not yet finalized but fully procured and contracted, with expected completion in December 2021. Table 5 below provides an overview of additional outputs for CETESB and SIMA for Disaster Risk Management platform under implementation or execution. Table 5: Additional Outputs under component 3 Agency Description Services for preparing, digitizing, optical character recognition, image processing, indexing and inserting document metadata, as well as providing electronic document search and management system Acquisition of Inductively Coupled Argon Plasma Mass Spectrometer (ICP-MS) CETESB Acquisition of sensors for multiparameter probes Acquisition of a Gel Permeation Chromatography System (GPC) Acid distillation system with a subboiling unit Cloud computing service Acquisition of two data RACK-DATAGEO servers Acquisition of server-type computer equipment Acquisition of 25 workstations for the Environmental Military Police Acquisition of refrigerated BOD incubators Acquisition of disk drawer for data storage SIMA Hiring four MS-SQL database manager licenses Hiring services to expand the high availability environment of geographic applications that make up the DataGEO platform Acquisition of control panel for the management of the automatic air quality monitoring network (Video Wall) Access service to the submetric orbital image collection Page 18 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Table 6: Summary table PDO 3 and Intermediate Results Indicators by Component PDO 3: Enhancing the Borrower’s capacity in disaster risk management Achievement at Indicator Name Baseline 2013 End Target 2021 March 2021 PDO Indicators Contingency plans established for selected UBAs 0 3 3 Intermediate Results Indicators by Component Component 3: Increasing State’s resilience to natural disasters Development and operationalization of a Platform No Yes No connecting the State Disaster Risk Management monitoring systems Number of Municipalities in the Sao Paulo 12 38 45 Metropolitan region with a concluded Disaster Risk mapping Justification of Overall Efficacy Rating 32. Overall, the Project delivered an improved and more resilient road network with enhanced quality, efficiency, and safety. In addition, the TA institutional strengthening components achieved all their PDO objectives and virtually completed all their intended activities, resulting in a significantly improved capacity of the State of Sao Paulo in environmental and disaster risk management as explained above. The overall efficacy of the Project is therefore rated Substantial. Page 19 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) C. EFFICIENCY Assessment of Efficiency and Rating 33. The economic analysis at appraisal was carried out for the Program and updated at the time of the first restructuring when the scope of the Project was expanded (830 km of State roads rehabilitation and upgrading). The evaluation followed a traditional approach assessing benefits and costs of the investments using the Highway Development and Management Model (HDM-4). The analysis was conducted on all 24 state-road sections identified as candidate to be financed under the Project. The estimated NPV, utilizing a 12% discount rate, and the related IRR of the investments were estimated respectively to be R$1,318 million and 41.4% at appraisal. This same methodology was applied for this analysis at closing (see Annex 4) considering the Project completed 680.13km of road improvements. As a result, the NPV at completion was estimated at R$1,080 million, representing a positive internal rate of return of over 25%. 34. The complexity and scope of infrastructure designs slightly affected the efficiency of improving the State's transport and logistics efficiency and safety (PDO1). Given the nature and scope of the Project, some engineering designs had to be adjusted due to resettlement requirements, utility relocation, and other issues that led to delays in procurement and implementation. This caused a slight efficiency reduction since total implementation time for the delivery of roadworks under Component 1.1 had to be extended through project restructurings. 35. Based on this analysis, the overall efficiency of the Project is rated “Substantial”. This assessment may be considered conservative as it does not explicitly consider benefits from the road safety improvements or the bridge reconstruction, which were also not included at appraisal. Initial findings from the iRAP study demonstrated high NPV and IRR for the road safety investments, but these results are not expected to substantially change the Project NPV and IRR given the relatively limited size of the road safety components compared to the total investment in roads. D. JUSTIFICATION OF OVERALL OUTCOME RATING 36. Based on the ratings of relevance (High), efficacy (Substantial) and efficiency (Substantial), the overall outcome of the Project could be considered either as “Satisfactory” or as “Moderately Satisfactory”. Based on the failure to complete some activities before Project closure (although fully contracted and underway) and minor shortcomings on the M&E framework, the overall outcome of the Project is rated Moderately Satisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Institutional Strengthening 37. As part of the multi-sectoral approach of the Project and the activities envisioned in components 2 and 3, the SoSP has developed significant capacity in environmental management and disaster risk management and resilience for the road sector. Processing times for environmental assessments have been significantly reduced, there is an improved monitoring capacity thanks to the underwater stations created by the project, and 45 municipalities have developed DRM plans. Page 20 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Mobilizing Private Sector Financing 38. The Project mobilized US$300 million of private financing from Banco Santander covered by a MIGA Non-Honoring of Sovereign Financial Obligations (NHSFO) guarantee for subcomponent 1.1 as a credit enhancement instrument. This additional financing from an international private lender allowed the scope of activities under the Project’s physical component to be scaled-up and accommodated additional road rehabilitation and upgrading works, and the US$300 million loan was fully disbursed. This innovation for Brazil was triggered by Brazil reaching the Single Borrower Limit and a World Bank Group effort to support the wider objectives of the State’s road program. To this end, the State promoted a public consultation with interested banks, through a Letter of Invitation to the market, prepared jointly by MIGA, IBRD, STN, PGFN and the Legal Department of the Secretariat of Revenue, inviting banks interested in participating in the process. Nine international financial institutions submitted proposals to the State. This was considered a very positive development at the time. The Guarantee Period for MIGA’s cover to Banco Santander is effective until October 15, 2026. During this period, MIGA will continue to monitor Component 1 of the Project, specifically ongoing adherence to Performance Standards and Environmental Guidelines through their Guarantee Holder’s contractual obligations. Wider Economic and Development Impacts 39. The rehabilitation and upgrading of the State’s transport networks is believed to have generated significant positive economic and development impacts. In addition to reduced transportation costs and improved road safety, the Project generated positive impacts on land values, helping to create jobs and raise incomes throughout the State, as well as in neighboring states. There are two main types of project beneficiaries that will benefit from economic and development impacts: (i) local and regional producers and industries that will benefit from improved reliability of transport services and reduced logistical costs, and road users that will benefit from safer road transport, and (ii) populations located in areas subject to potential disasters, such as floods and landslides, that will benefit from risk reduction initiatives in mapped areas, in addition to the improved alerts and management systems and procedures in the event of a disaster. Most of the beneficiaries in this second group belong to the poorest and most vulnerable population groups in the State. In addition, the application of MIGA's Performance Standard 2 - Employment and Working Conditions resulted in an improved condition and safety in the workplace by promoting fair treatment, non- discrimination and equal opportunity for workers, as well as compliance with national employment and labor laws; and promoting safe and healthy working conditions, protecting employee’s health. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 40. The Project took nearly two years to prepare, and the loan was declared effective 6 months after Board approval, in December 2013. Key factors during Project preparation were: a. Prior experience. The Project built on the experience gained from the Sao Paulo State Feeder Road Project (P106663, closed in June 2014), both in its implementing arrangements (centralizing the fiduciary management in the Road agency) and the articulation of an integrated approach (a civil works component with two institutional strengthening components). b. Appropriate selection of stakeholders and adequacy of risk and mitigation measures identification. To Page 21 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) mitigate the fiduciary and social and environmental risks identified, the Project was designed to centralize administrative responsibilities within the road agency (DER) where most of the investments took place. The DER (project implementing agency for roadway components) was already experienced with Bank fiduciary and safeguards requirements. Each agency benefited from training and regular dialogue between the Bank team and the State’s team to strengthen its capacity in handling Bank procedures. c. Monitoring and evaluation: M&E design presented some weaknesses that created issues during implementation. The results framework had to be modified significantly during the second and third restructurings due to changing of government priorities and attribution issues. Even at Project closing, the PDO indicator of State roads in good and fair condition was not meaningful for the Project since it considers all classified roads in the State, whereas the improvement to Project roads has been very significant. B. KEY FACTORS DURING IMPLEMENTATION (a) Factors subject to the control government and/or implementing entities 41. Environmental and social safeguards: the significant changes introduced in the first restructuring (almost doubling the Project size with the private financing from Banco Santander covered by MIGA) were underestimated in terms of management capacity of safeguards issues. While on the environmental side there were no big issues, the new road segments covered by the Project caused some unforeseen social safeguards issues regarding lengthy expropriation and compensation cases that could have perhaps been anticipated more explicitly, and with concrete action plans to monitor and swiftly resolve. (b) Factors subject to World Bank control 42. Adequate supervision: Despite having a relatively high turn-over of TTLs, the Project maintained adequate supervision, including proactive identification of opportunities, appropriate follow-up and resolution of implementation issues and appropriate adaptation to changing conditions. Being a complicated Project with significant safeguards and implementation issues, the task team and implementing agency managed an orderly closure of the Project with strong measures in place to ensure that all targets and development objectives will be met. (c) Factors outside the control of government and/or implementing entities 43. The impacts of COVID-19 and related lockdowns since March 2020 significantly slowed the implementation of roadworks and judicial procedures for the resolution of pending resettlement cases for several months, since construction companies couldn’t operate, and courthouses were closed during lockdowns. Page 22 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 44. At appraisal, five PDO indicators were specified to assess the achievement of the PDO and eight intermediate results indicators to measure main Project outputs. Four PDO indicators were specified to assess the achievement of the PDO 1 and one PDO indicator for PDO 3, while there was no PDO level indicator for PDO 2. The theory of change was clear, and the proposed results framework had specific, measurable and achievable indicators: (i) PDO 1: improve the State’s transport and logistics efficiency and safety (a) “Roads in good and fair condition as a share of total classified roads” was a mandatory core results indicator at the time of appraisal but lacked complete relevance since most of the roads considered where outside of the scope of the Project. However, improved road conditions significantly contribute to improved efficiency and safety. (b) “Increased intermodality measured by increased proportion of exported biofuel transported by waterway in the State on the Tiete river” was adequate at the time of appraisal but eventually lost relevance for factors outside of Project control (i.e. changing of government’s priorities on biofuel). (c) “Improved transport management measured by the increased proportion of State road network managed by performance” was relevant for measuring the PDO 1, but it measured results that could not be attributed to Project outputs (except for the SP-463 pilot performance-based road rehabilitation contract, the Project had no activities that could not be expected to result in such an outcome). Therefore, it was dropped 2018. (d) “Improved road safety measured by the reduction in fatalities on the 100 most critical spots of the State road network” was substantively relevant and adequate for measuring the safety element of the PDO. (ii) PDO 2: enhance the Borrower’s capacity in environmental management. There was no indicator on the PDO level. However, the Project measured “Improved environmental control measured by the reduction of average delay for licensing transport works” on the component level. This indicator was substantively relevant for measuring PDO 2 as explained in the efficacy section. (iv) PDO 3: enhance the Borrower’s capacity in disaster risk management (e) “Improved transport management proactively coping with disaster through the establishment of contingency plans in a sample of DER’s local representations for traffic management (pilot UBA)” was substantively relevant for measuring the PDO 3. 45. The M&E design included a detailed implementation plan with the frequency, data source and methodology and responsibility for data collection of all PDO and intermediate results indicators. The baseline data for monitoring was collected by the time of approval, arrangements were put in place for the remaining data collection during implementation and targets were set for each indicator. Page 23 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) M&E Implementation 46. The UCPR and consultants implemented the M&E system as designed through systematic data collection, analysis, and reporting. The UCPR regularly prepared and shared semiannual reports, which provided information on the project, including highlights on progress and key issues for attention. Progress on the PDO indicators and the intermediate indicators was provided by the client and tracked by the Bank as part of regular implementation support missions and reported in Aide Memoires and Implementation Status and Results Reports (ISRs). 47. The Results Framework was adjusted to try to accommodate the changes in the project. During the second and third restructuring, some PDO indicators were refined or modified due to their limited attribution to the Project and changing national priorities. (i) The revised PDO 1 indicator “Percentage increase of export products transported on the Tiete river” (2018) was linked to Project activities (the bridge reconstruction enabled navigation of larger vessels) but it was not closely measuring Project results and depended on factors outside the Project control (focus on biofuel was no longer relevant since the government cut subsidies on ethanol and the production and transportation of biofuel dropped to almost zero). The new revised indicator “Change in the average Vehicle Operating Costs on Project roads (2020)” was directly relevant for measuring transport and logistics efficiency (PDO 1). (ii) The revised PDO indicator “Number of fatalities on the 100 most critical spots of the State road network” for PDO indicator was substantively the same but more relevant and attributable to the project. (iii) The PDO indicator to assess “good and fair condition as part of total classified network” added a supplement considering only Project roads to try and better reflect the reality of the improvements caused by the project (iv) The intermediate outcome indicator “Improved environmental control measured by the reduction of average delay for licensing transport works” that was made a PDO 2 indicator in the 2018 restructuring was directly relevant for measuring PDO 2 because it supported the modernization of the State’s environmental licensing system and improved its capacity to efficiently process environmental licenses and environmental strategic assessment services. (v) The revised PDO indicator “Number of contingency plans established for selected UBAs” for PDO 3 was substantively the same but more relevant and attributable to the project. M&E Utilization 48. M&E data and results were used to inform management and for decision-making, and the Project was monitored and restructured accordingly utilizing M&E reports and documents. Some of the original PDO indicators had attribution issues due to factors outside of project control, and others were refined and reworded to be more specific and measurable, with no substantial changes in what they measured. However, the weaknesses in the results framework design mentioned above could have been better identified and mitigated as part of M&E reviews and improved through the restructurings. Justification of Overall Rating of Quality of M&E 49. Based on the above discussion, the overall quality of M&E is rated ‘Modest’. Page 24 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 50. The SoSP undertook an integrated Environmental and Social Impact Assessment (ESIA) to assess the potential risks and benefits of Project activities, examine alternatives and identify ways to enhance positive impacts by minimizing, mitigating, or compensating for any adverse social or environmental effects. During Environmental Screening, the Project was categorized as Category 'B', with the overall safeguard risk being 'Substantial.” The ESIA was updated by the State of Sao Paulo to consider the expanded scope of works during the first restructuring. No new safeguard policies were triggered, and no major additional adverse impacts were expected due to the inclusion of three new roads as well as the replacement of a bridge. Overall, the Project largely complied with the triggered environmental safeguard policies. Environmental Safeguards 51. At appraisal, the ESIA concluded that the Project was not expected to cause significant and/or irreversible negative environmental impacts on a large scale and should have an overall positive environmental impact, as it would contribute to the State policy aimed at promoting the modal shift of the polluting road transport mode towards a more ecological mode (inland waterway transport). The following policies were triggered: OP/BP 4.01 (Environmental Assessment); OP/BP 4.04 (Natural Habitats); OP 4.09 (Pest Management); OP/BP 4.11 (Physical Cultural Resources); and OP/BP 4.36 (Forests). 52. The Project had effective environmental management and adequate capacity and procedures in place to handle the expected environmental impact mitigation measures. The environmental management of the Project's works was supervised by independent supervision firms, in line with the best practices of other transport projects in Sao Paulo and Brazil, who carried out periodic visits to the construction sites and documented, classified and sought solutions for all the typical occurrences of environmental impacts during the Project execution (control of erosion, inadequate disposal of solid waste and effluents, siltation of water courses, etc.). 53. The Bank and DER agreed on a Post-Closing Action Plan in early 2021 to resolve all outstanding safeguards issues, including small environmental adverse occurrences that were not resolved by Project closure, as well as ensuring social and environmental Project oversight improvements until all the works are completed. Mitigation measures included in the Action Plan required ongoing dialogue with the DER and close supervision after closure. DER has committed to continue Project oversight, with the support of the current Project Management Consultant (under contract until at least October 5, 2021) and to provide monthly reports to the World Bank until completion of the remaining road works and the solution of all pending environmental occurrences. Social Safeguards 54. At appraisal, The ESIA determined that the proposed civil works would require the acquisition of 14 small strips of rural land (on average 0.139 hectares) and the relocation of 15 informal structures occupying the road right-of-way (ROW). Bank policy OP 4.12 (Involuntary Resettlement) was triggered. Site-specific Resettlement Plans were prepared to mitigate these adverse impacts. Moreover, a Resettlement Policy Framework (RPF) that set out the guidelines, procedures and criteria to avoid, minimize, mitigate and/or compensate any additional resettlement impacts that could potentially result from any eventual design modifications in the course of the Project being prepared. Although Component 3 did not finance the resettlement of any persons living in natural disaster-prone areas, it could potentially support preparation of resettlement plans for populations living in these high-risk areas. In such cases, the RPF also Page 25 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) guided the design of the resettlement planning activities included in disaster risk management frameworks that were prepared. The ESIA also determined that the land use planning and environmental enforcement activities under Component 2 could potentially result in involuntary restrictions of access to natural resources located in legally designated parks or protected areas causing adverse impacts on the livelihoods of the displaced persons. For this reason, a Process Framework (PF) was prepared to ensure that consultations with land users will identify relevant restrictions and design appropriate mitigation measures to enable preservation of their socioeconomic conditions, particularly of the most vulnerable populations. 55. Although none of the Project road works were located in areas occupied or claimed by indigenous peoples, some of the roughly 3,000 indigenous people living in the 33 Indigenous lands located throughout the state could potentially be affected by the land use planning and environmental enforcement activities under Component 2. For this reason, the Project triggered operational policy OP 4.10 (Indigenous Peoples) to ensure that specific actions were designed to provide indigenous peoples with appropriate opportunities to participate and voice their concerns in the regional planning processes. This outcome was achieved through application of the Indigenous Peoples Planning Framework (IPPF) prepared by the State and disclosed on January 23, 2013. 56. At Project closing, indigenous peoples were not affected by the Project and a lot of the initially expected cases of resettlement in the A-ISDS did not materialize (i.e. Involuntary restrictions of access to natural resources located in legally designated parks or protected areas causing adverse impacts on the livelihoods of the displaced persons), thus the prepared framework was not used. 57. Restructuring of the Project to include roads enlargement on SP 270, SP 250, and SP 088 lead to around 500 cases of expropriation, including some physical relocation of informal small business. There is systematic and complete information about expropriation cases in several project reports prepared by the client. At Project closing, over 95% of all resettlement cases were resolved. There were about 16 pending expropriation cases on small plots along some of the road works financed by the Project, but almost all were in the final stage of compensation. The impacts of COVID-19 and related lockdowns since March 2020 significantly slowed the judicial procedures for the resolution of pending resettlement cases for several months. These and any other issues relating to social safeguards after Project closing are being closely monitored under the agreed Post-Closing Safeguards Action Plan mentioned above. More information about the pending expropriation cases cand be found in Annex 6. Financial Management 58. The Unidade de Coordenação de Projetos (UCPR) under the DER was responsible for the management of all administrative and fiduciary aspects for all activities under the Project. Primary responsibilities included: (i) submitting disbursement requests and documentation of expenditures to the Bank; (ii) preparing and submitting Interim unaudited financial reports (IFRs) to the Bank; (iii) preparing and providing all financial documentation and Project reports requested by external auditors and Bank staff; and (iv) preparing, updating and ensuring that all implementing entities were in compliance with the Project Operational Manual. 59. During the project’s implementation time, there were a total of ten Financial Supervision missions. The FM arrangements for the first mission were rated Satisfactory (S). It was changed to Moderately Satisfactory (MS) after the second mission. And during the third to the sixth mission, the risk was changed to Satisfactory (S). Since the seventh mission until Project closing, the arrangements have been considered Moderately Satisfactory (MS). The Page 26 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Project Financial Management risk was assessed as Moderate during the first three missions. After that and until Project closure, the Project’s risk was rated Low. 60. Based on a total of fourteen IFRs submitted to the Bank during the life of the Project, eleven IFRs were received late. There have been no cases of ineligible expenses identified in the Project time. All audit reports, for the years of 2014 to 2019 were received by the Bank, considered acceptable and expressed unqualified opinions on the project’s financial statements. Only one report for the 2014 year was received late (< 4 months). 61. The financial team was composed of appropriately qualified staff that have experience working with the World Bank and government projects, and FM supervision throughout the Project verified that the FM arrangements provided (i) accurate and timely financial information; and (ii) reasonable assurance that Project funds are being used for the purposes intended. Procurement 62. Procurement management of the Project was overall moderately satisfactory. The UCPR and the bidding commission of the DER (Comissão de Licitação, CdL) were responsible for the procurement of all activities under the Project. The CdL prepared the procurement documentation up to contract award (invitation for shortlisting and bidding, bid documents/RFP, evaluation reports, and any other required document), the UCPR, with support from each procurement support units in the beneficiary entities, reviewed the TOR and technical specifications, issued the bidding documents and responded to the bidder’s/tenderers’ request for clarifications. The UCPR was also responsible for updating and monitoring the procurement plan. 63. Given the complexity of designs, the Project experienced contract implementation delays caused by resettlements, changes to Project design and delays caused by the need to relocate utilities. Additionally, during the MTR it was noted that part of the delays resulted from DER procurement unit being short-staffed, but the PIU always remained in close contact with the Bank’s procurement specialists for hands-on support and guidance. C. BANK PERFORMANCE Quality at Entry 64. During Project preparation, all the key parameters such as strategic relevance, country and sector context, lessons from other projects, and environmental and social development issues were adequately identified and appropriately addressed. However, as noted earlier, the result indicators were not consistently relevant for measuring Project outputs or outcomes. 65. Risk assessment carried out during preparation rated the overall implementation risk as moderate, considering the Bank’s ongoing partnership, the known capacity of the State and the good performance of the past Bank operation. Mitigation measures adopted included designating DER as the centralized administrative agency responsible, training and regular dialogue between the Bank team and the State’s team to strengthen its capacity in handling Bank procedures, and a detailed evaluation of the potential social and environmental negative impacts of the Project that provided a clearer understanding of the scope, extent and magnitude of the potential social impacts of the project, Page 27 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) as well as the appropriate instruments in line with Bank policy to avoid, minimize, mitigate and/or compensate any unavoidable adverse social impacts. 66. The task team worked closely with the State and DER to introduce the environmental management and the DRM & resilience components trough a multi-sectoral approach in close coordination with the respective State agencies involved in these matters. The project’s three components were to be implemented by multiple implementing and coordinating agencies (namely, Secretaria de Transporte e Logistica, Secretaria de Meio Ambiente, Instituto Geologico), one of which was unfamiliar with Bank’s processes, which posed a minor implementing risk. 67. The environmental and social assessments were done at the very beginning, with a partial EA assessment done on 27 December 2012. The triggered policies were identified, and appropriate mitigation measures were designed. 68. Overall, quality at entry was adequate, but the impacts of the changes introduced in the first restructuring (almost doubling the Project size with the private financing from Banco Santander covered by MIGA) were underestimated. While on the environmental side there were no big issues, the new road segments covered by the Project caused some unforeseen social safeguards issues regarding lengthy expropriation and compensation cases that could have perhaps been anticipated more explicitly, and with concrete action plans to monitor and swiftly resolve. Quality of Supervision 69. The task team provided proactive support to DER and worked to identify and resolve bottlenecks and threats to the achievement of the development outcome and to improve overall Project performance. The Bank team conducted fourteen implementation support missions at an average of two missions per year, which were supplemented with interim missions and technical visits. The impact of COVID-19 resulted in virtual missions during 2020 and Project closing. 70. Multi-disciplinary Bank teams comprised of procurement, FM and environmental and social management specialists reviewed all relevant documents before, during and after supervision missions and restructurings. Documents were prepared and issued after each mission, highlighting progress and critical issues candidly, along with timelines for agreed actions. Action plans were prepared to speed up implementation and improve Project performance, including compliance with safeguards and fiduciary requirements. Key implementation issues were regularly brought to the attention of SoSP and DER, as well as Bank management. 71. The December 2016 Mid-Term Review found that only one of the thirteen indicators included in the Result Framework had reached its MTR target (even though others had progressed, but not up to the expected level). The team and the State of Sao Paulo agreed at MTR to reprioritize Project activities and adapt the Result Framework to put the Project back on track. 72. The task team ensured that transition arrangements and action plans were in place for ongoing activities at Project closing, especially in the E&S front. All remaining activities, for both civil works and TAs, had been procured and contracted at Project closing, and a specific timeline was agreed with the implementing agency for the total completion of Project targets. Also, the on-going efforts of the institutional strengthening will continue to contribute to better operational capacity and an improved environmental management and disaster risk management and resilience for the SoSP. Page 28 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Justification of Overall Rating of Bank Performance 73. Based on above discussion on Quality at Entry and Quality of Supervision, Bank performance is rated ‘Moderately Satisfactory’. D. RISK TO DEVELOPMENT OUTCOME 74. The sustainability of the road assets created under the Project is assured by the fact that DER has maintenance plans and periods in place. However, there is a small risk that the State may not provide adequate budgetary allocations for road maintenance in an event of fiscal constraints. The SoSP is committed to completing the on-going rehabilitation of 9 km of State roads targeted under the Project by August-September 2021. 75. The institutional and capacity building initiatives under the Project are also expected to be sustained, given the high priority of the State to environmental and disaster risk management. The ZEE has been drafted and the validation was submitted in June 2021. The final zoning proposal is expected to be presented to CONSEMA in November 2021. The development and operationalization of a Platform connecting the State Disaster Risk Management monitoring systems is expected to be fully achieved by December 2021. The technical services were contracted on December 2020 for 12 months and the State is committed to completing the activity. 76. The Bank will monitor status of the above remaining activities and E&S issues on an on-going basis until they have been completed, as agreed in the Project closing action plan. Their timely completion could be impacted by the resurgence of the COVID-19 pandemic. V. LESSONS AND RECOMMENDATIONS 77. The Project had a number of lessons learned that could help the design and implementation of future projects: (a) The WBG’s guarantee products can help leverage public investments and maximize finance for development. The Project included MIGA’s Non-Honoring of Sovereign Financial Obligations (NHSFO) guarantee as a credit enhancement instrument which allowed additional private funding in the amount of US$300 million at favorable market conditions. This additional financing from international private lenders allowed the scope of activities under the Project’s physical component to be scaled-up and accommodated additional road rehabilitation and upgrading works. Nine international financial institutions submitted proposals to the State for the co- financing, which demonstrates the advantages of leveraging a wider array of WBG instruments to attract private financing, particularly in IBRD and IDA-graduate countries. Another feature of this collaboration with MIGA was that the Project roads were not separated into IBRD and MIGA lots but were treated the same project. This was done not only for financial management, but also to streamline environmental and social safeguards supervision. Page 29 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) (b) A multi-sector project approach delivers better results in the long term but requires greater coordination and capacity by the implementing agencies. Although most of the investment went to road upgrading and rehabilitating, this was a multi-sector Project that included critical aspects of road safety, environmental management, and DRM and resilience in the transport sector involving numerous State agencies. While the outputs in terms of improved roads are important, the Project has also delivered a significantly increased capacity for the client in environmental and disaster risk management, which are critical for the achievement of long-term outcomes such as sustainable development preserving the environment and protecting the population from natural disasters. While DER is a capable and competent implementing agency with broad experience with Bank projects, the project’s three components were implemented by separate implementing agencies, one of which was unfamiliar with Bank’s processes. Burdensome bureaucratic processes within some the implementing entities, lack of prioritization of Project activities, and understaffed teams led to substantial delays in Project execution. This could be improved by centralizing responsibilities in one PIU with the adequate authority to avoid bureaucratic delays caused by several layers of approvals. (c) Complex infrastructure projects benefit from an assessment of scope and designs as early as possible to minimize changes and delays during implementation. Given the nature and scope of the Project, some engineering designs had to be adjusted due to resettlement requirements, utility relocation, and other technical design issues that led to delays in the procurement, implementation, and/or time and price amendments. A more comprehensive assessment of designs as early as possible or alternative project delivery approaches (such as a design-and-build contract) could be considered in the future for appropriate works to help minimize delays. (d) Bank-supported projects can help clients adopt new technologies and practices in environmental and social management. Examples from this Project include the expansion of public communication channels and grievances redress mechanisms through digital and social media, improving field supervision through remote sensing (particularly during periods of travel restrictions because of the pandemic), increasing the agility and reducing the process time for environmental licensing, expropriation and resettlement processes, greater participation of environmental and social specialists from partner implementing agencies, and adoption of specific procedures for fortuitous findings during construction. (e) Investment projects that close before civil works are fully completed or with pending safeguards compliance issues require a post-closing action plan and a commitment from the client to carry it out. This requirement was made clear to the Sao Paulo State client and implementing agency as part of the final restructuring of this Project in December 2020. A post-closing safeguards action plan was jointly developed and agreed to before the closing of the Project in March 2021. The Bank team monitors progress on the action plan and relies on the client to provide the required updates. . Page 30 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improving the State’s transport and logistics efficiency and safety Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reduction in average Vehicle Percentage 0.00 5.00 5.54 Operating Costs on Project roads 02-May-2013 30-Mar-2021 01-Oct-2020 Comments (achievements against targets): Indicator was achieved and is largely due to the improved pavement and design conditions of the Project road network. In the last restructuring, this indicator replaced the previous indicator of "increase of export products transported on the Tiete river" because it could not be directly reported (only estimated) until the new bridge is open to larger vessels and vehicles. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads in good and fair Percentage 34.00 48.00 40.00 condition as a share of total classified roads in the State 02-May-2013 31-Dec-2020 30-Mar-2021 Page 31 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Share of Project roads in Percentage 34.00 48.00 74.00 good and fair condition 02-May-2013 30-Mar-2021 24-Mar-2021 Comments (achievements against targets): The measurement of this indicator was updated after December 2020 to reflect recent road pavement surveys for the entire classified State road network of 13,000km. The pandemic greatly limited maintenance and rehabilitation activities of the State road network for most of 2020, thus explaining the unexpected decrease in road pavement condition since the previous measurement at the beginning of 2020. Moreover, the scope of this parent indicator (a mandatory core sector indicator at the time of appraisal) goes much beyond the Project, which represents only 5% of the State road network. Consequently, a much more attributable sub-indicator was added for the share of Project roads in good and fair condition considering only the 690 km network of Project roads. This change in methodology should have been made in the last restructuring to make this indicator more attributable and relevant to the Project. The share of State roads rehabilitated under the Project (690km, including the one bridge rehabilitated under the Project) in good and fair condition increased from 34% at appraisal to 74% as of March 2021 compared to the target of 48% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of fatalities on the Number 124.00 62.00 58.00 100 most critical spots of the State road network 02-May-2013 31-Dec-2020 31-Dec-2020 Number of fatalities on the Number 124.00 62.00 62.00 100 most critical spots of the State road network (2016-2020 average) Comments (achievements against targets): Page 32 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) The fatalities on the targeted State roads (that were rehabilitated under the subcomponent 1.1) decreased from 124 per year at appraisal to 58 at the end of 2020, under the targeted 62 per year. The rolling average of yearly fatalities during 2016-2020 remained at 62, which shows continued improvement of road safety. Objective/Outcome: Enhancing the Borrower’s capacity in environmental management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of days required for Days 156.00 117.00 116.50 SMA/CETESB to approve Environmental Preliminary 02-May-2013 30-Mar-2021 31-Dec-2020 Reports (Relatório Ambiental Preliminar – RAP) for roadworks Comments (achievements against targets): It takes 116.5 days as of December 2021 for SMA/CETESB to approve Environmental Preliminary Reports for road works compared to 156 days at appraisal and to the targeted 117 days. SMA/CETESB will continue monitoring this indicator. The number of days has varied a lot throughout the years (110 days in 2018, 129 days in 2020) since it depends on the number of projects that go through SMA and other factors. Objective/Outcome: Enhancing the Borrower’s capacity in disaster risk management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 33 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Contingency plans Number 0.00 3.00 3.00 established for selected UBAs 02-May-2013 06-Apr-2021 06-Apr-2021 Comments (achievements against targets): The Project established the targeted three contingency plans. The plans were delivered to the Bank on April 6, 2021 by the Geological Institute within the State Secretariat of Infrastructure and Environment (SIMA). The contract (Contrato nº 20.595-3, Elaboração de Planos de Contingência frente a riscos de eventos geodinâmicos em trechos rodoviários das Unidades Básicas de Atendimento (UBAs) Caraguatatuba, Mogi das Cruzes e São Vicente) was administratively managed by DER-SP but the technical content was managed by the competent agencies within SIMA. A.2 Intermediate Results Indicators Component: Improving the State's transport and logistics efficiency and safety Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads rehabilitated, Non- Kilometers 0.00 377.00 690.00 680.13 rural 02-May-2013 30-Mar-2018 30-Mar-2021 15-Mar-2021 Comments (achievements against targets): Original target of 377km was increased with the additional $300m of financing from Banco Santander. Indicator virtually achieved. At Project closing, only 9km remained to reach the target. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 34 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Target Completion Number of bridges Number 0.00 2.00 1.00 1.00 reconstructed 02-May-2013 30-Mar-2018 31-Dec-2020 30-Mar-2021 Comments (achievements against targets): Target achieved as the construction of the new bridge is completed. Remaining civil works relate to the demolition of the old bridge which is expected to be completed by August 2021 Component: Enhancing the Borrower’s capacity in environmental management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Presentation of the Yes/No No Y No Ecological and Economic Zoning (ZEE) to CONSEMA 02-May-2013 30-Nov-2021 15-Mar-2021 Comments (achievements against targets): The ZEE was drafted but not formally submitted to CONSEMA because it is under a review and validation process with the State ZEE Commission (12 government clearance levels). The validation was completed in April 2021 and the zoning proposal submitted in June 2021. After that, the zoning proposal will be presented in consultations and public hearings from July to September 2021. The final zoning proposal are expected to be presented to CONSEMA in November 2021. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 35 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Target Completion Number of underwater Number 21.00 57.00 57.00 monitoring stations 02-May-2013 31-Dec-2020 13-May-2020 Comments (achievements against targets): Target fully achieved. Component: Enhancing the Borrower’s capacity in disaster risk management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development and Yes/No No Y No operationalization of a Platform connecting the 02-May-2013 31-Dec-2021 15-Mar-2021 State Disaster Risk Management monitoring systems Comments (achievements against targets): This indicator is expected to be fully achieved by December 2021. The technical services were contracted on December 2020 for 12 months and the State is committed to completing the activity. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 36 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Target Completion Number of Municipalities in Number 12.00 38.00 45.00 the Sao Paulo Metropolitan region with a concluded 02-May-2013 31-Dec-2020 10-Dec-2020 Disaster Risk mapping Comments (achievements against targets): Target has been overachieved, 45 municipalities carried out a disaster risk mapping. Part of the results have been published by the State at: https://www.infraestruturameioambiente.sp.gov.br/2020/12/governo-de-sao-paulo-finaliza-mapeamento-de-riscos-em-38-municipios-da-regiao- metropolitana-do-estado/ Page 37 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Improving the State’s transport and logistics efficiency and safety 1. Change in the average Vehicle Operating Costs on Project roads (%) 2. Roads in good and fair condition as a share of total classified roads in the State (%) Outcome Indicators 3. Share of Project roads in good and fair condition (%) 4. Number of fatalities on the 100 most critical spots of the State road network 1. Roads rehabilitated, Non-rural (km) Intermediate Results Indicators 2. Numbers of bridges reconstructed 1. 680.13 km of roads rehabilitated, with additional lanes, improved designs, and enhanced road safety features Key Outputs by Component 2. Bridge over Tiete river reconstructed, with wider span, allowing for bigger water (linked to the achievement of the Objective/Outcome 1) convoys, more efficiency of waterborne transport and, ultimately, higher export products Objective/Outcome 2: Enhancing the Borrower’s capacity in environmental management 1. Number of days required for SMA/CETESB to approve Environmental Preliminary Outcome Indicators Reports (Relatório Ambiental Preliminar – RAP) for roadworks 1. Presentation of the Ecological and Economic Zoning (ZEE) to CONSEMA Intermediate Results Indicators 2. Number of underwater monitoring stations 1. A study has been drafted and it’s under review and validation process with the State ZEE Commission. Zoning proposal will be presented in consultations and public Key Outputs by Component hearings from July to September 2021. The final zoning proposal expected to be (linked to the achievement of the Objective/Outcome 2) presented to CONSEMA in November 2021 2. 57 underwater monitoring stations installed in Aquíferos Bauru and Guarani Aquifers area, resulting in improved environmental monitoring Page 38 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Objective/Outcome 3: Enhancing the Borrower’s capacity in disaster risk management Outcome Indicators 1. Contingency plans established for selected UBAs 1. Development and operationalization of a Platform connecting the State Disaster Risk Management monitoring systems Intermediate Results Indicators 2. Number of Municipalities in the Sao Paulo Metropolitan region with a concluded Disaster Risk mapping 1. Technical services were fully procured and contracted on December 2020 for 12 months to operationalize the platform Key Outputs by Component 2. 45 municipalities carried out a disaster risk mapping, providing them with data and (linked to the achievement of the Objective/Outcome 3) tools for disaster prevention and protection of over 9 million people living in the municipalities Page 39 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Eric R. Lancelot Task Team Leader(s) Etel Patricia Bereslawski Aberboj Procurement Specialist(s) Susana Amaral Financial Management Specialist Jason Jacques Paiement Social Specialist Marcio Cerqueira Batitucci Social Specialist Supervision/ICR Georges Bianco Darido Task Team Leader(s) Luciano Wuerzius, Efraim Jimenez, Danilo Pereira de Procurement Specialist(s) Carvalho Susana Amaral Financial Management Specialist Silmara Moreira Da Silva Financial Management Specialist Paula Genis Team Member Graciela Sanchez Martinez Social Specialist Michele Martins Procurement Team Maria Bernadete Ribas Lange Team Member Gregoire Francois Gauthier Team Member Satoshi Ogita Team Member Marcio Cerqueira Batitucci Environmental Specialist Frederico Ferreira Fonseca Pedroso Team Member Angela Alice Dengo Team Member Nicolas De Leon De Maria Team Member Page 40 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 12.319 128,191.65 FY13 55.459 439,192.48 FY14 0 - 83.37 Total 67.78 567,300.76 Supervision/ICR FY14 48.816 277,589.40 FY15 67.936 318,349.82 FY16 47.071 201,890.61 FY17 30.461 137,514.88 FY18 24.835 178,019.71 FY19 24.895 172,460.94 FY20 14.929 95,799.54 FY21 0 342.00 Total 258.94 1,381,966.90 Page 41 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1: Improving transport and logistics 394.25 694.25 176.1% efficiency and safety Component 2: Strengthening sustainable environmental and land use planning and 18.00 18.00 100% territorial management capacity Component 3: Increasing State’s resilience to natural 16.00 16.00 100% disasters Total 428.25 728.25 170% Page 42 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) ANNEX 4. EFFICIENCY ANALYSIS BRAZIL: Sao Paulo Sustainable Transport Project 1. The present analysis updates the PAD economic evaluation given the Project expanded scope because of the Joint-Financing. The same methodology was used as at Project appraisal, focusing on the Project’s physical subcomponents, (i) State roads rehabilitation and upgrading, and (ii) bridge reconstruction for inland waterway transport on the Tiete river. For each subcomponent, similar methodologies as at appraisal were applied. 2. This assessment does not consider benefits resulting from road safety improvements as the selected improvements expected under the Project were not yet defined at the time of the restructuring. Initial findings from the iRAP study demonstrate high NPV and IRR for the potential investments. While this is not expected to substantially change the Project NPV and IRR given the relatively limited size of the investment compared to the overall investments in roads, Project NPV and IRR is expected to be slightly hirer than the values below. Road Rehabilitation and Maintenance Works (including CREMA) 3. For the State road rehabilitation and upgrading and CREMA works, the evaluation followed a traditional approach assessing benefits and costs of the investments using the Highway Development and Management Model (HDM-4). The analysis was conducted on all 24 state-road sections identified a candidate to be financed under the Project for the 830 km that were initially considered after the first restructuring. 4. Based on the calculation for the selected sections of 830 km, over 20 years, the net present value (NPV), at a 12% discount rate, and the related internal rate of return (IRR) of the investments in this subcomponent are respectively estimated as R$1,318 million and 41.4%. 5. Given that the Project targeted a total of 690km of selected roads in the candidate roads, the NPV is estimated at R$1,080 million. Bridge Reconstruction 6. Cost/benefits analysis, resulting from improved conditions of navigability in the Tiete inland waterway in the State of Sao Paulo assessed reduction in transport costs due to the modal shift of freight from road to inland waterway transport. 7. Considering that the estimated capital investment is R$107.3 million for reconstructing the bridge, and the annual costs on maintenance and rehabilitation are assumed at 1% of the capital costs, NPV at a 12% discount rate is R$37.4 million and the related IRR of the investments in this subcomponent is 16.5% over 20 years. Conclusion 8. Overall, the Project IRR and NPV of both road rehabilitation/maintenance and bridge reconstruction works are estimated respectively at 38.7% and R$1,229 million based on the above analysis. Page 43 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Comments from Borrower’s representative The Bank Team reached out to DER Sao Paulo (as the Borrower’s representative) for comments on the ICR and received the following comments on September 21, 2021 (translated from Portuguese): Positive Aspects: • Expansion of communication channels through digital media (remote conferencing, WhatsApp, Telegram, Facebook, DER Official Site, etc.) • Alignment of information for expropriation processes • Implementation and improvements in SCDES – expropriation control systems • Improvements of complaints and grievances mechanisms through several digital channels • Greater participation of environmental and social specialists from the Program Management Agency • Implementation communication and social interaction programs • Adoption of specific procedures for fortuitous findings • Inclusion and agility of resolution of environmental and social issues • Experience with CREMA contract • Project works continuation during the pandemic • Continued support from Bank’s teams • Examples of similar projects around the world Negative Aspects: • Loan divided in more than 1 beneficiary but only 1 coordinating agency. Poor experience for all beneficiaries when there was lack of coordination. • Lack of appropriate tools for the executor, such as physical and financial planning and project monitoring. • Improvement of STEP system with more user friendly platform. Verbatim comments to ICR Report: - To average yearly death toll, paragraph 3: Data from Infosiga (government database of São Paulo) showed an annual average of 2,050 fatal accidents (without informing the number of deaths linked to the accident) as of 2015. - To delays in implementation, paragraph 15: This fact occurred due to government interference and also due to the exchange of works between the financing agents IBRD/IDB. - To IRI of roads of 74%, paragraph 22: In the biannual report, the data presented was 79%. - To pending environmental occurrences, paragraph 53: Due to the receipt of the works, it will be necessary to discuss the proposed solutions between the parties involved for execution at an opportune time. - To Annex 1: Data can be updated with the numbers presented in the semi-annual report sent to IBRD in August. Page 44 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) The Client also sent the Task Team the Relatorio Semestral de Progresso (semi-annual progress report) from January to July 2021, including the post-closing period from 03/31/2021 to 07/31/2021 with information on: - Project progress during the semester, disaggregated by component - Social and Environmental issues, including health and safety in the workplace - Environmental supervision of works - Social safeguards issues - Actions carried out in the period by the Social Specialists of the Construction Companies with the people affected by the works in progress - Procedures for the control of traffic, safety and signaling of the road works - Follow-up of the Socio-Environmental Plan after Project Closure - April/2021 - Actions carried out by DER/UCPR to meet the Social and Environmental Plan after Project closure - Period April to July/2021 - Loan Disbursements information - Project indicators achievement (PDO and intermediate indicators) - Monitoring of civil works execution Comments from MIGA The Bank Team reached out to MIGA (guarantor to co-financier) for comments to the ICR and received comments on September 21, 2021. The following is a summary of the main comments: - The Guarantee Period for MIGA’s cover to Banco Santander will be effective until October 15, 2026. During this period, MIGA will continue to monitor Component 1 of the Project, specifically ongoing adherence to Performance Standards and Environmental Guidelines through their Guarantee Holder’s contractual obligations. - One of the main features of the way that the deal was structured was that it was not separated into “WB Roads” and “MIGA Roads” – all components were treated the same as part of the ‘SP Sustainable Transport Project.’ While this was mostly for financial reasons, it also helped streamline E&S and allowed MIGA to rely on the Bank’s monitoring. Page 45 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) Image 2. Selected Sao Paulo State STP Corridors at Appraisal (377km) Image 3. Updated map of Sao Paulo State STP corridors after the 2015 restructuring ($300 mn co-financing) Page 46 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Image 4. Updated map of Sao Paulo State STP corridors at Project closing Page 47 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Table 7. List of Planned Roads at Appraisal and in the 2015 restructuring Page 48 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Table 8: Progress made on pending compensation cases Page 49 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Image 5: Status of pending compensation cases Page 50 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Image 6. CREMA works in 2019 Image 7. CREMA works in 2020 Page 51 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Page 52 of 53 The World Bank Sao Paulo State Sustainable Transport Project (P127723) Table 9. Changes to Result Framework during the 3 restructurings First Restructuring The target of the intermediate indicator “Km of rehabilitated and/or upgraded State paved roads” under the subcomponent 1.1 on “Rehabilitation and upgrading of the State’s transport networks” was increased from 377 km (including approximately 100 km of CREMA) to approximately 830 km (including approximately 100 km of CREMA) thanks to availability of additional financing (see the change to financing plan). Second Restructuring The Result Framework did not fully capture the theory of change supported by the Project and most Project indicators were refined to be more specific and measurable, with no substantial changes in what they measured. (i) Subcomponent 1.1: one original intermediate outcome indicators for was dropped as no longer relevant6; (ii) Component 2: “Strengthened land use planning and territorial management" (with the target of ""Execution of three Government actions based on the policy instrument") was revised to "Approval from CONSEMA (State Council of Environment) of the Ecological and Economic Zoning (ZEE) for the State of Sao Paulo (Yes/No)" because the original indicator was beyond the scope of the Project as the execution of these Government actions would have required the vote of the State of Sao Paulo legislative assembly, which went beyond the prerogatives of the State executive branch. The proposed new end-target was seen to be within what the State of Sao Paulo can achieve as the planning instrument was expected to be prepared and approved by the relevant State authority (CONSEMA - State Council of Environment); (iii) Component 2: "Improved environmental monitoring measured by the expansion of the of underwater monitoring capacity in the State territory" (from 21 to 57 days) was revised to "Number of underwater monitoring stations" (from 21 to 57); (iv) Component 3: “Improved monitoring of climate risk factors measured by the number of automatic monito ring stations installed in the State” (from 0 to 40) was revised to “Development and operationalization of a Platform connecting the State Disaster Risk Management monitoring systems” (Yes/No). The indicator was adapted accord to the modified Project activity. Initially, the State of Sao Paulo planned to procure and install new weather monitoring stations. During implementation, the State of Sao Paulo decided to change its strategy: instead of procuring new stations, the State decided to make the most of existing weather monitoring stations by networking them within a common system. The targets for the subcomponent 1.1. were revised, as follows: (i) Subcomponent 1.1: the target of the intermediate indicator “Km of rehabilitated and/or upgraded State paved roads” was decreased from 750 km (including approximately 100 km of CREMA) to 700 km due to higher construction costs (investment involved more road duplication than initially expected, leading to higher construction costs); (ii) Subcomponent 1.1: the “Number of bridges reconstructed” was reduced from two to one bridge for the SP-147 crossing over Tiete because DER/SP could not get the environmental licensing for the second bridge initially planned to be built. Despite only one bridge being rebuilt, the target traffic increase on river Tiete, measured by the PDO indicator "Percentage of export products transported on the Tiete river" was expected to be reached. (iii) Component 3: "Increased number of Municipalities in the Sao Paulo Metropolitan region with a concluded Disaster Risk mapping" (from 12 to 39) was revised to “Number of Municipalities in the Sao Paulo Metropolitan region with a concluded Disaster Risk mapping (Number)” (from 12 to 38). End-target value reduced from 39 to 38, as the Sao Paulo municipality is part of the Sao Paulo metropolitan area but is not included in the risk mapping exercise developed within the Project. Third Restructuring See Paragraph 15. Extend the end-of-target dates of the indicators with the extended closing date. (i)The intermediate outcome indicator for Component 2: "Approval from CONSEMA (State Council of Environment) of the Ecological and Economic Zoning (ZEE) for the State of Sao Paulo" was revised to “Presentation of the Ecological and Economic Zoning (ZEE) to CONSEMA" because the Project had no control over the approval process by CONSEMA. (ii)The target for "Roads rehabilitated, Non-rural" under the subcomponent 1.1 was revised from 700km to 690km to reflect the current state of the project. 6 Deleted “Improved navigability of the Tietê-Paraná waterway system as measured by increased use of Tiete river for freight” (from 1.5 million Tons/year to 2.2 million Tons/year) that was redundant with the current PDO indicator "Percentage increase of export products transported on the Tiete river". Both indicators were considered strongly correlated and covered the same outcome, “the efficiency of State's transport and logistics under the waterway transport dimension”. Page 53 of 53