Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Report Number: ICRR0022225 1. Project Data Project ID Project Name P095485 AR Metropolitan Areas Urban Transport Country Practice Area(Lead) Argentina Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-77940,IBRD-87000 29-Feb-2016 194,679,985.45 Bank Approval Date Closing Date (Actual) 20-Oct-2009 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 150,000,000.00 0.00 Revised Commitment 194,679,985.45 0.00 Actual 194,679,985.45 0.00 Prepared by Reviewed by ICR Review Coordinator Group Elisabeth Goller Kavita Mathur Victoria Alexeeva IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The overall development objective of the this adaptable programmatic loan (APL) was to improve the quality and sustainability of urban transport systems in Argentine metropolitan areas, through the improvement of sectoral decision making frameworks and by giving priority to public transport modes in the urban transport sector (PAD, para 31). The project development objectives (PDO) of this first loan (APL1), as stated in the loan agreement (LA), were to: (i) support the design and creation of a multijurisdictional Metropolitan Transport Agency (MTA) for Page 1 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) the Buenos Aires Metropolitan Area (AMBA); (ii) strengthen the institutional capacity of the transport authorities in decision making, planning, priority setting, and resource allocation in urban transport; (iii) improve the quality and performance of urban transport infrastructure and/or services in medium-size metropolitan areas; and (iv) improve the physical integration and access to public transport networks in AMBA. In this review, the achievement of the PDO of APL1 will be assessed and each of the four sub-objectives will be assessed separately. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components Component 1: Creation of a MTA for the Buenos Aires Metropolitan Area (estimated costs US$28.5 million, without contingencies; estimated cost according to the 2017 additional financing (AF) with contingencies US$11.14 million; actual cost US$11.21 million - Note: the Bank task team provided the estimated costs according to the 2017 AF and the actual costs by project end in this section). This component was to strengthen the capacity of transportation planning and management at the metropolitan level and consisted of the following sub-components: 1.1 Agency Structuring: to develop the legal and institutional frameworks for the new MTA; 1.2 Consolidation of Transportation Planning in Buenos Aires to: (i) improve AMBA's urban transport model, (ii) prepare AMBA's transport master plan, (iii) conduct feasibility studies and design studies for a major multijurisdictional project in AMBA, and (iv) transport planning data collection and updating; 1.3 Urban Transport Observatory: to create an observatory to collect, process, and maintain all urban transport and air quality databases; 1.4 Integrated fare system (Sistema Unico de Boleto Electronico - SUBE): to facilitate the creation of a single fare media payment technology across all transportation modes in AMBA; and 1.5 Staff and Operational Expenses: to finance necessary staff, office and other operational expenditures of the MTA for the duration of the project. Component 2: Urban Transport Improvements in Argentina’s Medium-Size Metropolitan Areas (estimated costs US$62.90 million without contingencies; estimated cost according to the 2017 AF with contingencies US$22.56 million; actual cost US$20.05 million). This component was to enhance the mobility conditions of selected Argentina’s medium-size metropolitan areas (Mendoza, Posadas, Tucumán, Córdoba and Rosario) by financing the implementation of priority projects or feasibility studies and final designs for projects to be implemented in the second APL. It included the following sub-components: 2.1 Metropolitan Area of Mendoza: to pave the Colector Papagayos; 2.2 Metropolitan Area of Posadas: to construct a segregated busway on Uruguay Avenue; 2.3 Metropolitan Area of Tucumán: to pave and provide storm water drainage for Barrio 11 de Marzo; 2.4 Metropolitan Area of Córdoba: to prepare studies for Mitre Intermodal transport terminal; 2.5 Metropolitan Area of Rosario: to prepare studies for High Quality transit for the Corridor Norte- Sur Project; 2.6 Technical assistance: to finance feasibility and/or design studies or other studies supporting interested cities in attaining eligibility criteria for the program; and 2.7 Page 2 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Strengthening Environmental and Social Management Capacity: to finance institutional capacity strengthening of participating metropolitan areas in environmental and social management. Component 3: Public Transport Access and Modal Integration in AMBA (estimated costs US$53.6 million without contingencies; estimated cost according to the 2017 AF with contingencies US$220.92 million; actual cost by project end US$221.74 million). This component was to improve accessibility and mode integration of the public transport network in AMBA and consisted of the following sub-components: 3.1 Transfer centers: to prepare detailed design studies for improvement infrastructure works in three intermodal centers of AMBA; 3.2 Station Accessibility: to carry out works to improve the access conditions around train stations in the AMBA; 3.3 Grade separated crossings at critical railroad intersection; and 3.4 Train signaling System: to design the best strategy to improve the train signaling system. Component 4: Sectoral Training: Urban Transport Planning (estimated costs US$2.5 million without contingencies; estimated cost according to the 2017 AF with contingencies US0.65 million; actual cost by project end US$0.65 million). This component was to design a postgraduate program (masters) in urban transport, including all the necessary subjects, such as engineering, planning, economics, etc. It consisted of the following sub-components: 4.1 Institutional and Academic Development: to prepare the institutional and academic design of the course and to obtain accreditation by the Ministry of Education; 4.2 Implementation of the course: to finance education fees and a limited amount of scholarships for students; and 4.3 Teacher training program: to finance the teacher training program for the teaching staff that was expected to participate in the masters course. Component 5: Project Management (estimated costs US$4.6 million without contingencies, estimated cost according to the 2017 AF with contingencies US$9.10 million; actual cost by project end US$10.72 million). This component was to finance the project implementation unit to carry out the proposed operation. Several activities under components 1, 2, and 3 were revised in the 2014 restructuring as follows: Component 1 formally remained unchanged but its cost was reduced to reflect that the MTA was created utilizing a different model. The ICR para 26 mentions that the urban observatory under subcomponent 1.3 was replaced by the creation of the urban transport planning office and the contract involving the supervision of the integrated fare system (SUBE) supervision under subcomponent 1.4 was dropped in 2012 because of allegations of fiduciary irregularities. The Bank task team pointed out that the urban transport planning office is not the same as the observatory, which was expected to centrally collect and use urban transport data. However, the underlying objective of both was to strengthen urban transport planning. Under component 2, infrastructure improvements originally envisaged in Mendoza, Posadas, and Tucuman were dropped primarily due to delays in the initiation of the civil works. Instead, the component was revised to finance high impact infrastructure projects in four medium-size cities, including smaller scaled bus rapid transit (BRT) projects in Santa Fe and Rosario, and two bike lanes in the cities of Tucuman and Salta. Technical assistance was to be provided to various other medium-size cities, such as origin destination (O-D) surveys, updating of planning tools, and preparation of project portfolios, and to strengthen their environmental and social management capacity. Page 3 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Under component 3, the originally envisaged activities, such as transfer centers, station accessibility, grade separated crossing and train signaling systems, were maintained, but their location was modified. In addition, two emblematic works were included, which were the La Matanza BRT and the Saenz transfer station (ICR, para 28). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost was US$264.86 million, which is 141 percent of the appraisal cost estimate of US$187.60 million and in line with the 2017 AF cost estimate. Financing: The project was expected to be financed through two IBRD loans of US$150 million (original financing) and US$45 million (AF), respectively. These loans were nearly fully disbursed (the actual disbursements were US$149.68 million and US$45 million). Borrower Contribution: The expected borrower contribution at appraisal was US$37.60 million (PAD, datasheet). The ICR does not include the borrower contribution. The Bank task team pointed out that the actual borrower contribution was US$70.17 million. Dates and Project Restructuring: The project was approved on October 20, 2009, became effective on November 26, 2010. It was expected to close on February 29, 2016, and was extended four times for a total of 46 months. It closed on December 31, 2019. The project had five level-two restructurings and an AF accompanied by restructuring. The first restructuring took place on August 18, 2014, when an amount of US$47.23 million of the loan amount was disbursed. This restructuring (i) changed project activities (see section 2.d above), (ii) adjusted the implementation arrangements, (iii) reallocated funds between components and disbursement categories, and (iv) changed the results framework. The changes to the results framework included replacing the 12 original PDO indicators with five new ones and changing most of the intermediate outcomes or targets. The second restructuring took place on September 29, 2015 to extend the loan closing date by 11 months, to January 31, 2017, to enable the completion of a new activities introduced with the 2014 restructuring and account for project implementation delays. The third restructuring took place on September 15, 2016 to (i) extend the loan closing date by nine months, to October 31, 2017, to account for delays caused by adjustments in the civil works scope, and (ii) to reallocate funds between disbursement categories because of cost overruns. The AF took place on January 6, 2017, when US$92.76 million of the loan amount were disbursed. The AF (i) added US$45 million to finance cost overruns under the Matanza BRT contracts because of scope changes and a higher bid price than budgeted, (ii) reallocated funds between components and disbursement categories, and (iii) added two additional PDO and one intermediate indicators for two activities introduced with the 2014 restructuring. The fifth restructuring took place on April 8, 2019 and increased the Bank financing percentage for civil works to 100 percent to accelerate project implementation. The last restructuring took place on June 21, Page 4 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) 2019 to (i) extend the project closing date by six months, to December 31, 2019, to complete the Saenz transfer station and (ii) reallocate funds between disbursement categories. Need for Split Rating: The PDO-level indicator changes introduced with the 2014 restructuring reduced the number of indicators and replaced most of them. Some of the new PDO indicators seem less ambitious, such as the indicator "Existence of a legal agreement for creating the AMT, signed interjurisdictionally", which replaced the indicator "AMBA MTA legally created, adequately staffed according to the organization chart designed in the project, with adequate offices and equipment, and its own operating budget". In addition, the original and revised PDO indicators are not fully adequate to measure the PDO, several of them are output indicators in nature, hence the PDO indicator changes introduced in 2014 do not have a significant bearing on the achievement of the PDOs. The 2017 AF was to cover a cost overrun and introduced two new PDO indicators, which did not increase the project's level of ambition but improved results measurement. Therefore, this review will not apply a split rating and will assess the four sub- objectives based on the revised indicators. 3. Relevance of Objectives Rationale Alignment with Strategy: The PDO remained highly relevant under the FY2019-2022 Country Partnership Framework, which in area 2 aims at addressing key institutional constraints for better governance and service delivery and in area 3 aims at supporting Argentina to implement its National Determined Contributions (NDC). The latter is to be implemented through (i) improving metropolitan coordination, (ii) strengthening decision-making frameworks, policy formulation, and planning capacities within the urban transport sector, (iii) improving the quality and performance of critical transport infrastructure, addressing a sector that directly supports and enables the urban economy and contributes to strengthen domestic markets, and (iv) improving access to public transport networks, tackling congestion and building key transit infrastructure to curb private vehicle use and lay a path towards more sustainable public transit (ICR, para 36). The PDO is also fully aligned with the government's 2016 resolution (ICR, para 38). Country Context and Previous Sector Experience: The PDO was outcome-oriented in substance and appropriately pitched for Argentina's development status and capacity. In the light of the longstanding engagement of the Bank in the urban transport sector, the balanced share of institutional, capacity, and infrastructure related subobjectives was adequate. The project continued what had been achieved in previous projects, including extending its scope to medium-size cities (IEG PPARs- Buenos Aires Urban Transport Project and GEF Sustainable Transport and Air Quality Program, 2017). Given the PDO's close alignment with the Bank and country strategy, and its appropriate pitching for a country like Argentina, the relevance of objectives is rated high. Rating Relevance TBL Rating High Page 5 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Support the design and creation of a multijurisdictional MTA for the AMBA. Rationale The theory of change for subobjective 1 was that the activities related to the creation of and support to the multijurisdictional MTA would have as output a legally created MTA. In terms of outcomes, this was to lead to a functioning MTA carrying out coordinated planning of major multijurisdictional infrastructure and services restructuring, including infrastructure provision and bus and subway service restructuring and multimodal integration, especially fare integration, smart card compatibility, and subsidy and uniform tariff policies (PAD, paras 38 and 60 and annex 1). Outputs  MTA (Agencia de Transporte Metropolitano, in Spanish) legally created with decrees of 2012, 2013, and 2014, meeting the revised target. The MTA was originally envisaged as agency, technical secretariat, or body with adequate staff, equipment, and an operational budget. The project subsequently realized that it was more realistic to set up a tripartite arrangement to carry out metropolitan transport planning and coordination and formalized this change with the 2014 restructuring. The Bank task team clarified that the legally created coordination arrangement for AMBA consists of a representative of the municipal, provincial, and national governments and an executive director. The government representatives, the executive director, and the staff of the planning office are paid by the public administrations of their origin. AMBA's executive director chairs the MTA meetings and leads the urban transport planning office set up within the national government. The planning staff in this office are officials of the municipal government of Buenos Aires and the national government. Although such an coordination arrangement does not comply with the originally envisaged formalities, as seen under the outcomes below, it carried out the key functions of a metropolitan coordination body.  Statute of internal MTA organization, meeting the revised target.  MTA representatives appointed, meeting the revised target.  The Bank task team pointed out that the project supported a study on the legal arrangements for the MTA, provided international best practices on metropolitan transport authorities, and supported a study to further upgrade the MTA. The team also clarified that the studies to inform the MTA's work (2014 PP) consisted mainly of technical assistance to support MTA's planning capacity listed under sub-objective 2 below and its decisions making during project implementation. Outcomes Page 6 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) In terms of functionality, the MTA held 19 meetings between 2016 and 2019, meeting the revised project target. The task team explained that the MTA held regular meetings since its establishment, but in the absence of a project indicator, which was introduced in 2014, did not provide the Bank with information on these meetings. It is important to highlight that the MTA continued to regularly meet even after the recent government change, which brought the national and municipal governments under different political parties. The Bank task team clarified that since its establishment the MTA facilitated the introduction of targeted subsidies and integrated tariff policies in AMBA. The MTA approved the AMBA transport master plan and prioritized the civil works to be implemented by each jurisdiction in AMBA. The MTA was in charge of the two AMBA investments with inter-jurisdictional participation under the project, exceeding the target that was to provide evidence on the MTA's performance. The MTA representatives took decisions on BRT extensions, bus reform, bus fleet renewal, and the metropolitan rail service extension in AMBA. However, not all these decisions were necessarily formalized as MTA decisions because the decisions were also part of the representatives' other functions and hence this route was more straightforward. The urban transport planning office, chaired by the MTA's executive director, has been in charge for all metropolitan transport planning in AMBA. Although AMBA's MTA did not comply in form with what was originally planned, by (i) carrying out metropolitan planning, (ii) coordinating the interjurisdictional urban transport infrastructure and service provision, and (iii) facilitating several key transport policy measures, it carried out the functionalities of an MTA. Given that the MTA sustained its coordination functions beyond the government change and continues to move the metropolitan coordination agenda forward, the efficacy of subobjective 1 is rated substantial. Rating Substantial OBJECTIVE 2 Objective Strengthen the institutional capacity of the transport authorities in decision making, planning, priority setting, and resource allocation in urban transport. Rationale The theory of change for subobjective 2 was that the activities related to improving and producing urban transport planning tools and establishing an urban transport planning master course would have new and improved planning tools and trained staff as outputs. In terms of outcomes, this was to lead to the use of the planning tool and an enhanced planning and urban transport management capacity. Outputs  AMBA transport model developed and updated in 2017, with data from the integrated fare system SUBE and the new O-D surveys, and technical staff of the urban transport planning office trained to use and improve the model. The Bank task team pointed out that the model was of unprecedented complexity for Argentina and the update innovative. Page 7 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485)  Institutional Strengthening Plan for Mobility and Transport in the city of La Rioja prepared, which included (i) a diagnosis of the public transport situation in the city, (ii) an action plan and investment priorities, and (iii) the training needs for municipal staff.  Integral Urban Mobility Study for the Metropolitan Area of Salta prepared, which included (i) a diagnosis of mobility patterns in the metropolitan area, (ii) a list of investment projects, (iii) a sustainable mobility plan, (iv) the prefeasibility design of a priority BRT project, and (v) training for Salta's new metropolitan transport authority staff. The production of these three planning instruments is in line with the revised target of three planning instruments.  O-D household surveys in eight metropolitan areas produced, which exceeds the revised target of 6 O-D surveys. These surveys provided information on travel patterns, including trip motives, transport modes used, perceptions on public transport, as well as socio-economic information. They were used as basis to design the project interventions (ICR, para 42).  Studies and designs for BRT projects in Rosario, Santa Fe, and Salta, for bike lanes in Salta and Tucuman, and for the first phase of an integrated public transport system in Corrientes carried out, meeting the revised target of a portfolio of six projects developed (ICR, para 43).  Masters course in urban transport planning in line with the original and revised targets, which the Bank task team helped design in one of the most prestigious universities in Argentina, bringing together three different faculties. This is in line with the target. According to the Bank team, the course was sponsored by the project through scholarships to participants in 2014, 2015, 2016. The next course is scheduled to take place in April 2021. Outcomes The project enhanced the transport planning capacity in AMBA and medium-size cities through mobility data collection activities, which helped start evidence-based policy making in these cities (ICR, para 62). According to the ICR (para 41), the AMBA transport model became the key tool for the urban transport planning office to carry out investment planning at metropolitan level. This office used the model to prepare the fare integration policy, the economic evaluation of several transport interventions, and the rationalization of bus routes in the AMBA. According to the Bank task team, the preparation and update of the AMBA transport model and the establishment of the urban transport planning office created a planning capacity within the government, which is new in Argentina. The team stressed that this office used the model to prioritize the two projects implemented under this project, but also to carry out analyses for other government units and policies. This included the bus fleet rationalization policy, smart card fare analysis for cities outside Buenos Aires, and analyses for urban projects that required the estimation of the transport demand in an area, such as for the transit-oriented development project in AMBA. The latter was publicly announced, had the zoning law changed to support it, but has not yet been implemented because of the economic slowdown in Argentina. With respect to the Integral Urban Mobility Study for the Metropolitan Area of Salta, the Bank task team pointed out that this planning exercise not only enhanced the capacity of the city's planning staff but also Page 8 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) provided them with credibility vis-a-vis their superiors. Based on the plan, the city constructed a bike lane with project funds. The plan was also the basis for the city's implementation of a bus reform, pedestrianization, and a BRT system. The latter was not exactly implemented as planned, but it was inspired by the study. The Bank task team mentioned that the Institutional Strengthening Plan for Mobility and Transport for the city of La Rioja was less successful, but because of the collaboration under the project, a city official attended the master course and is now the city's urban mobility leader. In Santa Fe, the BRT project helped implement an integrated urban development corridor with a BRT system, bike lane, vegetation for flood control, and measures to incentivize denser development and mixed land use. The city used their local design for the corridor, and the project assisted in improving it. Therefore, although there were no formal capacity strengthening activities in this city, important capacity building took place through learning by doing. The Bank task team also confirmed that the city officials and university staff who worked in the project were still working in areas related to urban mobility in Santa Fe, some in different positions. The transport agencies in the metropolitan areas used the O-D household surveys as baseline to design the Bank-financed interventions (ICR, para 42). The Bank task team added that the O-D survey data generated under the project helped place the cities on a level playing field in the bus service negotiations with private bus operators, who previously were the only ones to have mobility data. Similarly, according to the Bank task team, the preparation of project studies and designs helped create institutional capacity in transport authorities through the technical discussions on project elements. The team pointed out that, for instance, in the city of Corrientes, the designs of a transfer station and a segregated bus lane, and the redesign of the bus circulation patterns spurred lots of discussions and consequential learning. The BRT system design in Santa Fe helped create a vision around this project. In Misiones, the preparatory activities for a BRT system created knowledge even if the project was not implemented because the mayor took too long to take a decision. During the three years in which the project provided scholarships, the master course in urban transport planning trained 94 students, of which 60 received scholarships. The Bank task team clarified that several of these students worked in AMBA and many others in municipalities. According to an estimate by the Bank task team, at least half of them remained in their positions after the government change. Because of the project's data collection activities, planning tools, capacity building through urban mobility projects, and the masters course in urban transport planning helped strengthen the institutional capacity of the transport authorities, the efficacy of subobjective 2 is rated substantial. Rating Substantial OBJECTIVE 3 Objective Improve the quality and performance of urban transport infrastructure and/or services in medium-size metropolitan areas. Page 9 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Rationale The theory of change for subobjective 3 was that the activities related to designing and constructing urban transport infrastructure in medium-size cities would have road improvements and new public transport and non-motorized transport (NMT) infrastructure as outputs. In terms of outcomes, this was to lead to better quality urban transport infrastructure and reduced generalized costs of travel*. Outputs  BRT systems and associated improvements in the corridor, such as sidewalks, lighting, and bike lanes, in Santa Fe and Rosario of a total of 8.2 km, carried out. This exceeds the revised target of 7.8 km.  Bike lanes and associated improvements, such as lights and sidewalks, in Tucuman and Salta of a total of 8.8 km (the project had no target for bikeways) implemented. Outcomes The BRT systems in Rosario and Santa Fe greatly improved the bus service performance by reducing travel times and enhancing the quality of the public transport user experience (ICR, para 46). The systems reduced the generalized cost of travel by 13.34 percent in Santa Fe and by 14.30 percent in Rosario, exceeding the targets of 10 percent (para 45, ICR). The Bank task team pointed out that this type of infrastructure not only facilitated the flow of buses, but also helped organize the traffic in the corridor, enhanced walkability, and highlighted the importance of road safety measures. The Bank task team did not have data on public transport ridership in Rosario and Santa Fe and mentioned that the impact of these systems on ridership was likely small because the BRT corridors were short, and it is difficult to shift transport users to other modes. The two BRT systems were among the first BRT projects outside AMBA and, according to the Bank task team, helped showcase this urban transport solution to other medium-size cities. The Bank task team mentioned that although Rosario did not implement a planned second phase of the project corridor because of the low transport demand, it continued with the BRT solution. The ICR (para 46) pointed out that the project provided an additional sustainable transport solution for medium-size cities and helped advance NMT solutions, which were almost nonexistent in medium-size metropolitan areas in Argentina. The ICR, para 46, considered the NMT agenda even more relevant under the COVID-19 crisis and mentioned that the project experience could serve to further extend these initiatives in the country. The Bank task team pointed out that particularly in Salta bicycle use increased, but they did not have data. The project enhanced urban transport infrastructure quality and its performance in terms of travel speed, accessibility and safety features. It also helped create awareness for the BRT solution in medium-size cities in Argentina. Therefore, the efficacy of subobjective 3 is rated substantial. *Note: The Bank task team clarified that the generalized cost of travel multiplies travel time saving in the corridor by the number of people using the BRT corridor and the value of time. Rating Page 10 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Substantial OBJECTIVE 4 Objective Improve the physical integration and access to public transport networks in the AMBA. Rationale The theory of change for subobjective 4 was that the activities related to improving public transport infrastructure and introducing integrated ticketing in AMBA would have as output improved integrated transport infrastructure and ticketing. In terms of outcomes, the improved integrated infrastructure was expected to reduce the generalized cost of travel and increase the ridership on the La Matanza BRT corridor, reduce the travel time at the Saenz transfer station, and facilitate public transport access. Outputs  La Matanza BRT corridor constructed (actual 11.3 km against the revised target of 10 km), including street widening, 40 bus stops, ramps, sidewalks, 17 stations, 1,390 planted trees, signaling, passenger information, lighting, and transfer center.  Elevated Saenz transfer station, in line with the revised civil work's completion target of 100 percent.  25 train stations in AMBA with improved surroundings, including paved or repaved accesses to stations, access ramps for people with disabilities, new or reconstructed sidewalks and pedestrian walkways/crossings to facilitate access and transfers, improved train signaling systems, grade- separated crossings in critical railroad intersections, public lighting, and upgraded transfer points between transport modes improved bus stops and bike shelters, among others. This is in line with the original and unrevised target of 100 percent completion.  Integrated fare system SUBE introduced in AMBA in 2018 and in 15 other cities (the support for the integrated fare system SUBE in AMBA was cancelled from the project scope in 2012, but according to the ICR, para 51, the project played a key role in pushing this agenda in its initial phases, supporting analytical work and providing strong technical assistance and international experiences on integrated fare/ smartcard systems). Outcomes The public transport travel speed in the La Matanza BRT corridor increased by over 105 percent, from an average travel speed of 12 km/hour to 25 km/hour, which is substantial but realistic for a segregated corridor in a suburban environment. This increase in travel speed represents a reduction in the average time of travel from 45 minutes to 23 minutes. The generalized cost of travel went down by 34 percent, exceeding the revised target of 10 percent. The ridership during the first year of implementation increased by 4.8 percent and according to the Bank task team corresponds to about 300,000 new passengers. Although this is an important achievement by international standards, it is significantly below the target of a 10 percent. The ICR, para 48, points out the benchmark used to set this target and its applicability to the La Matanza corridor were questionable. The Bank task team clarified that the La Matanza neighborhood was a very poor area with people already using public transport, therefore, the margin for modal shift was low. Page 11 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) In terms of integration, the Bank task team highlighted that the La Matanza BRT starts at a transfer center in the outskirt of the city, which was built as part of the project. The station links two main roads, the railways, a small informal bus station, and a motorcycle parking area. The BRT corridor has 16 intermediate stations, which provide integration with local bus lines. The elevation of the railways at the Saenz transfer station led to 53 percent travel time saving, largely exceeding the revised target of 10 percent (ICR, para 49). The Bank task team explained that this was previously a very busy at grade rail crossing, hence the large travel time savings were due to the elevation of the station. The team also pointed out that the Saenz station currently provides for bus to train transfer for a significant number of bus lines that operate on Av. Saenz. The team pointed out that in future, the Saenz transfer station will enable the connection between the rail, buses, and the planned terminal station of the subway line H, thus will provide excellent connectivity to the city center by rail. The planned subway station was approved by the legislative as part of the subway extension plan. The train station improvements in AMBA enhanced the overall accessibility to these stations (ICR, para 50). The integrated fare system SUBE had a direct impact on promoting modal and fare integration among different public transport modes in AMBA, resulting in an overall reduction of passengers' travel cost. It improved access to public transport because the integrated fare system SUBE provided a 50 percent discount on the second public transport trip and a 75 percent discount on further trips, mainly benefiting the poorest users who undertake longer trips with more transfers. Similar benefits took place in the other 15 cities in Argentina, to which system was extended (ICR, para 51). Overall, the project improved physical and fare integration and enhance access to public transport in AMBA. Therefore, the efficacy of subobjective 4 is rated substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project helped establish a functioning metropolitan transport coordination mechanism in AMBA, strengthened the planning and decision making capacity of transport authorities, improved the quality and performance of urban transport infrastructure and services in medium-size metropolitan areas, and improved the physical integration and access to public transport in AMBA. The achievement of all four subobjectives is rated substantial. Therefore, the overall efficacy is also rated substantial. Overall Efficacy Rating Substantial Page 12 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) 5. Efficiency Economic Analysis At appraisal, cost benefit analyses were carried out for the investments in Tucuman, Posadas, and Mendoza under component 2 (all dropped later) and the grade separated rail crossings under component 3. The analyses used a 22-year time horizon (2 years of construction and 20 years of operation) and a 12 percent discount rate. The benefits considered in each case were different. They included (i) savings in vehicle operating costs, (ii) savings in public transport operating costs, (iii) reduction in travel times for all users, public transport users, pedestrians and cyclists, and (iv) benefits due to enhanced accessibility to health care facilities, schools, and work places. The benefits were quantified using (i) values of the vehicle operating cost methodology of the National Road Directorate of Argentina, (ii) average salaries, and (iii) estimates of inaccessibility days to urban facilities, among others (PAD, para 83, and annex 9). For the station accessibility improvements under component 3, no cost benefit analysis was carried out, but the stations to be intervened were selected based on a multicriteria analysis, including the poverty index, number of passengers boarding and alighting at the stations, and the relative importance of the corresponding train line (PAD, para 83). For Tucuman, the analysis showed a net present value (NPV) of US$14.73 million and an economic internal rate of return (EIRR) of 23.1 percent. For Posadas, it showed a NPV of US$37.41 million and an EIRR of 16.8 percent. For Mendoza, it showed a NPV of US$ 32.17 million and an EIRR of 60.8 percent, and for the grade separated railways crossings, it showed a NPV of US$1.23 million and an EIRR of 13.4 percent. The sensitivity analyses showed that the economic evaluations were robust since no investment yielded a negative NPV (PAD, annex 9 and paras 4 and 5). The 2017 PP includes the economic analyses for the La Matanza BRT and the Saenz transfer station. The benefits considered were time savings for public transport and private vehicle users and road safety improvements. The analyses showed a NPV of US$46.82 million and an EIRR of 17.5 percent for the La Matanza BRT and a NPV of US$ 115.00 million and an EIRR of 42.6 percent for the Saenz transfer station (the PP has slightly different figures for the latter in the text and a table). The ICR, para 54, however, mentions that at appraisal an economic evaluation was only carried out for the Santa Fe BRT, which showed a NPV of US$9.3 million and an EIRR of 22 percent. By project close, ex-post cost benefit analyses were limited to the three most cost intensive investment for which the necessary data was available. This included the (i) Santa Fe BRT, (ii) Rosario BRT, and (iii) La Matanza BRT. These analyses used the same time frame and discount rate as in the ex-ante analyses, the actual investment costs with a revised value of time and time savings, and updated vehicle operating costs. The benefits comprised of time and vehicle operating cost savings. The analyses also looked at road safety and emission reduction benefits, but these benefits were not quantified (ICR, para 55 and annex 4). For the La Matanza BRT, the ex-post analysis showed a significant higher return, with a NPV of US$206 million and an EIRR of 50 percent. This compares to a US$46.82 million and an EIRR of 17.5 percent estimated in 2017. According to the Bank task team, the high return was due to the unexpectedly large travel time saving, which as mentioned in the Efficacy section seems realistic for this kind of infrastructure. For the Santa Fe BRT, the ex-post analysis showed a NPV of US$10.60 million and an EIRR of 31 percent, compared to an ex ante NPV of US$9.30 million and an economic internal rate of return of 22 percent. For the Rosario BRT, the Page 13 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) analysis showed a NPV of US$10.20 million and an EIRR of 62 percent. No ex-ante analysis is presented in the project documents. The Bank task team explained that for the Saenz transfer station no cost benefit analysis was carried out by project close because the station opened a week before loan closing, and it was difficult to get the necessary data or economic analysis from the government staff, which had recently changed. Administrative and Operational Efficiency The project's efficiency suffered from a 46-month-extension of the implementation period, more than doubling of the project management costs, and significant civil works cost overruns for the La Matanza BRT and the Saenz transfer station, which required an additional financing of US$45 million. The cost overruns were caused by high inflation, design adjustments to increase the contract scope requested by the new administration, and a bid price significantly higher than the budget estimate for the La Matanza BRT. According to the ICR, para 56, the design adjustments resulted in more paving, including of the two lanes for general traffic, better quality pavements to increase resistance, and road safety measures. Although the administrative and operational efficiency of project implementation was modest, the three main project investments showed very high returns. Therefore, on balance, the efficiency of project implementation is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 34.30 Appraisal  28.50  Not Applicable 64.60 ICR Estimate  47.70  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project's relevance of objectives is rated high. The project had minor shortcomings in efficacy and efficiency, which are both rated substantial. Therefore, the overall outcome is rated satisfactory. a. Outcome Rating Satisfactory Page 14 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) 7. Risk to Development Outcome The ICR (paras 108 to 110) does not identify specific risks to development outcomes, but simply mentions that sustainable urban mobility remained a key priority for the country and that the government and the Bank were discussing a potential new urban transport operation. Based on discussions with the Bank task team, IEG identified the following risks to development outcomes:  Metropolitan coordination in AMBA. The MTA has continued to meet under the new government after project closing. This government is from a different political party than the previous one. The government asked the Bank for technical assistance to help evolve the MTA under a possible new loan under discussion. Therefore, the risk that the MTA, as metropolitan urban transport coordination mechanism, will stop operating is low. There is a substantial risk that the MTA will not be transformed into a fully autonomous body because this would require that the three jurisdictions in AMBA transferred their responsibilities and powers to it.  Urban transport planning in AMBA. The urban transport planning office has become the metropolitan transport planning body in AMBA. The office and staff remained in place after the government change. The Bank task team reiterated that having a metropolitan urban transport planning capacity in the government is new in Argentina. According to them, the capacity is highly valued, and the new administration has been fighting for it. As with government change, however, there is a risk that the progress in metropolitan planning might not continue at the pace it progressed under the project.  Planning in medium-size cities. The Bank task team pointed out that they still have contacts with some of the cities involved in the project. This is, for instance, the case with Salta, where the city has continued to strengthen the transport team. For the other cities, especially where the project established a less solid planning base, there might be a risk to the sustainability of the project's efforts.  Masters course in urban transport planning. The masters course established under the project continued to be offered even without the project's scholarships, so there is a low risk to its sustainability.  Public transport integration at the Saenz transfer station. There is a moderate risk that fiscal constraints might delay the extension of the subway line to this station, hence also delay its full integration potential.  Infrastructure maintenance. Fiscal constraints could negatively impact the maintenance of the infrastructure built under the project, but the BRT design used requires very little maintenance because it was conceived as "cheap system to operate". Similarly, the bikeways do not require major and costly maintenance.  Public transport demand. There is a risk that public transport demand might decrease, but the corridors intervened have low demand elasticity, which reduces this risk. Page 15 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) 8. Assessment of Bank Performance a. Quality-at-Entry The Bank task team intervened in a highly relevant sector for the economic and social development of major cities. The team designed the project as a natural continuation of the ongoing operations in the sector expanding its scope to medium-size cities. The use of the APL instrument was meant to further promote continuity in the Bank's involvement in urban transport in Argentina. The Bank's previous engagement in the sector was beneficial in terms of implementation arrangements because the existing project implementation unit was familiar with Bank policies and procedures and had the necessary capacity. In addition, funds of the ongoing project could be used to finance project preparation. This ensured that the technical, financial, and safeguard preparatory work was carried out adequately and timely (ICR, para 72). These funds were also used to preselect the medium- size cities, where the project works under component 2 were to take place (ICR, para 72). Although a combination of institutional and capacity strengthening with urban transport infrastructure and service performance improvements and the expansion of the project scope to medium-size cities was appropriate from a technical point of view, the original project design was overly complex, with many small interventions involving different and new implementation agencies. In addition, some of the preselected medium-size cities did not have the capacity and technical resources to justify the investments in terms of clarity of conceptual designs, economic analyses, and social acceptability of the interventions (ICR, para 98). The Bank task team ensured that social, environmental, and fiduciary frameworks were in place at the time of project approval (ICR, para 100). The team identified most implementation-related risks, such as limited capacity of municipalities, fiduciary and procurement risks, cost overruns, and problems with interjurisdictional coordination (PAD, para 78). However, the gravity of several of these risks was underestimated, leading to an overall risk rating of moderate. This is especially the case for setting up a MTA, which is difficult and hence highly risky. Not all risk mitigation measures proposed were sufficiently effective. For instance, the training workshops and other measures to cope with the limited capacity of medium-size cities were insufficient to avoid a need for project changes and implementation delays. In addition, the planned procurement-related mitigation measures were not sufficient to avoid bid prices significantly exceeding cost estimates in major civil works, which contributed to increase in project cost. As mentioned below, the project had shortcomings in M&E (for details, see M&E section). Mainly because of the shortcomings in the original design, which had to be drastically changed, and the weaknesses in risk severity identification and mitigation, Bank performance in ensuring quality at entry is rated moderately unsatisfactory. Quality-at-Entry Rating Moderately Unsatisfactory Page 16 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) b. Quality of supervision During the project's lifetime, the Bank task team was a key partner of the government on urban transport issues and supported a broad range of policy issues (ICR para 60). The team leveraged resources to carry out additional activities and studies, which helped, among others, (i) foster the Ministry of Transport Planning Secretary's digital data capacity, by supporting methods for data recollection through cell phones to better understand travel patterns, including of non-motorized transport, (ii) enhance road safety under the La Matanza BRT, and (iii) introduce interest for transit-oriented development in Argentina (ICR, paras 67 to 69). The team also carried out analytical work that contributed to key strategic policy discussions, including tariff and subsidy policies, and bus sector reform (ICR, para 106). The ICR, para 103, points out that the project team categorized the project as “problem project" in the initial implementation period due to long delays, low disbursements, and red flags in terms of project governance. In response to the governance challenges, the Bank task team carried out a forensic audit, a fiduciary action plan, and technical assistance as needed. This response helped mitigate obstacles in project implementation and achieved the PDOs, especially after the early 2016. The Bank task team also managed to adjust the project to the changing context and circumstances, especially through the 2014 restructuring. This was a turning point in project execution, and from 2014, the overall performance of the project improved considerably in terms of performance ratings and disbursements. As mentioned, the team carried out five additional restructurings, including the 2017 AF (ICR, para 75 and 76). The 2014 restructuring, however, had shortcomings. It did not fully adequately revise the results framework (see M&E section). The 2014 PP did not include the economic analysis for the new investments. The 2017 PP included the economic analyses for the La Matanza BRT and Saenz Station, two of the investments identified through the 2014 restructuring. The 2017 PP reports only limited methodological details on the economic analysis and includes slightly different figures of the NPV and EIRR for the Saenz transfer station. According to the ICR, para 102, continuous technical supervision of civil works, particularly for the BRT constructed in the complex environment of La Matanza, was a key factor to ensure the quality of this investment. The ICR, para 104, also points out that the governance issues related to the integrated fare system SUBE put in question the Bank's contract supervision. However, the Bank's Integrity Department (INT) found no wrongdoing of Bank staff. Finally, the ICR, para 105, mentions a TTL high turnover under the project, four in 10 years, but does not indicate that this had negative consequences on project supervision. The borrower expressed appreciation about the close collaboration between the government counterparts' and Bank task team (ICR, annex 5). In summary, the Bank task team used this project as a platform to provide broader support to the urban transport sector in Argentina. The team carried out an adequate supervision effort under a project that suffered from serious design deficiencies, lasted ten years, had different government counterparts and TTLs, and, during its initial years of implementation, took place in a difficult fiduciary environment. The above-mentioned shortcomings mainly in terms of restructuring are minor. Consequently, Bank performance in supervision is rated satisfactory. Page 17 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project was to be monitored by the participating cities under the overall supervision of the project implementation unit. This unit was also expected to carry out data collection, analysis, and evaluation, and it was said to have the necessary capacity (PAD, annex 3). The original results framework included 12 PDO and nine intermediate indicators. All indicators had baselines and targets, except for the two indicators measuring user satisfaction, for which the baselines were to be determined through a user survey in year one. The PAD, annex 3, provides some information on how results were to be measured. The PDO indicators, however, were not fully adequate to measure the project's achievement. Several of them were output indicators in nature, such as "metropolitan areas with O-D surveys" and "transport plan completed". Others were not necessarily fully attributable to the project, such as "percentage of primary school students absent on low transitability days" and "number of appointments in in community health centers cancelled on low transitability days". The PDO indicators were also closely linked to the project activities. Therefore, once certain project activities were changed, the indicators were no longer appropriate. The 2014 restructuring replaced the 12 PDO indicators with five new ones, two for subobjective 2, and one for the other subobjectives. It also changed most of the intermediate outcomes or targets. The 2017 AF and restructuring added two additional PDO and one intermediate indicators to improve results measurement for two activities introduced with the 2014 restructuring. The revised indicators had shortcomings. Although all indicators had baselines and targets, two targets were largely overachieved. This might be an indication that the targets were not sufficiently ambitious. Several other targets were obviously conservative, such as the ones measuring the number of planning instruments produced and the evidence of the MTA performance. The target for the increase in public transport passengers on the La Matanza BRT corridor, on the other hand, was highly ambitious compared to international standards for modal shift. In addition, many of these baselines and targets were in percentage terms only. At least for some of them, to specify also the absolute numbers, such as travel time in minutes or the public transport users in numbers, would have been useful, including to judge their level of ambition. The PPs do also not include a definition of the generalized cost of travel nor a description of the methodologies to measure Page 18 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) results. Several of the revised PDO indicators remained output indicators in nature. They also remained closely linked to project activities. Finally, the measurement of certain subobjectives would have benefited from additional PDO indicators, such as to improve the quality and performance of urban transport infrastructure and/or services in medium size metropolitan areas, which was only captured through the generalized cost of travel on the BRT corridor in two cities. b. M&E Implementation The project implementation unit's dependence on data from the participating cities in the project caused difficulties in M&E implementation, particularly because the cities originally selected were modified during implementation (ICR, para 80). According to the Bank task team, these difficulties were caused by delays in signing the agreements between the project implementation unit and the cities. These agreements set out the obligations of the latter, including on M&E. It was also difficult to have cities comply with these agreements in terms of data provision, especially once the civil works were completed and their incentives gone. To make things worse, during project implementation there were relationship frictions between the national government, i.e. the borrower, and the cities. For the four biggest project interventions, the project had adequate baseline and follow-up surveys to measure the project indicators (ICR, para 81). Indeed, although the cities did not provide the monitoring data in the agreed form and on time, they provided the information on the impact of the main project works towards project end. Finally, the Bank task team pointed out that in Santa Fe the project tried to measure the impact of the BRT system on road safety, commercial activities, and other aspects. The project created the baseline with the indicators, but Santa Fe did not measure results. The project carried out a similar exercise for the La Matanza BRT. c. M&E Utilization The ICR, para 82, mentions the project’s M&E system was not designed and implemented to inform and improve project implementation, hence it was not useful as feedback mechanism. The Bank task team also mentioned that cities did not care about project data, and after measuring the initial travel time savings, they did not see the utility to measure it each year. M&E Quality Rating Modest 10. Other Issues a. Safeguards Page 19 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) The project was classified as category B for environmental assessment purposes. It was expected to have positive environmental impacts in the long run, such as reduced congestion and emissions and improved road safety. The civil works were expected to have short-term negative impacts, which were considered mitigatable (PAD, para 98). No involuntary resettlements were expected, except for the investments in Mendoza (PAD, para 102). The following safeguards policies were triggered: Environmental Assessment OP/BP4.01, Physical Cultural Resources OP/BP 4.11, and Involuntary Resettlement OP/BP4.12. The project implementation unit prepared an environmental and social management plan, which included a resettlement policy framework. The cities, where the civil works took place, prepared environmental and social impact studies and abbreviated resettlement plans, as needed (ICR, paras 84 and 85). The ICR does not report any details regarding the environmental performance of the project. It only mentions that the "project maintained a satisfactory rating in the application of environmental and social safeguards, undertaking all required public consultations and adequately handling complaints from affected stakeholders as they arose (ICR, para 88)." The environmental performance of the project in the last four implementation status reports (ISRs) was rated satisfactory. In the previous two ISRs, it was rated moderately satisfactory. The ICR, para 88, mentions that the establishment of a solid social and environmental capacity within the Secretary of Transport Works was an institutional strengthening outcomes facilitated by the project. In terms of social safeguards, in the case of the Matanza BRT, 11 informal vendors were adequately resettled. The process lasted over three years and resulted in better working conditions for most vendors. According to the last monitoring report (2018), six vendors were using the new commercial posts provided by the project, two intended to occupy their posts in the short run, one continued to use the original post, one moved to another zone, and one passed away. The three vacant commercial posts were given to other vendors from the same area (ICR, para 86). For the Saenz transfer station, the project identified four informal vendors to be affected by the civil works, but two of them disappeared before the start of the works. The remaining two vendors were relocated, also resulting in better conditions, particularly given the increased movement of passengers at the station. A temporary resettlement plan was implemented in a satisfactory manner. The last monitoring report (2019) indicated that the income of these vendors was not affected at any point during the process. The vendors were offered relevant training, which was extended to other vendors in the area. The project implementation unit's social team implement a communication plan to accompany the civil works at the Saenz transfer station given their temporary impact on the circulation of passengers (ICR, para 87). The involuntary resettlement compliance was rated moderately satisfactory in the last six ISRs. b. Fiduciary Compliance The project had several fiduciary issues. According to the ICR (para 89), intellectual property rights and financial management assessments in 2012 revealed (i) contracts of staff working for the project pending signature by the Minister for two years, hence the respective expenditures were considered ineligible, (ii) suspected case of collusion in a survey contracts, where the price more than doubled compared to the cost estimate, (iii) concerns related to ongoing works in five train stations, and (iv) systematic deficiencies in cost estimates, among others. The project also had instances of double billing Page 20 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) for office furniture, leading to disbursements exceeding the overall cost of the contract (ICR, para 90). In addition, there were irregularities in the supervision contract of the fare integration system SUBE, which led to an INT case ending in sanctions (ICR, para 91). In response to the allegations of these irregularities, the Bank conducted a forensic audit, which covered all Bank projects implemented by the same project implementation unit. This audit confirmed irregularities and possible double billing for about US$7 million, but only US$281,000 were related to this project. The project implementation unit staff was changed in 2012. The new staff prepared and implemented a comprehensive fiduciary action plan. As a result, their performance improved significantly (ICR, paras 92 to 94). Financial management and procurement were rated satisfactory in the last five ISRs. The Bank task team pointed out that these fiduciary issues related to the first part of the project implementation period. From 2013 onwards, the project's fiduciary performance hugely improved, with the application of strict procedures and staff training. The project provided timely interim financial reports and financial audit reports. The latter had unqualified opinions and very little observation. In 2015, as reported, the fiduciary performance improved. c. Unintended impacts (Positive or Negative) The project was able to obtain trust fund resources to carry out complementary studies and technical assistance, which helped, among others, (i) foster the Ministry of Transport Planning Secretary's digital data capacity, by supporting methods for data recollection through cell phones to better understand travel patterns, including of non-motorized transport, (ii) enhanced road safety under the La Matanza BRT, and (iii) introduced interest for transit-oriented development in Argentina (ICR, paras 67 to 69). d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Shortcoming in M&E design, Quality of M&E Substantial Modest implementation, and use Quality of ICR --- Substantial 12. Lessons The following have mostly been derived and summarized from the ICR, with minor additions by IEG: Page 21 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Scattered small interventions have higher transaction costs than focused major investments. The original project design envisaged many small, geographically scattered interventions in different cities. These interventions had higher transaction costs to implement, were more difficult to supervise, and had limited impact. For instance, the works to improve the access conditions around 25 train stations in AMBA demanded lots of discussions, required a large supervision effort, and had limited impact compared to the La Matanza BRT and the Saenz transfer station. At least in Argentina's political and economic context, it seems that concentrating efforts on larger, emblematic projects is a better strategy in rallying support and political buy-in for a project. It is also more efficient in reducing Bank transaction costs in terms of procurement, safeguards, and supervision tasks. This does not mean that mean that focused small interventions are not important. For projects aimed at smaller cities, without previous Bank experience, addressing their technical and capacity constraints upfront and reinforcing the project-agreed implementation mechanisms might increase the likelihood of success and sustainability. Under this project, some of the cities originally expected to participate in the project did not have a strong technical expertise, so were not able to justify the project benefits and provide the necessary data. They had also no experience with World Bank procedures. Therefore, it was very difficult to move their projects forward. Therefore, project preparation for these cities has to be comprehensive, with detailed agreements on details and capacity strengthening in advance. Contract flexibility, transparency, and overall good governance is central to project implementation. Under this project, the flexibility of the Bank task team to adapt to a changing and challenging implementation context was critical to promptly take measures to turn around a problem project with deficiencies in its original design and deliver the expected development outcomes. The measures taken to respond to fiduciary irregularities, including the forensic audit and fiduciary action plan, significantly improved fiduciary performance in the later years of project implementation. The introduction of new BRT projects needs to be adapted to the context of the city, the institutional capacity of the local government, and the bus operators' business model. The discussions on the main characteristics of this type of transit infrastructure and the alternatives to structure the service provision led to a BRT design, which differs from the standards set by the Transmilenio BRT systems in Bogota, Mexico, and Brazil. The Argentine BRT model, (i) has low- floor doors on the right side, which provides more flexibility and integration with the rest of the bus system outside the BRT corridor, (ii) has longer and open stations, different from the closed station with ticket pre-payment typical in Bogota, which provide a perception of enhanced safety to passengers and lower maintenance costs, and (iii) rationalizes the existing bus services, but does not require the restructuring of the operators' business model, hence making continuity easier. Although this model foregoes some efficiency gains in service provision, (i) it had a huge success with municipalities with limited capacity and resources to maintain and operate a more complex system, (ii) had the support of bus operators that benefited from the reduction of vehicle operating costs, and (iii) was welcomed by users for the travel time and safety improvements. Therefore, the Argentine BRT model supported by the project might be a valid model for many cities worldwide exploring alternatives for sustainable mass transit infrastructure. The following is an additional lesson added by IEG: Page 22 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) Using PDO indicators closely linked to project activities reduces the flexibility to change project activities and makes results measuring difficult. Under the original project design, many PDO indicators were closely linked to specific project activities, such as "Availability of public bus service along the Papagayo corridor in Mendoza" to measure the subobjective of improved quality and performance of urban transport infrastructure and/or services. Therefore, when several project activities were replaced, such as in this case to improvements in the Papagayo corridor, the whole results framework had to be changed. A more appropriate indicator could have been one that measured improved quality aspects of public transport in general. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is thoroughly done. It is longer than the recommended 15 pages, but this is largely justified due to the project complexity, duration, and multiple changes. The ICR includes very useful details and summary tables with the project scope and indicators changes. On page 6, the ICR presents the theory of change, reconstructed based on the information in the PAD. The theory of change shows the project activities, outputs, subobjectives, program PDO, and long-term outcomes in respective columns. The long-term outcomes correspond to the Country Partnership Strategy priorities. It also lists the critical assumptions. The theory of change does not include the expected outcomes captured by the PDO indicators, which nearly completely changed during project implementation. In terms of outcome analysis, the ICR clearly presents the relevance of objectives. The assessment of the achievement of the PDO is concise but focuses too much on reporting of outputs also because, as previously seen, many so-called PDO indicators were output indicators in nature. This assessment would have benefited from additional evidence on the achievement of the PDO not captured through indicators to make a more convincing case. The Bank task team filled the gaps to adequately assess the project outcomes. The ex-post cost benefit analysis is comprehensive, includes the necessary methodological information, and appears to be thoroughly done from a technical point of view. For instance, the value of time of public transport uses, for which no public information existed, was captured through an analysis of average salaries correlated to average GDP per capita. However, the reporting on the results of the ex-ante cost benefit analyses under the project was incomplete. It did also not include an ex-post cost benefit analysis for the Saenz transfer station because of lack of data. The ICR includes many well thought-through lessons, which are based on the project experience. In terms of other minor shortcomings, the ICR does not identify any specific risk to development outcomes, but explains in general terms that this risk was limited. It does also no report on actual total project costs but only on the project costs financed by the Bank. Page 23 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Metropolitan Areas Urban Transport (P095485) On balance, given its laudable aspects and shortcomings, the quality of the ICR is rated substantial. a. Quality of ICR Rating Substantial Page 24 of 24