ICRR 13268 Report Number : ICRR13268 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 04/01/2010 PROJ ID : P059161 Appraisal Actual Project Name : Introduction Of US$M ): Project Costs (US$M): 12.2 13.3 Climate Friendly Measures In Transport Country : Mexico Loan/ Loan /Credit (US$M ): US$M): 5.8 5.6 Sector Board : ENV US$M ): Cofinancing (US$M): 1.0 3.8 Sector (s): General transportation sector (60%) Sub-national government administration (20%) Central government administration (20%) Theme (s): Climate change (29% - P) Environmental policies and institutions (29% - P) Pollution management and environmental health (28% - P) Other urban development (14% - S) L/C Number : Board Approval Date : 10/29/2002 Partners involved : Shell Foundation, Closing Date : 06/30/2008 03/31/2009 Japan (PHRD), German and Canadian Trust Funds, Bus Manufacturers (Electrabus, Allison and Ankai-Faw) Evaluator : Panel Reviewer : Group Manager : Group : Payton Deeks Peter Nigel Freeman IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: From the GEF Grant Agreement: The objective of the Project is to develop policies, regulations and measures to make transport in the Metropolitan Area more efficient, less polluting and less carbon intensive than it is as of the date of this Agreement. From the PAD: The PDO is to contribute to the development of policies and measures that will assist in the long -term modal shift toward climate-friendly, more efficient and less polluting, less carbon intensive transport in the Mexico City Metropolitan Area (MCMA). The global objective of the program is to reduce the emissions of greenhouse gases from passenger ground transport in large metropolitan areas . As the GEF grant agreement is more specific this ICR Review will use the objectives from the grant agreement . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Component 1. Harmonization of sector strategies on air quality issues and Integrated Climate Action Plan for Transport (CAP) in the MCMA (US $0.8 million at appraisal; $0.5 million actual according to the text of the ICR; Annex 1 of the ICR says US$0.4 million appraisal; US$0.5 actual). Component 2. Definition of an enabling environment to facilitate the implementation of sustainable transport strategies (US$4.8 million at appraisal; US$2.6 million actual according to the text; Annex 1 says US$2.9 million at appraisal; US$2.6 actual). Component 2 included the support and promotion of non -motorized transport through a campaign. This campaign was not implemented due to time constraints . Component 3. Field Test of Climate -Friendly High Capacity Vehicles (US$4.8 million at appraisal; US$1.53 million actual according to the text; Annex 1 says US$1.50 million at appraisal; US$1.53 actual). Component 4. Technical assistance and training for incorporation of climate change and air quality considerations in the desi gn and analysis of transport strategies (US$0.8 million at appraisal; US$0.1 million actual according to the text; Annex 1 says $0.4 million at appraisal; US$0.13 actual). Component 5. Public Awareness and Dissemination (US$0.3 million at appraisal, $0.25 million actual according to the text; Annex 1 says $0.17 million at appraisal, $0.25 million actual). Component 6. Project Management (US$ 0.7 million at appraisal, $0.64 million actual according to the text; Annex 1 says US$0.43 million and $0.64 actual). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost. At appraisal, total project cost was estimated at US$ 12.2 million, financed by a GEF grant of US$ 5.8 million supported by a further US$ 6.4 million from other sources. At completion, the total project cost was US$ 13.3 million. Borrower and cofinancier contributions . The borrower contribution at appraisal was estimated at $ 2.4 million but the actual amount was US$0.9 million. This was compensated by additional cofinancing of US$ 0.9 million from the Shell Foundation, US$1.5 million from Japan PHRD, US$0.3 million from the Canadian Trust Fund, and US$ 0.1 million from the German Trust Fund. Local business groups (Electrabus, Allison and Ankai-Faw) contributed US$3 million equivalent as foreseen at appraisal . Dates. The project’s original closing date was 6/30/08, but the actual closing date was 9 months later at 3/31/09. The project was extended twice (first for 6 months, then an additional 3). The delay were attributed to procurement processes and procedures in respect of the studies . 3. Relevance of Objectives & Design: The project objectives were consistent with Mexico City ’s development priorities on transport and air quality . They were also consistent with the latest Country Partnership Strategy (2008), which identified developing infrastructure, strengthening institutions, and promoting environmental sustainability among its core themes, and with GEF ’s Operational Strategy and Program 11. The objectives were thus highly relevant . The project design was also highly relevant and supported project objectives . The project team incorporated lessons learned from Bank work on air quality issues in Mexico, including : focusing on a long-range approach; allowing for flexibility to cope with political and technological changes; taking a participatory approach; and capturing global and local benefits simultaneously (i.e., addressing local air pollution issues and also global climate change ). The project's results framework and indicators were satisfactory and reflected project objectives . The ICR does not contain information on risk assessment and mitigation . The implementation arrangements were fully adequate to address the mainstreaming of climate concerns into transport projects. Since the transport sector contributes negatively to air pollution and GHG emissions, it was an appropriate design decision to include the Ministry of the Environment for Mexico City in the project . 4. Achievement of Objectives (Efficacy): The PDO -- to develop policies, regulations, and measures to make transport in the metropolitan area more efficient, less polluting, and less carbon intensive than it is as of the date of the Agreement --was substantially achieved. Through the project the following was achieved : • Completion and publishing of the first citywide Climate Change Action Strategy in Latin America . This strategy included goals, timetables, program outlines, and budget requirements . • Development of a Mexico City Climate Change Action Program, which is the operational tool of the strategy . • Establishment of a protocol for measuring the performance of alternative bus technologies and also for measuring emissions reductions from the BRT . • Publishing of reports on technical subjects related to climate change and transport . The Climate Strategy, its attendant Program including this project, and the BRT provide the policies, regulations, and measures that contribute to a more efficient and less polluting transport system in Mexico City . Additionally, these measures are underpinned by studies and the development of measurement protocols . The ICR mentions minor shortcomings in that a campaign under Component 2 (the support and promotion of non-motorized transport) was not implemented due to time constraints and the project also did not do an analysis on which to base the future promotion of ridership of the Metro . 5. Efficiency (not applicable to DPLs): An incremental cost analysis was prepared during project preparation to estimate the cost of additional national actions required to ensure environmental sustainability . The baseline scenario assumed the continued investment and operation of diesel buses . The implementation of the baseline scenario entailed costs estimated at US$ 4.1 million, while the GEF alternative incurred costs estimated at US$ 12.2 million at appraisal. The incremental costs associated with the implementation of the project were thus US$ 8.1 million. GEF funded US$5.6 million of this as part of the project. There was no attempt to assess whether the incremental cost spent was an optimum figure in terms of the benefits achieved. Nevertheless, the enabling environment created paved the way for substantial efficiency savings in the implementation of BRT . Efficiency is rated as substantial . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: With relevance high and efficacy and efficiency both substantial, overall outcome is rated as satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk that the development outcomes of the project will not be maintained is low . The Borrower demonstrates full ownership of the local climate change strategy which the project helped to develop, and will likely ensure that development outcomes are sustained . The project also strengthened the enabling environment for a climate -friendly transport system and undertook studies and collected data that support such a climate . User surveys of the BRT indicate high satisfaction, time savings, and greater security . All of this indicates a low risk that the development outcomes will not be maintained. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: Quality at Entry was highly satisfactory . The Bank worked with the federal government, local government, and transport operators in Mexico City from project conception, ensuring client buy -in and helping to address barriers to implementation. The selection of an implementing agency with high capacity helped to achieve the development outcome. The Bank built on a long-standing, cooperative relationship with the Mexico City authorities and this project was mutually regarded as part of a long -term solution to air pollution in Mexico City . Lessons from earlier Bank experience in Mexico were considered during preparation (for example, that air pollution is a long-term problem needing long-term solutions; and that a participatory approach is necessary ). The Bank provided three grants in support of project preparation . The studies they financed defined the appropriate transport corridors for the project and strengthened the Borrower ’s institutional capacity. Studies financed by these grants identified the transport sector as the key user of fuels in Mexico City and the greenhouse gas (GHG) inventory allowed the project to set a baseline against which reductions could be measured . The Bank helped to mobilize other partner and co -financiers such as the World Resources Institute, Shell Foundation, bus manufacturers and fuel suppliers which was useful during implementation as it made available not only funding but also a range of technical expertise . The performance indicators were realistic and useful for assessing the progress towards achieving the project objectives. Quality of supervision was satisfactory . The Bank team visited the project approximately twice a year . In addition to the review of progress reports provided by the counterparts and to continuous interaction with them, the visits further helped the team to address issues pro -actively and in a timely manner, and to support the achievement of project objectives. The Borrower rates the Bank's performance as satisfactory, although concerns were expressed that transaction costs might be increased by Bank procedures, especially with regard to procurement . These concerns were accentuated by the fact that the Borrower was obliged to comply with the differing requirements of three entities-- the World Bank, BANOBRAS, and the city government . The Bank team was flexible and proactive, and played a key role in developing Mexico ’s first transport project incorporating carbon finance (which involved using financial benefits from emissions reductions ). Bank support was also beneficial for capacity building and dissemination activities . at -Entry :Highly Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government performance was highly satisfactory. Although the ICR contains no discussion of government performance, this was discussed with the project team . The Environment Ministry was reported as having performed "outstandingly well" and the Minister’s promotion of the project’s advantages and organization of a team of capable professionals were major contributions to the operation ’s success. The Transport Secretary also brought transport and environment experts together for the first time in Mexico . Further, the Secretary demonstrated commitment to the establishment of Metrobus, the replacement of microbuses, and the regulatory framework for the new transport system (BRT). Implementing agency performance was highly satisfactory. The performance of the climate change team inside the Ministry of Environment was excellent . They led the development of the climate change strategy and significant mainstreaming of climate change concerns into other sector programs . a. Government Performance :Highly Satisfactory b. Implementing Agency Performance :Highly Satisfactory c. Overall Borrower Performance :Highly Satisfactory 10. M&E Design, Implementation, & Utilization: M&E Design is rated substantial. The indicators were appropriate given project objectives . Some might have been complemented by quantitative gauges such as number of buses converted; baseline emissions and pollutant levels; reduced health costs, and other cost savings associated with improved transport . These data are provided but not included in the formal indicators. The data collected by the project indicate that obtaining air quality compliance for ozone and airborne pollutants offers health benefits of approximately US$ 2 billion per year. M&E Implementation is rated high. The indicators identified in Annex 1 of the PAD were collected and reported by the Project Implementation Agencies, discussed by regular supervision missions, and were the focus of the assessments conducted as part of a Mid-Term Review as well as the final end-project evaluations. The M&E system included reporting requirements by the implementing agencies on a frequent basis to the project team . This entailed the development of a Design and Construction Summary Report of the BRT, bi -annual operational summary reports, dissemination and training reports, and strategic documents developed as part of the project activities (final field test report; local climate change action strategy and program; emission reduction reports ). The reports were delivered on time and in a satisfactory manner to the Bank team . M&E Utilization is rated high as data were used to inform decision making . The data on emissions and pollutants were both used by policymakers in making their decisions regarding the future of the bus system . The data were also used to support alternative bus technologies . a. M&E Quality Rating : High 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards. The project was rated category C . It focused on climate change -related policy and regulatory reform and the development of a Metropolitan Climate Change Action Plan . The only activity with a potentially negative environmental impact was the comparative field test of buses . However, this involved established routes, did not require any new works and did not therefore involve resettlement . The only environmental issues were related to standard maintenance of the vehicles . Fiduciary. No financial reports are pending . All the audit reports were submitted and accepted by the Bank . Unintended consequences . The project provided the basis for a larger Bank operation, the nationwide Urban Transport Transformation Project . The nationwide project is expected to lead to a reduction of 2 million tons of carbon a year, which is equal to 10% of the total urban transport footprint . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Highly Satisfactory Satisfactory Given minor shortcomings in efficacy and efficiency, the project ’s outcome is rated as satisfactory. Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Highly Satisfactory Highly Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Bank knowledge and previous experience in a sector or country should be leveraged to produce a strong design. The involvement of many stakeholders adds to project complexity but also ensures greater adoption and buy-in. Political opposition can be mitigated by vesting implementing agencies with the authority to work at a technical level across the boundaries of state and other jurisdictions where opposing political parties may be involved . For example, in this case, while political opposition between the parties in power in Mexico State and Mexico City respectively threatened to hamper the project, one commission was given technical level jurisdiction in both zones, a factor which assisted the achievement of the project ’s objectives. A well-designed and implemented M&E system is a valuable aid to policy makers . The success of a project can be greatly enhanced by an implementing agency with high capacity The creation of an enabling environment for public transport – including appropriate regulatory, institutional and technical frameworks, is an essential precondition for realizing substantial gains in environmental management and performance. Among the major benefits of environment -enhancing transport projects are improved public health and reductions in health care expenditures . These benefits should be identified and quantified wherever possible . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is generally well-written and for the most part thorough . The discussion of Government performance is, however, largely absent and had to be obtained through feedback from the project team . Descriptions are sometimes too detailed and not pithy enough, and a more robust efficiency analysis would have been welcome . a.Quality of ICR Rating : Satisfactory