ICRR 13132 Report Number : ICRR13132 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 07/24/2009 PROJ ID : P098299 Appraisal Actual Project Name : Mx Competitiveness US$M ): Project Costs (US$M): 300.8 300.8 Dpl Country : Mexico Loan/ Loan /Credit (US$M ): US$M): 300.8 300.8 Sector Board : FPD US$M ): Cofinancing (US$M): 0 0 Sector (s): General industry and trade sector (50%) General energy sector (20%) Ports waterways and shipping (10%) Central government administration (10%) Adult literacy/non-formal education (10%) Theme (s): Trade facilitation and market access (25% - P) Regulation and competition policy (25% - P) Export development and competitiveness (24% - P) Other accountability/anti-corr uption (13% - S) Small and medium enterprise support (13% - S) L/C Number : L7367 Board Approval Date : 03/14/2006 Partners involved : Closing Date : 06/30/2006 07/30/2006 Evaluator : Panel Reviewer : Group Manager : Group : Jorge Garcia-Garcia Rene I. Vandendries Ismail Arslan IEGCR 2. Project Objectives and Components: a. Objectives: The Programmatic Competitiveness Development Policy Loan (DPL) supported the first stage of the Government of Mexico (GOM)’s medium term program to improve Mexico’s lagging competitiveness. The loan supported faster growth and poverty alleviation in Mexico. Specifically, the loan aimed at: (a) Building on past competitiveness efforts, strengthen new and existing programs, and promote a holistic, cross-governmental approach to improving productivity in order to accelerate growth and increase export and employment levels; and (b) Contributing to the formulation of a medium -to-long term competitiveness policy agenda and help lay a solid foundation of competitiveness programs in advance of the administration that took office in December 2006. With the interventions the loan is expected to : (i) Promote business innovation; (ii) Improve training and education levels of the work force; (iii) Improve regulation to facilitate new business entry; (iv) Reduce transactions costs for businesses and individuals; (v) Strengthen the economic competition policy; (vi) Streamline trade procedures and lower trade costs; (vii) Facilitate customs clearance; (viii) Improve the quality of port services; and (ix) Implement transparency measures and regulatory instruments in the energy sector . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The loan supported policy and institutional reforms in : A. Innovation, technology development, and training (to achieve objectives i-ii). B. The investment climate (to achieve objectives iii-v). C. Logistics and trade facilitation (to achieve objectives vi-viii). D. Energy (to achieve objective ix). E. In addition, a satisfactory macroeconomic framework would have to be maintained for the Bank to be able to proceed with a second loan The interventions consisted of eight prior actions that were met . Component A . Innovation, technology development, and training Prior action 1. The loan required that (a) CONACYT (National Science and Technology Council ) launch and carry out the AVANCE program (Program of High Value Added in Business with Knowledge and Businessmen ); and (b) the Secretaria de Economia (Ministry of Economy) create a national business incubation system (SNIE) and launch an internet portal for it in Spanish and English . Prior action 2. The action consisted of Secretaría del Trabajo y Previsión Social (Ministry of Labor and Social Security) and the National Institute for Adult Education (INEA): (a) carrying out an evaluation of the Training Support Program (PAC) between 2002 and 2004; and (b) issuing at least 200,000 secondary education certificates to the out-of-school population aged 15-34 years between January and December 2005. Component B . The investment climate Prior action 3. The Federal Regulatory Improvement Commission (COFEMER) would: (a) evaluate at least 355 Regulatory Impact Assessments in 2005; and (b) put in place the rapid business opening system (SARE - sistema de apertura rápida de empresas - quick system to start a business ) in at least 85 municipalities. Prior action 4. To promote homogeneity of anti-trust criteria the Federal Competition Commission (CFC) would execute separate cooperation agreements with 15 selected states, COFEMER, the Energy Regulatory Commission (CRE), and the Federal Consumer Protection Agency (PROFECO). Component C . Logistics and trade facilitation Prior action 5. Secretaría de Economía would: (a) reduce by between 3 and 10 percentage points the duty on a total of 9,000 tariff lines (about 75 percent of items in the Borrower’s tariff schedule); (b) simplify the import requirements for export enterprises; (c) implement Operation Paperless in the Comprehensive Foreign Trade System (Sistema Integral de Comercio Exterior-SICEX); and (d) implement the Logistics Practices Innovation Program in 92 businesses. Prior action 6. The Administración General de Aduanas (Customs) would establish an Inter Secretarías Information System with the purpose of establishing inter -connectivity to link public entities involved in customs procedures, which program includes : (a) incorporation of Pitex, Maquila, and PROSEC information; and (b) incorporation of quota administration information . Prior action 7. The Secretaría de Comunicaciones y Transportes would: (a) move the control of the Federal Preventive Police (PFP) and the Attorney General (Procuraduria General de la Republica – PGR) outside port areas; (b) install scanners to expedite cargo processing in major commercial ports; (c) improve coordination among governmental agencies (Customs’ headquarters and local offices and Secretariat of the Environment and Natural Resources); (d) modernize customs facilities with new buildings and equipment (video and dynamic scales); and (e) carry out a study on efficiency and productivity in the port of Veracruz . Component D . Energy Prior action 8. The Borrower, through its Secretaría de Energía, would: (a) integrate an inter-institutional regulatory accounting working group (consisting of Secretaría de Energía, Federal Electricity Commission, Luz y Fuerza, and Energy Regulatory Commission ) to implement the measures established in the action plan to improve regulations aimed at improving competitiveness in matters of energy published in Diario Oficial de la Federación on August 11, 2005; (b) issue a status report on quality and performance indicators for the electricity sector; and (c) create a program for the evaluation of performance indicators for PEMEX . Component E . Satisfactory macroeconomic framework . The program document did not define what it considered satisfactory. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The loan was disbursed upon effectiveness on July 18, 2006 and closed on July 30, 2006, one month later than expected. 3. Relevance of Objectives & Design: The loan sought to help Mexico improve its competitiveness in the medium term . The Mexican economy lacks competitiveness because its enterprises operate in a high cost environment caused, among others, by little innovation, low educational levels of its work force, high costs of doing business, barriers to trade -tariffs, administrative, port infrastructure, and high real costs of providing energy . The relevance of this objective was substantial. The loan supported prior actions that in most cases would reduce the real costs of operating a business, and whose short-term impact could be observed and measured (i.e., reductions in tariffs, changes in port management). In some cases, like research and development and procedural conditions in the energy sector, the real cost reductions might happen in the medium -term or not at all. While most prior actions seemed relevant to achieve the objectives, the program document fell short of looking deeper into the problems, their causes, possible solutions, and how much they cost to Mexico . For example, the program document notes that social rates of return to innovation are 60 percent, much higher than the private rates of return, but did not explore why this is so. Two questions arise immediately: (a) what caused such distortion?; (b) is more government expenditure on research the answer to the problem rather than tackling the sources of the distortion? Regarding regulations, the program document quotes that the cost of regulation reaches 15 percent of GDP but does not connect that cost or some of its components to the actions that the program supported . The program document is rich in stories for each policy area, but the stories lack a clear message . At the end, the reader is left wondering: (a) what is the theory or process of change behind the stories; (b) what facts are the relevant ones; (c) how much does each problem cost; and (d) what is the expected gain from solving each problem? 4. Achievement of Objectives (Efficacy): Component A . Innovation, technology development, and training ii) (Objectives i -ii) Objective i: Promote business innovation. It is too early to associate results to the effort of promoting business innovation, but several intermediate indicators were achieved . First, total research and development (R&D) expenditure increased from 0.46 percent of GDP in 2005 to an estimated 0.53 percent of GDP in 2008; the contribution of the private sector to it increased from 42 percent in 2004 to 44 percent in 2008. Both indicators surpassed their target values . Second, the consortia operating under AVANCE increased from 2 at end of 2004 to 9 at end of 2008. Fifty three enterprises participated in the consortia at the end of 2008, surpassing the target of 24 set for 2009. Third, by the end of 2008 a total of 268 technology-based projects had attracted private investment, surpassing the 160 target set for 2009. Not all targets were achieved. Regarding the number of researchers in businesses, only 81 had been incorporated, below the target of 500 set for 2007 and of 1000 for 2009. The evidence from these intermediate indicators suggests that the objective was achieved . Objective ii: Improve training and education levels of the work force . The evaluation showed that the PAC had design and monitoring problems, which led the government to discontinue the service; as a result, less than 3,000 enterprises received training in 2008 and the target of 10,000 for 2009 will not be achieved. Missing the target was not a shortcoming because the program added little value and had design and monitoring problems . Regarding the goal of issuing secondary education certificates, the program exceeded the numerical target of 200,000 certificates issued, but INEA does not collect information on employment and wages of its graduates, which does not permit to evaluate impact . The evidence suggests that the objective was partially achieved . Component B . Investment Climate (Objectives iii -v) Objective iii: Improve regulation to facilitate new business entry (for ICR, reducing the cost of doing business ). The program established as result indicator that COMEFER would evaluate at least 60 and 70 percent of the regulations issued in 2007 and 2009 by the regulatory agencies . The ICR does not quote numbers but notes that the regulatory agencies submit all their draft regulation proposals to COMEFER for its assessment and approval before they are issued in the Official Gazette . While the numerical objective may have been achieved (i.e., percent of proposals reviewed by COMEFER ), there is no evidence that: (a) the quality of regulation has improved; (b) the number of regulations has decreased (to be expected if COMEFER analyzes the proposals thoroughly); (c) the cost of doing business has declined; (d) the benefits of regulation exceed their costs; or (e) more companies are formed as a result of the improved regulations . The review concludes that there is no evidence to prove or infer that regulation improved, and considers that the objective has been partially achieved . Objective iv: Reduce transactions costs for businesses and individuals (for ICR, facilitate business entry ). The prior action was met, and 137 municipalities had adopted the system by December 2008. As a result of SARE the number of days spent opening a business in these municipalities fell from 52 to three. While the numerical and procedural objective has been achieved, there is no evidence showing direct impact (e.g., absolute and percentage reduction in costs, the objective of the program ), let alone the ulterior impact of the action (e.g., number of businesses opened or the additional investment caused by the fall in costs ). The review concludes that there is no evidence showing that regulation improved; and considers that the objective has been partially achieved. Objective v: Strengthen the economic competition policy . The prior action was met, and was possibly reinforced by a law approved by the Mexican Senate amending the Federal Law on Economic Competition and giving more power to CFC to punish monopolistic practices . While the procedural actions were met, no evidence is presented to show that competition has increased or that CFC has encouraged competition . The ICR notes that Mexico’s antimonopoly policy fell in the world ranking of competitiveness and effectiveness; it also notes that the share of fines collected in 2007 was less than 7 percent of the value of fines issued . The review agrees with the ICR conclusion that the objective was not achieved. viii ) Component C . Logistics and trade facilitation (Objectives vi -viii) Objective vi: Streamline trade procedures and lower trade costs . The prior actions were carried out but the evidence on whether they produced the expected results is contradictory . For example, importers and exporters had to fill fewer documents in 2008 than in 2006 (five instead of six and eight ), and Mexicans seem to have more freedom to trade in 2009 than in 2006 (according to The Heritage Foundation indicator ). Despite these improvements the cost of importing and exporting a container increased, and it costs at least twice as much in Mexico as it costs in Brazil, Chile and Colombia . Lacking better information, the review concludes that the objective has been partially achieved. Objective vii: Facilitate customs clearance . The prior actions were carried out and Customs has improved some of its procedures and information systems, but the objective of inspecting merchandise faster was not met . In 2005 customs inspected 84 percent of the cases in less than three hours, but in 2007 it only inspected 80 percent of the cases in that time; moreover, that percentage falls below the 90 percent target set for 2009. The review concludes that the objective was not achieved. Objective viii: Improve the quality of port services . The prior actions were carried out but there is insufficient information in the ICR to conclude that the quality of port services improved . Time in port declined but Mexico fell in the international ranking of port infrastructure . The time a container stays in the four main ports --Manzanillo, Altamira, Veracruz, and Progreso--fell from 32 hours at the end of 2004 (no data for 2006) to an average (weighted by containers handled ) of 29 hours in the first half of 2007. According to the Global Competitiveness Report Mexico lost competitiveness in its port infrastructure; it ranks 94 out of 134 countries, three steps lower than in 2007. The review concludes that the objective was partially achieved. Component D . Energy (Objective ix ) Objective ix: Implement transparency measures and regulatory instruments in energy sector . Progress on the six outcome (intermediate) indicators set for this objective was limited . Specifically, the government (a) created the regulatory accounting framework but executed less than one third of it; (b) Secretaría de Energía published only 10 of the 20 performance and quality indicators; (c) all permit holders comply with the LP gas regulation; and (d) there is no information on the other outcome indicators (e.g., whether contracts comply with the terms and conditions of the agreement concerning the delivery and receipt of petroleum products, and the first -hand sale of petroleum products). The review concludes that the objective was not achieved. Component E . Satisfactory macroeconomic framework Mexico has had a strong macroeconomic performance during 2006-08. Annual inflation hovered around 5 percent, GDP grew at 3.2 percent per year, and the average current account deficit has been below one percent of GDP. Good macroeconomic management explains this good performance . The government has maintained fiscal balance and reduced its p ublic debt, including public external debt, and Banco de Mexico has adopted an inflation targeting regime that works well, developing strong anti -inflationary credentials. Primarily as a result of the good macroeconomic environment the government decided to not continue with the second programmatic competitiveness DPL. The program did not define indicators to evaluate whether the macroeconomic framework was satisfactory but this review concludes that the authorities managed macroeconomic policies well, and that this objective was fully achieved . 5. Efficiency (not applicable to DPLs): ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The DPL sought relevant objectives but the efficacy of its results was modest . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The risk that the government turns back the changes reviewed above is significant . The government’s record on business regulation is mixed. A first indication of these risks comes from the analysis in the ICR, which shows the slow pace of reform in the areas the loan supported and Mexico ’s fall in the ranking of global competitiveness in some of the areas selected for reform (e.g., ports). A second indication of the vagaries of reform in business regulation comes from an index of business regulation produced by the Fraser Institute for its index of Economic Freedom of the World . The index shows an uneven performance : regulation improved between 1995 and 2000, deteriorated in 2001, improved in 2002 and 2003, deteriorated sharply in 2004, and improved in 2005 and 2006. Moreover, the appetite for results -oriented reform seems low as attested by the modest results in three loan components that affect competitiveness directly, i .e. energy, investment climate, and logistics and trade facilitation . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: The Bank carried out analytical work and a substantive dialogue with the government in support of its competitiveness agenda. This work prepared the Bank well both to design a coherent operation and to select indicators to measure intermediate and long -term impacts of the changes the loan supported, but the Bank used its knowledge partially. The program document reflects the effort done to learn the issues but its diagnosis of the problems and the outcome indicators selected fall short of what could have been expected from the effort done. The program document presents lots of information but fails to explain in a clear and concise manner the nature of the problem, its causes, its costs to the country, and the incentives that lead to sub-optimal outcomes. Three examples illustrate some of these problems . First, the document quotes a study that found social rates of return to innovation in Mexico of 60 percent, some 6-10 times the private returns, but does not explain the reason for such large divergence . Is the reason excessive regulation, too little budget for R&D, or something else? Lacking that explanation the reader wonders if the proposed solution to the problem (i.e., the prior actions) is the correct one. Second, one of the prior actions consisted of reducing tariffs for 9,000 items in the tariff schedule, with the reductions varying between 3 and 10 percentage points. It is not possible to know how much costs fall because the program document does not provide the baseline values for the level and dispersion of tariffs; as a result, it is difficult to judge if the prior action constituted a significant change in trade policy . Third, the program document shows that electricity prices have increased since 1999 and that tariffs in Mexico are higher than in other areas of the world, but does not explain how much of that high price results from true, real, economic costs and how much from distortions and bad management . The review concludes that Bank performance in ensuring quality at entry is moderately satisfactory . The loan was disbursed upon effectiveness and did not require supervision . For this reason, this review does not rate quality of supervision . at -Entry :Moderately Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Not Applicable c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Government performance : The government worked closely with the Bank to design the Competitiveness DPL and to develop a strategy for competitiveness . It shares with the Bank the merits and demerits in the design of the program and in the development of the strategy . Therefore, this review rates government performance moderately satisfactory. Implementing agencies performance : The agencies carried out the prior actions required in the program document and several of them worked closely with the government and the Bank in designing the program and developing a strategy for competitiveness . The agencies did not collect the information required to determine the impact of the program. This review rates implementing agency performance moderately unsatisfactory . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Unsatisfactory c. Overall Borrower Performance :Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: In most cases the program used intermediate and process indicators to monitor and evaluate impact . Except for the evaluation of the PAC, which led to the program's closure, there was little effort to gather information to analyze the impact of the loan, such as measuring the reductions in costs to businesses . Overall, the quality of M&E was modest. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately The ICR considers that the program Satisfactory Unsatisfactory produced better results in investment climate and energy than this review does. Risk to Development Moderate Significant The ups and downs in the recent Outcome : history of regulation in Mexico indicates that reform in this area is fraught with risks. The ICR overlooks this recent history. Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately The prior actions were carried out, but Unsatisfactory the agencies did not collect the information required to determine impact of the program. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: An economy gains competitiveness when its real costs of producing goods and services fall . When the Bank supports a program that seeks to improve competitiveness and neglects to point out the reductions in costs arising from the program, a flaw might be present in the diagnosis of the problem, the design of the program, or the indicators selected to evaluate impact . Adopting procedural conditions is unlikely to lead to visible and immediate outcomes, but they do not preclude thinking about desirable medium or long -term outcomes that could help strengthen the design of the project . The prior actions in the electricity sector exemplify this situation, whereby setting a pre -determined goal for the sector might have helped evaluate the potential effectiveness of prior conditions . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR presents a good summary of the project ’s design, implementation, and achievements . It has two shortcomings. First, it identifies project achievements with Mexico ’s competitiveness in world ranking rather than the direct impact from the government's actions . Comparing with other countries misses the point of looking at what Mexico did and achieved in its quest to reduce costs and become more competitive; it also overlooks the need to compare the situation before and after (e.g., costs of handling containers in 2004 and 2007). Second, by including the triggers for DPL 2 in the discussion of performance the ICR makes the reader lose perspective of what it is evaluating and what outcomes it associates with DPL 1. If the ICR wanted to discuss the triggers to show the Borrower’s commitment to the program it should have done it in the section on Borrower performance . If it wanted to show that the design of DPL 1 was appropriate, it should have discussed them in the section on program design. Last, since DPL2 was dropped this review does not see the necessity of the paragraph in section 2.4. a.Quality of ICR Rating : Satisfactory