ICRR 13230 Report Number : ICRR13230 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 10/23/2009 PROJ ID : P101311 Appraisal Actual Project Name : Gt Dpl Iii US$M ): Project Costs (US$M): 100 100 Country : Guatemala Loan/ US$M ): Loan /Credit (US$M): 100 100 Sector Board : EP Cofinancing (US$M ): US$M): 0 0 Sector (s): Central government administration (40%) General finance sector (35%) Aviation (10%) General industry and trade sector (10%) Sub-national government administration (5%) Theme (s): Standards and financial reporting (29% - P) Public expenditure financial management and procurement (29% - P) Infrastructure services for private sector development (14% - S) Regulation and competition policy (14% - S) Tax policy and administration (14% - S) L/C Number : L7482 Board Approval Date : 08/02/2007 Partners involved : Closing Date : 12/31/2008 12/31/2008 Evaluator : Panel Reviewer : Group Manager : Group : Jorge Garcia-Garcia Kris Hallberg Ismail Arslan IEGCR 2. Project Objectives and Components: a. Objectives: The Third Programmatic Development Policy Loan (DPL3) was part of a programmatic series of three DPLs intended to support the fundamentals and three main pillars of the government’s development plan “Vamos Guatemala�. Specific objectives of the program were: (a) promoting growth and improving the investment climate; (b) enhancing capacity for public spending in priority sectors; and (c) achieving greater transparency in public sector management . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The DPLs focused on the three main policy areas that encompass the three objectives . The loans had prior actions associated with each policy area . DPL3 had 10 prior actions. In addition, it had the general condition of maintaining a stable macroeconomic framework . The program covered eight sub-topics distributed among the three objectives as follows : I. Promoting Growth and Strengthening Investment Climate 1. Macroeconomic stability 2. Promotion of trade expansion 3. Promoting investment and business efficiency 4. Strengthening Infrastructure through private sector participation 5. Strengthening, modernizing and deepening of the financial sector In a section of the program document this objective was stated as promoting equitable growth and strengthening the investment climate. II. Enhancing the capacity for public spending in priority sectors 6. Improvement in tax collections 7. Improving budget allocations III. Transparency and Public Sector Management 8. Improving transparency and efficiency of public resource use d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The loan was disbursed in its entirety . It was approved in August 2, 2007, and became effective in May 29, 2008. 3. Relevance of Objectives & Design: The program documents sought relevant objectives and the prior actions mapped to the objectives were likely to cause the achievement of the objectives . The loans supported changes in areas where policies and the administration of the state constituted a barrier to grow faster and reduce poverty more effectively. Main constraints to growth were the incentives for the private sector to develop fully, the low level of human capital, the inefficient use of government resources, and the inadequate volume of government resources supporting the expansion of human capital . The loans sought to help alleviate such constraints. The program documents fell short in identifying good results indicators for some of the objectives of the program; the review will point the deficiencies when discussing achievement of objectives in section 4. The relevance of objectives and design was substantial. 4. Achievement of Objectives (Efficacy): I. Promoting Growth and Strengthening the Investment Climate 1. Macroeconomic stability The program documents for the three loans did not set standards for this objective . IMF staff reports of Article IV consultation (March 2007, June 2008) conclude that macroeconomic stability has been maintained despite tensions in the financial sector in late 2006 and early 2007. The reports also note that the rate of growth in 2006-2008 averaged 5.2 percent, higher than the historical average. Despite a current account deficit of 5 percent of GDP the central bank accumulated international reserves equivalent to about 4 months of imports and 100 percent of public external debt. The fiscal position remained solid, with the central government contained and the public debt estimated at about 22 percent of GDP. Annual inflation, which had averaged 7 percent in 2006-2008, fell to less than 2 percent in 2009 as a result of the fall in commodity prices and weak domestic demand . This review concludes that efficacy in achieving this objective has been substantial . 2. Promotion of trade expansion To achieve this objective the program supported the government’s decision to participate in the DR-CAFTA trade agreement and to simplify and harmonize procedures to conduct trade with El Salvador and Honduras. The program documents selected as medium-term outcome indicator an increase of 9 percent in the dollar value of nontraditional exports after the DPL program ended . The ICR notes that nontraditional exports grew 44 percent since 2004, and concludes that the objective was achieved. This review considers that this indicator is inadequate to measure the results of a trade agreement and that its selection exemplifies the problem of the quality of indicators noted in section 3 above. This indicator is inappropriate to measure results . First, it defines impact in nominal terms, when what matters for growth and poverty reduction is the impact on real income . Second, it neglects the trade creation and trade diversion aspects of a customs union . Third, it neglects the potential impact on domestic and foreign investment of a change in the rules of the game accompanying the trade agreements. The ICR suggests that the indicator is not adequate to measure impact but does not bring information to overcome this shortcoming . Moreover, the ICR does not bring information on the growth of trade with Honduras and El Salvador and the real income gains from such growth . Lacking an adequate results indicator for this objective, this review concludes that efficacy in achieving the objective has been modest . 3. Promoting investment and business efficiency To achieve this objective the program supported actions to reduce processing times and costs in customs, establishing offices to promote investment and to register a business, to carry out a cadastral survey, and to title and register land . This review discusses the results associated with these actions in two groups, as the ICR does. The expected outcome for the first group of actions (customs and establishing offices to register businesses ) was met. Customs administration clears air and seaport cargo in one and 24 hours, while it did so in 12 and 96 hours in 2004. The government established a one stop shop to register business, and according to Doing Business reports it takes 26 days to register a company, fewer than the 42 days it took in 2004; although the target was 25 days, this review considers that the target was substantially achieved . Did these actions and results promote investment and business efficiency? There is no information to claim this was the case, although the ICR suggests that FDI tripled since 2005, among other reasons because of the administrative effort. This review considers the ICR claims more than it should for two reasons . First, gross domestic investment in 2005-08 remained unchanged at around 20 percent of GDP suggesting that the impact of the actions taken was limited. Second, FDI may have increased because the Berger administration (January 2004-January 2008) reduced the political and economic instability that the Portillo administration created. The second group of indicators covered land registration . The expected outcome for this group was partially met. The cadastral survey covered 30 percent of the territory, not the 50 percent expected, but land titling and registration had reached 33.1 percent in 2006, meeting the 33 percent target. No information exists for the status of titling and registration in 2008. This review concludes that efficacy in reaching this objective has been modest. 4. Strengthening infrastructure through private sector participation To achieve this objective the program supported actions to establish a legal framework for public-private partnerships (PPP) to invest in infrastructure, and to achieve international security certification of all Guatemalan seaports and a Category 1 rating for the Aurora International Airport in Guatemala City. Congress did not approve the PPP law, so the first result indicator was not met . According to the ICR the second result indicator, a 7 percent reduction in handling times and costs in ports and airports, was exceeded, and the actual reduction achieved was 15 percent. A gain seems to have been caused by the security certification of G uatemala's seaports and the upgrading of the Aurora International Airport in line with the standards of the International Civil Aviation Organization, but it is not clear if the 15 percent is part of the gains discussed in 3 above. The ICR does not explain what caused the gain and whether it differs from the one discussed in 3 above. The review concludes that efficacy in achieving this objective has been modest.. 5. Strengthening, modernizing and deepening the financial sector To achieve this objective the program supported actions to strengthen the payments system and the supervision of financial institutions and conglomerates, to facilitate SMEs access to credit, and to resolve the situation of two failed banks. In addition, DPL1 accepted as prior action the steps the government took in 2004 to ensure that Guatemala was taken off the list of Noncooperative Countries of the Financial Action Task Force. The program expected to achieve two outcomes: (i) Improve and implement legal and regulatory framework to facilitate access to credit by SMEs . Guatemala has a better functioning regulatory framework today than in 2004 as witnessed by: (a) application of risk-based supervision on a pilot basis to selected financial institutions; (b) better information on relationship between financial groups, their subsidiaries, and related parties; (c) more and better quality of data on large corporate debtors reported to the Banking Superintendency; and (ii) Payments system implemented and in operation (Banco de Guatemala). The ICR reports that the system is in operation but does not explain what is better today than in 2005. Guatemala has improved some aspects of its financial sector to which the prior actions may have contributed. First, according to the Doing Business report Guatemala ’s ranking in access to credit went from 61 in 2008 to 28 in 2009 (both dates not covered by the DPLs but, possibly, this is a residual impact of the measures taken earlier ). Second, credit to the SME sector grew 60 percent between 2005 and 2007, but this does not convey if access improved; a better measure of access would have been the share of total credit going to SMEs or the rate of interest paid by SMEs before and after the program. Perhaps the best indication that the financial sector has improved comes from the evolution of key indicators between 2004 and 2008. Coverage over assets at risk increased from 44 percent to 73 percent; profitability over capital increased from 14.4 percent to 16.3 percent; and the capital asset ratio increased from 8.9 percent to 10.3 percent. The ICR does not provide information indicating that deepening took place. Although the DPLs and the ICR do not spell out the logical link between the prior actions and the results discussed, it is clear that the actions taken should have helped improve the overall situation of the sector. This review concludes that efficacy in achieving this objective has been substantial . II. II. Enhancing the capacity for public spending in priority sectors 6. Improvement in tax collections To achieve this objective the program supported actions to change tax law and to improve tax administration at the central and municipal levels . The program expected to achieve two outcomes: (i) strengthened tax administration as assessed by independent agencies; and (ii) tax revenues to equal or exceed 11 percent of GDP (national accounts base year 1958). Regarding the first outcome, two indicators not shown in the ICR could support its conclusion that the administration has improved : first, the number of taxpayers increased from 740 thousand in 2003 to 1.025 million in 2006; second, the percentage of late filers of VAT and ISR for large and medium taxpayers fell from about 9 percent in December 2004 to about 1.6 percent in June 2007. Regarding tax revenues, they increased from 11.5 percent of GDP in 2005 (IMF data) to 11.9 and 12.1 percent of GDP in 2006 and 2007 (Ministry of Finance Third Fiscal Report, February 2009). Because of the financial crisis, tax revenues, especially customs revenues, fell to 11.3 percent of GDP in 2008. In summary, tax administration has strengthened and tax revenues exceeded 11 percent of GDP. This review concludes that efficacy in achieving this objective has been substantial. 7. Improving budget allocations To achieve this objective the program supported actions to increase budgetary allocations . As outcomes the program documents indicated a better targeting of resources for social sectors and an increase in social expenditures from 5 percent of GDP in 2004 to at least 6 percent of GDP in 2007. Social (and other peace related) expenditures increased from 5.6 percent of GDP in 2004 to above 6 percent of GDP for every year since 2005. Regarding targeting there is no evidence that the resources are better targeted. A July 2008 report from the Ministry of Finance about subsidies in the budget concludes, when looking at the budget for education and health in 2007, that there is a “problem of regressiveness in the budgetary allocation and in particular of the presence of the state in the least favored regions� (p. 38); the report also notes that there is evidence of problems in budget management (administración presupuestaria , p. 39), as poorer regions receive lower budget allocations for health and education. This evidence indicates that the objective of more resources for social programs was achieved, but that a better targeting of them did not occur . The review concludes that efficacy in achieving this objective has been modest. III . Transparency and Public Sector Management III. 8. Improving transparency and efficiency of public resource use To achieve this objective the program supported actions to increase the coverage of the integrated financial system (SIAF) and to use a transparent, web-based, procurement system (Guatecompras) in most public sector purchases. The outcome indicators from these actions would be an enhanced performance of public financial management and more transparency and efficiency in public procurement. These "outcomes" correspond more to objectives, and lack measurable indicators that the program document mentions but does not define. The assessment that follows is based on achievement of outputs. Regarding public financial management, SIAF covers all public entities at the central and local level and the central government has greater control of budget execution and outcomes. Civil society notes that the transparency in the use of public funds has improved now that the information from SIAF is on-line. Regarding procurement, Guatecompras has expanded coverage, and it is mandatory for all public agencies, while in 2004 they barely used it. Guatecompras serves to inform potential bidders but does not allow yet for procurement on -line. Although it is not yet known if as a result of SIAF and Guatecompras public expenditure is more efficient the review concludes that the expansion of the two programs constitutes a step to improving transparency and, eventually, as the public acquires more information, to increasing efficiency . The review concludes that efficacy in achieving the objective has been substantial . 5. Efficiency (not applicable to DPLs): ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The review concludes that relevance of objectives and design was substantial, efficacy in achieving the objectives was modest, and rates the outcome as moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The review considers that risk to development outcome is moderate. Macroeconomic risks are low, as the authorities have shown a prudent management of macroeconomic policies, with low fiscal deficits, low external debt and heir decisiveness in solving problems in the financial sector to prevent their spillover to the rest of the economy. Microeconomic risks are also low, as the likelihood is small that the government will reverse the changes carried out in customs, tax administration, procurement and public financial management, and on the allocation of resources for social expenditures . This is so because after the signature of the Peace Accords in 1996 a consensus has been growing that the reforms the agreements supported (and which the prior actions of the DPLs largely reflect ) are essential steps to promote growth and improve the standard of living of the poor . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: The Bank identified relevant objectives to support, and the objectives were consistent with the Berger’s administration program and the Bank’s country assistance strategy. The prior actions were appropriate to help deal with the problems identified, although in one instance, the investment promotion office, it is not clear why a prior action was required when no particular result was associated with it. The Bank carried out one supervision mission for DPL 1 and DPL3, and two for DPL2 (although the information in the ISR and in accounting for staff time and costs does not reflect it ). Bank staff participated in a workshop for the Financial Committee of Congress to explain the nature of a DPL loan when the first one was approved. In subsequent loans Bank staff in conjunction with the executive explained the nature and parameters of the loan to relevant congressional committees . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The government sought support from civil society and members of Congress for the different elements of the DPL program and carried out the actions that it had agreed to in the loan agreements. Despite temporary setbacks caused by delays in congressional approval of actions the DPLs supported, the government maintained its focus on the program and met most of the triggers agreed. The Ministry of Finance was the principal implementing agency and kept the DPL agenda on track . Progress in land administration has been uneven; land titling met the targets in the program but the cadastral survey fell short of them as a result, among others, of weak execution capacity in the Registro de Información Catastral (RIC). a. Government Performance :Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: The prior actions and triggers for loans constituted the main tools to monitor the development of the program. In general, the triggers and prior actions were well thought out and most triggers were met . The medium-term indicators and the outcome indicators could have been designed better; some did not have a relationship with the actions taken (e.g., growth of nontraditional exports and FTAs), and others could have been defined with more precision to inform or measure better the expected benefits (e.g., land, reductions in processing time in customs, efficiency gains from better public financial management). a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Ratings ICR IEG Review Reason for Disagreements /Comments Outcome DPL1 Satisfactory Moderately satisfactory See section 4 above DPL2 Satisfactory Moderately satisfactory See section 4 above DPL3 Satisfactory Moderately satisfactory See section 4 above Risk to Development Outcome DPL1 Moderate Moderate DPL2 Moderate Moderate DPL3 Moderate Moderate Bank Performance DPL1 Satisfactory Satisfactory DPL2 Satisfactory Satisfactory DPL3 Satisfactory Satisfactory Borrower Performance DPL1 Satisfactory Moderately satisfactory See section 9 above DPL2 Satisfactory Moderately satisfactory See section 9 above DPL3 Satisfactory Moderately satisfactory See section 9 above ICR Quality DPL1 Satisfactory DPL2 DPL3 Satisfactory 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately See section 4 above Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Moderately Moderately See section 9 above Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: DPLs are more likely to achieve their objectives when they support the government ’s reform program and the time frame for the actions they support takes into account the political barriers to reform and the capacity of the government to carry out the reforms. Trying to push for reforms beyond what is politically feasible may endanger the potential success of the program . Broadly designed DPLs are more likely to be more effective when complemented with technical support from Bank-financed projects, projects financed by other International Financial Institutions, government programs, or local institutions . This technical support for the DPLs reviewed made it easier to achieve their development objectives in a timely manner and on a yearly schedule coordinated with the national budget. When the government owns the reform agenda and its process, the Bank can design the program to accommodate risky environments as that ownership permits targeting other areas for reform when some part of the program proves intractable. As reforms in PPP became difficult, the DPL program gave more weight to reforms in the financial sector, which needed the reforms and the government supported it. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR presents well the general development and the potential results of the program . This review found particularly useful the section on achievement of results, with its attention to results rather than to processes. With its analytical approach to results, though, the ICR could have explored alternative ways to elucidate impact when the results indicators of the program were inadequate to measure it (e.g., on FTAs or land). The sections on M&E and on risk to development outcomes do not match the quality of other sections of the document, and the lessons section could have been sharpened by separating findings from lessons. The section on Bank supervision could have explained what the supervision missions did and why there is no supervision cost for DPL 2 when two supervision missions were carried out. The overall quality of the ICR is good. a.Quality of ICR Rating : Satisfactory