ICRR 13690 Report Number : ICRR13690 IEG ICR Review Independent Evaluation Group 1. Project Data : Date Posted : 09/13/2011 Country : Haiti Is this Review for a Programmatic Series? Yes No How many operations were planned for the 2 series? How many were approved? 1 Series ID : S117944 First Project ID : P117944 Appraisal Actual Project Name : Ht 3rd Econ. US$M ): Project Costs (US$M): 12.5 12.5 Governance Reform Operation L/C Number : Loan/ Loan US$M): /Credit (US$M ): 12.5 12.5 Sector Board : Economic Policy US$M): Cofinancing (US$M ): Cofinanciers : Board Approval Date : 12/08/2009 Closing Date : 12/31/2010 12/31/2010 Sector (s): Central government administration (90%); General public administration sector (10%) Theme (s): Public expenditure; financial management and procurement (65%); Tax policy and administration (21%); Administrative and civil service reform (14%) Evaluator : Panel Reviewer : ICR Review Group : Coordinator : Clay Wescott Rene I. Vandendries Ismail Arslan IEGPS2 2. Project Objectives and Components: a. Objectives: This was intended to be the first in a series of two operations supporting the medium term strategy set forth in the National Strategy for Growth and Poverty Reduction (DSNCRP). However, following the earthquake of January 12, 2010, the series was discontinued . The objectives of the 3rd Economic Governance Reform Operation (EGRO 3) are to contribute to: (i) reducing inefficiencies in the electricity sector in order to support the Government’s effort to reduce the fiscal transfer to the electricity company (EDH); (ii) modernizing and strengthening public financial management to raise and use public resources in a more efficient and transparent manner; and (iii) strengthening and implementing the legal framework for public procurement to contribute to more transparent and cost-effective public expenditure (Program Document, paragraph 57). b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: The operation supports three areas of policy reform : fiscal policy, public finance management, and public procurement. These, in turn, support the two broader objectives set forth in the DSNCRP : (a) macroeconomic stability; and (b) sustained growth and poverty reduction, and the four pillars : (1) strengthening political governance and promoting dialogue and reconciliation; (2) improving economic governance and modernizing the State; (3) promoting economic growth, including by maintaining a stable macroeconomic framework; and (4) improving access to quality basic services, particularly for the most vulnerable groups . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The single tranche of $12.5 million equivalent from the grant was fully disbursed . Effectiveness was January 12, 2010, ahead of the planned date of March 16, 2010. The operation was closed on schedule on December 31, 2010. 3. Relevance of Objectives & Design: a. Relevance of Objectives: The objectives were highly relevant . They were aligned with the DSNCRP as set out above, with the joint matrix for budget support agreed upon with all key donors, with the action plan for the reconstruction and development of Haiti (March 2010), and with the Bank's Country Assistance Strategy for FY 2009-12, with its two-pronged approach of longer-term institution building and economic governance reform, combined with helping government to deliver quick and visible results . They built on reforms supported by EGRO 1, EGRO 2 (both separate operations, not a series ), the two Governance Technical Assistance Grants, and the Electricity Loss Reduction Project. The objectives were also aligned with work funded by, inter alia, the Bank's Low Income Countries Under Stress Implementation Trust Fund, the United States Agency for International Development, the Inter-American Development Bank, the United Nations Conference on Trade and Development, the Canadian International Development Agency, the European Union, the International Monetary Fund, and the French Cooperation Agency. While the second operation in the series (EGRO 4) was canceled due to the new challenges arising from the earthquake, the Emergency Development Policy Operation continued support to improved public expenditure management and procurement, as they are of great importance to facilitate aid inflows that finance post-earthquake reconstruction . b. Relevance of Design: The design was substantially relevant . Building on reforms starting under EGRO1 and 2, there was a clear statement of objectives, and the specific actions supported by the operation were relevant to achieving the objectives. There are time-bound, measurable indicators in the form of triggers that were expected to be met by the end of EGRO 3, and expected outcomes at the planned end of the series (December 2011). There is a clear and convincing causal chain linking funding from the joint donor matrix for budget support, including the Bank's funding for EGRO 3 and related operations mentioned above, and the expected outputs and outcomes . The design takes into account the severe capacity challenges of Haiti's post -conflict context with features such as single tranche disbursement, and a greater focus on a smaller number of actions supported than was the case in EGRO 1 and EGRO 2. There had also been progress in strengthening the policy and institutional frameworks underpinning the actions supported by the operation, as documented by the 2008 Public Expenditure Management and Financial Accountability Review , so it was plausible to believe that most of the proposed triggers and outcomes could be achieved . By project completion, many of the actions supported by the operation continued to be relevant, but, not surprisingly, some actions became impractical due to the challenges of earthquake reconstruction, and were overtaken by other policy priorities . According to the Task Team Leader, the approach with politically sensitive power sector reforms was to carry out extensive dialog with high level officials, helping them to understand that DPG -supported actions such as greater transparency could lead to more budget support from development partners . This helped to mitigate the opposition to reforms from vested interests, and helped pave the way both for greater transparency and for the adoption of politically sensitive electricity rate increases. 4. Achievement of Objectives (Efficacy): The achievement of efficacy of objectives will be measured by considering achievements of prior actions, triggers, and progress towards achieving planned end 2011 outcomes. 1. Reducing inefficiencies in the electricity sector in order to support the Government ’s effort to reduce the EDH ): Two prior actions were important parts of the result chain : fiscal transfer to the electricity company (EDH): design and implementation of a cost recovery policy by EDH, and competitive appointment of a provider of new customer and technical management systems in EDH . There were two triggers indicating achievement for this objective. The first one was that a monitoring table would be published monthly on the MEF website for the period November 2008 - October 2009 showing the extent of fiscal transfers to EDH . An earlier system was implemented under EGRO 2, but it had been implemented erratically . The result under EGRO 3 was that partial data was published for the agreed period . In addition, incomplete data were published between January 2010 and March 2011, but not on a monthly basis. The second trigger was that information contained in the monitoring table would be audited on a biannual basis by an independent firm . This trigger was not met. Subsequent to project completion (as of May 2011), the Government is finalizing TORs for the first audit . There was one outcome indicator for this objective : by end 2011, EDH's cost recovery index would have improved to 45 percent in FY2011, from less than 32 percent in FY2008. According to the ICR, in FY2010 the index declined to 19.5 percent in August 2010, such that transfers exceeded the budgeted amount . Contributing factors to this outcome included reduced revenue collection capacity due to damage to the grid from the earthquake. Given these achievements, IEG rates the efficacy of this objective as negligible . 2. Modernizing and strengthening public financial management to raise and use public resources in a more efficient and transparent manner : The ICR lists tax reform as a separate objective, but it was part of objective 2 in the Program Document. The program focused on the area of clarity and comprehensiveness of tax liabilities, which had already been given the highest "A" rating by the 2005 PEFA assessment (indicator PI13(i)). Three prior actions included setting up a customs information system in 11 customs posts, an information system for all other tax units, and the interconnection of the systems, all crucial steps towards achieving the objective . The trigger in this area -- submitting a Fiscal Code to the lower chamber of Parliament, was not achieved due to the destruction of the Tax Directorate and other factors . In public expenditure management, the prior action of staffing the general financial inspectorate was an important step towards achieving the internal audit triggers --- strengthening the internal audit function, and applying regular controls to the discretionary accounts --- which were partially achieved. According to the ICR, the trigger --- Ministry of Economy and Finance (MEF) adopt and disseminate a procedural manual for budget preparation --- was partially met. The ICR doesn't say whether the action --- integrate the preparation of current and investment budgets --- was achieved or not. The final two actions: submission to the Supreme Audit Institution (CSCCA) of the 2008-9 public accounts, and submission to the Parliament of the CSCCA's opinion on the 2007-2008 audit report, were achieved. In terms of outcomes, the indicator of improved tax collection rate was not available since the Directorate General of Taxes lost a large portion of its records due to the earthquake . Two ministries have multi-year expenditure plans, but most ministries lacked an appropriate multi year expenditure estimate (3 were expected to have one by end 2011. The General Finance Inspectorate (IGF) had begun some key tasks in its audit work program, but there is no evidence of distribution of audit reports to ministries expected by end 2011. New budget and accounting classifications are being provided to ministries, and are hoped to be fully effective in FY2011-12, as expected. The audit of 2008-9 accounts was delayed by earthquake, and 2009-2010 accounts have been submitted; Thus, the expected outcome of timely submission of accounts by the Government was met, and efforts are underway to enable both audits to be prepared by September 2011, in which case the expectation of timely audit would be satisfied for the 2009-2010 accounts. Given these achievements, IEG rates the efficacy of this objective as modest . Some prior actions were important prerequisites, triggers were mostly partially achieved, and there has been some progress towards expected end 2011 outcomes. 3. Strengthening and implementing the legal framework for public procurement to contribute to more transparent and cost -effective public expenditure : Four implementing decrees required for the Public Procurement Law to become effective were issued as prior actions . However, neither of the two triggers was achieved: submission to National Procurement Commission (CNMP) procurement plans from key contracting authorities for FY2009-2010, and implementing a CNMP database on contract awards that is operational, up-to-date, and accessible to the public . However, in early 2011, the Government initiated selection of consultants to achieve these actions . In addition, the expected outcome of increased share of competitive procurement is not possible to measure, since equipment and records disappeared in the earthquake . Given these achievements, IEG rates the efficacy of this objective as modest . 5. Efficiency (not applicable to DPLs): 6. Outcome: EGRO 3 sought highly relevant objectives, and the design was substantially relevant . However, achievement of objectives is rated modest or negligible, due mainly to the extraordinary challenges of the 2010 earthquake that could not have been anticipated . Overall, based on relevance and achievement, the outcome is rated as moderately unsatisfactory. a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: According to the ICR, all prior actions and the modest additional progress on triggers and planned outcomes have been sustained, despite the calamitous earthquake that, inter alia, led to the collapse of MEF's office building and deaths of or disruption to the lives of key staff . This is in part due to the risk mitigation measures used: single tranche disbursement, concentrating on a limited number of areas, ensuring that the most difficult reforms were taken up-front, and strong alignment with and cross support from other development partners . However, it isn't clear how the project intended to address opposition to reform from vested interests . PFM and procurement weaknesses persist despite modest achievements in improving procurement, supported by the DPO. Taking also into account the enormity of additional challenges facing reconstruction, IEG rates the risk to development outcome as significant . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: EGRO 3, building of lessons learned in EGRO 1 and 2, was single tranche, well focused, and well aligned with other related support from the Bank and other development partners . The operation built on a government program that the Bank had worked closely with the government to prepare, and which had a good record of recent achievement . There was a strong analytical foundation including a PEFA assessment and Public Expenditure Management and Financial Accountability Review , though no evidence of political economy analysis to understand how to mitigate resistance from vested interests, nor a convincing plan for ensuring the integrity of funds as they passed through the budget . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: The Bank's team deserves credit for supporting the operation under exceptionally difficult conditions that ISRs monitored planned outcomes. Since the Government was not in a position to participate in the supervision mission preceding the firts ISR, the first ISR could not provide a detailed assessment of the status of the triggers, except for general statements like : "most of the indicative triggers...were significantly delayed" . The second ISR did provide details of progress on triggers . The PDO ratings of "moderately satisfactory" on the two ISRs are hard to reconcile with the weak performance reported on planned outcomes and triggers. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Despite the devastating consequences of the earthquake, the government made a determined effort to retain achievements under prior actions, to make some headway on triggers and planned outcomes, all while contributing to unplanned tasks including reconstruction needs assessments, and managing a vast increase of donor supported reconstruction assistance . Government Performance Rating : Moderately Satisfactory b. Implementing Agency Performance: The MEF performed well in coordination and implementation of the reforms and participating agencies in the program. They provided leadership to address bottlenecks, and assure the attention of political decision-makers when needed, despite the challenges from the earthquake and a difficult political and security situation. According to the Task Team Leader, nearly all PFM and procurement functions supported by the operation are carried out by Haitian civil servants, with consultants acting only as advisors . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: There are time-bound, measurable indicators in the form of 10 triggers for the planned EGRO 4 that were expected to be met by the end of EGRO 3, and 7 expected outcomes at the planned end of the series (December 2011). b. M&E Implementation: There was specific reporting on the expected outcomes in ISRs, but only generalized reporting on the triggers in some areas. More attention to detailed monitoring of progress on triggers could have identified bottlenecks and follow up actions, and might have led to better performance on the reform agenda . According to the Task Team Leader, there were monthly meetings of the donor working group on budget support to monitor that reforms supported by this, and other operations were on track, thus ensuring improved fiduciary standards of country systems . The possibility of using civil society monitoring was discussed, but it was decided that monitoring by development partners would be more effective . c. M&E Utilization: Outcome indicators were not designed to be met until the end of the planned series . Although it was decided to replace EGRO 4 with the Emergency Development Policy Operation (P118239) responding to priority reconstruction needs, most intermediate results indicators are the same or closely aligned with PFM and procurement triggers and outcomes in EGRO 3. Thus, a full accounting of M&E utilization awaits completion of the ICR for that operation. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The operation supported economic governance reforms that do not have direct effects on environment, resettlement, or other safeguard issues . b. Fiduciary Compliance: While there was no Governance and Anti -corruption Action Plan per se, fiduciary risks were mitigated through enhanced supervision and the monthly donor working group meetings mentioned above . However, the Program Document states that the fiduciary environment is weak, and there is not a clear statement that PFM and procurement systems are adequate to support the DPO . c. Unintended Impacts (positive or negative): There were no unintended impacts . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Significant Significant The actual ICR rating was "substantial", Outcome : a category not available for this area . Bank Performance : Satisfactory Moderately There was no evidence of political Satisfactory economy analysis. Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The lessons cited in the ICR -- the need for a simple approach, need to specify better the responsibility of every agency, the need for strong institutional support to back up reforms, and the importance of close donor cooperation --- are all supported. Using DPLs in Haiti's fragile, post-conflict setting is fraught with high risks . Yet if the risks can be properly mitigated, using country systems for donor funds can build public trust in the Government . Improving the effectiveness of formal PFM and procurement systems, and demonstrating that donors have confidence in them, also opens the possibility that there will be greater incentive for elites to invest political capital in these systems, and to reduce their reliance on unclear, informal patronage systems . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provides a useful summary of the strengths and weaknesses of the operation . It could have elaborated more on gaps in program design, including lack of political economy analysis, and incomplete fiduciary review . a.Quality of ICR Rating : Satisfactory