Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) Report Number : ICRR0021519 1. Project Data Country Practice Area(Lead) Grenada Macroeconomics, Trade and Investment Programmatic DPL Planned Operations: 3 Approved Operations: 3 Operation ID Operation Name P147152 1st Programmatic Resilience Building DPC L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-55230 29-Jun-2015 14,991,447.00 Bank Approval Date Closing Date (Actual) 30-Jun-2014 29-Jun-2015 IBRD/IDA (USD) Co-financing (USD) Original Commitment 15,000,000.00 0.00 Revised Commitment 15,000,000.00 0.00 Actual 14,991,447.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Paul Holden Robert Mark Lacey Malathi S. Jayawickrama IEGEC (Unit 1) PHPROJECTDATATBL Operation ID Operation Name P156761 Grenada Resilience Building DPC 3 ( P156761 ) Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-55230,IDA-59340 30-Nov-2017 9,088,880.00 Bank Approval Date Closing Date (Actual) 16-Dec-2016 30-Nov-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 9,340,000.00 0.00 Revised Commitment 9,340,000.00 0.00 Actual 9,088,880.00 0.00 PHPROJECTDATATBL Operation ID Operation Name P151821 Grenada Resilience Building DPC 2 ( P151821 ) L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-85460,IDA-55230,IDA-57340 30-Nov-2016 14,993,960.00 Bank Approval Date Closing Date (Actual) 28-Oct-2015 30-Nov-2016 IBRD/IDA (USD) Co-financing (USD) Original Commitment 15,000,000.00 0.00 Revised Commitment 15,000,000.00 0.00 Actual 14,993,960.00 0.00 2. Program Objectives and Policy Areas a. Objectives This was a series of three development policy operations. The Program Development Objective (PDO) for the first operation (DPO-1) as stated in the first Program Document (PD1, p. vi)) was to: (i) create conditions for private investment in a sustainable manner; (ii) support improved public sector management (PFM) and better Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) targeting of social safety net programs; (iii) enhance resilience against natural disasters; and (iv) facilitate debt portfolio restructuring and enhance debt management. The PDO for DPO-2 (PD2 p. vi) was identical for the first two elements, with (iii) changed to (iii) enhance resilience against natural disasters and key elements of resilience in the banking sector (i.e., adding reference to resilience of the banking sector). The fourth element was dropped. For DPO-3 (PD3 p. vii) the PDOs were reworded again to: (i) improve the investment climate and competitiveness; (ii) improve public resource management; and (iii) enhance resilience against natural disasters. The goal of increasing the resilience of the banking sector became a sub-objective of objective (i). There are no substantive differences between the three sets of objectives, the reasons for the rewording are unclear and add very little to refining the development objectives. This Review will be based on the statement of objectives in PD3. b. Pillars/Policy Areas There were three pillars: Pillar 1 focused on creating conditions favorable to private investment through strengthening the governance of the tourism sector, improving competitiveness of agribusinesses, improving trade logistics, developing a public-private partnership (PPP) policy framework, and strengthening the regulatory environment in the energy sector. Pillar 2 supported fiscal consolidation by reducing fiscal, financial and social vulnerability through public service modernization; strengthening public procurement; strengthening the regulation and supervision of the financial sector; strengthening social safety nets; and enhancing the capacity to manage institutional debt and the restructuring of the public debt portfolio. Pillar 3 focused on enhancing natural disaster resilience through a strengthened regulatory system governing physical planning.　 c. Comments on Program Cost, Financing, and Dates The first operation (P147152) was approved on June 30, 2014 in the amount of an IDA credit of SDR9.7 million (US$15 million equivalent). It became effective on July 9, 2014 and closed on schedule on June 29, 2015.　 　 The second operation (P151821) was approved on October 28, 2015 in the amount of an IDA credit of SDR7.2 million (US$10 million equivalent) and an IBRD loan of US$5 million. It became effective on November 19, 2015 and closed on schedule on November 30, 2016.　 　 The third operation (P156761) was approved on December 16, 2016 in the amount of an IDA credit of SDR6.8 million (US$9.34 million equivalent). It became effective on December 23, 2016 and closed on schedule on November 30, 2017.　 Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) All Credits and the Loan were fully disbursed. 　 3. Relevance of Objectives & Design a. Relevance of Objectives Instead of a strategy framework specific to Grenada, the World Bank Group has a Country Partnership Strategy (CPS) for the Organization of Eastern Caribbean States (OECS), of which Grenada is a member. The PDOs were and remain relevant to the two main pillars of this strategy: (i) building resilience, including climate resilience, debt and fiscal sustainability, and human capital, and (ii) enhancing competitiveness and stimulating growth through strengthening the financial sector and improved business environments. The PDOs are also relevant to the three areas of engagement of the Bank’s OECS Regional Partnership Strategy FY2015-FY2019 - competitiveness, public sector modernization and resilience. They were also consistent with Grenada’s 2014-2018 Growth and Poverty Reduction Strategy, which had as its pillars, building resilience, developing competitiveness, reducing vulnerability and improving governance.　 Rating Modest b. Relevance of Design The causal chain between the series-supported actions and the intended results was partly convincing. Regarding the first objective of an improved investment climate and competitiveness, establishing an effective Tourism Authority might be expected to contribute to improving the sector’s institutional governance framework (the ICR points out that, prior to the series, the lack of a governance mechanism for the tourism sector impeded the development of a shared and holistic vision and common implementation strategy), However, agricultural productivity and competitiveness would not necessarily improve by leasing out inefficient government-owned estates and strengthening the marketing board, in the absence of accompanying measures to ensure that private sector lessees instituted productivity enhancing measures. It is plausible that better quality infrastructure through public-private partnerships, modernized trade logistics, and reduced electricity costs, would make Grenada more competitive, although it was less clear whether the specific prior actions would be strong enough to achieve such results (see Section 4 below). The second objective – improved public resource management – focused on reducing costs (better human resource management to lower the wage bill, more competitive procurement, better targeting of social safety net programs) and potential future liabilities (debt restructuring, financial sector reform). Again, while the causal chain may be persuasive in principal, there could be questions concerning the adequacy of specific prior actions. Similarly, with the third objective – it is unlikely that new building codes and establishing procedures for the professional practice of architects and engineers would suffice to enhance resilience against natural disasters. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) The program is also very wide ranging, covering tourism, agriculture, trade logistics, electricity, the financial sector, public human resource and expenditure management, resilience to natural disasters, and other areas, thereby increasing the coordination and implementation burden for a relatively weak administration. Provision for technical assistance was inadequate, notably in the areas of public sector management, trade logistics and physical planning regulation (ICR, page 7). The macroeconomic framework, while initially weak, benefited from IMF support and was considered adequate for program purposes. Following an extended period of low growth, Grenada faced large fiscal and balance of payments deficits, with a concomitant debt burden, when DPO1 was approved. The authorities had, however, sought assistance from the IMF, and a 36-month arrangement under the Extended Credit Facility was approved in June 2014. By 2015, growth had resumed. Fiscal consolidation and debt restructuring had yielded a primary surplus. Growth continued into 2017, and the budget primary surplus expanded, but the current account deficit was still about 16% of GDP, and the risk of renewed debt distress remained significant. The speed with which the program was prepared and the need to coordinate with other donors provides justification for a rating of substantial for relevance of design.　 Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve the investment climate and competitiveness. This PDO had 5 sub-objectives: Improving the governance of the tourism sector; increasing the productivity and competitiveness of agriculture; improving trade logistics; promoting investment through public private partnerships; strengthening the policy and regulatory environment for the energy sector. Rationale The development of tourism, accounting for about a quarter of Grenada’s GDP and employment, was thought to be impeded by the lack of a governance mechanism to encourage a shared and holistic vision and an agreed implementation strategy, outlined in the Government’s Tourism Strategic Plan (ICR, page 12). To address this, the Grenada Tourism Authority Act was passed (prior action, DPO-1), and a new Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) regulatory framework established (prior action, DPO-2). The associated results target was an increase in tourism receipts from a baseline of EC$307 million to EC$332 million in 2018. This was exceeded (receipts were EC$448 million), and Grenada's tourism performance was substantially better than regional competitors. The Government considered that low agricultural productivity stemmed in part from inefficient publicly-owned estates and an ineffective marketing board. It was decided, therefore, to lease the estates to private operators, and to transform the marketing board into a service organization providing commercial information, product standards and logistics. A leasing oversight committee and criteria for commercialization of the estates were established (prior actions, DPO-1); and two agreements with private entities for 20 leases were entered into (prior action, DPO-3), A strategic plan was approved for the transformation of the marketing board (prior action, DPO-2). Bank staff expected that these measures, by increasing agricultural productivity, would enable agribusinesses to be become more competitive. This was to be assisted by the passage of the Food Safety Act (prior action, DPO -3) empowering the inspection and licensing of food premises by the newly-established Food Authority. There were four targets in the results framework. First, by series closure, at least three out of four estates would have been leased; this was not achieved – two estates were leased. Second, the area leased on estates would be at least 186 acres out of a potential 379; this was exceeded, with 205 acres leased. Third, the number of farmers served by the marketing board would rise from 1,623 to at least 2,500; this was not achieved – by series closure, 2,000 farmers were so served. Fourth, the output of crops produced by the leased estates would double from 350 tons to 700 tons; this target was revised to 500 tons under DPO-2 and the indicator was dropped altogether under DPO-3. Despite modest success in meeting the outcome targets, there are issues with the targets themselves. It is questionable whether there would be sufficient time for private operators to achieve the expected impact on output; and the marketing board target consisted of a strategic plan indicating the number of farmers who would be served, but with no indication of whether or how it would be implemented. Any impact of the Food Safety Act on output and competitiveness would be difficult to achieve by series closure, since it was a prior action for the third and last operation.　 　 The Authorities considered that Grenada’s outdated trade logistics were an impediment to competitiveness. A draft Law to modernize and automate the Customs Administration was presented to Parliament (prior action, DPO-1), The Bureau of Standards in the Ministry of Finance was connected to the Automated Customs Clearing System (ASYCUDA, World), and the Inland Revenue Department, and the Central Office of Statistics were provided with data access to ASUCUDA (prior actions, DPO-3) These measures were intended to reduce Customs clearance time from six to three days, but this results indicator was dropped during DPO-3 because of slow progress in adopting ASYCUDA. It was replaced by a results indicator on the number of agencies using ASYCUDA with a target to increase the number from two to six, and the number of agencies sharing information through ASYCUDA from which was to increase from zero to two. 　Neither was achieved. These are, in any event, input targets that reveal little about the system’s efficiency. The number of transactions processed, and the time taken to clear goods (the dropped indicator) would have been more appropriate. According to the ICR, technical assistance from the World Bank for the use of ASYCUDA was inadequate. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) To promote new investment through PPPs,　the government developed in 2015 "principles and processes" for developing and implementing PPPs (prior action, DPO-2). A PPP Unit was established in the Ministry of Finance (prior action, DPO-3). The chosen results indicator target – conformity by all new PPP projects with the new policy framework – was met. However, only one　project materialized (a telephone service provider).　 High and volatile electricity costs (stemming from dependence on imported diesel) and reduced competitiveness. To address this, the Public Utilities Regulatory Commission Act was approved, and the Ministry of Finance’s participation in the Eastern Caribbean Regulatory Agency was endorsed (prior actions, DPO-3). The aims of the new policy were to reduce tariffs and promote renewable energy investments. There is little evidence of attributable progress towards these aims. The Public Utilities Commission was not fully operational by 2018 as intended, though the ICR states that it would be by January 2019. A targeted increase from 1% to 2.3% in the share of renewable energy in total power generation was exceeded (the actual share in 2018 was 6.3%). However, the causal chain from the prior actions to this achievement is not clearly set out in the ICR. There were no targets related to the cost of electricity. In summary, evidence of an improved investment climate resulting from series-supported actions is weak. The policy framework for the tourism sector improved, but attribution of increased tourist numbers is unclear. There is no evidence that agricultural productivity improved, nor that customs clearing times fell. ASYCUDA was not operational at series closure. Only one PPP was underway at the closing of the third operation. 　 There are no indications of declining electricity costs. Efficacy of this objective is rated modest. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improve public resource management Rationale The program aimed to improve public resource management through (i) modernizing public service personnel policies, (ii) better procurement processes, (iii) stronger financial sector supervision (to reduce fiscal risks), (iv) reducing ineligible participants in social safety nets, and (v) restructuring public debt.　 The Authorities endorsed a policy to modernize the public sector by realigning employment, strengthening management of key agencies, and developing results-focused planning and budgeting (prior action, DPO1). There were no series-supported measures to implement this policy except establishing computerizing personnel records and job descriptions (prior action, DPO3). Although the single chosen results indicator target (formally established job descriptions) was met, this is clearly insufficient to modernize public service management.　 Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) A new Procurement Law, "consistent with international best practices" was approved, a Chief Procurement Officer appointed, and a Public Procurement Board established (prior actions, DPOs 2 and 3). The results indicator target - the publication of all contract awards by 2018 (2013 baseline zero) - was not met.　 To enhance financial sector stability, Grenada participated, with the series’ support, in efforts to bolster the resolution powers, loan classification, and provisioning requirements of the Eastern Caribbean Central Bank (ECCB). These were encapsulated in a uniform Banking Act and Amendment to the ECCB Agreement Act, approved in 2015 (prior action, DPO2). By June 2018, provisioning as a percentage of non-performing loans (NPLs) had risen to 71% from 30% in 2013, exceeding the target of 60%.　 Given the sensitivity of the social program reform, it was implemented gradually: a policy framework was adopted (prior action, DPO1), eligibility criteria began to be systematically enforced (prior action, DPO2), and a phase out plan executed (prior action, DPO3). By 2018, all ineligible recipients had been removed, exceeding the target of 95%. This was expected to allow increased coverage of the poorest households, though no data are available.　 Restructuring public debt had become urgent by 2013 - the debt to GDP ratio was 107%, 17% of outstanding debt had a maturity of 90 days or less, and there had already been a default on dollar-denominated bond coupon payments. A medium-term debt strategy was approved (prior action, DPO2). Negotiations with creditors led to a 50% repudiation of principal. Debt falling due within 90　days was reduced to 4.2% of the total　in 2018, below the target of 10%.　 In summary, there is evidence that the program succeeded in reducing financial sector risks, removing ineligible participants from social programs, and in restructuring public debt. However, there is little indication of improved public service and procurement management. 　 Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Enhance resilience against natural disasters Rationale This objective was to be achieved by improving the quality of new construction through improved building standards, better zoning and planning, and by professionalizing engineers and architects. A new building code was approved and legislation to professionalize architects and engineers submitted to Parliament (prior Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) actions, DPO-1). A physical planning bill incorporating the approval of building codes and guidelines was enacted (prior action, DPO-3). It was expected that these measures would increase the share of engineers that were members of the Grenada Institute of Professional Engineers from zero in 2013 to 40% of practicing engineers in 2018, and to the establishment of a Building Inspection Unit in the Ministry of Finance. However, due to delays in harmonizing building codes within the OECS, and the lack of qualified personnel to staff the Building Inspection Unit, these expectations were not met. The ICR (p. 23) reports that in a pilot project, only 2% of buildings constructed under permits issued from 2017 onward were inspected for compliance with the codes. However, more generally, in the period following the adoption of the new codes, 65% of buildings constructed in 2017 were inspected. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Outcome The relevance of objectives was rated substantial given consistency with the Bank strategy for the Eastern Caribbean and was germane with respect to country priorities, although the rewording of objectives in successive loans seemed unnecessary. Relevance of design is rated substantial although the theory of change was less than robust for several objectives, the breadth of the series’ scope presented coordination and implementation challenges for the government’s limited capacity, and provisions for technical assistance were inadequate. Nevertheless, the need for rapid preparation of the series to coordinate with development partners was a positive. Efficacy of the first objective - improving the investment climate and competitiveness – is rated modest in view of the weakness of evidence concerning achievement of results and their attribution to series- supported actions., Efficacy of the second objective – better public resource management - is assessed as substantial given reduced financial sector risks, removal of ineligible participants from social programs, and a restructuring of public debt. The efficacy of the third objective is rated negligible since there is no evidence of improved resilience to natural disasters. Shortcomings are considered significant, and outcome is assessed as moderately unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 6. Rationale for Risk to Development Outcome Rating Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) The risk to development outcome declined as the operations unfolded. At the outset Grenada faced serious debt and macroeconomic imbalances, and there was substantial global uncertainty that threatened tourism and the revival of remittances. Subsequently, macroeconomic risk fell as the budget improved and public debt was reduced. Grenada, however, like most small, open economies in the Caribbean, remains highly vulnerable to international commercial vicissitudes and to natural disasters. The ICR (p. 25) considers that the government remains committed to the objectives of the series and has continued to pursue reform under follow-on operations Many reforms supported by the series are grounded in legislation, which needs to be applied for them to bear full fruit. The main domestic risk to sustainability is limited institutional capacity.　 a. Risk to Development Outcome Rating Modest 7. Assessment of Bank Performance a. Quality-at-Entry The series was designed to respond to Bank and government priorities and to Grenada’s difficult macroeconomic circumstances. The areas covered had been identified in earlier analytical work. Coordination with other external partners, notably the IMF and the Caribbean Development Bank, was adequate. There were some shortcomings in Quality at Entry. First, lessons from previous operations included the need for simple program design focused on a relatively select set of issues, this program ranged over a wide range of sectors and policy areas, thereby adding to coordination and implementation challenges. Moreover, Bank plans for technical and advisory support, and interventions from other partners were to prove insufficient to address capacity weaknesses. Several policy areas needed more hands-on assistance that was not foreseen at the preparation stage (for instance, processing trade transactions through AYSCUDA would require extensive coordination and negotiation and inter-agency agreements, and sufficient funds to finance the necessary technical assistance. Second, although the ICR reports that there were extensive consultations, political resistance was a delaying factor for some reforms, notably public employment management and the phasing out of ineligible beneficiaries from social programs. Third, the causal chain between the prior actions and the intended results was less than fully convincing in a number of cases (see Section 3b above). Fourth, there were serious weaknesses in M&E design, especially with regard to the choice of indicators (see Section 9a below); several of the targets were over-ambitious.　 Nevertheless, the program was prepared against a background of crisis in Grenada, and the need to coordinate with development partners. The Bank team was rolling out the CDF in the Caribbean region at the time the first operation was prepared, and the Grenada authorities requested this approach to complement the IMF program. Bank performance is therefore rated moderately satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) The ICR (p. 26) states that supervision was timely in preparing the second and third operations, and pragmatic in adjusting targets to what might realistically be achieved in the program’s time frame. However, there were shortcomings. First, little was done to address the problems related to the results framework, which persisted until series closure. Second, the team could have been more proactive in mobilizing technical assistance in certain key areas, and more could have been done to leverage IDA’s investment portfolio to support capacity building. Third, the ICR refers to lack of continuity in the World Bank task team, especially in terms of leadership, after the approval of DPO-2. No details are provided, but there is a reference in the lessons to the need for "a solid handover [from one task team leader to another] to ensure sustained technical support throughout implementation and especially in the final phases through completion." The ICR refers specifically to the "lack of implementation support in the final phase of the program, due to staff illness;" clearly steps could and should have been taken to substitute for staff unable to perform their duties.　 With a split rating of moderately satisfactory for Quality at Entry and moderately unsatisfactory for Quality of Supervision, IEG rates Bank Performance as moderately satisfactory. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Satisfactory 8. Assessment of Borrower Performance a. Government Performance According to the ICR, , the government remained committed throughout the program to achieving its objectives, and ensured the passage of key legislation that provided a foundation for many of the reforms. There was continuity in the personnel connected with the program and government officials pushed back against some of the design complexity (ICR p. 27). Lack of technical and institutional capacity to pursue some of the reforms, together with difficulties in recruiting qualified staff, were the principal negative dimensions. 　These led to delays in implementing reforms in Customs, agriculture and the promotion of PPPs.　 The scope and range of the program proved challenging. As main interlocutor with the World Bank, the Ministry of Finance and Energy (MoFE) had to coordinate activities across ten sectors and thirteen agencies. Despite the inherent difficulties, some political resistance, and lack of adequate technical support, the Ministry’s efforts were sufficient to elicit some degree of response in most areas except for the Physical Planning Unit, which remained virtually inactive in implementing the natural disaster resilience program. They could not, however, lead to the attainment of most of the program's goals. Overall, shortcomings in government performance are considered moderate. Government Performance Rating Moderately Satisfactory Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) b. Implementing Agency Performance The program was implemented by the government as a whole, and there is no separate assessment of implementing agency performance Implementing Agency Performance Rating Not Rated Overall Borrower Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The design of the M&E framework contained weaknesses in several of the indicators or did not use indicators that would have identified the success of the reforms. For example, attributing an increase in tourist numbers to the series is clearly deficient because they are affected by exogenous factors in originating countries. The performance of Grenada’s tourist arrivals relative to other countries in the Eastern Caribbean might have been more appropriate. There was no indicator that would have measured the quality of service to farmers; useful measures of improved trade logistics were either not included (for example, the number of transactions being processed through the ASYCUDA system) or dropped (the number of days to clear goods); instead of one PPP that was in accordance with the processes established, the number of PPPs initiated would have been a clearer indication of the program’s success or otherwise; and there was no link established between series-supported actions and the specified goal of reducing electricity tariffs, for which no measure was included in the results framework. b. M&E Implementation The ICR (p. 8) provides little information on M&E implementation, beyond saying that it was "carried out through ongoing dialogue with the government and was supported by complementary investment projects, hands-on technical assistance and analytical activities."　Having to deal with thirteen　different agencies across such a wide-ranging program must have presented considerable coordination difficulties in the collection, analysis and presentation of M&E-related data. However, no details are presented on this. It is striking that little, if any, attempt appears to have been made to address the design weaknesses outlined in Section 9a above. c. M&E Utilization As the programmatic operation evolved, M&E was used to modify the prior actions and indicators in the policy matrix. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) M&E Quality Rating Modest 10. Other Issues a. Environmental and Social Effects No safeguard policies were triggered. Reforms to the social safety net would in principal allow increased benefits to the poor. b. Fiduciary Compliance No fiduciary compliance issues were reported. c. Unintended impacts (Positive or Negative) N/A d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment There were significant design shortcomings leading to Moderately Moderately Outcome modest and negligible efficacy Satisfactory Unsatisfactory ratings for two out of three objectives. Risk to Development Modest Modest --- Outcome There were significant Moderately shortcomings in both Quality Bank Performance Moderately Satisfactory Satisfactory at Entry and Quality of Supervision Technical and institutional Borrower Performance Satisfactory Moderately Satisfactory capacity weaknesses and the Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) coordination challenges of such a wide-ranging series, led to delays; Some reforms foundered against political opposition. Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons The following two lessons are taken from the ICR with some adaptation of language: • When designing a programmatic series, it is important at the outset to be realistic about what can be accomplished, taking into account capacity of government counterparts. Excessive optimism or complexity, such as that involved in the design of some components of this programmatic series, results in having to adjust subsequent operations to reflect the reality on the ground. Political economy factors in small countries, such as Grenada, may be of an order of magnitude different from those in large economies. For example, downsizing government departments must take account of scarce alternative opportunities; only a limited pool of potential investors might be interested in investing in PPP projects in a small economy; and skills and capacity may be scarcer. • Creation of new administrative agencies can lead to mission overlap and higher fixed costs. This series required the creation of numerous new units and agencies, where assigning responsibilities to dedicated staff in existing units might well have sufficed. IEG draws the following three additional lessons:　 • Adequate recognition at the design stage for technical assistance (TA) requirements can assist the successful development and implementation of a reform program. In this case, relying on "regular" support from the World Bank team, and other external partners proved insufficient. More could have been done to mobilize resources for targeted TA for certain particularly complex initiatives, and through leverage of the investment lending portfolio. • While including legislation, new regulations, and other "pieces of paper" within prior actions is appropriate, a program that does more to ensure practical application on the ground stands a greater chance of success. Here, much new legislation was approved, but in several cases the capacity or political will to apply the modified legal and regulatory framework was not manifest. • Particularly in small economies with limited implementation and coordination capacity, a relatively simple reform program focused on a few key areas stands a greater chance of success than a wider-ranging initiative. In this case, the relative lack of success was due in no small part to an over-ambitious design involving ten sectors and thirteen agencies. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 1st Programmatic Resilience Building DPC(P147152) • 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR rated achievement of objectives in terms of pillars rather than development objectives. However, the analysis and description of a complex series of operations is clear and given the fact that it was a series of three operations, the document is relatively concise. It does outline the issues well and explains clearly the challenges confronting Grenada that the DPL series was designed to address. Nevertheless, the treatment of M&E issues is weak, in that it does not assess　fully the quality of the M&E framework or implementation challenges. The discussion of the program indicators is incomplete. The ICR’s assertion that the achievements of the program led to the strong macroeconomic performance is not based on analysis and is hard to justify.　 a. Quality of ICR Rating Substantial Page 15 of 15