ICRR 12574 Report Number : ICRR12574 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/08/2007 PROJ ID :P069923 Appraisal Actual Project Name :Emergency Recovery And Project Costs 6.20 7.12 Disaster Management US$M ) (US$M) Project Country :St Vincent & Grenadines Loan/ US$M ) Loan /Credit (US$M) 5.90 5.65 Sector (s):Board: ): UD - Central US$M) Cofinancing (US$M ) government administration (60%), Information technology (20%), Other industry (10%), Irrigation and drainage (10%) L/C Number :C3644; L7116 FY ) Board Approval (FY) 2 Partners involved : Closing Date 01/31/2006 06/30/2006 Evaluator : Panel Reviewer : Division Manager : Division : Elliott Hurwitz Ridley Nelson Jaime Jaramillo IEGCR 2. Project Objectives and Components a. Objectives Objectives: The primary development objective of the project was to strengthen St . Vincent and the Grenadines' resilience, preparedness, and ability to respond to natural disasters . This objective was to have been achieved by : 1. Building the capacity of the Office of Disaster Preparedness (ODP) and other concerned public agencies to perform their respective responsibilities more effectively prior to and during a crisis; 2. Strengthening the country's institutional disaster preparedness and management procedures, rehabilitating a designated Emergency Operations Center (EOC) which would also serve as ODP headquarters, and procuring specialized and emergency equipment; 3. Implementing selected priority mitigation and prevention measures to protect key infrastructure and lifeline facilities before disasters strike, on a pilot basis; 4. Retrofitting shelters and health centers to improve the safety of the populations, particularly in poor areas; 5. Improving environmental protection through better standards with regard to disaster management, and environmentally-sound prevention measures; and 6. Promoting community involvement through organizing, training and equipping community disaster committees, and enhancing their role in disaster preparedness measures, and mitigation and recovery operations . b. Components (or Key Conditions in the case of Adjustment Loans ): 1. Strengthening Disaster Management Capacity (Appraisal US$3.07 million, actual US$2.37 million); 2. Preventive Physical Mitigation (Appraisal US$2.45 million, actual US$4.2 million); 3. Project Management (Appraisal US$0.68 million, actual US$0.55 million). c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project closed 5 months late on June 30, 2006. In nominal terms, project expenditures were around US$ 0.9 million above those envisioned (US$7.12 million versus US$6.2 million), causing the government to contribute US$1.0 million more than it anticipated. However, some activities originally intended to be funded under the project were funded by other agencies (e.g., early warning systems equipment and training ), which produced savings that were applied to the overruns on civil works . 3. Relevance of Objectives & Design : Relevance is substantial . The project is consistent with the Country and Bank strategies, in particular building human and institutional capacity as well as reducing vulnerability and income insecurity . The project was expected to have a disproportionate impact on the poor since their dwellings are often more exposed, and the project specifically targeted rehabilitation of shelters in poor areas . The project addressed an important development barrier insofar as the country's resilience to natural disasters is expected to facilitate the attraction of service industries by providing more reliable services and infrastructure . Project design was straightforward, with capacity -building in disaster-related agencies (but not in the Ministry of Works), civil works to mitigate damage from natural disasters, and the rehabilitation of shelters for the benefit of nearby residents. 4. Achievement of Objectives (Efficacy) : Efficacy was overall substantial , but with some shortcomings: 1. Building the capacity of ODP This was achieved to a substantial extent. TA was provided to strengthen emergency procedures; a robust physical facility was rehabilitated as headquarters; emergency equipment and supplies were purchased; and staff training provided. 2. Strengthening institutional disaster preparedness and management procedures This was achieved to a substantial extent. ODP (renamed NEMO, or National Emergency Management Office ) was established as a line agency, with a permanent staff and a dedicated budget (see also item 1 above). However, minimal progress was made in upgrading the capacity of the Ministry of Works, which would be charged with rapid rebuilding following a disaster . 3. Implementing selected priority mitigation and prevention measures to protect key infrastructure and lifeline facilities Progress in this area was substantial (though final costs were 320% above appraisal estimates). These consisted of civil works to improve sea defense and bolster river banks and flood protection works . 4. Retrofitting shelters and health centers, particularly in poor areas Progress in this area was modest; seven shelters out of a planned 20 were refurbished. The ICR does not provide information on whether the improvements were in poor areas, or to what extent the beneficiaries are likely to be the poor. 5. Improving environmental protection through better standards with regard to disaster management Progress in this areas was negligible. It had been intended that the capacity of the Ministry of Works to perform environmental impact assessments would be strengthened . While some environmental analysis and materials were "delivered," no training or capacity enhancement took place, and the environmental capacity of the Ministry did not improve. 6. Promoting community involvement and enhancing their role in disaster preparedness Achievement in this area was substantial . A community management training program has been initiated, NEMO is actively working with local communities , and local disaster committees meet regularly, inter alia, to disseminate key information. 5. Efficiency : The PAD economic analysis defined the project benefits as damages averted in a hurricane or storm surge as a result of project activity. This was posited as a percent of GDP --assumed to be 10% in the base case--although there was little knowledge on which to base such an estimate . The PAD NPV was US$26.3 million. In the ICR re-estimation of the NPV, increased project costs due to overruns of 17% were used, and the percent of GDP representing damage avoided was reduced to 9%, even though shortcomings in project performance seemed to be greater than this small incremental change (sec. 4). With an estimate of 9%, the NPV was calculated as US$21.3 million. While it would have been desirable to perform a more rigorous economic analysis of project benefits, the ICR Guidelines (Appendix G) do not require that the ICR do so . 6. M&E Design, Implementation, & Utilization: The M&E design was good, but was only partially implemented . Data were collected on monitoring indicators dealing with the institutional strengthening of NEMO, improved flood -protection infrastructure, and studies for further infrastructure investment. It is not clear to what extent these data were utilized in project management . However, data on other indicators were apparently not collected : improved performance of weather information and warning; number of people served by shelters enhanced under the project; Ministry of Works capacity to carry out works during emergencies and prepare environmental assessments . 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory Shortcomings in building capacity in the Ministry of Works, on the number of shelters constructed, and on environmental assessment (sec. 4) detract from the project's positive achievements. Also, no evidence is presented that the project addressed the poverty goals outlined in the PAD . Institutional Dev .: Substantial Modest While efforts to build capacity and the enabling environment for agencies involved with emergency response were overall successful, those regarding the Ministry of Works were not. This was important, as the PAD notes, because the Ministry of Works would be responsible for rapid rehabilitation of infrastructure following an event, as well as maintaining the civil works constructed under this project. Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Bank performance overall was satisfactory, with good performance in supervision overcoming project weaknesses. Contrary to the ICR assessment, Quality at Entry was unsatisfactory : Appraisal estimates of the cost of works were very inaccurate Agency capacity assessment, especially regarding the Ministry of Works, was deficient Readiness for implementation was inadequate, especially with regard to PCU capabilities The project design did not take sufficient account of lessons from previous projects with regard to : --more detailed preparation of project components and activities; --clear and agreed project management responsibilities for the PCU; --focus on strengthening the capacity of existing institutions to effectively discharge their responsibilities with regard to disaster preparedness . The PAD (p. 18) includes these items as "Lessons learned and included in the project design, " when in fact they were not. However, despite the poor QAE and the project's slow start, the Bank supervision team persevered, and worked closely with the government to achieve most relevant objectives . Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: A dedicated and activist supervision team can overcome poor quality at entry and a slow start, and thereby enable a project to achieve important benefits When a project is put in abeyance during preparation and later reactivated on a rapid schedule, the Bank needs to pay closer attention to revising design and cost estimates for civil works . This would limit the potential for major cost overruns as experienced on this project . Closer attention to the PAD monitoring indicators by the supervision team might have increased project benefits . While the PAD indicators were sound, in some cases they seem not to have been addressed . 10. Assessment Recommended? Yes No 11. Comments on Quality of ICR: The ICR is satisfactory, but only barely so; deficiencies include : Absence of information on the extent to which the shelters that were refurbished were likely to benefit the poor Inconsistency between the text and Annex 1 regarding the number of shelters originally planned . Text: "about a dozen shelters." Annex 1: "original 20 shelters." No information provided on shelter management and degree of local staffing (monitoring indicator, Annex 1) Inadequate information on: performance of meteorological information and warning ("at least 72 hours warning of impending tropical events"); capacity of Ministry of Works to erect on -site bridges (span of 10-15m) within 4 days. These are monitoring indicators defined in the PAD . Misstatement of the project objectives (ICR, pp. 5-6). This, despite a correct statement of the objectives on ICR p. 2. Ambiguity about the extent to which project environmental objectives were met