ICRR 13213 Report Number : ICRR13213 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/16/2011 PROJ ID : P094205 Appraisal Actual Project Name : Sri Lanka Tsunami Erl Project Costs (US$M): US$M ): 150.00 140.50 Country : Sri Lanka Loan /Credit (US$M): Loan/ US$M ): 150.00 141.00 Sector Board : UD US$M): Cofinancing (US$M ): Sector (s): Housing construction (55%) Roads and highways (35%) General public administration sector (10%) Theme (s): Natural disaster management (50% - P) Access to urban services and housing (25% - S) Social analysis and monitoring (25% - S) L/C Number : C4039; CH147 Board Approval Date : 02/24/2005 Partners involved : Closing Date : 09/30/2008 09/30/2008 Evaluator : Panel Reviewer : Group Manager : Group : George T. K. Pitman Robert Mark Lacey IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: In response to the emergency, the Bank simultaneously restructured 10 ongoing operations on February 15, 2005 for immediate emergency recovery needs calling the project the Tsunami Emergency Recovery Program 1. In each of the ongoing operations the Legal Agreements had the following objective added : 1. "Part Z: Emergency Recovery, Rehabilitation and Reconstruction Program Assistance for recovery, rehabilitation and reconstruction activities in the Affected Areas including, but not limited to, various sectors, including health, education, infrastructure, roads, and housing . These activities will also include Cash Grants, capacity building, support for small and medium enterprises, and establishing a civil society fund." At the same time the Bank undertook a formal appraisal of The Tsunami Emergency Recovery Project proposal and this was presented as a Technical Annex to the Memorandum of the President (MOP) for the Tsunami Emergency Recovery Project 2 (TERP-2). The project objectives were to support the Government of Sri Lanka to : 1. "Reduce the immediate suffering resulting from the effects of the tsunami and restore livelihoods destroyed by the tsunami; 2. Restore basic services to the affected population; and, 3. Start the recovery and reconstruction process ." Subsequently the DCA for TERP-2 (dated 04/01/2005) stated the project's objective to be : "to assist the Borrower in carrying out the second phase of an emergency recovery and reconstruction program in the Affected Areas as a consequence of the massive destruction and devastation following the tsunami and tidal waves in December 2004 and other similar incidents." Because both TERPs were differing facets of the same emergency operation and emergency funding from each TERP was used for for the same range of activities, the more general objective defined in the MOP for TERP -2 is used in this Review. To determine achievement of project objectives the MOP listed key performance indicators linked to each of the six main components as described below . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The Tsunami Emergency Recovery Program included the following components : 1. Housing reconstruction (Appraisal: US$60.0 million; Actual: US$65.0 million). The component was designed to include housing reconstruction grant assistance; capacity building, including training of construction workers, training of homeowners for auto-reconstruction, the design and implementation of a communications initiative, establishment of a grievance response mechanism at the local, divisional and district levels, continuous social and environmental monitoring; and technical assistance . Key performance indicators (KPI) were (i) percentage of eligible families assisted by the program; (ii) percentage of (fully and partly damaged) houses reconstructed; (iii) number of masons and carpenters trained; (iv) number of community-based housing societies established ; and (v) percentage of beneficiary grievances and complaints redressed . 2. Roads, Water Supply and other Infrastructure (Appraisal: US$41.0 million; Actual: US$34.6 million). The component was designed to finance the repair /reconstruction of priority infrastructure systems in tsunami-affected areas. These included roads, water supply and sanitation (these latter two were later dropped from the project), debris removal, and electricity connections to houses, if needed . It would finance consultancy services associated with infrastructure reconstruction, such as detailed needs assessments, feasibility studies, environmental and social analyses, preparation of engineering designs and bid documents, construction supervision and monitoring. A limited amount of office and communication equipment was financed to support project implementation. The KPIs were: (i) the number of water supply and sanitation facilities repaired/reconstructed; (ii) the number of km of road rehabilitated and or reconstructed; (iii) the number of households that have regained access to power; and (iv) the number of tons of properly disposed debris . 3. Livelihood support (Appraisal: US$35.0 million; Actual: US$32.9 million). The component included pure cash transfers to tsunami-affected households that lost their ability to earn a living . It also included a cash for work program to generate employment through labor -intensive public works construction involving unskilled work for many activities. The component also included a program to provide training and business services to micro-enterprises as well as provision of grants and /or loans to institutions financing micro -enterprises. There was a broad range of KPIs (TA page 30) but the most easily measured ones were : (i) number of households benefited; (ii) amount of money transferred; (iii) number of project approved and completed; (iii) no of person-days of employment generated; and (iv) number of community infrastructure rehabilitated; (v) number of jobs created; and (vi) amount of loans and grants extended and (vii) amount of loans repaid. It was expected that the outcomes would be (i) improvement in poverty measures, and child and women's health and nutrition indicators; (ii) reduction in the levels of displacement; (iii) increase in the ownership of productive assets and (iv) reduction in the level of indebtedness . 4. Capacity Building (Appraisal: US$10.0 million; Actual: US$8.0 million). The component was designed to include : the provision of human resources and consultancy inputs at the central and district levels; a continuous social impact assessment; the design and delivery of training programs for district administrators on financial management and streamlined implementation processes; the provision of information technology connectivity and the establishment of management information systems (MIS); and the supply of basic office supplies and infrastructure and urgently needed vehicles . 5. Social Sector s (Appraisal: US$8.0 million; Actual: US$ 1.27), The component was designed for the reconstruction and recovery needs for health and education : Health . Three types of activities were to be financed : (i) strengthening existing, undamaged health care delivery services; (ii) supporting additional preventive, promotive and rehabilitative health needs; and (iii) reconstructing and re-equipping partially or fully damaged institutions . The first activity was to support the provision of health services in undamaged healthcare centers identified by central, provincial and district health authorities in the tsunami -affected areas. IDA funds would be used to strengthen those health care centers not covered by other international agencies . The second activity was to help meet preventive, promotive and rehabilitative health gaps identified in affected areas . This was to entail: (i) support for a Logistics Unit to coordinate the reconstruction of all health care centers in the tsunami -affected areas; (ii ) support for a Bio-medical Engineering Unit to coordinate the procurement and maintenance of essential medical equipment; (iii) strengthening of the Medical Supplies Unit to restore the supply of urgently needed drugs in the affected areas; (iv) support for the rabies control program; and (v) strengthening the Epidemiology Unit to improve disease surveillance, emergency preparedness, and disaster mitigation activities. The third activity would support central, provincial and district level health authorities to reconstruct and re-equip partially and fully damaged health care centers not identified for support by other donors in the affected areas. Education . This was to assist efforts to rebuild, equip and upgrade the quality of schools affected by the tsunami either through damage caused by the tidal waves or through their use as shelters for displaced persons. Included were financing for: (i) support for architectural, engineering and consultancy services and the monitoring of education outcomes; (ii) civil works; and (iii) equipment, technology, machinery and tools, books and reading materials, and essential school furniture . The objective was to reinstate the tsunami-damaged capital stock of the education system with quality upgrading so that equitable access to basic, secondary, and tertiary education was restored to pre -tsunami levels. At the time of appraisal the Government was considering relocating some schools inland utilizing, where possible, state -owned land for new schools sites to eliminate the involuntary acquisition of private land and reduce costs . Schools were only to be relocated if the coastal population from which these students are drawn was relocated and ethnic sensitivity was taken into account . If the event that there was no relocation of the coastal population, care was to be taken to ensure that existing students are not denied access to schools . 6. Contingencies (Appraisal: US$4.0 million). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs : Project costs were less than estimated at appraisal as there were undisbursed funds at the close of each phase except for housing. The funds allocated to the social component for emergent financing gaps in health and education were left largely unused . The water supply and sanitation activities under the infrastructure component were dropped. Financing : IDA carried out a portfolio restructuring of 10 credits/grants to release up to US$50 million and provisionally set aside US$25 million to provide immediate financing for urgent tsunami -related reconstruction efforts (this was Phase-I). The follow-up Tsunami Emergency Recovery Operation, Phase II or TERP 2, added US$75 million to the US$75 million financing initiated under the Phase I restructuring . Total IDA financing for TERPs 1 and 2 included a US$120 million credit and a US$30 million grant. At appraisal, it was anticipated that other donors would be identified in the near future to support some of these priority needs . If other donors were to provide grant funds to finance any activity earmarked for support by IDA credits, IDA resources would not be used . The ICR (page 29) reports that, by closure, US$ 141.0 million had been disbursed, of which US$ 115.20 million was disbursed from the credit and US$ 25.80 from the grant. According to the Operations Portal (11/08/2010) the total amount disbursed under the project was US$ 84.71 million and US$13.99 million was cancelled. The difference in reported financing and disbursement is because most of the early financing under TERP -1 was disbursed directly from other projects' funds. The ICR does not break-down project financing and disbursement by project source . In mid-2005 the Bank agreed to administer a US$ 25 million Trust Fund established by the International Federation of the Red Cross (IFRC) for housing reconstruction in the former 'buffer zones " (a 100-200 m strip next to the sea). Borrower Contribution : There was no contribution from the Borrower . Dates : The project closed on the original closing date (September 2008) even though the government had requested closing date extension of both TERP -1 and TERP-2. The delayed submission of a project audit report prevented IDA from agreeing to the extension of the closing date for TERP -1 in 2007 and the same problem prevented the extension of the closing date for TERP -2 in 2008. There were extensive issues over disbursement of the IFRC Trust Fund (see Section 11 below). In consequence it took until October 2010 to satisfy all the Bank's fiduciary concerns . 3. Relevance of Objectives & Design: Relevance of Objectives : High. The tsunami hit Sri Lanka during the implementation period of the previous CAS (2003). The disaster had such a profound impact on the country and a direct impact on the extent of achievement of the existing CAS pillars and the overall Bank portfolio that it was decided to add the tsunami as a new pillar and associate outcomes to this pillar. The objectives of the project are very much in line with this pillar and the two outcomes associated with it : (i) restored housing in the affected districts; and (ii) restored livelihoods of the tsunami affected families . The project objectives also remain relevant to the current CAS which was approved in 2008. Under CAS Outcome 3.2, Improved Health and Social Protection Services, there is direct mention of the need to develop capacity for effective delivery of cash transfers in post -disaster situations in Sri Lanka . Under CAS Outcome 3.3, Strengthening Environmental Protection, there is a direct mention of the need to combat the adverse impacts of environmental degradation. The project objectives are also consistent with the Government ’s Poverty Reduction Strategy (PRS). Relevance of Design : Substantial. The emergency nature of the operation precluded attention to the institutional requirements to ensure targeting and accountability, and subsequent extensive reorganization of government agencies responsible for project implementation (see section 9 below) made this task more difficult. Thus it was decided early in the project to set up a new institutional unit to implement the reconstruction efforts (Task Force for Rebuilding the Nation, TAFREN). However, it had no prior experience to take on such a complex task in a short period of time . It also had difficulty in proving itself to existing line agencies and coordinating them . As a result the project lacked an effective Project Management Unit for the period 2005-2008 while responsibility was shifted from one agency to another, losing institutional memory in the process (ICR page 45). Project design gave priority to the restoration of livelihoods and reconstruction of basic infrastructure, particularly private housing, which was appropriate as these were the two main challenges facing those affected . The allocations under the project components were kept notional and flexible so that there would be the ability to respond to emerging needs. The project design included stringent requirements for the livelihood cash grants and housing programs to ensure accountability and transparency in order to maximize the benefits of the project. Overall, the flexibility of the design, the possibility of directing funds to identified priority areas through contingent sectors/components, and the decentralized implementation approach allowed for a greater chance of achieving the development objectives . Finally, the decentralized design and the ability to scale down contributed to the mitigation of potential risks associated with the very serious threat of civil conflict, particularly in the North . 4. Achievement of Objectives (Efficacy): Reduce the immediate suffering resulting from the effects of the tsunami and restore livelihoods destroyed by the tsunami . Substantial . Livelihood Cash Grants . Through cash grants, the project contributed to the recovery process by directly assisting for up to four months tsunami-affected families that had lost their capacity to earn a livelihood . The Livelihood Cash Grants Program was completed ahead of time in 2005 under TERP-1 with the four tranches paid to an estimated 191,293 confirmed beneficiary families. There were some instances of over -inclusion, however. The first tranche assisted 255,761 households with grants of US$12.1 million, the second 267,829 households for US$14.6 million. Following a beneficiary survey the Bank put an upper limit on reimbursement of US$ 7.2 million to exclude beneficiary households outside the target group. Accordingly the Bank reimbursed US$ 14.4 million of government expenditures of US$ 16.8 million for the 3rd and 4th tranches. The project also included cash -for-work opportunities providing temporary employment for many who lost their jobs in the disaster. Training provided for construction and supervision increased the capacity of the communities and increased skills thereby contributing to the economy . The number of masons and carpenters trained is not known although it was a KPI . The project design included training and assistance to micro -enterprisers in order to help regenerate the economy, but these activities are not covered in the ICR . There is no data reported in the ICR on other KPIs : (iii) number of project approved and completed; (iii) no of person-days of employment generated; and (iv) number of rehabilitations of community infrastructure; (v) number of jobs created; and (vi) amount of loans and grants extended and (vii) amount of loans repaid. There is no information on outcomes : (i) improvement in poverty measures, and child and women's health and nutrition indicators; (ii) reduction in the levels of displacement; (iii) increase in the ownership of productive assets and (iv) reduction in the level of indebtedness . Restore basic services to the affected population . Substantial . Housing . Through the homeowner-driven reconstruction program, the Bank supported approximately 48,159 beneficiaries (ICR page 42; on page 7 the number is 47,864) or 68 percent of beneficiaries included in Phase I of the homeowner program (Phase II is not covered in the ICR ). This assisted cohort comprised 47.8% fully damaged houses, the balance partly damaged . However, about 6% of total Phase I beneficiaries were found to be ineligible in ex-post follow up surveys. The physical completion of Phase I reached 94% of the potential beneficiaries (ICR page 8). There was a time gap between the first housing mobilization installment (April 2005) and actual commencement of project activities in August 2005. The lack of a good information dissemination strategy regarding housing grants meant that a "good percentage of families used this money for consumption " (ICR, page 46). Due to implementation problems the housing backlog accumulated early in the project could not be cleared by the project closing date . Independent 3rd party technical audits of a 10% sample of newly-built houses commissioned by the government found that 90% of the houses met a satisfactory level of technical quality for foundation, superstructure and roofing (ICR page 33). Many houses did not have sanitation facilities because beneficiaries gave this a low priority (ICR page 12). However, the ICR also states (page 34) that in some instances that technical building requirements cost more than available grant financing and this adversely affected the poor. Thus in the North and East beneficiaries classified as poor accounted for 93% of incomplete houses. Similarly 70% of incomplete houses belonged to female -headed households. There was an uneven picture also in the regional distribution of housing outcomes, with affected communities in the North and East showing a lower completion rate (92%) than in the South and West (97%) in part the result of security issues in the North and East resulting from civil conflict (enforced moves or destruction that were outside of the project's control ), and shortage of building materials (ICR page 46). In addition to the IDA support, NGOs provided top -up grants for housing reconstruction and these were similarly skewed by the effects of the civil conflict in favour of the South and West (where 77% of beneficiaries received top-ups). In contrast only 30% of beneficiaries received top -up grants in the North and East (ICR page 12). The project was initially slow in setting up the social monitoring of beneficiary identification as well as the grievance redress mechanism. When the grievance redress mechanism was in place, it showed that the level of grievances was particularly high in the South West Housing Reconstruction Unit (SWHRU). The beneficiary eligibility audits revealed that most of the rejected cases were due to two main factors : (a) lack of land ownership documents; and (b) beneficiaries availing a second home assistance through the donor-housing program. Several housing-related KPI have no record of achievement because of the poor management of M&E : the number of community-based housing societies established; the number of water supply and sanitation facilities repaired/reconstructed; the number of households that have regained access to power; and the number of tons of properly disposed debris . Transport . The project completed construction of nearly 114 km of tsunami-damaged southern coastal road from Kalutra to Matara but at 23% greater cost than estimated at appraisal due to design changes and material cost escalation. Since completion of the project these roads and associated work have been taken over by the Road Development Authority . The quality of these roads upon completion was satisfactory, but it has been observed that certain sections of the roads have been dug up by utilities and are yet to be restored to their original state . This is mainly due to the lack of coordination between road maintenance and the utilities . The project also undertook construction of a storm water drain along the Galle Road in Mirrissa to improve the poor drainage conditions in the local area . Only one of the KPIs ( number (ii) the number of km of road rehabilitated /reconstructed) is reported in the ICR. Health . An ambitious government program included reconstruction of 3 District Health Offices, procurement of 36 vehicles, procurement of emergency life -saving drugs and consultancy services . While there was limited procurement of drugs and consultancy services for reconstruction /refurbishment of all donor-funded health institutions, and some work on epidemic preparedness, physical repair and procurement of vehicles was extensively delayed due to poor coordination within the Ministry of Health Care and Nutrition (ICR page 43) and delay in Bank approvals . The delays were so extensive that only 16% (US$1.27 million) of available financing was disbursed before project closing, the balance being financed by the government. Education . There were no interventions required in the education sector . Start the recovery and reconstruction process and to assist Sri Lanka in carrying out the first phase of an emergency recovery and reconstruction program in the affected areas . Rating : Modest . Although the capacity building component contributed to keeping government agencies up and running by financing their operating costs, actual capacity -building that took place during implementation was somewhat limited. Financing provided much-needed office equipment as well as training of local level staff, such as technical officers, who played an important role in housing damage assessment and inspection . Ultimately, the limited implementation capacity to handle the massive reconstruction activities on the part of these agencies slowed the recovery and reconstruction process . The disconnect between other government departments and TAFREN created difficulties in information flow, and eventually it was replaced by a new implementing agency ultimately to be absorbed by a government ministry. These transitional periods inevitably stalled implementation and reduced accountability . Government's reversal over creation of a coastal belt "buffer zone" (a 100 to 200 m wide non-construction zone next to the sea) and weakness in communicating this to the public had negative consequences . It delayed housing reconstruction for those in the zone, and those that sold land cheaply before the buffer zone was abolished were the main losers (ICR page 12). 5. Efficiency (not applicable to DPLs): No formal ERR was calculated because it was an Emergency Response Project . However, the poor coordination among participating agencies on some of the activities (e.g. health) and differing criteria and standards (e.g. for beneficiary family identification and housing grants ) led to inefficient use of Bank funds. Efficiency is rated modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Considering the high relevance of project objectives for the reconstruction and recovery of Sri Lanka in the wake of the devastating tsunami, and substantial relevance of design, coupled with substantial success in achieving recovery objectives despite myriad complexities and a weakened and modestly efficient use of bank funds, the overall project rating is Moderately Satisfactory . Both housing and road construction targets were met under the project, and the cash grants were delivered . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Risk to development : Moderate. Political: While these risks were high, the cessation of the conflict in mid -2009 has reduced this risk to low. Social: The project had some experience with TERP homes having been destroyed in civil -conflict by the time communities were able to return but this risk has now abated . Another social concern is the completion of construction of housing for vulnerable groups (e.g. female-headed households) and whether they will be completed outside the project or maintained without continued assistance . Government ownership and commitment: The disaster has reinforced the commitment of the government to be better prepared to deal with future disasters . The government has expressed interest in the enhancement of their disaster preparedness capabilities to ensure the sustainability of project outcomes and therefore poses little risk. Institutional: The Ministry of Nation Building (MoNB) was the implementing agency at the time the project closed . This ministry is the successor to two prior agencies . These transfers in responsibility pose some risk to the project because many government staff that were part of the tsunami response and the project are now posted to different agencies within the government and some have left the government entirely . Natural disasters: The major disasters faced by Sri Lanka include floods, cyclones, droughts and landslides . Since the country is an island with settlements relatively close to the coast there is limited response time as well as limited shelter options when these disasters do strike . Sri Lanka is yet to develop effective disaster preparedness and emergency response systems . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: at -Entry . The project was expeditiously appraised and approved in 8 weeks. The Ensuring Quality -at- post-tsunami damage and needs assessment mission was fielded January 9-22, 2005 and the team completed appraisal by February 13, 2005. The Board approved the project on February 24. The task team leader for the project changed quite early during the implementation cycle, though only once during the life of the project. During the design phase, the project team ensured that development objectives and scope of the project were appropriate to the emergency conditions . The team also ensured that the initial project implementation was rapid and responsive to identified priority needs through a phased approach . While more attention could have been given to institutional support for M&E, this was difficult in the rushed appraisal process. Risks were, for the most part, clearly identified and appropriately mitigated . However, the attempt to mitigate limited national capacity to handle the massive reconstruction efforts by government agencies through adding a capacity building component to the project had limited success . The component was not well defined, and although project financing kept the agencies up and running, the ICR (page 5) reports that little actual capacity -building took place. Although environmental impact assessment mechanisms were identified, not enough attention was given on how to manage environmental issues in an emergency recovery situation . In contrast, attention to social assessment and establishing grievance mechanisms was done well. Quality of Supervision . The Bank required independent reviews, third party technical and eligibility audits, and continuous social assessments to ensure accountability and transparency . The Bank team conducted regular supervision missions, with at least one and sometimes two missions per year, including a mid -term review mission. From May 2007, the task team leader was stationed in Colombo allowing for continuous supervision. Task team members involved with procurement changed a few times which affected implementation, though not significantly, and the Borrower notes Bank approvals sometimes took a long time (ICR page 43). With respect to the Housing Reconstruction Program, there were reported cases of people resorting to illegal sand mining, tree cutting to meet the local /personal demand. While assessing the environmental impact of the Housing Reconstruction Program, especially during project design, might have been difficult given the imperative to respond fast to the emergency, the Program was sufficiently large (amounting to US$65 million, or 46.4%, of an actual total project cost of US$ 140.5 million, according to Annex 1 of the ICR), to have justified monitoring its effects during implementation to flag any observed adverse environmental effects and to agree on how they would be address and by whom . a. Ensuring Quality -at-at -Entry :Moderately Satisfactory b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Government Performance . The government successfully facilitated urgent needs assessment and the beginning of the reconstruction phase . The department of Treasury Operations, Ministry of Finance (MoF) successfully executed the cash grants program in a timely fashion . The housing reconstruction design formulated by the government was based on both a homeowner -driven model and a donor-driven model which proved to be very successful . In the design of the road construction component, the government contributed to a smooth implementation process by using the Project Implementation Unit from the existing Roads Sector Assistance Project which already had experience with road reconstruction . However, some inadequately informed, though well -meaning, decisions made early in the project negatively affected implementation of the reconstruction program . After the tsunami, the government announced the use of a coastal buffer zone as a disaster prevention mechanism .The decision came without proper technical advice or consultation of stakeholders, was met with an adverse reaction, and was ultimately reversed . There was no effective strategy from the government for communicating reconstruction news . After the buffer zone was reversed, the already complex reconstruction program was further complicated by poor communication of the policy change to the public . When the government decided to close down the South West Housing Reconstruction Unit, its computerized database of beneficiaries in the South along with payments details to beneficiaries was lost, and this compounded the weak supervisory role played by the implementing agency. Subsequently, delayed submission of covenanted audit reports led to the failure to extend the project closing date which could have produced further results . Implementing Agency Performance . Several changes in the implementing agency affected implementation negatively. The main implementing role was initially with TAFREN which was then changed to Reconstruction and Development Agency (RADA) and thereafter transferred to the Ministry of Nation Building (MoNB), when RADA was absorbed into the Ministry . The decision of MoNB to restructure and establish a new unit brought about another transition period, this time stalling the project for almost six months. The appointment of auditors was delayed by RADA and this affected timely transfer of IDA funds to the commercial banks. Subsequently, MoNB did not maintain a consolidated database of beneficiaries and made no attempt to cross-verify the accuracy of the disbursement reports generated by the commercial banks to request for reimbursements . These serious lapses in the control framework meant that the only risk mitigation measure for IDA was to adjust disbursement amounts based on the findings of the third party beneficiary and financial audits . As a result of procurement and financial management problems, the Country's Auditor General became involved at one point . The Bank's Final report on the IRDC Grant (March 2010) was strongly critical of the poor management by RADA (para 2.1): "Lack of adequate coordination between the commercial banks and the divisional administrations and poor oversight by the Reconstruction and Development Agency (RADA) have significantly affected the project implementation performance . A large part of the ineligibility captured by the beneficiary and financial audits could be attributed to the noncompliance with the program operations guidelines and inadequate monitoring oversight of beneficiary eligibility and payment approval and insufficient internal controls." Environmental Impact Assessments (EIAs) reported cases of people resorting to illegal sand mining, tree cutting to meet the local/personal demand. These impacts should have quantified and addressed . a. Government Performance :Moderately Unsatisfactory b. Implementing Agency Performance :Unsatisfactory c. Overall Borrower Performance :Unsatisfactory 10. M&E Design, Implementation, & Utilization: Design . The Technical Annex for TERP II included a results framework with appropriate key performance indicators . Some targets were difficult to determine prior to implementation, for example in the case of housing, the Bank committed to cover all eligible units outside the buffer area for Phase I, a figure unknown at appraisal . The design included limited provision for evaluation and assessment of outcomes, but there was provision for periodic surveys and audits, particularly for livelihood cash grants and the housing components . However, design overlooked the needs for capacity -building in M&E and centralized data bases . Implementation . Surveys and audits were conducted by the Bank and government . The IDA-assisted housing program was monitored by the North East Housing Reconstruction Unit and the South West Housing Reconstruction Unit for the North East and South West Districts respectively . However, the government’s capacity constraints in implementation of an effective MIS based monitoring and evaluation system at all levels posed problems . This was compounded by the fact that the custodians of the data were frequently changing, which led to computation errors, lack of clarity in the M&E arrangements, data loss and infrequent reporting from various field entities . Utilization . Housing eligibility and verification surveys, third party quality audits, livelihood cash grant audits were used to inform project decision-making, though not always accurately . Poor communication of data between the different levels of implementation, particularly to the central implementing agency, negatively affected decision -making capability. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards . In the case of environmental safeguards, an Environmental and Social Screening and Assessment Framework (ESSAF) was prepared. This included an assessment of the Government ’s review and approval process for environmental impact assessments (EIAs) and the ability to monitor the implementation of environmental mitigation measures. The key environmental issues directly relating to TERP were those associated with extraction of natural resources as construction materials . The EIAs identified possible sources of material supply for the selected divisions but there was no system in place to verify such information during or after reconstruction, since housing reconstruction was home -owner driven and it was impractical to verify the origin of construction resources available in the open market . There was no demand for new land in the project as houses were built on the same land as before . According to the ICR (page 11) "As such, on-site environmental impacts were either none or marginal ." Environmental Impact Assessments (EIAs) reported cases of people resorting to illegal sand mining, tree cutting to meet the local/personal demand. No assessment was undertaken to identify or quantify the environmental impacts of these activities either under TERP or nationally . The current CAS highlights the need for more attention to environmental protection . In the case of social safeguards, inclusion of a Continuous Social Impact Assessment (CSIA) in the project design was to facilitate communication of community perceptions, grievances and feedback, particularly in relation to beneficiary selection for the housing and livelihood program components of the project . The Bank’s resettlement Policy, Sri Lanka’s National Involuntary Resettlement Policy (NIRP) and “Guidelines for Land and Asset Acquisition, Entitlements and Compensation � were included in the framework to deal with resettlement and land acquisition issues . As noted in section 4, the project was initially slow in setting up the social monitoring of beneficiary identification as well as the grievance redress mechanism, but once set up they were effective . Fiduciary . Overall there were significant fiduciary concerns . The overall responsibility of Financial Management (FM) for the project rested initially with TAFREN, then RADA and, thereafter, with the Ministry of Nation Building (MoNB), when RADA was absorbed into the Ministry . RADA / MoNB played a coordinating and monitoring role albeit with weaknesses, while the day to day financial management related activities were carried out by the implementing units of the different programs under the project. FM capacity issues led to: (i) consistent delays by RADA in meeting the reporting and auditing requirements of IDA, including the delay in appointing internal auditors for the Tsunami Housing Program; and (ii) misuse of operating expenses by RADA as observed by the auditors, leading to a request for a refund by IDA following its own investigations. IDA eventually received this refund on May 21, 2009. Three of the 10 Audit Reports contained Special Opinions and all have been satisfactorily resolved according to the Project Portal (November 2010). The Bank did not close the IFRC Grant Account on March 31, 2010 due to the fact that there were undocumented advances of about US$ 1.5 million that needed to be refunded by the Government to the Bank . In addition the Bank still held about $5 million in undisbursed funds pending closure of the loan . Following Bank pressure to resolve the situation (a letter to the Government dated July 1, 2010 stated that no new Designated Accounts will be allowed for new loans under negotiation until the advance has been refunded ), the Government finally submitted the Audit by the Auditor -General on October 5, 2010. This Audit confirmed a number of financial discrepancies. Finally the Bank closed the IFRC Grant on October 22, 2010 when an additional US$1.54 million that had been disbursed to ineligible beneficiaries was refunded . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Significant Sri Lanka is yet to develop effective Outcome : disaster preparedness and emergency response systems. Many government staff that were part of the tsunami response and the project are now posted to different agencies within the government and some have left the government entirely. Resources for completion of construction of housing for vulnerable groups and subsequent maintenance is uncertain. Bank Performance : Satisfactory Moderately There should have been more attention Satisfactory to M&E and ensuring an effective PMU . There were reported cases of people resorting to illegal sand mining, tree cutting to meet the local/personal demand. Bank supervision should have ensured that the environmental impacts of the housing reconstruction program were quantified and agreed on how they would be addressed, and by whom. See Section 8. Borrower Performance : Moderately Unsatisfactory The implementation delays caused by Satisfactory multiple changes to the implementing agency, generally poor supervision on the part of the Borrower, and fiduciary concerns warrant the lower rating . See Section 9. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Of the ICRs 13 lessons the most important six are presented : In disaster recovery homeowner -drive approach coupled with necessary technical assistance fosters ownership of the program among participants lending to strong results; however, the strategy needs to include special assistance to more vulnerable groups – female -headed households and seniors – in order to prevent disparities in construction completion . Homeowners under this approach had the incentive of constructing houses of their choice while adhering to specified safety guidelines set by the project . The approach also gave beneficiaries an opportunity to receive construction and supervision training increasing skills development within communities . This training will contribute to an increased likelihood that project outcomes will be maintained as more people have skills to construct and repair homes in the future . However, as was the experience of this project, female -headed households and those of seniors were disproportionately behind in construction. It is better to establish common standards and procedures for implementation in disaster recovery projects . Allowing varying approaches in implementation by different players in the same reconstruction program can contribute to inequity in the distribution of benefits among those affected by the disaster . The IDA-financed housing component utilized the homeowner -driven reconstruction model while at the same time a donor-driven model was also being used under the wider housing reconstruction program . The donor-driven model was financed by other bilateral donors and implemented primarily through NGOs . The latter model included the possibility of “top-ups� which involved homeowners receiving additional cash grants above the normal government housing grant . The distribution of “top-ups� was disproportionate between the South West and the North East, where the former received more . Greater consideration to the potential effects of using different approaches and increased communication between players with regard to distribution of beneficiaries throughout the project could prevent uneven results . Effective strategic communications, training and continuous social assessment and are critical components for project success and consolidation of gains . The lack of information dissemination under the housing component made it difficult in achieve commitment to implementation in some areas . Furthermore, poor dissemination of information created complications surrounding the reversal of the buffer zone policy and how this would affect households living in it . Training increased the capacity of homeowners and enabled communities to successfully reach target reconstruction and the skills developed by beneficiaries increased the pool of skilled labor contributing to the economy . In disaster recovery projects a comprehensive and institutionalized monitoring and evaluation system is critical for enabling informed decision making for policy and operations . While the project did involve reporting and monitoring through audits, surveys, and field visits among other mechanisms, the flow of information between the different levels of implementation was poor, and regular communication of this data to the central implementing agency, where policy decisions were being made, was not realized . This line of communication was further crippled by institutional reorganization midway through implementation . Without an effective grievance redress mechanism and associated documentation of grievances, the concerns of beneficiaries can go unattended and these eventually cause delays in implementation and disparities in outcomes . Grievance redress institutions related to land ownership and other housing complaints at the village, district and divisional level did not function at its full potential under the project . Grievances were also poorly documented . In the wake of such a large-scale disaster, the range of land ownership issues calls for consistent and strategic policy for assisting beneficiaries in dealing with related complications. In the case of this project, land demarcations were destroyed as well as titles, and in some cases landowners were forced to sell land (especially within the former buffer zone ) and without an effective grievance mechanism or system for monitoring beneficiaries it is difficult to address these issues adequately, thus compromising project outcomes . Accountability is an important component of emergency reconstruction . This project highlights the importance that must be given to establishing sound and transparent M&E arrangements for inputs and outputs, as well as outcomes . Given that frequent reorganization of the implementing agencies frustrated sound M&E, the Bank partially addressed this failure by putting in place alternative and independent verification and reporting arrangements . 14. Assessment Recommended? Yes No Why? The project may be of interest to IEG for future disaster or conflict studies as it presents a unique combination of the two. It would also be very interesting to compare and contrast the success of Bank -assisted tsunami recovery efforts around the Indian Ocean . The Region has suggested that the Assessment should also include two parallel housing projects (the North East Housing Reconstruction Project and the Puttalam Housing Project), where the owner-driven reconstruction approach was also used . 15. Comments on Quality of ICR: The ICR covered the comprehensive program unevenly : in great detail in some areas whilst in others clarity was lost . Some areas that could have benefited from further attention are the outputs by component which did not address all the activities planned in the MOP's Technical Annex, and the key performance indicator chart which was not complete in the ICR. Although outcomes were among the KPIs these were not discussed . a.Quality of ICR Rating : Satisfactory