ICRR 13886 Report Number : ICRR13886 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/13/2012 Country : Pakistan Project ID : P099110 Appraisal Actual Project Name : Pk: Pakistan US$M ): Project Costs (US$M): 400 700 Earthquake Erc L/C Number : C4134 Loan/ US$M ): Loan /Credit (US$M): 400 700 Sector Board : Urban Development US$M): Cofinancing (US$M ): Cofinanciers : Board Approval Date : 12/15/2005 Closing Date : 06/30/2009 05/31/2011 Sector (s): Housing construction (40%); Sub-national government administration (30%); Central government administration (20%); Other social services (5%); General education sector (5%) Theme (s): Natural disaster management (40% - P); Rural services and infrastructure (20% - S); Participation and civic engagement (20% - S); Social safety nets (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Peter Nigel Freeman Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The project development objective in the Technical Annex (page 4) to the Memorandum and Recommendation of the President (approved by the Board on December 15, 2005) was "to support the efforts of the Government of Pakistan to: (i) reduce the immediate suffering resulting from the effects of the earthquake and restore livelihood destroyed by the earthquake; (ii) restore basic services to the affected population and rebuild public infrastructure; and (iii) start the recovery and reconstruction process ." The statement of objectives in the Development Credit Agreement was to assist the Recipient in carrying out its emergency recovery and reconstruction program in the Affected Areas, following the massive destruction and devastation that occurred as a result of the Earthquake . The PAD's statement of objectives will be used in this Review since it is more monitorable . In 2010 devastating floods struck Pakistan . The Bank responded with an additional credit of US$ 300 million, while this Emergency Recovery Credit was restructured to accommodate the Government ‘s emergency resource needs to address flood related recovery and reconstruction costs . There was a greater demand for imports, and in particular, fuel used by helicopters, jets, and other equipment during the emergency operation . One of the country's largest refineries was closed in August, 2010, because of flood damage to roads and bridges providing access to the refinery. Repairs were expected to take at least 7 weeks, suspending operations until the second -half of September. The refinery, according to the Project Paper, accounts for some 20 percent of Pakistan’s refinery capacity, producing each month 60,000 tons of motor gasoline, 60 tons of jet fuel and 100,000 tons of diesel fuel. The closure translated into a loss of fuels worth over US$ 150 million, which would need to be imported in order to sustain adequate fuel supplies in the country. Consequently, an additional US$ 300 million was provided under component 3 on import financing, increasing the overall project cost by more than four -and-a-half times from US$ 85 million to US$385 million. There were, however, no changes made to the original design and implementation modality of component 3, as defined and appraised during preparation . The project objective remained the same except for an additional clause "to enhance the resources available to adequately meet the early recovery needs of the people affected by the 2010 floods". Therefore, no split evaluation prior to and following restructuring is necessary . b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 09/30/2010 c. Components: Component 1: Housing Reconstruction (US$220 US$ 220 million estimated and actual ). This component was to finance : (a) housing assistance in the form of cash grants for replacement of destroyed homes with new seismic resistant core units and the restoration and strengthening of damaged homes to seismically acceptable standards, with eligibility determined by a detailed damage and eligibility survey; and (b) technical assistance and capacity building to support the Earthquake Reconstruction and Rehabilitation Authority (ERRA) and provincial and local governments in overall project management, reporting, monitoring and evaluation, and compliance with the social and environmental framework, development of a grievance redress mechanism, among other activities . Component 2: Livelihood Support (US$85 US$ 85 million estimated and actual ). This component was to address the dual objectives of protecting the most vulnerable households in the short -term through the provision of cash grants, and rejuvenating economic activity by reviving small businesses and replacing assets lost in agriculture and livestock . Cash grants were to be provided for monthly income support to affected households (for example, households that were displaced due to housing damage or lost their main source of livelihood – shop/business, livestock or agricultural land). Preference was to be given to needy or vulnerable households, for example those with a high number of dependents to support, such as the disabled, elderly, children, and injured . Component 3: Import Financing (US$85 US$ 85 million estimated; actual US$ 385 million ). This component was to assist the GOP in early reconstruction and rehabilitation efforts while maintaining macroeconomic stability by financing critical imports related to reconstruction and rehabilitation of earthquake affected areas . The component was based on a quick-disbursing mechanism to help to finance petroleum related imports . The imports would in turn support early reconstruction, rehabilitation of assets, and restoration of economic activity, while mitigating pressure on the balance of payments. Imports financed under this component were to include fuel and petroleum products and other construction material, prefabricated and temporary structures for providing medical services, and construction and earth-moving machinery and spare parts . Component 4: Capacity Building (US$10 US$ 10 million estimated and actual ). This component was to support the strengthening of the existing implementation capacity of all levels of government to implement and to monitor the recovery program, as well as to meet the heightened demand for project management and oversight . These measures were to include the provision of human resources and consultancy inputs at the federal, provincial and district levels, technical assistance for continuous social impact assessment and third party evaluation, sector-specific training programs, the provision of information technology connectivity and the establishment of management information systems, as well as the purchase of basic office supplies, infrastructure, and urgently needed vehicles. In addition, technical assistance was to be provided to develop a strategic and effective hazard risk management system in Pakistan. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost: The estimated project cost at appraisal was US$ 400 million and the figure at closure was US$ 700 million. Financing: The entire cost was funded by the Bank as an Emergency Recovery Credit . An additional US$300 million was provided in 2010 to help cope with the subsequent flooding disaster . The credit was fully disbursed . Many other donors participated in assisting the Government . Their individual contributions are not recorded, but in total the pledges amounted to US$5.9 billion of which US$2.0 billion took the form of grant support and US$ 3.9 billion was in the form of concessional loans . Borrower Contribution: None Dates: The project closing date was extended by 23 months prior to the restructuring even though 97 percent of the work (at that time) had been completed. GOP indicated it wanted to complete the disaster risk management aspects and the Bank acceded. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High The 2005 earthquake and the more recent 2010 severe floods were among the worst natural disasters in Pakistan ‘s history. Tens of thousands of lives were lost and there was severe public and private property damage and destruction. Resource requirements to address the recovery and reconstruction needs of Pakistan were consequently huge. The disaster had a vast impact on the country and directly affected the achievement of the Country Assistance Strategy (CAS) pillars and the Bank program current at the time . The objectives of the project, specifically to restore vital economic and social infrastructure were nevertheless in line with pillar I, strengthening macroeconomic stability and pillar III, supporting pro -poor and pro-gender equity policies. Through supporting the Government in reconstruction the project was able to soften the macroeconomic effects of the disaster and most importantly, support those affected by the natural disasters . The following CAS (2005—2009) took the impact of the earthquake disaster into account and highlighted the vulnerability of the poor to shocks by natural disasters under pillar III, "Improved Lives and Protection of the Vulnerable ". The current Country Partnership Strategy (2010—2013) extends this concept by focusing on improving disaster risk management and reducing the country ‘s vulnerability to natural disasters. The project objectives were also in line with the Government ‘s poverty reduction strategy, which recognized the susceptibility of the poor to natural disasters and the need for their protection . Finally, the new National Disaster Risk Management Framework calls for disaster risk reduction through structural and nonstructural mitigation measures. b. Relevance of Design: High The design of the project conformed to the required rapid response to disasters and emergencies (O.P. 8.0) . The causal chain between outputs and intended outcomes was clear . The housing component directly supported the Rural Housing Reconstruction Strategy of the GOP . The livelihood component aimed at addressing the immediate basic needs (including food items) of those affected by the earthquake . The import financing component directly supported the financing of imports of items required for reconstruction and rehabilitation activities . Capacity building support helped the Government to better manage the recovery and reconstruction phases of the disaster response . Under the project, all structures were to be constructed under the "Build Back Better" model and were seismic resistant in nature. 4. Achievement of Objectives (Efficacy): Reducing the immediate suffering resulting from the effects of the earthquake and restore livelihood destroyed by the High . Cash grants were provided for monthly income support to affected households . According to the earthquake. High. ICR (page 34) preference was given to needy or vulnerable households, such as those with a high number of dependents to support, including the disabled, elderly, children, and injured . The initial target population was 250,000 eligible households, but over 280,000 families benefitted from the cash grants by the end of the project with full disbursement. Restoring basic services to the affected population and rebuild public infrastructure . Substantial . Under the ERRA program, a total of 353,931 houses were reconstructed . Out of these, 337,177 houses (95 percent) were completely reconstructed and certified as compliant with the hazard resistant construction standards . Under the quick disbursing arrangements 82 percent of the allocated amount had already been disbursed by October, 2006, less than 9 months after effectiveness. The target of 528,000 beneficiaries living in earthquake resistant houses was reached . Under the project, individual families used the financial assistance to rebuild their houses . The money was disbursed in installments and nearly all construction was in compliance with earthquake resistant building standards . High . Assistance with import financing accelerated the early Starting the recovery and reconstruction process . High. reconstruction and rehabilitation efforts by providing quick-disbursing assistance to partially finance the imports of items required for reconstruction and rehabilitation activities . Following the 2010 floods, it also helped to maintain macroeconomic stability through US$ 300 million budget support to finance critical imports by mitigating pressure on the balance of payments. Imports financed included fuel and petroleum products, building materials, prefabricated and temporary structures for providing medical services, and construction and earth -moving machinery and spare parts. The additional US$ 300 million supplemented the GOP‘s emergency resource needs to address the floods, in particular, fuel used by helicopters, jets, and other equipment during the emergency operation . The amounts concerned were fully disbursed . Stocks of motor gasoline and high speed diesel met the targets of 10 and 15 day's supply respectively. In addition to meeting its specific development objectives, the project contributed to pilot testing a number of innovative approaches especially in terms of participatory monitoring and evaluation through on -line and real time systems which are expected to be adopted by the Government in future disaster response operations . 5. Efficiency: Modest The project was an emergency response operation and thus no economic analysis was undertaken at appraisal . The exact cost of the earthquake damage was difficult to quantify . Approximately 73,000 people died and more than 70,000 sustained severe injuries or were rendered disabled . Over 2.8 million persons were left without shelter and 2.3 million persons without adequate food . More than 400,000 units of housing were destroyed or damaged, but with an investment of US$ 400 million under the project, almost the entire housing stock was restored and all households affected received some relief in the form of cash grants . Similarly, the 2010 flood disaster and needs assessment showed that the overall economic impact of the floods on the Pakistan economy was equivalent to about 5.8 percent of GDP. The relief program was a multi-donor effort, but the timely intervention of a further US$ 300 million for emergency import financing to mitigate some of the potential effects of the disaster was a sound humanitarian intervention for the Bank and its partners to undertake . No least cost analysis of these expenditures was undertaken at closure . Some inefficiencies may have resulted from immediate action on essential emergency procurements at the outset of implementation since there was, at the time, no procurement plan, a procurement officer had not yet been appointed, and the Bank's prior approval was not obtained. Moreover, there were some start-up delays of about four months in the finalization of the eligibility criteria and methodology for the launch of the livelihood component . Otherwise, most activities were quick disbursing and succeeded in bringing about relief as speedily as possible . No reconstruction work was undertaken in about 3.8 percent of the houses (13,383 houses), and, around 0.95 percent of houses (or 3,371 houses) were found noncompliant with ERRA‘s construction standards upon inspection to the extent that the deficiencies could not adequately be addressed through retrofitting . No explanation is given for the reasons why no reconstruction work was undertaken in some instances, but according to the ICR (page 10 ) a challenge which emerged was the reluctance of the Government to invoke the penalty clause that required the house-owners to complete reconstruction within a stipulated time . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of the objectives and design were high . Two of the objectives were highly achieved and one was substantially achieved. However, it is not possible to say how efficient the project was since no cost benefit or least cost analysis was carried out and some deficiencies were recorded . Efficiency is therefore rated as modest and the overall outcome as moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The overall risk to development outcome at the time of project closing was rated by the ICR as substantial - a rating that does not conform with the terminology of the guidelines - but this review takes this to mean significant . Institutional risks are moderate . ERRA has been established and given the mandate to oversee future disasters . Although ERRA built significant capacity over the course of the project and displayed satisfactory performance, it is currently thin on staffing as many of its employees have left . However, natural disasters risks remain substantial in Pakistan. The affected regions have a history of major disasters, including floods, seismic events and landslides . Although the program helped to establish seismic resistant construction standards aimed at safeguarding against a similar scale of damages, it is difficult to ascertain their sustainability in the event of any future disaster . However, it appears that the program has contributed to stronger Government ownership and commitment toward disaster risk reduction and management. The Government has shown strong interest in enhancement of its disaster preparedness capabilities and considers the project a success . At present, the risks due to lack of government ownership and commitment are small. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: During the design phase, the project team ensured that development objectives and the project scope conformed to the emergency nature of the needs that had arisen due to the disaster . The Bank team also maintained full liaison with the government entities to ensure that the project activities were in line with the government priorities in response to the earthquake . The Bank drew on its experience in supporting emergency operations in Pakistan and elsewhere . Approaches applied in these and other operations were adopted in the project design. These included flexible and demand driven design, continuous oversight by local and state authorities, use of supervisory and implementation consultants to augment capacity, enforcement of performance based criteria, effective procurement planning and contract management arrangements . In addition to the conventional contractor driven reconstruction, community -driven reconstruction model, particularly the home-owner driven program, were also catered for in the project design . Other useful approaches applied in this project design include training of local builders, formulation of operational manuals clearly defining roles, responsibilities and procedures for implementation by multiple actors, formulation of construction guidelines and requirement of supervision mechanisms . The key risks were the limited capacity and the highly inaccessible environment due to inclement weather . The capacity risks were mitigated through ongoing capacity support . at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: The Bank team conducted regular supervision missions with at least one and sometimes two or three missions per year. In addition, the Task Team regularly discussed key issues and the way forward with government, stakeholders and counterpart staff through face to face meetings and audio conferences . The Task Team also regularly reviewed various technical reports and financial audits and discussed findings and required actions with the government. Supervision missions regularly met with various stakeholders, periodically inspected physical works in the field and conducted interviews with beneficiaries and community groups . However, efficiency in the use of project resources should have been better monitored and measured during implementation and at project closure . Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government managed the significant challenge created by the earthquake in a timely and effective manner. Realizing the importance of looking beyond relief and focusing on reconstruction of destroyed houses and provision of livelihood support to those affected was a strategic decision that paid dividends over the longer term. The government provided support to the project and ensured timely completion of activities as well as meeting the targets outlined as part of the PDO . Government Performance Rating Satisfactory b. Implementing Agency Performance: The implementation agency (ERRA) was given the challenging task of managing recovery and reconstruction on a massive scale in the aftermath of the earthquake . It was a complex task considering the scale of the disaster, inclement weather conditions and difficulties of access to all those affected due to diverse terrain . Furthermore, it had to coordinate with a large number of stakeholders including the local governments, the army, donors, the United Nations agencies and the NGOs to ensure coherence and equitable distribution of the assistance. However, the role of the army‘s monitoring team in the reconstruction program was difficult to coordinate. The two formations of the army in the two provinces were reporting to different channels making it difficult to harmonize communication on housing standards . But, In spite of such challenges, market conditions and the capacity of industry to meet increasing raw materials demand, ERRA managed to complete the project activities and meet the targets set for the project . Since the nature and geographic scope of the flood event of 2010 was beyond the mandate of ERRA, the Planning and Development Division (PDD) of the GOP became the implementing agency for the import financing component. The PDD was the main institution at the federal level responsible for flood response coordination, as well as donor interface. This arrangement did not impede the achievement of the intermediate outcomes under component three or the attainment of the PDO . Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Due to the complex nature of the activities in the post -earthquake scenario and the large number of beneficiaries targeted over a diverse geographic area along with access challenges, the program established an integrated M&E system that would enable mid-course corrections and promptly identify any bottlenecks to undertake corrective measures. The Bank and ERRA created an effective reporting, monitoring and evaluation system that enabled the decision makers to utilize the information in order to make informed decisions . b. M&E Implementation: The M&E system was initially geared towards the housing reconstruction component . A number of monitoring indicators were tracked during the implementation including physical reconstruction (number of houses reconstructed in the affected areas), disbursements (the flow of funds and the number of beneficiaries ), and the number of master trainers educated. The central reporting, monitoring and evaluation (RME) system housed at ERRA received inputs from a number of sources, including amongst others; (i) ERRA and army monitoring units/mobile teams in the field; (ii) a National Database and Registration Authority; (iii) database of banks and; (iv) partner organizations operating at the union council/community level. The M&E arrangements for the livelihood support component primarily monitored disbursements and the number of beneficiaries . To ensure support to the beneficiaries, the Bank and ERRA, in partnership with the National Database and Registration Authority, created an on -line Grievance Redress Mechanism to identify and address any grievances and issues of the beneficiaries . c. M&E Utilization: Through the RME, ERRA was able to identify trends, particularly related to the quality of housing in given areas or the number of disbursements vis -à-vis number of houses reconstructed in a given district . To deal with the issue of noncompliance and to take corrective measures the integrated system was able to identify, document, analyze and provide solutions for noncompliance leading to higher levels of compliance in the reconstruction of houses . M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The project was classified as Category "B". The ICR reports that safeguard policies as required under Bank policy were adequately integrated in the project design . An Environmental and Social Screening and Assessment Framework was prepared to undertake environmental impact assessments for all rehabilitation and reconstruction activities. The assessments were aimed at ensuring that adverse environmental and social impacts were minimized and that appropriate mitigation measures were included . In addition, a continuous social impact assessment (CSIA) was recommended to facilitate the articulation of community perceptions, grievances and feedback vis -a-vis identification of beneficiaries for the housing and livelihood components and the mobilization of resources . However, the CSIA was not carried out. Instead the Government implemented a grievance mechanism to receive and address any complaints lodged by those affected by the earthquake .The system provided a "helpline" facility via telephone, fax and email for those affected . According to the ICR (page 21), this mechanism was effective and robust . b. Fiduciary Compliance: Several Financial Management risks were identified at the appraisal stage of the project, which highlighted the dearth of financial management expertise in ERRA, weak auditing capacity, potential mismanagement of livelihood cash grants and an inability to comprehend the complexities in funds flow arrangements . To mitigate these risks, the project had provisions for hiring appropriate staff to build capacity at the implementing agencies . It was thus able to address most of these issues with support from the Bank . ERRA also issued an Accounting Procedures Manual to provide guidance regarding general and specific payments and internal control processes . Appropriate staff was placed in the Internal Audit Unit. However, various Bank missions reported that under -staffing remained a persistent issue. The submission of annual audited Financial Statements for FY 06-07 were delayed by approximately four months. However, subsequently these delays were avoided and the Bank found the accounts to be acceptable . Bank Implementation Status Reports rated financial management moderately satisfactory throughout implementation (except the first report which gave a a satisfactory rating ). Quarterly Financial Monitoring Reports were prepared by ERRA for all operations undertaken by the entity, which also separately identified the sources and uses of funds relating to major donors. This approach was fully subscribed to by the donors and presented a complete picture of reconstruction activities managed by ERRA and its subsidiaries . The procurement risk identified for the project was rated as high, and at the inception of the project there was delayed recruitment of a procurement officer and preparation of a procurement plan .However, ERRA adopted the Pakistan Procurement Regulatory Authority Public Procurement Rules 2004, with a few specific adjustments to allow it to deal with the nature of the emergency project . In addition, ERRA developed a standard operating procedure for handling of procurement and set up appropriate bid evaluation and procurement committees . A separate procurement cell, housed within the Administration and Procurement Wing was established at ERRA . The ICR does not specifically say that the emergency procurements made at the outset without Bank authorization were eventually recognized. The first two Implementation Status Reports rated procurement as moderately satisfactory, but the nine subsequent ones assessed it as satisfactory . c. Unintended Impacts (positive or negative): The interventions in disaster risk management under the project informed the country -wide disaster risk management programs especially in the area of community -based disaster risk management . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately It is not possible to say how efficient Satisfactory the project was since no cost benefit or least cost analysis was carried out and some deficiencies were recorded . Risk to Development Significant Significant Outcome : Bank Performance : Satisfactory Moderately Efficiency in the use of project Satisfactory resources should have been better monitored and measured during implementation and at project closure . Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons are taken from the ICR with some adaptation of language : Clarity of Implementation Arrangements : One of the major factors that contributed towards timely completion of the project was the clearly defined implementation mechanism covering the federal, provincial /state, district and local level governance of the earthquake response . For the reconstruction activities, ERRA created parallel structures at the provincial /state levels and at the district level in the form of reconstruction units . In addition, partnerships were established with national and international NGOs with clearly defined roles and responsibilities which helped in ensuring timely and equitable assistance to the communities . Importance of coordination with all stakeholders : The earthquake response operation at its peak involved more than 80 stakeholders involved in housing reconstruction alone . To ensure coherence and avoid duplication, a Housing Strategic Working Group was established . By doing so, ERRA ensured strong commitment to common principles from among a very diverse range of partners . Coordination mechanisms were also set in place at federal level between ERRA, the army, and institutional partners . At village level coordination was carried out between the army, assistance and inspection teams, training mobile teams and village reconstruction committees . Owner Driven Reconstruction : The dividends of such a large reconstruction project can only be fully realized if an owner-driven, pro-poor policy ensuring equity is uniformly applied across an entire disaster affected area . Under the project, individual families used the financial assistance to rebuild their houses . The money was disbursed in installments and in accordance to compliance with earthquake resistant building standards . Anticipating and planning for cost escalations and capacity of construction industry : The increased investment in construction of destroyed houses placed pressure on the local capacity of the industry and saturation of the market due to increased demand and ultimately resulting in cost escalations . The project had to face serious risks in meeting the high demand for materials by allowing importation from neighboring countries. Mainstreaming disaster preparedness in emergency reconstruction projects can be of high value provided it does not delay the provision of urgent assistance to populations in need . This project was able to focus on reconstruction and livelihood support, while also supporting the creation of a culture of preparedness in the earthquake affected areas that was later replicated in the entire country . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is easy to read, admirably concise and contains some useful lessons . However, there is insufficient detail under efficacy to properly evaluate the outcome and this information has to be pieced together from other sections of the report. There was also insufficient information in respect of efficiency since neither a cost benefit nor a least cost analysis had been carried out . Some minor discrepancies in the figures are unexplained e .g. Annex 2 says the livelihood disbursements amounted to US$ 81 million, but Annex 1 says it is US$85 million. The overall risk to development outcome at the time of project closing was rated by the ICR as substantial - a rating that does not conform to the terminology in the guidelines . a.Quality of ICR Rating : Satisfactory