Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) Report Number : ICRR0021013 1. Project Data Project ID Project Name P125805 HT Neighborhood Housing Reconstruction Country Practice Area(Lead) Haiti Social, Urban, Rural and Resilience Global Practice L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-99644 30-Jun-2015 65,000,000.00 Bank Approval Date Closing Date (Actual) 04-May-2011 31-Dec-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 65,000,000.00 65,000,000.00 Revised Commitment 64,871,115.93 64,871,115.93 Actual 64,871,115.93 64,871,115.93 Prepared by Reviewed by ICR Review Coordinator Group Maria Shkaratan Lauren Kelly Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives Project Development Objective (identical in the Emergency Appraisal Paper and in the Haiti Reconstruction Fund Grant Agreement): “To help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by supporting them to repair and/or reconstruct their houses and improving basic community service infrastructure.” (Grant Agreement, p. 6) The revised PDO was: “To help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by supporting them to repair and/or reconstruct their houses and/or return to improved Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) housing conditions and improving basic community service infrastructure.” (Restructuring Paper, May 3, 2011) (the phrase in italics was added at revision, no other change was made as compared with the original PDO). The PDO was revised in October 2012, during the first restructuring (level 1). As such, this ICRR conducts a split rating. The first part of the objective statement changed significantly from "repair and/or reconstruct [their] houses" to "repair and/or reconstruct their houses and/or return to improved housing conditions." The second part of the objective statement, "improving basic community service infrastructure" did not change. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 11-Oct-2012 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components There were four components: (Please note that IEG was not able to obtain disaggregated component costs. Actual costs for component 1 and 2 were combined in the ICR). Component 1: Debris Removal and Housing Repair and Reconstruction (appraisal: US$37.00 million; actual costs are reported as total of Component 1 and Component 2 costs: US$59.88 million): (i) the removal of debris; and (ii) the safe demolition of Red-tagged Houses and Buildings, and houses considered unfit for repair. The component was supporting residents to: (i) repair and/or retrofit the Yellow-tagged houses and buildings (“yellow” means repairs were needed), through hiring of local contractors; or (ii) the on-site reconstruction of Red-tagged Houses and Buildings (“red” means that the houses had to be destroyed and rebuilt), and houses unfit for efficient repair through the provision of Cash Grants to Beneficiaries. “To the extent feasible, works will not be limited to the repair of damages but include structural reinforcement of houses to withstand possible future earthquakes.” (Appraisal Paper, p. 18). The component would also provide TA to support communities, including Neighborhood Development Councils (NDCs) in reconstruction activities. Component 2: Community Service Infrastructure Repair, Improvement and Extension (appraisal: US$20.00 million; actual costs are reported as total of Component 1 and Component 2 costs: US$59.88 million). Carrying out of: (i) repairs, improvement and extension of basic community infrastructure, including roads, walkways, drainage ditches and channels, solid waste management facilities, water supply systems and sanitation facilities; and (ii) construction of Community Reconstruction Centers (CRCs) to become one-stop shop for consultations and training to serve the targeted Port-au- Prince Neighborhoods. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) Component 3: Institutional Capacity Support and Studies (appraisal: 4.0m; actual costs combine the actual costs of Component 3 and 4 and are reported as 4.58 million). (i) the development of neighborhood-level urban restructuring plans and natural risk maps; (ii) the preparation of medium and long-term urban development and housing studies and strategies, and associated policy and administrative measures; (iii) the carrying out of technical and environmental/social safeguard audits; and (iv) the provision of capacity building to GoH agencies and municipalities. Component 4: Project Management (US$2.00 million; actual costs are reported as total of Components 3 and 4 costs: US$4.58 million). Supporting the implementing agency Bureau of Monetization of Development Aid Programs (BMPAD) in coordinating the Project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project costs: The total project cost estimated at appraisal was US$65.00 million. Actual cost was US$64.79 million (ICR, Annex 1, p. 21). However, the actual disaggregated component costs are not provided (i.e. for the specific activities). Project financing: The project had one financing source: the Haiti Reconstruction Fund (HRF). The financing was tagged as an USAID/United States (US) Foreign Disaster Assistance (OFDA) contribution. HRF was set up as a partnership between the GOH and the international community to finance reconstruction after the earthquake. The World Bank became a partner entity for the implementation of the HRF’s funds. Dates: The project was approved on April 4, 2011 and became effective on July 28, 2011. The mid-term review was originally scheduled for July 15, 2013 but was later rescheduled for June 30, 2014. The original closing date was June 30, 2015, the actual one was December 31, 2016. The project was restructured three times: • The first restructuring (Level 1) was on October 11, 2012 when the amount disbursed equaled to US$4.98 million. The restructuring was initiated in response to a request from the GoH (Ministry of Finance) to accelerate the return of people to safe neighborhoods by introducing new support mechanisms in the form of Return Cash Grants. These Grants included a rental subsidy, livelihood subsidy and transportation assistance, and an associated package of services including health, psycho- social, protection, water, sanitation and hygiene assistance, and vocational training. The introduction of the Grants did not require reallocation of funds among project components. The first restructuring had two objectives: accelerated return of the population home from camps and increased infrastructure re- building. In relation to the former, the PDO was amended (see section 2.b). To accomplish the latter, some of the funds in Component 3 (Institutional Support and Studies) and 4 (Project Management), as well as contingencies, were moved to Component 2 (Repair and Improvement of Community Infrastructure). As a result, Component 2 gained US$4.6 million. Also, the target value of one key indicator was increased: the number of displaced households who have returned to neighborhoods upon housing reconstruction was changed from 12,000 to 16,000. In addition, a new intermediate Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) indicator was added: Indicator 8: Number of households provided with Return Cash Grants (see description of these grants above). • The second restructuring (Level 2) was on February 12, 2015 when the amount disbursed was US$49.04 million. It included a further increase in the emphasis on infrastructure (by re-allocating funds to Component 2) and an extension of the project closing date from June 30, 2015 to June 30, 2016. The ICR explains that the need for extension is due to the challenge of working in the post-earthquake environment and the extremely limited capacity in the country (ICR, p. 12) and by social unrest and tense political circumstances (ICR, p. 42). Target values for a number of intermediary indicators were adjusted: some were increased, some reduced, and some defined to replace the “TBD” label • The third restructuring (Level 2) was on May 20, 2016 when the amount disbursed was US$62.86 million. It was needed to extend the project closing date from June 30, 2016 to December 31, 2016. The ICR explains that this extension was needed to accommodate delays caused by the political situation related to the postponed presidential run-off (ICR, p.vii). 3. Relevance of Objectives & Design a. Relevance of Objectives Owing to the restructuring and the revised objective, the PDO assessment is spit against the two objective statements, pre and post the level one restructuring. The PDO prior to the Level 1 restructuring was: "to help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by supporting them to repair and/or reconstruct their houses and improving basic community service infrastructure.” (Grant Agreement, p. 6). The PDO was in line with the “Neighborhood Return and Housing Reconstruction Framework” (NRHRF) prepared jointly by the GOH and the donor community’s Haiti Reconstruction Fund (HRF) which sets out the principles of the earthquake restoration. The framework emphasizes improved safety of houses and neighborhoods on the principle of “build back better”; it promotes the idea that reconstruction contributes to urban renovation and regional development, as envisioned in the government’s long-term rebuilding plan; and it aims at closure of temporary camps and return of the population to the original neighborhoods and livelihoods. The PDO is also in line with the IDA and IFC Interim Strategy Note for The Republic of Haiti (CY 2012) whose first pillar is reducing vulnerability and increasing resilience. In particular, the strategy aims to strengthen Haiti’s capacity to respond, manage and prevent disaster related crises. It sets out a framework for sustainable reconstruction in post-earthquake Haiti with a focus on upgrading neighborhoods, housing repair, and transferring services from camps to communities to encourage people to return. While the project was in line with these strategic frameworks, the PDO language that focuses on the reconstruction or repair of housing was not directly in line with the needs of the target population most of whom were not homeowners. The intention to focus on repair or reconstruction was relevant in so far as it played to the Bank's comparative advantage of longer term resilience building ("build back better"), but this was an emergency operation confronted by physical, political and social challenges. The PDO was constructed in the absence of a sufficient understanding of home ownership prior to the crisis i.e. the number of vulnerable persons that owned versus rented homes. As such, the PDO was designed in a way that did Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) not allow for flexibility in a crisis situation as this information came on stream and different demands were placed on the project team. The PDO was therefore relevantly revised (Level 1) to reflect the reality on the ground given this new information. The PDO after restructuring was to "help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by supporting them to repair and/or reconstruct their houses and/or return to improved housing conditions and improving basic community service infrastructure." It included an additional focus, and hence more flexibility, by adding language related to the return of persons to improved housing conditions, which allowed for broader interpretation. It omits the need to rebuild or repair houses in recognition that the objective needs to be aligned with the most pressing needs of the persons affected by the earthquake (many of whom were at the time still living in camps and were not homeowners). The initial objective was cautious in its approach because of the Bank's hesitation to appear as if it was engaging in activities solely in the humanitarian domain. But in so doing, it limited its own flexibility. The revised objective, on the other hand, sufficiently places the Bank in a position that bridges the Humanitarian-Development nexus, an aim that it can pursue through its emergency operations. Therefore, the revised objective is rated Substantial. It is not rated high because it lacks specificity and it includes a metric ("improved") that would have required data from the pre-crises period for the beneficiaries. A better metric would have been housing that conforms to agreed upon standards (safe and resilient). Rating Revised Rating Modest Substantial b. Relevance of Design Project design is rated against the Original and Revised design. Original Design. The project was relevantly designed as an emergency operation that allowed it to have flexibility. However, the PDO was somewhat rigid since it featured an exclusive focus on the reconstruction of housing - versus other means of achieving safe housing - as was later identified and added under the restructuring. This is a lesson for future ERLs. The design was based on insufficient data regarding housing economics in Haiti so that, more than a year into project implementation, it became known that about half of the affected population and 78 percent of camp residents were renters. As such, the main project activity of rebuilding /repairing houses was not benefitting most vulnerable people, was leading to increased inequality, and was unable if carried our to achieve the main project objective and the associated PDO targets. The absence of agreed upon reconstruction standards present at design, between the GoH and the development partners - weakened the original design aim of "building back better" from a systems level perspective. However, it is noteworthy that the project incorporated a copious amount of structural and systems level analysis into the project's early stages, as part of the needs assessment: this includes the financing of the Structural Assessment of 400,000 buildings, the multi-hazard mapping of Port au Prince, and a full LIDAR assessment of the affected area. The result framework was adequately aligned with the two PDO(s) was but it was mostly output based. The results framework was not capable of providing meaningful information on the quality of the housing conditions provided for by the reconstruction, nor the quality of the community infrastructure that was built. It was also not developed in a way that could provide proxy indicators about the welfare of the persons that Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) received return support over time (did the subsidy smooth expenses for spending in areas such as health and education? are renters able to afford safe accommodation after the end of the subsidy? etc.) Revised Design. The revised project design introduced rent subsidization (which replaced housing reconstruction/repair) with a condition that privately rebuilt housing satisfies safety and resilience standards. The components stayed the same, and the only change to the results framework was a new intermediate level indicator that was added: Number of Households provided with Return Cash Grants. This was a relevant design change in line with the information required. The results framework remained relevant throughout the project life cycle. The new design also decentralized decision-making about the community infrastructure. The new design involved Neighborhood Development Councils (NDCs) who became the primary interlocutors for the coordination of project activities with neighborhood residents, helping the government identify the needs and priorities. Project implementation included households, NDCs, community organizations, municipalities, PMCs, government agencies, and other entities. The participatory process introduced for the selection of community infrastructure was very relevant form the point of view of ensuring fairness within and between neighborhoods affected by the crisis, to determine need, and to ensure ownership of the infrastructure. Overall, the revised design was relevantly in line with the urgent needs expressed by the GoH to move the displaced persons living in camps due to the disaster to safe housing conditions. The revised design incorporated checks on the "safety" of the private housing conditions. However, more attention should have been paid to putting in place a system to monitor and understand the ling term housing conditions and needs of the rental subsidy recipients. While project documentation indicates that most recipients of the rental subsidy did not return to camps, their present day whereabouts - and the longer term effects of the subsidies - are therefore less known. Rating Revised Rating Modest Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by supporting them to repair and/or reconstruct their houses. Rationale Outputs: 1. 16,099 displaced households have returned to neighborhoods upon completion of housing repair and reconstruction works and/or receiving return support. Of these, 2,078 households received housing repair. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) (This number was derived by subtracting the number of households that received return case grants from all housing beneficiary households). The original target was 12,000 households (for housing repair). 2. Against an original target of 8,300, the project repaired 1,134 housing unit were repaired. This target was twice revised downwards from 8,300 to 3,000 during the 1st restructuring to 1,162 as part of the 2nd restructuring. While it is clear that the 1st restructuring would have significantly reduced the target due to the fact it replaced housing repair with rental subsidies, it is not clear why roughly only two-thirds of the revised target (2nd restructuring) was not met. 3. Against an original target of 3,700 houses built (compliant with standards), 944 were built. This target was twice revised downwards from 3,700 to 1,000 during the initial restructuring to 938 during the second restructuring. 4. 2,405 metric tons of debris was cleared in the targeted neighborhoods. No original target was set. A revised target was put in place one year before closing of 1,154 metric tons, however it appears this target was the actual amount of debris removed in 2015. 5. The project directly benefited 348,163 people (through the implementation of all project objectives). The original target was 210,000 people. The actual achievement constituted 166 percent of the target. The project had 82,089 female beneficiaries. The original target was 105,000 female beneficiaries. The actual achievement constituted 173 percent of the target Outcomes: The original PDO sought to help earthquake victims return to their communities. The project partially achieved this original aim. It helped a total of 16,099 displaced households return (against an original target of 12,000, or a 25% increase). However, the project was not able to help return all of these displaced households to their original communities. And, it achieved this through different means than originally envisioned. As discussed below, as part of the restructured first objective, the project replaced its housing targets (build and repair) with rental subsidies. As such, it achieved only a fraction of the housing repair that it originally planned (1,134 repaired units versus 8,300; and 944 houses were built as compared to 3,700 houses built (compliant with standards), 944 were built. This target was twice revised downwards from 3,700 to 1,000 during the initial restructuring to 938 during the second restructuring. A note on standards: The issue of "standards" was and remained contentious between the donors during project implementation. It was difficult to agree to a common set of standards. The Bank included standards related to the structural reinforcement of houses to enable them to withstand the effects of possible future earthquakes. However there is very little information in the ICR on this issue. Rating Modest PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by supporting them to repair and/or reconstruct their houses and/or return to improved housing conditions. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) Revised Rationale Outputs: 1. 16,099 displaced households have returned to neighborhoods upon completion of housing repair and reconstruction works and/or receiving return support. Against a target of 12,000, 14,021 households were returned with the use of Return Cash Grants - a modality added at restructuring (117%). 2. 2,405 metric tons of debris was cleared in the targeted neighborhoods. No original target was set. A revised target was put in place one year before closing of 1,154 metric tons, however it appears this target was the actual amount of debris removed in 2015. Outcomes Of the displaced households that received cash grants, none of the grantee households returned to the camps within one year. The cash grants allowed 100% of all grantees to autonomously find an accommodation solution. By the second year, only 25% of grantees remained in the same rental accommodation. In the absence of data, their whereabouts are unclear. Revised Rating Substantial PHEFFICACYTBL Objective 2 Objective To help residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by improving basic community service infrastructure. Rationale Outputs 1. The project directly benefited 348,163 people (through the implementation of all project objectives). The original target was 210,000 people. The actual achievement constituted 166 percent of the target. The project had 82,089 female beneficiaries. The original target was 105,000 female beneficiaries. The actual achievement constituted 173 percent of the target. 2. 75,035 households benefited from community-wide upgrading. The original target was 42,000 households. The actual achievement constituted 179 percent of the target. 3. 288 community infrastructure repair and rehabilitation contracts were completed. The original target was 277 contracts. The actual achievement constituted 104 percent of the target. These included roads and corridors, drainage ditches and channels, small bridges, electric panels, and sanitation. Most subprojects were properly executed. 4. 2,405 metric tons of debris was cleared in the targeted neighborhoods. No original target was set. A revised target was put in place one year before closing of 1,154 metric tons, however it appears this target was the actual amount of debris removed in 2015. 5. Construction of CRCs (Community Reconstruction Centers) was completed on time: CRCs were established on time and were fully staffed and effective. Outcomes Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) The project helped residents of selected Port-au-Prince Neighborhoods severely affected by the Earthquake return to their communities by improving basic community service infrastructure. Of the 288 demand driven local infrastructure activities implemented, most of these were properly executed. However, these activities lacked design standards. For example, while roads were constructed, they may have been constructed using cobblestones (less expensive and available) that are not resistant to heavy traffic. Other corridors were constructed with cement (which was more expensive, but also more resilient). However, it is unclear if infrastructure was rebuilt according to standards, the ICR provides mixed information on this matter. In relation to these activities, the ICR comments: “Most sub-projects were properly executed, however, there was no quantitative criteria for sub-project selection, which led to over or under-engineer sub-projects, i.e., some main roads were constructed using cobblestones that are cheaper, but are less resistant to heavy traffic, and some corridors were made with cement (more expensive and very resistant).” There is no other information on the subject making it hard to judge whether this comment means that Objective 2 was not fully achieved or not. Also, the ICR provides insufficient information regarding TA provision (Component 3) (ICR, p. 13, p. 14, p.16, p. 24) making it difficult to verify activity implementation within Component 3. Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Efficiency Ex-ante At appraisal, cost benefit analysis was not carried out as the reviewed project is programmatic, with an adjustable design (demand driven). Ex-post ERR, NPV, and benefit cost ratio: The project is economically viable with a net present value (NPV) of US$7.3 million, with an economic rate of return (ERR) of 19%, and a present value of benefit-cost ratio of 1.1. The discount rate of 6% was used. The project remains economically viable at discount rate 8% with NPV equaling to US$5.3 million, ERR of 19%, and present value of benefit-cost ratio of 1.1. Scope of the analysis: The ex-post economic analysis was prepared for the ICR. It covers the 2011-2026 period, up to 10 years after the project closure. The analysis used five indicators to quantify the benefit for the population of the return to the restored/repaired housing. The following indicators were included in the calculations: • Cholera avoided due to the return of over 16,000 displaced households (equivalent to over 75,000 people) in restored/repaired houses. • Land value appreciation associated with the 20,600 linear meters of road and corridor construction, as Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) well as the construction of small bridges. • Drainage and other improvements for up to 12,000 beneficiaries. • Houses repaired, retrofitted and constructed as additional “economic” rent collected and considered proxies for renters and homeowners living in their houses. • Construction of 367 latrines in houses. While these benefits are likely, the information in the ICR regarding the approach to the economic analysis and the assumptions tested are insufficient to validate the ERR methodology. Unit Cost Analysis A major weakness in the ICR is the lack of reporting on unit costs. Total costs, of all activities are reported together. There is no benchmarking of costs - for activities that varied from debris removal to housing repair to community infrastructure. Without unit cost analysis, and some benchmarking, this validation exercise is not able to determine the cost-effectiveness - or the value for money - of the activities carried out. It goes without saying that costs would have been high - but it is critical in large trust funded programs that use official development assistance that all costs are broken down, that there is learning about "what it takes" to respond to disasters and emergencies (across all cost categories) and that analysis be conducted on the value for money of the different activities to help future projects prioritize their investments. There is also no analysis of the relative value that accrued to the relatively few beneficiaries that received housing construction support, as compared to other activities. Administrative Efficiency The project started with a slight delay and implementation was also delayed by 18 months. The project was affected by such exogenous factors as additional catastrophic events, political and social instability, poverty prevalence, and low government capacity. ICR states that the administrative processes were as efficient as possible in this challenging environment. The extension of the Grant closing date by 18 months is therefore understandable. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate  19.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) 6. Outcome The overall outcome rating is Moderately Satisfactory. The relevance of the original objective is modest while the relevance of the revised objective is substantial. The relevance of the original design is rated modest while the relevance of the revised design, that allowed for greater flexibility in this emergency operation is substantial. The efficacy of the original objective including support for housing reconstruction is modest while the relevance of the revised first objective is substantial. The efficacy of the second objective, which did not change, is substantial. Efficiency is rated modest, owing mostly to the lack of unit cost analysis. This is a $65 million dollar project, of which US$5 million was spent prior to restructuring. Thus the bulk of the funds were spent after restructuring. The restructuring, coupled with the revised PDO and new design elements (e.g. rental subsidies) increased the relevance and effectiveness of this operation for vulnerable populations affected by the 2010 earthquake. Per the split rating formula, the original project scores unsatisfactory and the revised project scores moderately satisfactory. Since 10% was spent prior to restructuring, and 90% was spent afterwards, the calculation is as follows: 2 x 0.1 = 0.2 pre restructure and 4 x 0.9 = 3.6 post restructure leading to an overall 0.2 + 3.6 = 3.8 rounded up equals an overall rating of 4. This is a Moderately Satisfactory outcome rating. a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating Overall sustainability of the development outcomes are substantial. Safe and Affordable Housing. This rating mainly relates to the lack of information that is available, at a project and system level, about the current condition of the housing that is being accessed by the project related beneficiaries. Rental subsidies supported the reintegration of households into neighborhoods. To increase the sustainability of this scheme, the project financed psychosocial support, vocational training, livelihood development programs, access to clinics, and connected the beneficiaries to services provided under other Bank programs, e.g., Education for All project (P124134), which provided tuition waivers to schools in poor neighborhoods. A review of the Rental Support Cash showed that, after the expiration of the one-year rental support only 25 percent of families stayed in the same neighborhoods, while none of them returned to camps. Information for the 75% who left the subsidized housing arrangements is not known. According to the ICR, the upkeep of coops or social housing projects’ communal space and the management of communal services would be assumed by the Ministry of Social Affairs and Labor. This would need to be explored in a project field assessment. It was also envisioned that, at the policy level, the Office of the Prime Minister would implement the Housing Policy, by promoting integrated buildings (such as coops) as a corner stone of earthquake and hurricane-proof construction. This would also need to be further studied in a project assessment. At a policy and institutional level, despite the reconstruction coordination efforts and technical support led by UN Habitat and actively supported by World Bank Staff the GOH at did not succeed in adopting a national Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) reconstruction policy containing standards. In this context, each development partner (e.g., USAID) and international NGOs (e.g., Habitat for Humanity) used its own reconstruction standards. it is unclear therefore whether there is a sufficient stock - after the reconstruction period - of safe housing conditions for the returnee population. Community Infrastructure. The Bureau of Monetization of Development Aid Programs prepared maintenance plans for the communal infrastructures with the municipalities, Electricité de Haiti and communities; management plans for the houses with MAST/EPPLS (Guide produced in 2015); and plans for transition/closing of the CRCs. An initial agreement was reached with EDH to assume the responsibility for the upkeep of the electricity network extension. Specific agreements have been reached with the municipalities of Port-au Prince, so that the newly built assets (roads, corridors, bridges, drainage systems, street lighting, commercial spaces, and amphitheater) would be maintained by the municipalities’ force account, while daily street sweeping and minor upkeep operations were to be assumed by the communities. a. Risk to Development Outcome Rating Substantial 8. Assessment of Bank Performance a. Quality-at-Entry • The project was based on the Bank’s experience with reconstruction in the country (Urban Community Driven Development Project) and with disaster housing reconstruction projects (in Indonesia, Pakistan, and India) globally. However, as discussed in the design section, more information was needed on the housing sector and the status of the target beneficiaries (who were renters not owners). recover loan, the Bank was quick to prepare the project and to administer the trust funds alongside international partners. In the absence of time- the kind of time that would have been needed to conduct a more thorough scoping of the housing sector - the Bank restructured the project relatively quickly after effectiveness. • Quality of entry was affected by the quality of the underlying data that was part of the post disaster needs assessment, partly funded by the World Bank's GFDDR. The problems were associated with the quality of "tagging" and the mapping of reconstruction needs (supplemental information provided by the TTL). This affected the ability of the project to adequately at entry plan and cost activities. For example, especially debris removal. • Since no agreement was reached on building standards, the Bank put in place, at entry its own housing standards, even though it was not able to fully meet its targets (see efficacy). • M&E was designed in a pragmatic way, although more instruments could have been designed to obtain qualitative data (including from beneficiaries) to learn lessons about the rental subsidy program. Social accountability tools could have also been applied to support more meaningful data useful for course correction and learning along the way. Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The supervision missions were conducted regularly, weekly or bi-weekly, which was ensured by the presence of the TTL in the country and Bank stuff based in the country. The missions visited the project sites on a regular basis. The Bank team undertook 13 implementation support missions. The Bank also undertook multiple restructurings, which significantly adjusted the PDO and design (during the first restructuring) ad which adjusted targets in the second and third - this allowed expectations and project risks to be updated based on the situation on the ground (vis a vis the data that was being received from the various contractors). There were challenges during implementation, reflected in the ISR ratings. In particular, this relates to compliance with safeguard and fiduciary policies. The location of the debris transferred was not fully documented and safeguarded. Financial management of the project was characterized by delays encountered in reporting on advances by the Implementing agency, which is not uncommon in a crises environment. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance The GOH worked closely with the Bank and other donors during project implementation and guided the implementing agency regarding coordination issues with government agencies. The GOH was also instrumental in stakeholder consultations. The GOH successfully implemented standard approaches for assessing post-disaster damages to infrastructure and housing. The GOH also internalized institutional and technical capacities in reconstruction and disaster risk management: The Ministry of Public Works, Transport, and Communications (MTPTC) became the lead agency for many reconstruction and disaster risk management activities including building safety assessment, debris clearance, and public infrastructure reconstruction. MTPTC implemented programs that created standard practices and de facto policies, which laid the ground for the consolidation of the 2012 national building code. (ICR, p. 14, p. 19 However, despite the reconstruction coordination efforts and technical support by donors, during project preparation and implementation, the GOH, at the technical and political levels, was not able to adopt national reconstruction policy with reconstruction standards or a unified mechanism for the delivery of subsidies to home owners for reconstruction. As a result, each development partner used its own reconstruction standards. (p.6) Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The Bureau of Monetization of Development Aid Programs (BMPAD) was the implementing agency for the project. BMPAD delegated the day-to-day project management in the project neighborhoods to the Project Management Contractor (PMC). The implementation relied on the Neighborhood Development Councils (NDCs) for consultations with the neighborhood population regarding community-based mapping, urban planning studies, selection of grant beneficiaries and management of grievances, and evaluation of results. NDCs were complemented by the Community Reconstruction Center (CRCs). The CRCs were located in project neighborhoods and used for consultations and training. BPMPAD was committed to the project and provided effective project management. The quality and regularity of reporting was not sufficient at start, but it improved over time. Although BMPAD was familiar with Bank procedures, only after the 2014 MTR it was able to resolve procurement and safeguard issues in a timely manner and to reduce implementation delays. After the first restructuring, the coordinating function of the BMPAD substantially improved (this relates to coordination with partners, contractors, NGOs, and communities). Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design M&E design was adequately constructed to measure the outputs envisioned by both the original and revised PDOs. However, it is limited to a recording of the outputs achieved by component and lacks causal linkages. There is also some language inconsistency. Whereas the original PDO indicated that displaced persons would return to their communities (this was the initial assumption), the associated PDO indicator refers to "the neighborhoods (e.g. not their communities, and not their neighborhoods). This language is in line with the revised objective - that reflects learning that - displaced persons in many cases did not always want to return to their original neighborhoods, or "communities." Standards were, on the other hand, incorporated into the housing indicators. The revised PDO seeks to return persons to "improved" housing conditions. The indicators on housing include : (1) number of houses repaired that are compliant with standards; (2) number of houses that are rebuilt compliant with standards. There is a need to review the criteria that was associated with the standards in a fuller assessment, but minimum conditions (such as space) were set and pursued by the Bank. With that said, similar expectations, in the form Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) of indicator language, was not used for the community infrastructure. An emergency operation, it is clear that some of the indicators will not have baselines. But the indicator on debris removal should have been updated at restructuring (with an estimate or target, based on anticipated area and costs). As it stands, the targets represent the actual amounts of debris that were removed, so it is not clear how effectively the project performed in this regard, especially in terms of cost-effectiveness (see efficiency section). M&E design does not disaggregate results for youth or other special needs groups. There is also no measure of institutional capacity building at the national and local level. b. M&E Implementation The implementing agency BMPAD submitted biannual and annual reports to the Bank on time. These reports included progress achieved on all indicators and identified key areas for action. Information on key project outputs and indicators was regularly collected and reported in ISRs as part of the Bank’s supervision visits. It is worth noting that neither a beneficiary assessment nor a stakeholder workshop was held to collect qualitative data or information on the quality of the works that were being built and the housing situation. c. M&E Utilization The conclusions and the recommendations of the MTR and technical audits were integrated in project implementation and restructurings. Technical audits to evaluate the technical quality of works guided BMPAD in following up with MDODs and executing firms. M&E Quality Rating Substantial 11. Other Issues a. Safeguards The project was classified as an environmental Category B at appraisal and triggered OP/BP 4.01 Environmental Assessment, OP/BP 4.11 Physical Cultural Resources, and OP/BP 4.12 Involuntary Resettlement (for dwellers living in areas that needed to be reconstructed but not applicable to IDPs per World Bank Policy). The project was processed under OP 8.00 Rapid Response to Crises and Emergencies, and an Environmental and Social Management Framework (ESMF) and a Resettlement Policy Framework (RPF) were developed, consulted and disclosed within the first year of project implementation. The project funded neighborhood reconstruction activities (which led to some impacts under OP 4.12) and a number of people were relocated for the period of time while their houses and neighborhoods were Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) rehabilitated. Affected people received compensation according to OP 4.12, which covered loss of assets, moving expenses, and economic losses. A resettlement audit at the end of the project confirmed that the compensation was provided before the start of works. The resettlement audit noted that the resettlement could have been handled better, including (i) keeping the geographical advantages of PAPs prior to their resettlement and (ii) paying compensation for moving expenses based on the size of the household, rather than on a standard basis. b. Fiduciary Compliance Financial Management The project suffered from financial management issues during implementation. In particular, there were delays in submitting IFRs and audit reports to the Bank, delays in submitting disbursement reports while advances were made without documentation, and delays in submitting withdrawal applications to document advances made to the designated accounts. Throughout the life of the project, delays between the provision of advances and the full accounting of spending prevented accurate readings of project financial status. The quality of the audits was acceptable. All suppliers were paid by April 30, 2017 for services provided. All advances to the designated accounts were fully documented by mid-May 2017. Procurement Throughout project implementation, procurement management remained weak due to: (i) high staff turn-over and lack of qualified procurement staff; (ii) long delays in the procurement of works, goods and services; and (iii) contract management characterized by multiple contract amendments and cost overruns c. Unintended impacts (Positive or Negative) --- d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Moderately The outcome rating is Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) Satisfactory changed because of the split rating methodology and because efficiency is rated modest, mainly due to the lack of information on unit and activity costs, that would allow for an assessment of relative cost-effectiveness. Risk to Development Modest Substantial --- Outcome Moderately Moderately Bank Performance --- Satisfactory Satisfactory Moderately Moderately Borrower Performance --- Satisfactory Satisfactory Quality of ICR Modest --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons As an instrument to transit displaced persons out of post-disaster camps, rental cash grants can be used to provide a rapid response. The project had a positive experience with Rental Support Cash Grant and supported the development of institutional tools for future disaster efforts to address the displacement of affected people. The project Operational Manual could be used a tool, in other post disaster projects, to develop shelter solutions for displaced persons other than camps. In the future, targeting, and the sustainable and gradual phasing out of a rental subsidy program, should be better considered. Regular monitoring and follow up on the renters and clear alternative plans for the renters, who could not afford to stay after the phasing out of the rental subsidy, need to complement this instrument. Using a programmatic approach to project design in emergency situations can speed up implementation. In the reviewed project, the programmatic approach simplified project restructuring when it turned out that many of the relocated people were renters and not owners and it would be therefore necessary to support rent rather than housing. Community engagement can be instrumental to managing disaster risks appropriately and efficiently, and support sustainability of the reconstruction efforts. However, in a post-crises environment, it is difficult to ascertain information about the make-up of the pre-crisis community. its identity, and inter and intra community perceptions and tensions that need to be mitigated alongside investment decisions. Community engagement can also have the ill-effect of creating a perception that international donors are "delegitimizing" the role of the state, and the way that it would like to be perceived. These political economy issues need to be handled with sensitivity. Disaster preparedness is critical and requires upfront investment in disaster risk management capacity at several levels (national, local and within communities). The lack of available and updated information on Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Neighborhood Housing Reconstruction (P125805) the quality of Haiti's housing stock and other related housing data undermined project effectiveness. 14. Assessment Recommended? Yes Please explain Yes, together with other Haiti HRF projects as a way to continue to learn lessons about (1) the emergency- development assistance nexus; (2) considerations over housing - with an aim to build back better versus rental subsidies or cash transfers that may be needed to provide emergency and temporary housing or shelter; (3) standards (building and construction) should also be a consideration of the PPAR; (4) there is also no beneficiary feedback provided in the ICR - reaching out to beneficiaries who received rental subsidies would increase the learning around this tool that is being rolled out elsewhere (e.g. currently in Senegal; (5) to better understand the role of the WBG when it is part of a donor coalition, including its ability to leverage the overall effort; (6) and as part of its programmatic approach going forward. 15. Comments on Quality of ICR The ICR is well written and supplies an easily understood narrative of the anticipated outcomes of the project and the adjustments that were made throughout. However, the ICR should have conducted a spit assessment, since there was a Level 1 restructuring, since the PDO and the PDO indicator was changed significantly. The change altered the method by which the project would achieve its first objective. IEG has conducted a split assessment per the ICRR Guidelines. . The ICR lacked disaggregated component financing. It included all project activities in Component A. As such, it is not possible to obtain or analyze unit costs from such differentiated activities as rubble removal, housing repair or reconstruction, or other community infrastructure activities. This is the main weakness of the ICR, and one that matters from the point of view of both understanding and assessing efficiency. a. Quality of ICR Rating Modest Page 18 of 18