ICRR 14594 Report Number : ICRR14594 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/09/2015 Country : Guyana Project ID : P103539 Appraisal Actual Project Name : Conservancy US$M ): Project Costs (US$M): 5.0 5.0 Adaptation Project L/C Number : Loan/ US$M ): Loan /Credit (US$M): 3.8 3.8 Sector Board : Urban Development US$M): Cofinancing (US$M ): Cofinanciers : Board Approval Date : 10/11/2007 Closing Date : 06/30/2011 08/30/2013 Sector (s): Flood protection (97%); Central government administration (3%) Theme (s): Natural disaster management (67% - P); Water resource management (33% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Samantha Sauny Robert Mark Lacey Christopher David IEGPS1 Mignotte Nelson 2. Project Objectives and Components: a. Objectives: According to the Global Environmental Facility (GEF) Trust Fund Grant Agreement (TFGA), “the objective of the Project is to reduce the Recipient ’s vulnerability to the catastrophic flooding of its low -lying coastal area due in part to the rise in sea level as a result of global climate change .� According to the Project Appraisal Document (PAD, page 9), the objective of the project is to "reduce the vulnerability of catastrophic flooding in Guyana's low -lying coastal area that is currently threatened by sea level rise resulting from global climate change .� The PAD describes the Global Environment Objective (GEO) as follows:“The project would serve as a template that could be applied to countries with similar geographical attributes and aims to raise awareness to promote the world wide application of physical infrastructure upgrades to reduce vulnerabilities brought on by climate change .� (The GEO is not stated in the GEF Grant Agreement ). This Review is based on the statement of objectives in the GEF TFGA . The key associated outcome targets in the PAD (page 10) were: Development of a hydraulic engineering foundation critical for flood control management . Identification of at least ten drainage regimes for follow -on intervention. Increase in the drainage relief capacity of the East Demerara Water Conservancy (EDWC) to the Demerara River by 35%. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: There were three components. According to the ICR (Annex 1), actual costs by component and total actual project cost were identical to appraisal estimates Pre -investment studies for engineering design of works (Estimated Cost at Appraisal and Actual Cost : US $2.0 1. Pre- million). This component focused on the creation of a hydrologic baseline for future interventions . It sought to achieve this by: producing detailed topographic and land use maps, completing hydrologic modeling of the coastal lowlands, assessing the EDWC system integrity, completing EDWC hydraulic modeling, conducting pre -feasibility studies for coastal lowland interventions, and through operational capacity building . 2. Investments in specific adaptation measures (Estimated Cost at Appraisal and Actual Cost : US $1.7 million). This component focused on investments that widened key drainage relief canals, improved water flow system within the EDWC, and upgraded water control structures . It also included the purchase and installation of selected equipment . 3. Institutional Strengthening and Project Management (Estimated Cost at Appraisal and Actual Cost : US $0.1 million). Component 3 focused on strengthening the institutional framework for flood control through the production of a contingency plan for flood events, consolidation of flood control actors, monitoring and evaluation of project progress, and project management . Changes in the Components At a Level 2 Restructuring in March 2011, the widening of key drainage relief canals was dropped . The corresponding funds were reallocated to Component 1 to support the improvement of the EDWC water system, the upgrading of flood control structures, , the purchase and installation of selected equipment, and the development of a contingency plan . Capacity building activities originally included under Component 1 were later moved to Component 3. In order to be consistent with the Results Matrix in the PAD (Annex 3), the ICR includes another activity in Component 2 -- "major infrastructure civil works and operational improvements " -- that does not feature in the main text of the PAD or in the Detailed Project Description (Annex 4). The ICR also states that activities under Component 3 that related to institutional analysis of drainage were dropped, and that the scope of the Implementation Secretariat was scaled back . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs : According to the ICR, the actual project cost, at US$ 5 million, was identical to that estimated at appraisal. The estimated and actual costs for each Component were also identical . Table B in Annex 1 of the ICR indicates total financing by the GEF grant and total financing from the Government . It does not indicate which components were actually financed by the Government; however, ICR Section 1.5 (pages 18-19), shows that at appraisal, the Government committed to contributing $ 1.2 million to Component 2, which is equivalent to the actual Government financing detailed in Annex 1. Nevertheless, the March 2011 Restructuring Paper shows a reallocation of US$0.8 million from Component 2 to Component 1 (US$0.7 million) and Component 3 (US$0.1 million). Financing : GEF funding of US$3.8 million was committed and fully disbursed . There were no other external sources of finance. Borrower Contribution : The Government of Guyana committed US $ 1.2 million at appraisal, which was fully disbursed. Dates : The closing date, originally June 30 2011, was extended twice for a total of 26 months. The first extension, to March 31 2013, was made at the March 2011 restructuring to allow for time to complete engineering studies . Only half of the GEF Grant had been disbursed at that time . The second extension, for five months to August 31 2013, was granted in March 2013, to allow for quality control and for the preparation and completion of a contingency plan to be used in the event of a breach of the EDWC . 3. Relevance of Objectives & Design: a. Relevance of Objectives: High The project development objective was highly relevant to the 2009-2012 Country Assistance Strategy (CAS) for Guyana. The project directly supported the first pillar of the CAS, “Strengthening Environmental Resilience and Sustainability,� and the related outcome, “Improve the Government’s ability to reduce exposure to natural disasters and global climate risk.� The objective is also relevant to Guyana's National Development Plan, its 2013-2018 Disaster Risk Management Plan, its Drainage and Irrigation Master Plan for Region 4, and its National Sector Policy for Sea and River Defense. The Government of Guyana has also been working on the challenge of sea level rise with other development partners such as the European Union, UNDP, the UK aid agency DfID, and the Inter -American Development Bank (IDB). The relevance of the global environment objective was substantial . The project contributes to a number of outcomes laid out in the GEF Adaptation Strategy 2010-2014, including: " Awareness raised and communities involved in disaster planning, preparedness and prevention, " "Climate change and variability induced disaster planning mechanisms developed and applied; " and "Adaptation practices developed and implemented to respond to climate change induced stresses in development sectors and vulnerable ecosystems ." b. Relevance of Design: Modest The statement of objectives was clear, but over -ambitious in relation to the resources committed to the project . The components were designed to work together to support both long term and medium term reductions in vulnerability . Component 1 was meant to indirectly reduce vulnerability to sea level rise by creating the hydrologic baseline necessary to guide the structural investments to be supported by Component 2, as well as future investments . Component 2 was designed to directly reduce vulnerability to flooding by making structural improvements in the management of water levels behind the East Demerara Water Conservancy (EDWC) dam (through increased drainage capacity and improved water flow systems ). Component 3 was designed to strengthen the institutional framework for flood control in order to better position the Government to respond to flood emergencies in the future . However, the complexities of, as well as the human and financial resources required for, the planned structural investments were underestimated, and many of these had to be dropped during implementation . The relevance of design to the Global Environment Objective was modest . Sub-component 3.4 (which called for a meeting of donors) was the only direct support provided to the GEO . The objectives of this meeting were not properly developed or explained in the PAD . An African Caribbean Pacific-European Union (ACP-EU) Global Facility for Disaster Reduction and Recovery (GFDRR) grant was used to fund the communication plan for the project (used to disseminate results, facilitate knowledge transfer, boost coordination and inform future activities ). However, this grant fell under a complementary project . 4. Achievement of Objectives (Efficacy): The efficacy of the project development objective -- to reduce the Recipient ’s vulnerability to the catastrophic flooding of its low -lying coastal area due in part to the rise in sea level as a result of global climate change – is assessed as substantial . Outputs : The project created a hydraulic engineering foundation to guide investments under Component 2, as well as future investments, which could reduce vulnerability to flooding . All of the pre-investment studies necessary to create a hydraulic engineering foundation for project areas were completed. One mapping exercise was 70% completed, but initially included non -project areas in the design that were later dropped from the study. Mapping for all project areas was completed . One of the main project outputs highlighted by the project team was a portfolio of suggested future interventions based on the analytical work completed under the project . The project enhanced the water flow systems, mainly through upgrading of the EDWC control structures, equipment purchase and installation, and through a study on ways to improve the systems further . The project also strengthened the operational capacity of staff in a variety of agencies through training in the use of Digital Elevation models, flow models, and monitoring equipment (including the Environmental Protection Agency, Ministry of Agriculture, the National Drainage and Irrigation Authority, Civil Defense Commission, and others). The project did not create a contingency plan for flooding, despite a five month project extension in 2013 meant to provide additional time to complete it . Outcomes : The pre-investment studies were intended to guide future investments to reduce flooding, while previously identified specific adaptation measures were carried out during the project period . However, some of these adaptation measures were dropped from the project and turned into a separate operation . As a result, the third PDO indicator (increase in the drainage relief capacity of the EDWC to the Demerara River by 35%, PAD page 10), was dropped. The ICR does not provide information on the extent to which drainage capacity was enhanced by project closure in the absence of these measures .The ICR states that the decision to drop the proposed works on the Cunha Canal was due to the finding of a 2010 Environmental and Social Assessment Report that the proposed works would trigger the Involuntary Resettlement Safeguard (OP 4.12). According to the project team, the additional time that would be needed to negotiate with the affected party and complete a resettlement plan would have resulted in considerable project delays. This led to the withdrawal of proposed funding for this activity by the UK ’s DfID, and its reallocation to further studies. According to the ICR, the dropping of the Cunha Canal works was in order to better reflect the “main focus of the project to provide technical engineering baseline studies for future interventions ." Although this would appear to involve a significant modification in the original purpose of the project, the ICR states that the project development objective was not substantially changed . The objective of reducing vulnerability to flooding was, therefore, achieved within the project period based on the definition of vulnerability used in the United Nations International Strategy for Disaster Reduction 2009. As the ICR acknowledges, following the dropping of activities related to increased drainage capacity and the reallocation of commitments to support pre-investment studies, project activities focused primarily on reducing vulnerability to flooding by providing technical engineering baseline studies and training for future interventions . Global Environment Objective : Outputs : The results of the project were disseminated through a complementary project, the ACP -EU Global Facility for Disaster Reduction and Recovery (GRDRR) grant. It created technically accurate animation videos explaining the results of the modelling study and describing the EDWC and East Coast drainage systems . It also created a booklet highlighting the technologies developed during the project and held a series of workshops to disseminate the information. Outcome : As a results chain and M&E system were not established for this objective, it is unclear whether these dissemination efforts were sufficient or not to raise awareness . 5. Efficiency: At appraisal, the cost effectiveness of the sub -components related to the proposed widening of the Cunha Canal (which represented 24 percent of total project costs ) was estimated. It would cost $1.2 million and increase discharge 3 3 capacity from 25 m /second to 150m /second. This offered the greatest improvement in discharge capacity for the least cost out of the 5 alternative methods assessed . Since this activity was dropped, the ICR did not create an estimate of actual cost effectiveness . The ICR estimated the efficiency of certain project expenditures using two different methodologies . The first was an Avoided Cost Approach comparing the costs incurred in enhancing the water flow systems (budgeted at $400,000, or 8% of final total project costs). It compared the costs incurred in the operation of two rehabilitated sluices and the use of equipment for maintenance of the dam before and after project implementation (the sluices were rehabilitated during the upgrading of the EDWC control structures, lowering operating costs ). The estimated rate of return on investments was 29%. The second method used a benchmarking methodology, which calculated the potential benefits of investing in a hydro -meteorological system. This method offers limited insight on efficiency as the results of the analysis lacked precision and potential returns were estimated to be anywhere between 21 and 293 percent. The ICR indicates that, despite dropping important activities from Component 2 and scaling back Components 2 and 3, the GEF Grant and Government counterpart contribution were both fully disbursed and spent . The project closing date was extended twice for a total of 26 months (59% of the original 44 month schedule). Despite the extensions, the Emergency Preparedness Plan had not been completed by closure . The delays were the result of a slow disbursement rate due mainly to procurement issues . Lengthy and unsuccessful bidding processes (due to a limited number of bids and/or excessive costs) resulted in the main contract for the engineering studies (which represented more than 60% of funds final project costs ) being awarded 2 years after effectiveness . Primary data collection was also delayed by procurement issues, and was another reason for slow disbursement . The capacity of the Project Execution Unit (PEU) was overestimated. Contrary to expectations during preparation, the PEU was not familiar with Bank procurement procedures, but was not provided with training on these procedures . Moreover, the PEU was not funded directly through the project, and thus had competing priorities . Efficiency is rated modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The objectives were highly relevant to both Bank and Government strategy, but relevance of design was rated modest: the Recipient’s capacity to implement the project as designed was overestimated . The consequent dropping of key specific adaptation measures during the implementation period meant that while the development objectives were achieved (vulnerability was reduced through other structural and non -structural measures, including a number of pre-investment studies), one of the development indicators had to be dropped . Significant operational inefficiencies, the full disbursement and expenditure of available funding despite the dropping of the largest activities, and the lack of a complete quantitative analysis, lead to a modest rating for efficiency . Outcome is assessed as moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The Government of Guyana has incorporated disaster risk management, drainage, and river defense into its national development plan for 2013-2018. It has thus demonstrated commitment to reducing vulnerability to flooding and climate change. The Government has also partnered with a number of external agencies, including the EU, UNDP, DFID and IDB to reduce further vulnerability to sea level rise . Follow up projects have already been planned and the activities dropped from Component 2 have been converted into separate projects . The risk of not sustaining commitment to reducing vulnerability thus appears low . However, there is a possibility that proposed investments and the identification of funding may not take place in a timely manner given the issues of slow disbursement experienced in this project . There is also a risk that major flooding may occur prior to the implementation of new projects, leading to incomplete or reversed progress in the reduction of vulnerabilities to climate change . Land tenure and resettlement issues related to the Cunha Canal could lead to delays in the implementation of the new projects focused on the dropped activities from Component 2. However, a progress update from the project team on May 22, 2014 showed that the government is moving forward . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: Preparation included a detailed review of a number of technical studies and vulnerability assessments . However, the economic appraisal was limited to considering the pros and cons of civil works alternatives that would decrease the likelihood of EDWC collapse . Other dimensions of the project were not analyzed due to the difficulty of quantifying the social benefits of interventions . The fact that some sub-components would trigger the Involuntary Resettlement Safeguard (OP/BP4.12) was not identified. This triggered the dropping of investment activities that were key to achieving the third PDO indicator . Project risks that were identified included those related to system failure and flooding, delayed implementation of follow on programs, poor implementation capacity at the local level, change in government priorities, sustainability of interventions and resistance from stakeholders . To address weak implementation capacity, the project team clustered technical work under one contract to reduce and optimize procurement timing and processes, though the complexity of the main contract and the required local technical and administrative capacity was underestimated . While the risk associated with the capacity of the Project Execution Unit (PEU) was identified as substantial, there were no appropriate mitigating measures . The Bank and the Government, in coordination with the Inter-American Development Bank (IDB), agreed to use the existing PEU that was housed within the Ministry of Agriculture and was carrying out the procurement and financial management activities for IDB projects . The procedures for IDB projects were considered to be “almost exactly the same as World Bank procedures � (PAD, page 18), and procurement and financial management activities were predicted to be relatively limited due to the size of the project. This conclusion was, however, inconsistent with the substantial risk assessment related to PEU capacity. This assessment turned out to be accurate, since the PEU ’s unfamiliarity with Bank procurement procedures contributed significantly to a slow disbursement rate . The PAD also refers to an Action Plan to provide procurement-related training to PEU staff, though it is unclear if the necessary resources were made available to implement this Plan. Moreover, because the PEU was not funded directly through the project, it had competing priorities. Implementation arrangements did not include a Designated Account to facilitate operational expenses which further contributed to slow disbursement . The project also conducted an Institutional Analysis of the drainage sector as one of its main capacity building activities. However, according to the project team, this was already being supported by other partners and was consequently dropped. The project objective was overly ambitious given weak implementation capacity and limited financial resources . This resulted in the dropping of key activities during implementation . The design of the M&E system was weak . It did not include indicators to measure actual changes in vulnerability to flooding. Several indicators chosen would not generate adequate information on whether project activities would contribute to achieving the development objective (see Section 10 below). at -Entry Rating : Quality -at- Unsatisfactory b. Quality of supervision: Given significant implementation difficulties and a slow disbursement rate, the mid -term review (MTR, June 2010) focused on progress in achieving outcomes, on the institutional arrangements for project implementation, and on the effectiveness and suitability of the monitoring and evaluation framework . It also thoroughly reviewed the implementation plan, disbursement schedule and Operations Manual . The four main recommendations were: (i) adhere to the project implementation schedule; (ii) drop the Cunha Canal rehabilitation activities and reallocate the related funding to Component 1; (iii) modify the development objective and the Global Environmental Objective (GEO), together with the related indicators; and (iv) extend the project closing date . At the subsequent restructuring, which became effective in March 2011, these recommendations were adopted with the exception of that related to the project development objective and GEO . The MTR had proposed a revision of the objective to reflect the intention to reduce vulnerability to flooding through "providing a comprehensive engineering baseline and analytical tools allowing Guyana to develop a program of strategic interventions and policies to address recurrent flooding and the anticipated impacts resulting from climate change and sea level rise," rather than through investments as had originally been intended . This proposed revision was a more accurate representation of the project's focus on analytical tools and an engineering baseline . However, it was decided that there would be no change in the statement of objectives for two reasons . First, it was thought that the revised wording did not substantially change the objective of reducing vulnerability . Second, a formal change in the objective would result in further implementation delays, since it would require approval from the GEF council. Supervision missions and Implementation Status Reports (ISRs) were timely and frequent. There were eleven ISRs filed, with two archived most years; the exceptions were 2007 (the first year of the project), 2010 (the year of the Midterm Review) and 2013 (the final year of the project). A mid-term review was planned no later than 1.5 years after the first project disbursement (which was completed) and a final evaluation in the last semester of project execution (the ICR does not state whether this was completed ). The ICR refers to a June 2010 QAG report stating that aide memoirs were candid and were well summarized in ISRs . However, the first four ISRs (between 2007 and 2009) rated Implementation Progress as satisfactory, although there was no disbursement for the first two years. Ratings did fall slightly between 2009 and 2011, but remained at moderately satisfactory, despite an MTR recommending a change in the PDO, the dropping of key project activities, and a 21 month extension of the closing date . The QAG report noted that there was less consistency between reporting and the ratings. Little information is provided in the ICR concerning M&E implementation, beyond the revisions recommended at the MTR (see Section 10 below). There were no major safeguards or fiduciary issues (see Section 11 below). Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government collaborated closely with the Bank to improve implementation performance, helping to strengthen financial management capacity by hiring additional staff within the Project Execution Unit . The Government also demonstrated strong commitment to the project . It financed additional projects supporting the development objective, such as the Hope -Dochfour Canal and the rehabilitation of the EDWC dam as well as interventions along the East Coast . It also used the project financed pontoon and excavator (with additional units financed by the Japan International Cooperation Agency ) to monitor and repair weak parts of the dam, and the hydrometeorological instrument and online database to monitor EDWC water levels . Government Performance Rating Satisfactory b. Implementing Agency Performance: The project had an Implementation Secretariat (composed of the National Drainage and Irrigation Authority, Civil Defense Commission, Sea and River Defense Commission and Lands & Surveys Commission ) responsible for project oversight and coordination, as well as a Project Execution Unit (PEU), housed in the Ministry of Agriculture, responsible for the administrative and fiduciary dimensions . The PEU was not directly funded from the project and thus had competing priorities . It also carried out procurement and financial management activities for IDB projects . As noted in Section 8a, despite perceived similarities between the procedures for IDB and World Bank projects, the PEU was found to be unfamiliar with Bank procedures and specific training was not provided to its staff . This lack of familiarity contributed to procurement problems and a slow disbursement rate for the project, eventually leading to two project extensions . For example, the ICR notes that the main contract covering engineering studies, which represented more than 60% of project funds, wasn’t awarded until almost two years after effectiveness . It also highlights delayed primary data collection due to procurement delays, as a further reason for slow disbursement (as noted in section 10b). The ICR notes that by the time of the mid -term review, the project had strengthened financial management capacity by hiring additional staff within the PEU and through capacity building for the PEU team (financed by IDB working with the Agriculture Sector Development Unit ). Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E system was designed to monitor (i) contract compliance for the development of pre -investment studies for engineering design of works; (ii) project implementation; and (iii) impact monitoring. An external engineering firm was recruited to monitor and evaluate progress and project results, while the PEU managed procurement, performance monitoring and acceptance certification . The indicators for Component 1 were all related to the completion of pre -investment studies. These outputs were intended to contribute to the reduction of vulnerability indirectly, by providing a guide for future investments with a view to reducing vulnerability to flooding . However, there was no indicator measuring whether or not these products were actually used. There was no single indicator measuring the reduction of vulnerability to flooding . Separate indicators measured increased drainage capacity and improved water flow for the EDWC system . The indicator measuring improvement of water flow only called for the creation of a flow model and a report on potential improvements, not the application of the model. Other indicators supporting the water flow system included equipment purchase and installation as well as an upgrading of EDWC control structures . The final indicator for Component 2 was listed as: "Major infrastructure civil works and operational improvements ." It was too broad to be of practical use . Component 3 was comprised of three main indicators . The first two indicators measured the reduction of future vulnerability by tracking the creation of analyses and plans to contribute to emergency preparedness . One of these was the completion of an Institutional Analysis of the Drainage Sector that was later dropped (as noted in Section 8a). The third indicator (Development of a Flood Control Thematic Committee ) was related to meeting attendance and bureaucratic functioning . There was no measure of whether these meetings created any productive outputs that could contribute to improved water management or reduction of vulnerability to flooding . A fifth indicator was added during implementation. It covered operational capacity building through the training of staff in specific technical areas. This indicator aimed to link institutional strengthening to improved water management . There were no indicators measuring the Global Environment Objective of changes in awareness about the implementation of adaptive measures to strengthen infrastructure . b. M&E Implementation: Several of the indicators in Component 2 were dropped following the Mid-Term Review and the restructuring of the project. Several indicators were dropped from Component 3, specifically those regarding the bureaucratic functioning of the Implementation Secretariat and the institutional analysis of the drainage sector . As noted previously, there were delays in primary data collection due to procurement delays, which contributed to slow disbursement. There is little additional information in the ICR concerning the quality of M&E implementation . c. M&E Utilization: There is no discussion in the ICR of the utilization of the project ’s M&E framework. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was classified as Category “B� for environmental assessment purposes . In addition to Environmental Assessment (OP 4.01), three safeguard polices were triggered at appraisal : Natural Habitat (OP 4.04), Forests (OP 4.36), and Safety of Dams (OP 4.37). During implementation, it was discovered that the Involuntary Resettlement safeguard (OP 4.12) would also be triggered. According to the PAD, the project was not expected to have negative impacts on the Forest and Natural Habitat safeguards, thus these safeguards did not need to be addressed in depth . Even though no civil works would be conducted on the EDWC dam, an engineering assessment of the dam and its associated drainage structures was planned to assist the Government in the design and development of any dam strengthening programs . The ICR reports that no further mitigation actions were required for OP 4.04, 4.36 and 4.37. According to the ICR (page 24), an Environmental Assessment was conducted as planned . In accordance with the Environmental Implementation Plan developed from the Environmental Assessment, a further assessment was conducted specifically on the proposed work on the Cunha Canal . It was found that lands originally believed to be vacant were actually occupied . The proposed realignment of the Cunha Canal was encumbered by a fence, a steel bridge and a shed. The evaluation found that planned activities on the Canal would trigger the Involuntary Resettlement Safeguard (OP 4.12) and would delay the implementation of the proposed investment . According to the project team, the resolution of these land tenure issues would have taken a considerable amount of time because the Government had few formal processes that could be relied on to address them . As noted in Section 4 above, DfID then reallocated funding from activities related to the Cunha Canal because of the delays . The project was restructured, and the proposed Cunha Canal widening was dropped . The triggering of OP 4.12 therefore became moot. The ICR does not indicate whether or not a new environmental or social assessment was conducted after the project restructuring, nor does it specifically state whether there was compliance with OP 4.01. b. Fiduciary Compliance: Financial Management . According to the ICR (page 25), "…no major fiduciary issues requiring Government or Bank attention emerged throughout the Project implementation period, as corroborated by audit reports and procurement post-reviews." However, slow disbursement rates were identified as a reason for the extension of the project closing date. In 2010, two years after project effectiveness, less than a million dollars had been disbursed . The ICR states that slow disbursement was directly related to problems in procurement and a series of related issues . Among them were: disbursement arrangements that originally did not include a Designated Account to facilitate operational expenses, and a PEU that was unfamiliar with Bank procurement procedures . The ICR (page 23) states that “audit reports were unqualified and there were no internal control issues .� Procurement. The ICR states that lengthy and unsuccessful bidding processes (due to a limited number of bids and/or excessive costs) contributed to the main contract for the engineering studies being awarded 2 years after effectiveness. It also identifies other secondary effects, such as delayed primary data collection due to procurement delays, as a reason for slow disbursement . Problems with procurement and disbursement were due in part to an overestimation of the capacity of the Project Execution Unit (PEU). However, other than delays, no major procurement issues arose as corroborated by post -procurement reviews (ICR, page 25). There were no reported cases of mis-procurement. c. Unintended Impacts (positive or negative): d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately . Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: IEG draws the following lessons from the experience of preparing and implementing this project : The outcomes of a project can be better evaluated if a restructuring, that significantly alters the focus of the operation, is accompanied by a formally approved change in the development objectives . This was recommended in this case by the project team at the time of the mid -term review, but was not carried out . Where, as here, the Global Environmental Objective is different from the project development objective, it is important to include activities in project design aimed at its achievement . Insufficient research on engineering and safeguard compliance requirements at the preparation stage can negatively impact project implementation and outcomes . The assumption that a project execution unit that has worked with other international development banks may be sufficiently familiar with World Bank procedures as not to require targeted training and capacity building may not be justified. A unit that is financed independently of the project, and continues to work on operations funded by other development banks, may not devote the required effort and attention to the project . A strongly designed M&E framework, with appropriate outcome and outputs indicators, is an essential tool for tracking progress as well as for evaluating results . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is satisfactory. It is candid in its analysis of shortcomings in Quality at Entry, and provides a generally informative narrative of the challenges that arose during implementation . The assessment of the achievement of objectives is output rather than outcome -oriented, although this reflects weaknesses in the M&E framework (which was not designed to measure the intended outcomes ) and, more fundamentally, the disconnect between the scope and depth of project activities and the expected results . The discussion of M&E implementation and utilization is sparse. The Lessons Learned are largely project -specific rather than of more general application . Annex 1 states that the final project costs by component and in total are identical to appraisal estimates, which is prima facie unlikely . There is no clear statement of compliance with OP 4.01. a.Quality of ICR Rating : Satisfactory