ICRR 13880 Report Number : ICRR13880 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 12/31/2013 Country : Kiribati Project ID : P089326 Appraisal Actual Project Name : Adaptation Program US$M ): Project Costs (US$M): 6.58 7.70 Phase II - Pilot Implementation Phase (KAP-II) L/C Number : Loan/ US$M ): Loan /Credit (US$M): 0.00 0.00 Sector Board : Agriculture and Rural US$M): Cofinancing (US$M ): 2.46 4.48 Development Cofinanciers : AusAID and NZAID Board Approval Date : 11/11/2008 (now called NZMFAT) Closing Date : 06/30/2009 06/30/2011 Sector (s): General public administration sector (45%); General agriculture fishing and forestry sector (26%); Central government administration (24%); Sub-national government administration (5%) Theme (s): Climate change (25% - P); Vulnerability assessment and monitoring (25% - P); Natural disaster management (24% - P); Other environment and natural resources management (13% - S); Participation and civic engagement (13% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Richard C. Worden Ridley Nelson Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: This was a stand-alone program supported by Global Environment Facility (GEF) trust funds. There was no World Bank Group financing involved . As stated in the Project Appraisal Document (p. 5) and the GEF Trust Fund Grant Agreement (p. 17), the project objectives were : “to develop and demonstrate the systematic diagnosis of climate-related problems and the design of cost-effective adaptation measures, while continuing the integration of climate risk awareness and responsiveness into economic and operational planning.� The Global Environment Objective of the project, as stated in the PAD (p. 5), was: “to assist the Government of Kiribati in enhancing its capacity to plan and implement adaptation measures to the climate-related issues facing the country, which will also reduce the detrimental impacts of climate change on the fragile atoll ecosystems of Kiribati.� Project objectives remained unchanged although expected project outcomes were modified at restructuring in 2009. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 08/27/2009 c. Components: The Project had five components and expected outcomes associated with them : 1: Policy, planning and information (appraisal estimate: US$1.17 million; actual cost: US$1.28 million): This was to provide improved consultation, planning and coordination mechanisms to support climate change adaptation . It supported three core elements of adaptation efforts in Kiribati : (a) awareness raising and consultation; (b) policy coordination and planning including technical assistance for mainstreaming and climate risk management, and; (c) generating scientific climate risk information . 2: Land use, physical structures and ecosystems (appraisal estimate: US$2.17 million; actual cost: US$1.79 million): This was to support improved management of climate related hazards to coasts, public assets and ecosystems, contributing to reducing the vulnerability of the coastline, including key public assets and ecosystems, and shifting management practice to a more preventative, technically varied and sustainable approach . 3: Freshwater resources (appraisal estimate: US$2.16 million; actual cost: US$2.95 million): This was aimed at improving sustainability of freshwater resources, and supporting the development and management of freshwater resources to reduce their vulnerability to climate variability and climate change . 4: Capacity at island and community level (appraisal estimate: US$ 0.55 million; actual cost: US$0.10million): This aimed to improve capacity for climate change adaptation at the island, government and community level by providing technical assistance to the Ministry of Internal and Social Affairs(MISA) to include adaptation in the Outer Island socioeconomic development profiles and climate risk management training for local governments, and by financing a pilot program of small scale adaptation investments in select Outer Islands . 5: Project Management (appraisal estimate: US$0.39 million; actual cost: US$1.58 million): This was to provide support to the Project Management Unit (PMU) to implement project activities, and manage the accounting, procurement, and other fiduciary responsibilities for the project . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : Total project cost was US$7.70 million, US$ 1.12 million, or 17% more than estimated at appraisal (US$6.58 million). This additional amount of financing was provided almost entirely by AusAID (US$ 1.94 million) to increase support for the Freshwater Resources (US$ 0.79 million) and Project Management (US$ 1.19 million) as well as to make up for the shortfall of US$ 0.90 million in the Borrower’s contribution. The project was a stand-alone Global Environment Facility program supported by GEF trust funds . There was no World Bank Group financing involved. Financing : There were three sources of co -financing support for the project : (i) a GEF Trust Fund grant of US$1.80 million, (ii) a US$ 3.43 million grant from the Australian Agency for International Development (AusAID), far more than the estimated US$ 1.49 million; and (iii) a US$ 1.05 million grant from New Zealand Agency for International Development, which was US $70,000 more than estimated. At project closing US$ 24,776 was cancelled. Borrower Contribution : The Borrower contributed US$1.42 million to the project. This was 61 percent of the amount estimated of US$2.32 million. Dates: Dates Although the project’s proposed restructuring was endorsed by the GEF CEO on July 16, 2009 and approved by the Bank Board on August 27, 2009, it did not change the Project Development Objective, Key Outcome Indicators or their targets. It focused the project’s activities on two key areas (freshwater resources and project management) and extended the project by two years . The extension was granted due to the additional time and cost involved in getting to and from geographically isolated islands spread out across 3.5 million square kilometers of the Pacific Ocean, shipping delays of materials and supplies, weak procurement capacity, foreign exchange loss, and higher labor and materials costs than estimated at appraisal . The project closed on June 30, 2011. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial . The project was a central element of the Bank ’s first Country Assistance Strategy (2011-2014) with the Government of Kiribati at the time of the project’s closure, and remains at the core of the Bank Group ’s strategy to help the country adapt to climate change impacts . , The Government’s most recent development strategy (the new Kiribati Development Plan (2012-2015)) strongly endorses the perception of climate change as an existential threat to its survival. However, the 38 percent shortfall in the Borrower’s actual contribution to the project and the 10-fold drop in its estimated contribution for the recently launched (KAP-III) follow-on project (only US$250,000 appear to indicate that the Government views slow on -set climate change adaptation as a donor responsibility and moral obligation, allowing it to focus its own resources on other more immediate challenges of economic and social development, such as health and education. The three projects comprising the KAP program have all taken the ‘precautionary’ approach in selecting climate change adaptation interventions, focusing on ‘no regrets’ actions which are productive even if the impacts of climate change never materializes as expected . While this has been useful in terms of focusing interventions on more immediate concerns rather than on potential impacts decades away, it has blurred the line between actions taken for climate risk management benefits with those associated with typical development challenges, such as maintenance and improvement of potable water and sanitation systems, solid waste management, and the operation and maintenance of public infrastructure . b. Relevance of Design: Modest . The design of the project was not well conceived in terms of being able to achieve its stated objectives within the project timeframe. Both the geographic and thematic breadth of the project had to be drastically reduced midway through implementation in November 2008 when it was realized that the scope of the project design and project implementation period were unrealistic and “overly ambitious with regard to both the range and technical complexity of activities, taking into account the implementation and management capacity of agencies in a small country such as Kiribati� (ICR, p. 5). The sub-objectives in the results framework were not well -aligned with the final outcomes. For example, how the mere “establishment of a lead agency to coordinate climate change adaptation strategies � would lead to achieving any one of the three project objectives, or how the “consistent use of best practice in the application of risk management, environmental assessment and options analysis to public infrastructure and CCA vulnerability reduction measures� was relevant to the “the design of cost-effective adaptation measures .� Conversely, there were no objectives or measures of increased institutional capacity that would be strengthened as a result of the project's activities, even though this was identified as a key constraint and “critical risk� to the project achieving its development objectives. Only the “percentage of climate-affected [Ministry Operational Plans] MOP programs that reflect systematic climate risk management � was directly relevant to the objective of “the integration of [climate change adaptation] awareness and responsiveness into economic and operational planning .� In addition, there was a lack of consistency about how those expected outcomes would be assessed and a lack of clarity in the causal chain between inputs like funding levels and activities or project components that would result in the desired outcomes. Exogenous factors, such as the widely held view among the Government and local population that the costs to mitigate the impacts of climate change impacts should be borne entirely by donors representing those countries principally responsible for emitting carbon dioxide and methane gases and thereby causing climate change, were not accurately assessed or realistically incorporated into the project design or funding allocations . These design flaws had serious implications later in terms of delays and problems encountered during project implementation. 4. Achievement of Objectives (Efficacy): The development objective was �to develop and demonstrate the systematic diagnosis of climate-related problems and the design of cost-effective adaptation measures, while continuing the integration of climate risk awareness and responsiveness into economic and operational planning .� This objective has three elements : (i) to develop and demonstrate the systematic diagnosis of climate -related problems; (ii) to design cost-effective adaptation measures; and (iii) to continue the integration of climate risk awareness and responsiveness into economic and operation planning. Achievement of these objectives is assessed below . (i) Develop and demonstrate the systematic diagnosis of climate -related problems . Rating : Substantial . Outputs : Staff members of the Ministry of Environment Lands and Agricultural Development (MELAD) and the Ministry of Public Works and Utilities (MPWU) received training in the application of coastal protection tools and a Coastal Hazard and Risk Diagnosis and Planning approach for communities on South Tarawa that was supported by the project. Subsequently, a Foreshore Management Committee, an inter -ministerial technical group, carried out and extended this work with little or no outside assistance from project staff or consultants . Coral reef benthic communities and coastal ecosystem monitoring training and demonstration programs were instituted concomitantly in late 2007. The Final Coral Reef Benthic Monitoring and Workshop Report developed a four-part “Roadmap� of coral reef monitoring protocols . The coastal ecosystem monitoring pilot activity was carried out by an international consultant working with staff of the Minerals Unit of the Ministry of Fisheries and Marine Resources Development (MFMRD). Groundwater quality, quantity, and sustainable yield assessments were successfully conducted using electro-magnetic induction surveys and 20 high quality salinity meters, supplemented by a survey of over 560 shallow household water wells to assess their condition as sources of potable and secondary water supply . This work provided the technical basis for establishing the sustainable extraction rates for the shallow groundwater aquifer lens underlying South Tarawa and the need for groundwater protection . In addition, rain gauges were installed on Outer Islands (the number of which were not indicated ) and staff were trained in data collection methods. The regulatory, permitting and enforcement activities to monitor and control beach mining activities were dropped following the Mid-Term Review since they had not even begun, and only one of four activities (that is, to develop island socio-economic and climate vulnerability risk profiles ) on six Outer Islands under Component 4 were carried out. Outcomes : The Coastal Hazard and Risk Diagnosis and Planning tools developed by the project were used by the Foreshore Management Committee, led by MELAD and MPWU, were used to prepare an island -wide hazard risk assessment of South Tarawa . Then later, without further technical assistance provided by the project, the Committee prepared village-specific assessments for all the villages on the island . These assessments were used to create a “Risk Map for South Tarawa� and were directly relevant to the formulation of the Shoreline Protection Guidelines, whose implementation has been stalled by the lack of funds to implement the guidelines . There is no indication that monitoring of coral reef benthic organisms has been applied or used by Ministry staff since no regular monitoring reports have been released since the initial baseline report on coral reef conditions was completed (the Results Framework target was 8 such reports). Similarly, there is no apparent follow-up or result that has yet to come from preparing the six Outer Island Risk Profiles . In terms of the effective results achieved by regulatory, permitting and enforcement activities, there is little evidence showing them to have had much, if any, impact . For example, land-use controls overlaying critically important groundwater reserves at Bonriki and Buota have not been strictly enforced with a number of illegal squatters, economic activities such as sand mining and artisanal production, and even cemeteries, located within those recharge zones, which are critical for ensuring South Tarawa ’s drinking water supply to its rapidly growing population. In sum, while tangible results have lagged far behind expectations and targets, the objective of “Developing and demonstrating the systematic diagnosis of climate -related problems� has been substantially achieved . A few activities, such as the six Outer Island Risk Profiles, have clearly met the objective without resulting in tangible outcomes while others, such as the work by the Foreshore Management Committee, have resulted in some useful, demonstrable outcomes like the community -level assessments as part of an island -wide Risk Map for South Tarawa. ii) Design cost -effective adaptation measures in Kiribati . Rating : Modest (ii) Outputs : Rainwater collection and storage system pilots were implemented at four public building sites, incorporating innovative flush designs and overflow systems back into groundwater via soak pits . The target for this Intermediate Outcome Indicator was 20, but the costs estimated at appraisal significantly underestimated the actual costs of installing these coastal civil works . Only two household systems were installed in Bairiki, but the households require additional training and follow -up monitoring to ensure their proper use and maintenance . Water resources assessments at 14 sites throughout South Tarawa and other Outer Islands were undertaken . However, only one infiltration gallery was installed at a site in North Tarawa against a Results Framework target of five such water supply improvements installed . Mangrove planting was a “soft� adaptation measure intended to protect Kiribati ’s coastlines and estuarine and marine resources with the intention of increasing their sustained replication by local communities . Mangrove planting pilots were undertaken in two phases in 2010 by Environment Ministry (MELAD) staff on six islands. Over 37,000 mangrove seedlings were planted on approximately 1,500 hectares, although survival rates ranged from 11% to 98% (Mangrove Activities Report: 2010, pp. 15-16 and 25-27; ECD/MELAD). However, this work simply augmented what the ministry was already doing, which plants an average of 20,000 seedlings every year, and was dwarfed by the size of the need to plant several times this amount of mangroves just to protect the most vulnerable areas of the islands . In addition, no cost-effectiveness studies were conducted as part of this activity . Outcomes : No further uptake of rainwater harvesting systems has occurred at either private residential, community, or government properties without project support and subsidies since their cost -effectiveness was not convincingly demonstrated. Their replication is considered dubious given their high cost relative to other less hygienic options (i.e., contaminated shallow hand-dug wells). Thus, they did not achieve their intended outcome of demonstrating the design of a cost-effective climate change adaptation measure even though the use of traditional pit toilets and shallow wells are not hygienic . Future water galleries were only implemented in one village, but are planned in at least three more communities during the third phase (2012-2016) of this three-project program of climate change adaptation projects, according to the ICR (p. 26). Thus, it is inconclusive whether they achieved their purpose of demonstrating their utility as a cost-effective climate-change adaptation measure. Finally, it is difficult to determine whether the ecosystem -based coastal protection activity undertaken on six Islands to plant mangroves has been sustainable . The Mangroves Activities Report : 2010 prepared by MELAD (p. 22) indicates that “Overall, the key driver behind mangrove promotion and conservation is community engagement and financial support, � but it fails to state whether or not mangroves planted with project support have been maintained and extended . These efforts represent a very small fraction of the much larger need for more mangrove forests to protect vulnerable coastal areas from intensified storm surges and sea -level rise caused by climate change. Given that this project activity only added to already on -going mangrove planting activities by MELAD, it had no appreciable demonstration effect and did not prove its cost -effectiveness as a climate change adaptation measure . iii ) Continue the integration of climate risk awareness and responsiveness into economic and operational (iii) planning by the Recipient : Modest . Outputs : The principal institution established to achieve the twin goals of increasing public awareness and responsiveness to the need to adapt to climate change impacts and to ensure the integration of climate change adaptation measures into the economic development and operational plans across Government ministries was the Office of the Presidency (the Office of TeBereti (OB)). This was expected to happen in the first year of project implementation, but was delayed by resistance by the Environmental Ministry . As a result, the OB wasn’t officially given the authority and responsibilities to act as the lead agency coordinating climate change adaptation and related strategies until 2009. The two mechanisms that the OB intended to use to integrate climate change adaptation considerations into national economic and operational planning processes were : (i) community consultations to raise public awareness about the threats posed by climate change, and (ii) Ministry Operational Plans developed by key affected ministries (e.g., health, public works and utilities, environment, lands and agricultural development ). The first mechanism to build “frameworks and processes for [public] participation and awareness� (Component 1.1) were only modestly achieved. Among its achievements: A Report on the Quality & Effectiveness of Existing Methods of Public Consultation in Kiribati and an Operational Manual for Consulting Citizens of Kiribati by Dr . Christine Hogan was completed. Radio, newsletters, and events related to building public awareness about climate change adaptation were discontinued as the outputs were “not directly linked to physical investments � (ICR, Annex 2, p.25). A baseline survey of public attitudes toward climate change was undertaken, but follow -up semi-annual surveys were discontinued after 2007. A bilingual glossary of climate change adaptation -related terms was produced, but the use of this output was characterized by the ICR as “limited.� The focused behavior change campaign on coastal resilience and water resource management was dropped during implementation since it was “not linked to specific physical investments .� Initial consultation with villages in North Tarawa in relation to proposed installment of infiltration water galleries was conducted to increase potable water supply . With regard to the second mechanism of Ministry Operational Plans being the principal institutional planning vehicle for mainstreaming and linking CCA to national development priorities, there was no verifiable evidence provided in the ICR supporting the assertion that 60% of these ministerial plans have incorporated the results of project activities and outputs . Outcomes : Neither the recommendations of the review of project -sponsored public participation processes nor the processes set out in the Operational Manual for Public Consultations were thoroughly incorporated into community consultations for large civil works investments that occurred later in the project since consulting firms were not aware of these reports . The results achieved from preparing those outputs were therefore limited . The bi-lingual glossary of climate change adaptation related terms that was developed by the project has been surprisingly useful in increasing public awareness and responsiveness to climate change issues in Kiribati since it has now given them the words to better describe causes and effects in their native tongue . The Office of the Presidency (OB) has not been able to effectively carry out its intended role of coordinating the integration of climate change considerations into ministerial operational plans (MOPs) and its role as the lead agency in such matters has recently been taken over by the Ministry of Finance and Economic Development (MFED), which has increasingly assumed this function as part of its role coordinating national economic development and operational plans across the Government . With regard to the extent to which MOPs have incorporated climate change adaptation measures within them, it is nearly impossible for IEG to ascertain the 60% target cited in the ICR with any precision (due to the fact that MOPs are not publicly available documents ). However, IEG did find evidence that the process is not functioning as envisioned. A 2009 final report (Development of an Integrated CCA-based Risk Diagnostic and Response Process ) prepared under KAP-II stated (pp. 18–19): “[There is] A lack of connection between the MOPs and the Kiribati National Development Strategy of 2008-2011 and the Climate Change Adaptation Strategy . [There was] alignment between the previous National Development Strategy 2004-2007 and the MOPs that were established in 2004. However, due to lack of implementation of the MOP reporting system, and of misalignment between the National Development Strategy and MOP objectives, this previous alignment does not now exist .� iv ) Progress toward the Global Environment Objective : “to assist the Government of Kiribati in enhancing its (iv) capacity to plan and implement adaptation measures to the climate -related issues facing the country, which will also reduce the detrimental impacts of climate change on the fragile atoll ecosystems of Kiribati .� The project had limited success in enhancing Kiribati ’s capacity to integrate climate change adaptation measures into its economic development and operational planning processes . The Government still sees climate change adaptation as a responsibility of donors to fund given that they are perceived as being responsible for climate change impacts due to their greenhouse gas emissions into the atmosphere . The “no-regrets� measures that the project supported in order to gain greater political and public support for their activities also blurred the lines between those measures taken for their climate change adaptation value and those associated with typical development issues of water supply and sanitation systems, infrastructure improvements, and strengthening local institutions . Thus, the project’s impact on enhancing the country ’s capacity to plan for and address climate change impacts was limited . 5. Efficiency: Modest . Traditional measures of efficiency such as cost -benefit analysis were not undertaken during project preparation or at the end of project implementation given the long lead -times (decades) before the full effects of slow -onset climate change impacts are expected to be experienced in the second half of the century . Only then could the cost-benefit ratio of actions taken now be determined with any precision . Instead, the project adopted a “precautionary� climate change adaptation approach of only undertaking ‘no regrets’ adaptation interventions, such as planting coastal mangroves, conducting water resources assessments prior to installing freshwater supply water galleries, building seawalls to protect coastal infrastructure, and piloting the use of rooftop rainwater collection systems for community buildings and private houses . While the adoption of the precautionary CCA approach was perceived as having resulted in very pragmatic, practical interventions being undertaken by the project, their cost -effectiveness was not clearly demonstrated nor were these project activities carried out in a particularly efficient manner (e.g., of the 14 water gallery site assessments undertaken, only one resulted in the actual construction of a water gallery, which is still not used by the local community /school due to bad odors and discoloration of the water ). A qualitative incremental cost analysis of climate change impacts was undertaken prior to project appraisal (Cities, Seas, and Storms: Managing [Climate] Change in Pacific Island Economies; 2000) indicated that in the absence of adaptation, the impacts of current climate change scenarios in Kiribati could be severe, disrupting major economic and social sectors. The expected economic benefits identified at appraisal included : maintenance of livelihoods otherwise threatened by climate change; avoidance of damage to coastal assets and ecosystems; avoidance of climate change and disaster-induced limits to economic growth; avoidance of public health costs, private sector productivity losses, and public sector investments due to insufficient and contaminated water supply . The project was extended from three to five years for the reasons given in Section 2.d, which were all evident at appraisal, and should have been taken into consideration more carefully in estimating the time and cost of carrying out those activities. Other examples of inefficiency involved numerous instances of activities that were “discontinued or dropped� after initial expenditures and effort had been made without having achieved any useful purpose or result . Finally, the Project Management Unit (PMU) was unprepared for the task of managing a World Bank project, particularly with respect to its weak compliance with the Bank ’s procurement procedures, despite assertions to the contrary in the PAD (p. 10) that such capacity had been developed under the previous climate change adaptation project (KAP-I). Apart from necessitating a high level of input from Bank staff based in the Sydney office, cost effectiveness was reduced by a quadrupling of project resources above the appraisal estimate (from US $0.39 million to US $1.58 million) for Component 5: Project Management. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The Relevance of Objectives was assessed as substantial due to the direct connection to the Government ’s latest three-year development strategy (KDP 2012-2015) and the Bank’s last CAS with Kiribati while the Relevance of Design was rated as modest due to the project ’s overly ambitious scope and unrealistically short duration . Nor did the Results Framework align project inputs and activities well with expected outcomes and the three elements of the PDO. The efficacy of the first project objective to systematically diagnose climate change impacts was rated as substantial, although it was noted in Section 4(i) that many of these project activities have not been sustained . The efficacy of the second project objective to assess the cost -effectiveness of designs promoted by the project is rated as modest because their cost -effectiveness was not convincingly demonstrated, nor was there any appreciable up-take of those designs by communities in the absence of substantial project subsidies . Achievement of the third objective to mainstream climate change adaptation into national preparedness planning processes is rated modest . In the absence of traditional measures of cost -effectiveness, efficiency of project design and implementation is also rated as modest due to wasteful expenditures for many activities that were dropped or discontinued following project restructuring, or which were simply not utilized by the project . Lengthy delays in implementation requiring a two -year extension of the project. Altogether, these weaknesses reflected significant shortcomings in the project . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: There are a number of political and institutional risks to maintaining the limited accomplishments achieved by the project due to low stakeholder support among the local population for the need to urgently prepare for and aggressively adapt to climate change impacts . The Government’s reluctance to dedicate its own resources to address the threat posed by climate change is due in part to the pressing need to address other, more immediate social problems to improve basic infrastructure, health delivery services, and educational opportunities for its rapidly growing population. These factors threaten to undermine the project ’s outcomes, but are mitigated to a large extent by the increasingly significant investment that is programmed for Kiribati over the next several years by a coordinated program of donor assistance, such as the joint Asian Development Bank /World Bank Airport Modernization Project, the Tarawa Road Rehabilitation Project, and the ADB’s South Tarawa Water Supply Sector Improvement Project . All of these projects (plus KAP-III) incorporate climate risk management as a core principle and should mitigate any inclination by the Government to back away from climate resilient investments as a core element of its national development strategy . While the financial burden placed on the Government by the current climate change adaptation project (KAP-3) is much less than it was under this project (KAP-2), the political will needed to turn proposals into policies, and policies into plans and programs with dedicated resources remains an open question . Thus, there remains a moderate likelihood that this risk will materialize, which would undermine project efforts to mainstream climate risk management considerations into Government planning, budgeting and programs, if it weren ’t offset by the significant increase in donor assistance . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The project carried forward the design elements developed under the first KAP phase (KAP-I), it was consistent with the Bank’s Country Assistance Strategy, and incorporated activities from six of the top ten priority action areas identified in the UNDP-executed National Adaptation Programme of Action (NAPA), including the two most expensive ones (water resources management and upgraded coastal defenses and causeway restoration). As mentioned earlier in this review, the project team also adopted the very pragmatic ‘precautionary’ approach of climate change adaptation by focusing only on ‘no regrets’ interventions, and moved the Project Management Unit (PMU) within the Office of the President from the Environment Ministry, which resulted in resistance and delays, but moved it closer to the central coordination functions of the Government . Exogenous factors, such as the widely held view among the Government and local population that the costs to mitigate the impacts of climate change impacts should be borne by donors representing countries responsible for emitting carbon dioxide and methane gases and thereby causing climate change, were not accurately assessed or adequately incorporated into the project design . Despite a sense of “inequity and lack of justice � about cost-sharing arrangements held by local populations and the Government as expressed in several publications (N. Teuatabo; Pilot Case Study: KAP and NAPA, p. 12; January 19, 2005), the Bank did not adequately address the Government’s reluctance to commit its own resources to staff the newly created PMU within the Office of the President (OB) during the first three years of the project .In addition, limited implementation capacity and inadequate resourcing of line ministries to manage the demands of project activities was not identified as a project risk at appraisal, nor was the lack of sufficient experience and capacity in the PMU to manage the complicated procurement arrangements for handling nearly 70 individual contracts and the Bank ’s demanding financial management requirements. The limited degree of technical capacity in Kiribati ’s private sector was overestimated and was not identified as a risk during appraisal . The combined effect of these shortcomings had significant impacts later during project implementation . at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: There was a lack of adequate supervision by the Bank during the first 21 months of the project (until the Second Joint Supervision Mission in April 2008), whose effects were exacerbated by an inexperienced PMU overwhelmed by the breadth and scope of project activities geographically and thematically .For example, it was surprising that the establishment of the special unit that was supposed to be established within the Office of the nd President was not noticed by the Bank for 18 months (from project effectiveness in July 2006 until the 2 Supervision Mission in April 2008, an average mission spacing of ten months ) given that this project was seen as a “flagship� World Bank project, being the first to focus entirely on climate change adaptation in the Pacific region� (ICR, p. 5), and despite the fact that its establishment was required within 12 months of effectiveness. As the ICR states (p. 17): “As a result of infrequent initial supervision, the full extent of capacity and implementation constraints that were limiting the acceptable implementation of the project – and the actions required to address those constraints - were only realized two years into the (initially) three year project.� Relatively inconsequential changes were made to two of the three Outcome Indicators when the project was restructured in 2009, but a large number of intermediate outcome indicators were dropped . Given that neither the project development nor the global environmental objectives had been changed and that the project was extended by two years, dropping these intermediate outcome indicators amounted to a significant shifting of the ‘goalposts’ in terms of assessing the project ’s performance. In addition, Bank supervision and involvement increased noticeably following the project ’s restructuring in August 2009. In attempting to get the project back on track, the Bank may have inadvertently intensified its supervision too much . Government officials and project staff referred to it as ‘heavy-handed’ during IEG field interviews. This caused morale and local initiative to falter, but restructuring the project did simplify and focus it on achieving tangible ‘hard’ outputs more in line with initial Results Framework targets. However, in the rush to get things done , it cut back on ‘soft’ local capacity-building and training and community engagement activities that undercut local ‘ownership’ of project interventions that may have later sustained and extended them, such as installing rooftop rainwater collection systems, planting mangroves on outer islands, or converting coastal inundation hazard maps and guidelines into a national policy framework and program. This very uneven approach to supervision was not constructive to achieving the goal of enhancing the Government’s capacity to plan and implement adaptation measures to the climate -related issues facing the country. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: Initially, the government showed strong interest in the project, and agreed to locate the PMU within the Office of the President (OB). However, the Government did not meet its financial or operational obligations to the project, as agreed in the Grant Agreement . Counterpart contributions lagged and the Government did not staff the PMU for the first 18 months of the project. Nor did the National Policy Steering Committee meet its responsibility to provide strategic guidance to the project . Even when the Government did comply with the Bank ’s insistence on staffing the position of Project Director in the OB, it did so by assigning those additional duties to the existing Secretary of the OB, a compromise arrangement that was largely cosmetic and ineffectual in terms of strengthening the PMU’s ability to promote the incorporation of climate change adaptation measures in Ministry Operational Plans or coordinating their insertion into four -year national development plans (i.e., the NDS 2008-2011 or KDP 2012-2015). More than any other factor, this severely impeded implementation progress . Only 8.5% of project funds had been disbursed nearly two years into a three -year project, according to the April 2008 joint supervision mission (pp. 1-2), which rated progress with implementation as “unsatisfactory.� Confirming this, an independent evaluation of the project conducted for the Government in 2011 stated that, “the project quickly fell behind schedule in commitments and disbursements . This is not unusual in the Kiribati public sector, but it was a serious matter for the institutions funding KAP -II [that is, the GEF, AusAID, and MFAT], and when it came to their notice it had important consequences for the project .� (T. Hughes; Unfinished Business, p. 8; May, 2011.) Subsequently, the Government indicated to the Bank and the other project donors that it did not intend to establish a special unit within the Office of the President any longer, but in a compromise agreement, it assigned the additional duties of the Project Manager to the Secretary of the Cabinet supported by three additional technical staff members: two of them funded by donors and the third transferred from the Environment Ministry . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: The implementing agency was comprised of two related, but distinct, entities : the PMU and the special unit created within the Office of the President . Both were unprepared to carry out their mandate of managing the day-to-day operations of the project, while complying with the Bank ’s fiduciary requirements. They were overwhelmed by the logistical and procurement challenges of the project ’s scope as well as its thematic and geographic breadth, and ill-prepared to handle the procurement and oversight of such a large number (69) of individual consultancies that “created a bottleneck preventing implementation of most physical investments � (ICR, p. 7). This eventually led to 19 contracted tasks being either dropped or discontinued, and 14 more individual consultants’ contracts being rolled up into two large international ‘firm’ contracts (FS6 and FS7) to carry out the coastal works and freshwater resource tasks . Following restructuring, the PMU functioned much better . Its steady support and oversight of analytical work in the water and sanitation sectors was cited in a number of reports as having made an invaluable contribution to having the National Water Resources Policy and National Sanitation Policy adopted by the Cabinet in 2008 and 2010 respectively. The PMU’s support of the Foreshore Management Committee ’s work in developing coastal inundation “hazard maps� not only for the island of South Tarawa (where half of Kiribati’s population lives) was critical to its success in independently conducting village -level climate risk assessments and developing ‘action plans’ of prioritized coping strategies in consultation with those communities . These achievements all benefited from the PMU’s strong and steady support . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project development objectives were stated in a specific and measureable manner while the global environmental objective was more ambiguous . The three Outcome Indicators and targets at the time of appraisal were designed specifically to measure project progress integrating climate risk diagnosis and cost -effective adaptation designs into climate -affected ministry operational plans and mainstream climate change adaptation considerations into national economic development planning processes . Intermediate Outcome Indicators were logically linked to the three main Outcome Indicators, and logically linked to the corresponding project component . The proposed data collection and analytical methods were appropriate for their purposes, the design for collecting baseline data was adequate in all of the major project areas, and institutional arrangements for collecting and analyzing data were clearly spelled out in the Results and Monitoring Framework . However, there was no attempt made to distinguish attribution between project activities from other unrelated exogenous factors . b. M&E Implementation: Overall project monitoring and reporting was the responsibility of the special Unit in the Office of the President, while the day-to-day monitoring and evaluation activities were to be carried out by the PMU . However, the ICR notes on page 8 that: “Data was not regularly collected by the PMU � due to low capacity to handle multiple project management demands and simultaneously implement the M&E program . The monitoring of the second Outcome Indicator was problematic for two reasons . First, Ministry Operational Plans (MOPs) are confidential Government documents and are not publicly accessible . Therefore, it is not possible to verify whether the assertion in the ICR (Annex 3, p. 37 and 40) that the target of 60 percent of climate-affected MOPs “reflect systematic climate risk management � was reached. In fact, there is unambiguous evidence (presented in Section 4) to the contrary. Secondly, the achievement of that second Outcome Indicator target was improperly measured as the percentage of KAP -II activities appearing in MOPs. As the ICR itself notes on page 8, “Although KAP-II activities were reflected in MOPs from 2007 to 2010 [unverifiable], this does not necessarily mean that climate risk management was sustainably integrated as a priority in regular ministry work . Success in this regard would have been better measured by how many additional climate risk -related initiatives were reflected in ministry planning, particularly those that utilized government funding sources . Further, although MOPs are intended to link Ministry work to the Kiribati Development Plan and national budgeting process, in reality they are not generally closely followed.� Thus, the measurement of the second Outcome Indicator was seriously flawed . c. M&E Utilization: The ICR’s treatment of how M&E data was used to direct the management of the KAP -2 Project, or to change its direction, is unclear and inadequate . However, there was evidence that M&E data collected from the PMU and special unit within the Office of the President, as well as from initial Supervision Missions, was not promptly acted upon by project staff or the Bank ’s (and other donors’) staffs. For example, the ICR states on page 9, “Because of limited data collection, the Results Management Framework was not utilized in most project progress reports, which although generally submitted, were not forward looking and results -orientated.� Later during the restructuring of the project, there was no evidence indicated in the ICR that shifts in the project ’s direction or outcomes by project or Bank staff or managers could be attributed to M&E information that had been collected and analyzed . M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: Environmental : The project triggered the Bank ’s Environmental Assessment safeguard policy (OP 4.01) and was screened as a Category B project, requiring only a partial Environmental Assessment . However, a full Strategic Environmental Assessment (SEA) was undertaken to examine the environmental issues that were likely to occur as a result of small-scale construction of seawalls and other civil works projects . An Environmental Policy Framework was also developed, and larger sub -projects, with potentially higher environmental impacts, were required to have Environmental Management Plans in place . However, it was the cumulative effects of many, small coastal infrastructure developments (the vast majority of them being illegal and not associated with the project ) that are likely to cause the most important adverse impacts over time . Therefore, the project undertook activities to help the Government develop an integrated coastal zone management policy and program to monitor and control coastal construction activities and impacts (both beneficial and adverse ). While those impacts are continuing to occur, the project did not put in place policies or programs to redress this problem . There was no mention of any incompliance issues associated with of either social or environmental safeguards . Social: Social The project also triggered the Bank ’s Operational Policy (OP 4.12) on Involuntary Resettlement since the project was expected to entail land acquisition for seawalls and freshwater abstraction galleries . A Land Acquisition and Resettlement Policy Framework (RPF) was prepared and disclosed based on the Social Assessment that had been conducted under the preceding Kiribati Climate Change Adaptation Project (KAP-I). According to the ICR (p. 9), there was only one instance in which OP 4.12 was triggered to install a freshwater abstraction gallery at the Taborio Catholic School in North Tarawa that required the acquisition of land and the removal of coconut trees from the area immediately surrounding the gallery. The proposed construction works at two sites in North Tarawa (Notoue and Tabonibara) were deferred to the follow-on project (KAP-3) due to complications and conflicts over compensation for land and livelihood restoration . The ICR noted that social safeguard procedures were not followed initially in these negotiated processes, but that once a social safeguards specialist was brought in to work on the project after a two-year gap in coverage, all social safeguard procedures were followed in compliance with Bank policy . b. Fiduciary Compliance: The ICR (page 10) notes that the “recipient complied with Financial Management (FM) conditions outlined in the GEF grant agreement," and two supervision missions rated the FM performance as ‘satisfactory’ with no material issues and no follow up recommendations identified .� Procurement issues bedeviled the project from the outset and were only partially resolved by restructuring the project to reduce its geographic and thematic scope, and to simplify contracting mechanisms . The PMU was not staffed with a Procurement Officer and Assistant until 2008, and due to frequent staff turn -over, this was not sufficient to keep procurement issues from becoming a “bottleneck� for project implementation. An international Procurement Adviser was also seconded to the PMU following restructuring, but according to project staff in Tarawa this resulted in many more unnecessary delays and problems, such as those encountered in building the Tungaru Hospital ’s new water tower and tank. c. Unintended Impacts (positive or negative): None apparent. d. Other: None known. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Relevance of Objectives was Satisfactory Unsatisfactory substantial, but Design was rated ‘modest’ due to unrealistic expectations and overly ambitious scope. Two of the three objectives were only modestly achieved. And the efficiency of project implementation lagged badly and encountered numerous roadblocks . See Sections 3, 4, and 6 for fuller explanation. Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Initial appraisal work misjudged the Satisfactory Unsatisfactory Government's commitment to the project, and supervision was very uneven during implementation (overly inattentive initially, and then overly intrusive following the MTR). Borrower Performance : Moderately Moderately Lack of initial Government commitment Satisfactory Unsatisfactory to and incorporation of climate adaptation measures in national planning, budgeting, and programming processes. Lack of experience among PMU staff and limited authority hampered implementation. Quality of ICR : Exemplary NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The Bank must ensure that the Borrower demonstrates strong leadership /ownership through its actions (not just words ), and that the Borrower has a clear understanding of its role and responsibilities . The project performed poorly due in large part to a lack of Government commitment and ‘ownership’ of the project, as evidenced by the Government ’s failure to staff the special unit within the Office of the President and meet its financial obligations. It appears that the Government was more interested in obtaining donor assistance than it was committed to making the tough decisions and devoting the requisite financial and human assets to adapt to climate change. This reflected a profound misunderstanding between the Borrower and the donors about their respective roles and responsibilities to the agreement . The Bank and other donors should have recognized this earlier and insisted on a tangible, demonstrated commitment from the Government before agreeing to continue the project following the Mid -Term Review, or to extend it into a third phase (recently launched). It ’s critical in climate change adaptation projects to address the difficult tasks of developing the It’ ‘software ’ of empowered local institutions and capacities rather than just focusing on infrastructure and civil works ‘hardware ’ because they demonstrate more tangible outputs and results . The fundamental purpose of a disaster-preparedness project like this is to build the local institutional capacities of communities and governments to plan and implement climate change adaptation measures to meet the climate -related risks and threats they face. Midway through the project, the Bank decided to shift its implementation approach by dropping almost all of the ‘soft’ attitude- and behavior-changing activities, such as community engagement and public consultations, in favor of concrete actions that would have quick and tangible results . The result of this approach was that while many outputs were attained, few outcomes were achieved. In aggressively pursuing its own agenda to meet its own internal needs to generate an impressive list of outputs, it lost the balance between ‘hard’ and ‘soft’ activities, and undermined the project ’s fundamental intent of empowering local ownership, strengthening institutions, and building local technical expertise . No-regrets � climate change adaptation interventions can easily be ‘captured ’ by immediate local needs to “No- fix or improve infrastructure that have little to do with climate change instead of focusing on longer -term tasks to protect local populations and assets from those slow -onset climate impacts . Most of the physical infrastructure civil works outputs of the project on South Tarawa were effectively ‘catch-up’ maintenance of essential infrastructure that were only marginally related to climate change adaptation . The advantage of this “no-regrets� approach (that is, interventions that would still be good investments even if climate change impacts do not occur as predicted ) is that it helped secure public and political support more easily than other adaptation investments with results that might be decades away . The disadvantage is that it can lead to a situation of ‘learned dependency’ on external assistance instead of developing the self -reliance to prepare for the consequences of anticipated impacts . Projects seeking to change people ’s attitudes and behaviors must first build a strong sense of shared ownership and partnership with governments, other donors, and affected communities about the nature of the problem, and then agree on an approach to address it . Projects affecting many different economic sectors, Government ministries, and peoples ’ lives cannot impose a solution already decided on without their input, but must start from a shared view of the challenge faced and the best way to address it . One-way public ‘consultations’ to inform communities about the problem and a pre -determined solution are not the same as community engagement . True two-way communication and community engagement require shared decision-making processes that are followed up by project or Government technical support and resources to implement the agreed-upon mitigation or adaptive measures to address challenge . When public consultations are not followed up by concrete actions, they result in empty promises, processes, and public apathy due to the perception of wasted time and effort . Actions that are decided upon must be seen as being in the community’s own interest or benefit (rather than serving the project ’s own agenda). If not, then those investments may be neglected, misused, abused, or vandalized by those same ‘beneficiaries,’ who often discontinue using those practices and behaviors once external funds are no longer available . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR was candid in describing the problems encountered in the preparation and appraisal of the project, as well as during implementation, and the reasons behind the need to restructure the project . The ICR was complete and consistent with OPCS’s ICR guidelines, particularly with regard to its focus on results and outcomes achieved, and the reasons why more lasting results were not achieved . The evidence presented was objective and relevant . The “Lessons learned� and “Key Factors Affecting Implementation and Outcomes � sections were exceptionally insightful . However, some of the self-ratings were overly generous given the seriousness of the shortcomings described . a.Quality of ICR Rating : Exemplary