ICRR 13246 Report Number : ICRR13246 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 05/24/2010 PROJ ID : P068121 Appraisal Actual Project Name : Environmental US$M ): Project Costs (US$M): 26.76 31.57 Protection And Sustainable Development Of The Guarani Aquifer System Project Country : Latin America Loan /Credit (US$M): Loan/ US$M ): 13.40 13.35 Sector Board : WAT US$M): Cofinancing (US$M): Sector (s): Central government administration (92%) Other social services (3%) General education sector (3%) Renewable energy (2%) Theme (s): Environmental policies and institutions (33% - P) Water resource management (33% - P) Participation and civic engagement (17% - S) Climate change (17% - S) L/C Number : Board Approval Date : 06/13/2002 Partners involved : Closing Date : 03/31/2007 01/31/2009 Evaluator : Panel Reviewer : Group Manager : Group : George T. K. Pitman Robert Mark Lacey IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: There are two slightly differing formulations of project objective in the agreements by the Bank as implementing agency for the GEF Grant: The GEF Trust Fund Agreement states : "The purpose of the Grant is to assist in the preparation of the Environmental Protection and Sustainable Integrated Management of the Guarani Aquifer Project, the main objective of which is to support the Members in the joint elaboration and implementation of a common institutional framework for managing and preserving the Guarani Aquifer (the Aquifer)." The Grant Agreement between the IBRD and the Organization of American States (OAS that executed the project) was: "The objective of the Project is to support the Beneficiaries to jointly elaborate and implement a common institutional and technical framework for the management and preservation of the Guarani Aquifer System (GAS)." The PAD states that the project objective was : "The long-term objective is the sustainable, integrated management and use of the Guarani Aquifer System . The Guarani Aquifer System is; situated in the eastern and south central portions of South America, and underlies parts of Argentina, Brazil, Paraguay, and Uruguay ." This ICR Review uses the PAD's statement of objectives because it is more monitorable . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): 1. Expansion and Consolidation of the Current Scientific and Technical Knowledge Base on the Guarani Aquifer System (Planned US$ 9.91 million, actual US$ 15. 15 .95 million ). The aim of this component was to synthesize, analyze, and expand the knowledge base on the GAS and had two major sub -components: (i) Aquifer studies for consolidation and expansion of the scientific knowledge base; and (ii) Technical socio-economic assessments of current and future use scenarios of the GAS . 2. Joint Development and Implementation of the Guarani Aquifer System Management Framework (Planned US$7 US$ 7.01 million, actual US$ 4.79 million ). The aim of this component was to develop a framework for the coordinated management (technical, institutional, financial and legal ) of the GAS and had five sub-components: (i) Design and implementation of an aquifer monitoring network; (ii) Development and integration of an Information System (SISAG - Sistema de Informacion del Sistema Acu fero Guarani ); (iii) Formulation of a Strategic Action Program (SAP); (iv) Institutional strengthening; and (vi) formulation of Transboundary Diagnostic Analysis (TDA) 3. Public and Stakeholder Participation, Education and Communication (Planned US$ 1.31 million, actual US$1 US$ 1.30 million ). The aim of this Component was to promote and support the participation and involvement of the public, and to foster environmental and water resources education, social communication, and the dissemination of knowledge. The component had four sub -components: (i) Design of a Regional Communications and Public Participation Plan; (ii) Establishment of GAS Citizens' Fund; (iii) Creation and dissemination of instruments to increase awareness, interest, and commitment among stakeholders; and (iv) Formulation of an Indigenous Peoples Strategy . 4. Project Monitoring and Evaluation, and Dissemination of Project Results (Planned US$ 0.48 million, actual US$0 US$ 0.23 million ). The aim of this Component was to create and implement a system for recording and analyzing progress achieved during the Project implementation period . Two subcomponents were identified under this component: (i) Development and implementation of a monitoring, evaluation and feedback system; and (ii) Dissemination of Project results throughout the region and beyond . 5. Development of Management and Mitigation Measures within Identified "Hot Spots " (Planned ( Planned US$ 3.73 million, actual US$ 3.92 million ). The aim of this Component was to design, apply, and evaluate the costs and feasibility of good management practices at specific sites within the GAS region . Four areas for pilot subprojects were identified to test the proposed local management practices . 6. Assessment of Geothermal Energy Potential (Planned US$ 0.28 million, actual US$ 0.01 million ). The aim of this Component was to evaluate the geothermal potential of the GAS and was designed to be implemented in two phases. During Phase I, existing geohydrological data were planned to be compiled and evaluated . In Phase II it was planned to create a task force comprised of representatives from the four countries, supported by world experts in the study and use of the enthalpic energy and to conduct a scientific evaluation of the geothermal potential of the GAS based upon the data acquired during Phase one . 7. Project Coordination and Management (Planned US$ 4.04 million, actual US$ 5.37 million ). The aim of this Component was to provide organizational and administrative support to the Project . The Project Secretariat (PS) was expected to carry out the traditional role of project implementation units and the National Coordinators (NC, composed of representative from each country ) was expected to act as a focal point for the dissemination of Project results, and coordination and communication with other GEF -IW projects within the Latin American region. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The project was planned to be financed by a US$ 13.4 million GEF Grant supplemented by co-funding of US$13.36 million from three development partners and the Borrowers . In April 2004 the GEF Grant amount was changed from a SDR denomination to US$ as part of Bank -wide action. And in March 2007 the project was restructured to reallocate Grant proceeds among components .Total Project costs were 18% more than planned at appraisal primarily because (a) studies and exploration to expand the knowledge base (component 1) were 61% more then planned and (b) project coordination and management was 33% more then planned. These increases were partly offset by reduced disbursement on development of a joint management framework (components 2) and on monitoring and evaluation (component 4). The project was supported by parallel financing from 3 development partners totalling US$ 1.85 million: the Bundesanstaltfuer Geowissenschaften und Rohstoffe (BGR) for US$0.90 million (planned US$0.60 million); the International Atomic Energy Agency for US$ 0.63 million (planned US$0.30 million); and the Organization of American States (OAS) for US$0.32 million, the same as planned. The Borrowers (the regional governments of Argentina, Brazil, Paraguay and Uruguay ) contributed US$15.32 million compared with the US$11.99 planned at appraisal. Similarly beneficiaries contributed US$ 0.76 million, significantly more than the planned amount of US$ 0.05 million. The Bank supplemented the GEF Grant with US$ 0.29 million grant from the Bank-Netherlands Water Partnership Program to provide additional TA to assist evaluation and quality control of project consultants' outputs In March 2007 the project closing date was extended for 22 months. Among the reasons for the delay, the ICR and the Project Team in subsequent discussions cite (i) the large geographic area to be covered; (ii) lack of capable firms to undertake the studies; (iii) the need for exhaustive and detailed work to address the rudimentary knowledge base available on the GAS; (iv) extended procurement arrangements and clearances involving four countries; (v) the need to address initial negative perceptions of the project among NGOs and others; (vi) the time required to harmonize OAS and Bank procedures and guidelines; (vii) fulfilling the GEF requirement of wide consensus building at various levels; and (viii) the large number of institutions involved in the project . 3. Relevance of Objectives & Design: Objectives : Relevance is High . The Project is in line with the GEF International Waters (IW) focal area that addresses, among other things, assisting "countries to collaborate with their neighbors to modify human activities that place stress on transboundary water systems ..." In conformity with the GEF IW focal area, this Project supported the countries in learning to work together on their key transbound ary concerns and set priorities for joint action as reflected in the SAP and other auxiliary declarations . Because groundwater sustains streams, lakes and wetlands in the dry season, the better management of the GAS enhances the health of dependent ecosystem and biodiversity thus also making a significant long-term contribution to GEF's environmental objectives . Current CASs affirm the Project's continuing relevance . The 2006 CAS for Argentina specifically stated that regional GEF activities are focusing, among others, on shared water (both surface and the Guarani aquifer ). The 2008 Brazil: Country Partnership Strategy (FY2008-2011) states that the Bank Group will assist Brazil in fostering coordinated efforts to address large transboundary issues in partnership with, among others, the Guarani aquifer riparian countries . Although the 2003 CASs and 2009 CPS for Paraguay and the 2005 CAS for Uruguay did not specifically mention the Guarani aquifer, both aimed to improve water resource management and enhance coverage of water supply and sanitation services . Design : Relevance is Substantial . Establishment of the regional organizational structure was highly relevant given the complexity of a four -country project. The multi-country Steering Committee provided inter -country and regional coordination supported by National Project Execution Units (NPEU) and the Project Secretariat (PS), the latter managed by the OAS that executed the Project, and this arrangement proved to be highly successful in managing project implementation . The focus on building knowledge of the GAS through exploration and research to mitigate development concerns proved to be highly relevant although more consideration should have been given to vertical connections with other aquifer systems that were at risk . The focus on identifying protective and preventive intervention to reduce future risks of pollution, overdraft, intra -country conflicts, and high cost of future mitigation and stabilization measures was and remains relevant . Given that the GAS is a transboundary resource the aim to establish a cooperative framework was relevant to achievement of sustainable and integrated management and use of the aquifer. However, the design was overly optimistic in terms of the time needed to implement a four-country project that covered a total area of over a million square km, and the Project schedule had to be extended. Project design underestimated the risk of adverse reactions to the project from some NGOs and the public who were agitating against the 'Washington Consensus' and the misperceptions that public goods were going to be privatized. However, as soon as this became evident a communications and information strategy was designed and implemented and it successfully mitigated public concern . From an operational perspective, the number of indicators was large, but deemed to be necessary in order to adequately monitor the different activities . Based on lessons from GEF IW projects, the design correctly identified that "a significant lag time may occur between the preparation of the ...Strategic Action Program (SAP) and its implementation. For this reason, the Project was designed as the first phase of a larger Program because it was expected that elaboration of projects to implement the SAP will take place during the last year of the proposed Project (PAD, p.18). In practice, each country latterly internalized the requirements of the SAP in their own operations . 4. Achievement of Objectives (Efficacy): long -term objective is the sustainable, integrated management and use of the Guarani Aquifer System The long- . Substantial . The Project successfully supported the countries in learning to work together on their key transboundary concerns and set priorities for joint action as reflected in the SAP and other auxiliary declarations . As a result the risk of over-extraction and contamination of groundwater resources and any transboundary impacts have been significantly reduced. The technical studies clearly demonstrated that the lateral movement of groundwater in the GAS is very slow and that impacts from over-pumping and pollution are highly localized . The ex ante concerns of governments, institutions and civil society that joint coordinated management was needed for the entire aquifer were shown to be unfounded. As a result regionally coordinated management is only required in a few small localities where transboundary effects were likely and, in most areas, management of the aquifer is essentially a local set of activities. A major outcome is that the risks of inter -country conflict over groundwater resources has been significantly reduced. Pollution risks are diminished or controlled . Knowledge generated by the project show that only 10% of the GAS is exposed to pollution and thus management efforts to control these risks are now focussed - indeed pilot studies in some areas (the pilot regions Rivera/Santana and Ribeirao Preto) are developing risk management strategies and have implemented enhanced monitoring and evaluation of groundwater quality (ICR pages 36-39). Overdraft risks are diminished or stabilized . The project demonstrated that the GAS does not show signs of overdraft except in very localized areas where mitigation measures such as well spacing standards, and zoning for protection and extraction have been put in place . Even so, recuperation of groundwater levels in over-exploited areas requires measures that are very difficult to implement . Enhanced understanding on the dynamics of the GAS allied with improved management mechanisms contribute to the efficient and reliable provision of water supply to areas within the aquifer region .The majority of the people who lack access to a reliable and safe water supply system reside in the rural areas of the four countries and 90% of the water withdrawn from the GAS is for water supply . Over 300 cities also rely on the GAS for domestic water supplies. Groundwater is generally the most efficient and cost effective method of improving coverage of access to water supply since scattered settlement patterns (the characteristic feature of rural areas ) would require huge investment costs for building surface water delivery networks and infrastructure . The Project involved a wide array of organizations through its participatory mechanisms, including indigenous people (ICR page 35), raised their understanding and awareness in groundwater management, and the need for protection and management of the GAS (ICR pages 34-36). As an outcome of the technical and scientific studies the adopted coordination mechanisms are light . The Project Secretariat model as a coordinating body has been retained ex post by the Borrowing governments and its responsibilities include : (i) reviewing policy and developing proposals; (ii) providing for comprehensive monitoring; (iii) coordinating institutional capacity improvements and provision of TA; (iv) developing and improving manuals, procedures and guidelines; and (v) continuously updating the knowledge base . Complementing this outcome, a new culture of cooperation among specialists, universities and institutions in the four countries in now in place, and NGOs are now actively involved in participatory management (ICR page 15). In Argentina, a new Directorate for Groundwater in the Sub -Secretariat for Water Resource was created as an outcome of the Project. The National Government created the Inter -ministerial Committee for Groundwater to help improve integrated water resources management . In Brazil, management of the GAS was included within the Implementation of Integrated Management of Surface and GW Program for 2008-2011 with a budgetary allocation of $ 18 million. State governments have invested in studies, monitoring and in general improving the institutional basis necessary for sustainable groundwater management. In Paraguay a Water Law was approved by the Paraguay Parliament in 2007 and the Government created a Guarani Aquifer Unit within the Ministry of Environment . In Uruguay a law to include the creation of Guarani Aquifer Management Unit to ensure participation and the sustainable use of the aquifer is is under consideration by parliament . The longer term sustainability of these improvements depends, inter alia, on adequate implementation of the SAP (see also Section 7 below). 5. Efficiency (not applicable to DPLs): No efficiency estimates were made at either appraisal or completion . The non-quantified benefits derived from the project were both national and global, and stemmed mainly from the avoidance of the future costs of aquifer mismanagement that could have included inter -country conflict, resource depletion and increased pollution . However, the project took nearly two years longer to complete and cost nearly 20% more than foreseen. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Given significant delays and higher costs than foreseen, Outcome is rated moderately satisfactory rather than satisfactory. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Support for the project continues at regional, national and local levels from both Governments and NGOs . Although GAS management has essentially devolved to the level of the individual countries, interest and enthusiasm are high, and the countries are both maintaining, and have agreed to strengthen, their cooperation and the coordination of their activities . There is, nevertheless, a moderate risk that the framework and mechanisms agreed to in the SAP may not be implemented in a timely and adequate manner and with sufficient funding . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: at -entry . The project was thoroughly appraised and considerable attention was given to its Quality -at- institutional and technical design because it was the first Bank project that addressed the management of transboundary groundwater resources . IEG's 2006 Regional Project Evaluation (RPE) noted that "stakeholders praised the substantial technical background of the first Bank Task Team Leader (TTL) who took an active role in Project preparation and oversaw the first two years of implementation ." The recognition that the OAS had a comparative advantage in specific areas of managing a multi -country project in LAC was both realistic and pragmatic and proved to be successful . The appraisal team was, however, too optimistic about the time needed to implement such a complex project . Supervision . Despite having 3 TTLs, supervision provided coherent support and guidance to the implementing agencies and collaborated well with development partners . It was assiduous in bringing the Bank's convening power and resources to bear at high diplomatic levels to mitigate and reverse adverse public opinion about the project . And at the other end of the spectrum it introduced notable innovations in the pilot projects to build local knowledge and support (as noted in IEG's 2006 RPE (ICR page 19)) and develop Local Action Plans. Supervision reports were thorough and well documented . a. Ensuring Quality -at- at -Entry :Satisfactory b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government . The four country governments were represented through the Steering Committee that strongly supported the project and remained neutral to some bilateral water issues, including adverse publicity, that could made the SC ineffective at coordination . The SC approved annual plans on time, successfully facilitated counterpart contributions by each country and oversaw the technical direction . As the Project closing approached, the SC proactively established a Working Group to study and recommend transition mechanisms, the financing of future activities, cooperation arrangements, and the division of responsibilities . Counterpart financing reached 127% of the appraisal estimate and clearly demonstrated governments' wide support and ownership. There were, however, some variations in performance among the four Governments . The Bank and OAS differed in their assessment of the degree of completion and readiness of enabling institutional arrangements at the regional level at project closure, with the OAS assessing that these were only partially in place (ICR, pages 46-50). Implementing Agency . The OAS successfully coordinated multi -country project inputs and activities and worked with the different donors supporting the project . The OAS partnered with the Bank in harmonizing the two institutions' procedures in financial management, contract awards and applicable operational guidelines and rules. It was particularly successful in maintaining an equitable staffing from the four countries and managing the internship and twinning programs. OAS's experience in international water greatly facilitated the preparation of transboundary diagnostic analysis and SAP . The OAS Argentina Office Chief, who was responsible for Project implementation coordination, was instrumental in ensuring that SAP preparation reflected the relevant knowledge generated from the vast information collected and that it focused on practical actionable items . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design . There was substantial attention to all aspects of M&E at appraisal and the PAD included a special Annex 18 describing the objectives of M&E, system structure, M&E activities and a long list of indicators including those relevant to tracking compliance with GEF's IW (process, stress reduction, and environmental status), as well as those required to monitor inputs and outputs . This was supplemented by the traditional logical framework in Annex 1 of the PAD. Detailed design of the M&E system was supported by the GEF Grant under component 4. Implementation . During implementation, the logical framework was updated following the results framework that introduced a temporal element into the M&E process . Periodic reports from the four pilot sites and from other project activities, including the Citizen Fund and the University Fund, were fed into the project M&E system managed by the PS. The Oracle-based system of the OAS facilitated the monitoring of financial utilization and commitments and performed satisfactorily for project supervision and planning . Several evaluations were carried out in conjunction with the MTR and with preparation of the trans -boundary diagnostic analysis and the SAP . In addition, the OAS contracted an external consultant to do the final evaluation of the Project and the BNPP supported TA to undertake quality control reviews of project outputs . Utilization . According to the ICR national reports were consolidated into project progress reports for PS for SC Project oversight. As a final product, the project deployed the GAS Information System (SISAG), with each country having access to the system and an interface to directly update data (Component 2). Through SISAG the GAS now has a state-of-the-art information system that could be instrumental in future monitoring and management of the aquifer. Evidence of actual utilization is weak . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards . According to the ICR, three safeguards issues that apply to the project : Environmental Assessment (EA, OP 4.01, the project was classified as Category B ), Indigenous People (OP 4.10) and International Waters (OP 7.50). EA was systematically applied to the project as an integral component of project design . As such environmental issues are reported in detail, both in the ISRs and project output reports . International Waters were similarly internalized among the four riparians that share and coordinate the management of the GAS . The Project developed an Indigenous People's Strategy (IPS) that was disseminated widely and earmarked resources from the Citizen Fund to ensure the participation of indigenous organizations in awareness building . According to the ICR, as a result of the development of the IPS, the SAP was able to identify and incorporate actions that the three countries should follow to benefit and ensure the participation of indigenous people in the future use and management of the Guarani aquifer . Fiduciary . The ICR reports that the project met all the Bank's fiduciary requirements . All audits and post-procurement reviews were unqualified . Unintended Impacts . The GEF grant leveraged greater Borrower and beneficiary buy -in than planned (actual was US$16.08 million vs US$12.04 anticipated) and they continue to finance activities initiated through the Project (ICR para 5.2.3). 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Given significant delays and higher Satisfactory costs than foreseen. Risk to Development Negligible to Low Moderate There is a moderate risk that the Outcome : framework and mechanisms agreed to in the SAP may not be implemented in a timely and adequate manner and with sufficient funding. Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Regional projects tend to be administratively complex and considerable attention has to be given to project scheduling and coordination . This project showed that it is often difficult and time consuming to reach consensus and move forward, a task made more difficult by fragmented responsibilities and differing institutional approaches in each country . In the absence of any other existing regional framework, regional projects have to set up their own project implementation framework . The success of such frameworks is highly dependent on the unbiased selection of staff and good governance at all levels and their willingness to put the common good above national interests. Only thus can trust in the objectivity of the framework be established at country level . A regional project has to continuously engage national line -agency counterparts to maintain the relevance of the project objectives and outcomes to their work programs . Once the riparian countries of the Guarani aquifer realized the importance of the project's outputs to their work, they shared scientific and technical information and actively participated in developing management approaches . Regional projects should develop a communication strategy early in project preparation and continuously updated it to reflect the changing circumstances . Using NGOs, civil society organizations and learning institutions (especially those which have a presence at the local level ) for awareness building is effective because it leverages their local credibility and specialized knowledge to support project development objectives. Water sector policies need to be aligned with national and regional development policies . This project highlights the importance of planning and coordinating surface and ground water development within a comprehensive socio-economic framework that includes the views and input from civil society and indigenous peoples . Economies of scale and scope do not always require regional action . However this lesson was only possible after a systematic assessment of water resources and their inter -reaction, and the risks posed by development proposals had been thoroughly completed, and after mechanisms were put in place to enable riparian stakeholders to reach a consensus . 14. Assessment Recommended? Yes No Why? To verify the ratings. Also, given the differences in the Bank and OAS assessments of institutional capacities. 15. Comments on Quality of ICR: A very comprehensive document that is, at times repetitious . The annexes are excellent and the OAS's contribution adds considerable value and insight on project achievements . a.Quality of ICR Rating : Satisfactory