Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) Report Number : ICRR0021137 1. Project Data Project ID Project Name P114810 BR GEF Amazon Region Prot Areas Phase 2 Country Practice Area(Lead) Brazil Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-12073 30-Nov-2015 85,890,000.00 Bank Approval Date Closing Date (Actual) 23-Feb-2012 31-Jul-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 15,890,000.00 15,890,000.00 Revised Commitment 15,890,000.00 15,890,000.00 Actual 15,890,000.00 15,890,000.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD) (p. iv) and the Grant Agreement of March 21, 2012 (p. 7) the objective of the project was “to expand and consolidate the system of Protected Areas in the Amazon Region, and to strengthen the mechanisms for its financial sustainability.” The Global Environmental Objective is the same as the Project Development Objective (PDO). This project is the second phase of the three phase Amazon Region Protected Areas Project (ARPA). Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project included four components: Component 1: Creation of New Protected Areas (appraisal estimate US$1 million, actual US$0.30 million): This component was to finance the creation of approximately an additional 13.5 million hectares of new Protected Areas in the Amazon Region to those created under ARPA 1 by i) identifying new areas in the Amazon Region to be designated as Protected Areas; ii) carrying out environmental, socioeconomic, and land tenure assessments as needed, including public consultations and workshops, in respect to the new areas identified in the Amazon Region to be designated as Protected Areas; and, (iii) establishing new Protected Areas by enacting the necessary decrees, followed by demarcation of the boundaries of the Protected Areas in question. Component 2: Consolidation of Protected Areas (appraisal estimate US$11 million, actual US$12.08 million): This component was to finance the consolidation of approximately 32 million hectares of Protected Areas, including new Protected Areas established under Component 1 of the project and other Protected Areas already established before the implementation of this Project by (i) providing technical assistance to strengthen the recipient’s capacity in managing the consolidation of Protected Areas and carrying out works to build certain structures, such as visitor centers, office space, guard-posts, to support consolidation in selected Protected Areas; (ii) preparing, implementing and evaluating Management Plans for Protected Areas; (iii) promoting better coordination and institutional enhancement of local communities and organizations; and, (iv) providing training to relevant staff in the management of Protected Areas. Component 3: Long-Term Sustainability of Protected Areas (appraisal estimate US$0.40 million, actual US$0.26 million): This component was to finance technical assistance to develop and implement strategies to raise additional revenue for the Endowment Fund and strengthen the Recipient’s procurement capacity and to support the development of effective and transparent mechanisms for disbursement of the proceeds of the Endowment. This component was also to finance the carrying out of studies to identify management and funding options to support the long-term economic sustainability of Protected Areas. Component 4: Project Coordination, Monitoring, Management and Communication (appraisal estimate US$3.49 million, actual US$3.25 million): This component was to finance the establishment of efficient management arrangements, strengthening of communication and coordination among stakeholders, and management of financial resources, procurement and budget allocation. Also, this component was to support the development and implementation of a communication strategy, and M&E activities. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$85.89 million. Actual cost was US$87.89 million. Financing: The project was financed by a US$15.89 million Global Environment Facility grant and US$40 million co-financing by donors (US$30 million by the KfW Development Bank and US$10 million by the Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) World Wildlife Fund). Borrower Contribution: The borrower (state and federal level) was planned to contribute US$30 million. Actual contribution was US$29 million. Dates: The project was approved on February 23, 2012 and effective on June 19 the same year The project was restructured twice: • On September 22, 2015, the project was restructured to extend the closing date from November 30, 2015 to November 30, 2016. Political and administrative changes and demobilization of teams at the end of the previous phase of the ARPA Program (Phase I of this project) resulted in project implementation and disbursements delays. The project extension was to allow for the completion of all pending activities to enable the full achievement of the project’s targets and outcomes, and to utilize the entire grant. • On November 1, 2016, the project was restructured to extend the closing date from November 30, 2016 to July 30, 2017 to allow the grant co-recipients to finalize ongoing project activities, conduct final independent project evaluation, and reallocate grant proceeds between disbursement categories. The original closing date was November 30, 2015 and the actual closing date was July 31, 2017, i.e. a 20 months’ long extension, or 1 year and 8 months in total. 3. Relevance of Objectives Rationale The Amazon region represents the largest area of remaining tropical rain forest in the world (approximately 30 percent) and is estimated to contain carbon stores of approximately 120 billion tones. However, despite the significant importance of the region to the regional and global climate, its sustainability is threatened by deforestation, degradation of watersheds, and overexploitation of wildlife and fisheries resulting from poorly planned and managed economic development. The government of Brazil has been developing policies and implementing regulations to expand the Protected Areas and reduce deforestation and greenhouse gas emissions since 1997. Critical policies included the passage of the National Protected Area System passage which put the management of federal, state, and municipal Protected Areas within a single national system, the establishment of the 2010 National Goals for Biodiversity, and the 2009 adoption of a National Policy for Climate Change. The project’s objective is in line with the Bank’s most recent Country Partnership Framework (FY2018-2023) which focuses in one of its thematic areas on inclusive and sustainable development by supporting land-use targets, and promoting inclusive rural development and protection of vulnerable groups. The objective of the project is also in line with the Bank’s 2016 Systematic Country Diagnostic which identifies well managed land, forest, and water resources as key to economic returns, provision of livelihoods, and environmental services. Furthermore, the project’s objective is also aligned with the Bank’s Biodiversity Roadmap which stresses the importance of sustainable use and conservation of biodiversity to end extreme poverty and promote shared prosperity. Also, the Bank’s Climate Change Action Plan 2016 emphasizes climate-smart land use, water and food security as one of six high-impact action areas. Finally, the project’s objective Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) supported the Global Environmental Facility Amazon Sustainable Landscape program which aims to protect globally significant vegetation cover in Brazil, Colombia and Peru. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To expand the system of Protected Areas in the Amazon region: Rationale Outputs: • The amount of areas studied for the creation of Protected Areas increased from 5 million hectares in 2012 to 30 million hectares in 2017, achieving the target. The studies conducted included analyses of the integrity of ecosystems, presence of local communities and indigenous people, and land tenure status. • The map of priority areas is being updated based on the evaluation of the representation of the current system of Protected Areas, public areas without destination, endangered species, traditional communities, and indigenous people. However, since this process has not been completed, the target of completing the updating the map was not achieved. Outcomes: • The amount of newly protected areas in the Amazon increased from 2.4 million in 2012 to 3.75 million in 2017, not achieving the target of 13.5 million hectares. Officially, 5.5 million hectares were created resulting in the establishment of 24 new Protected Areas. However, only 16 Protected Areas, covering 3.75 million hectares, were eligible to be counted toward the ARPA targets according to the program’s guidelines, which only allows the inclusion of strict protection categories (ecological stations, biological reserves, national parks, natural monuments, and wildlife reserves) and two sustainable use categories (extractive reserves and sustainable development reserves). • 33.95 million hectares of protected areas were consolidated in the Brazilian Amazon, surpassing the target of 32 million hectares. Of the 114 Protected Areas supported by the ARPA program, 30.05 million hectares were consolidated in stage 1 and 3.9 million hectares were consolidated in stage 1. Rating Modest Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To consolidate the system of Protected Areas in the Amazon region: Rationale Outputs: • Six integrated Protected Area Management Models were developed and tested, surpassing the target of three models. Outcomes: • The amount of existing protected areas consolidated increased from 8.5 million in 2012 to 33.88 million hectares in 2017, surpassing the target of 32 million hectares. A consolidated Protected Area is defined as one which has all necessary material, financial, and human resources to ensure that its protected status is being respected and land-use restrictions are being enforced. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To strengthen mechanisms for the system of Protected Areas’ financial sustainability: Rationale Outputs: • 23 Protected Area Action Plans were implemented with communities and indigenous people in accordance with the Indigenous People Policy Framework and Process Framework, surpassing the target of 20 Action Plans. These action plans focused on increasing capacity of traditional communities and indigenous people to participate in the management of Protected Areas and to improve the sustainability of their natural resource use. • The project’s capacity building plan was prepared and its implementation started in 2014, achieving the target. 692 Protected Area Staff were trained in technical and administrative capacity. • The ARPA endowment fund is fully operational and makes annual disbursements to selected Protected Areas, achieving the target. • Six strategic studies and four protected Areas financial sustainability plans were implemented, Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) surpassing the target of six studies and plans. • The Strategy for Conservation and investment (ECI) and Protected Areas Evaluation Tool (FAUC) are being annually updated and used to guide operations. The ECI assesses, among others, the biodiversity significance of and the level of threat to individual Protected Areas. The FAUC assesses the management performance of Protected Areas. Outcomes: • The ARPA endowment fund (FAP) increased from US$27.2 million in 2012 to US$121.3 million in 2017, surpassing the target of US$42.8 million. Rating High PHREVDELTBL PHOVRLEFFRATTBL Rationale Achievement of the objective "to expand the system of Protected Areas in the Amazon region" was Modest, achievement of the objective "to consolidate the system of Protected Areas in the Amazon region" was Substantial. Achievement of the objective "to strengthen mechanisms for its financial sustainability" was High. Overall, this results in a Substantial efficacy rating. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency: The PAD (p. 12) included an economic analysis of ARPA Phase 2 which was based on incremental costs and compared a baseline scenario with the ARPA Phase 2 proposal. The baseline scenario was based on the progress of Phase 1. Soares-Filho et al. (2008) found that the creation of 13 Protected Areas in the Amazon under ARPA from 2003 to 2007 was associated with the offset of emissions equivalent to 430 million tons of carbon by 2050 as compared to the business-as-usual scenario. Assuming the value of US$ 5 per ton of carbon, these PAs were to account for US$ 2.2 billion dollars of emissions reductions by 2050 or about US$ 54 million dollars per year. The ARPA Phase 1 total cost was of US$ 84.5 million, thus the IRR for this investment was 22%. The ICR (p. 10) mentioned the analysis for the Brazil Amazon Sustainable Landscapes project which includes the third phase of the ARPA program. The analysis shows that the program is economically viable even when only stimulated over the project life time. However, the ICR did not conduct an Economic analysis for this project. Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) Operational Efficiency: The project’s closing date had to be extended twice which might indicate an inefficient use of project funds. Overall, the project’s efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the objective was High since it supported the government’s priorities and the Bank’s strategies to expand the Protected Areas and reduce deforestation and greenhouse gas emissions. Overall efficacy was Substantial. Efficiency was Modest. Overall, this results in a Moderately Satisfactory outcome rating. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Since the new government came in place in 2014, commitment to the creation of new Protected Areas in the Amazon is less favorable than it used to be. In May 2017, the Brazilian congress approved, for the first time since the establishment of the 1988 Constitution, a reduction and degazetting of federal Protected Areas. The president vetoed this vote, however, in 2016 a constitutional amendment was passed which freezes federal spending for the next 20 years. Any increase in budget line items such as spending on Protected Areas requires budget cuts of an equal amount in another area. This is a major setback to the government’s previous commitment to increase the budget for ARPA annually. This political environment presents a risk to the sustainability of project outcomes and the overall ARPA. Firm donor commitments and growing international Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) recognition of the importance of this program will continue to be critical. 8. Assessment of Bank Performance a. Quality-at-Entry The project design was built on the experience of ARPA’s phase 1 and the achievement of pilot programs to conserve the Brazilian rain forest and previous Global Environment Facility operations in the country. The project benefited from the experience of key Bank team members already having worked on the implementation of phase 1, allowing for continuity. The Bank identified relevant risk factors, however, the risk of political changes was underestimated, resulting in implementation delays due to changes in internal procedures for the creation of new Protected Areas. During project preparation, the Bank took critical aspects such as securing community rights to land and resource access, the interest of indigenous people, and providing opportunities for income generation associated with Protected Areas, into account. The team designed a robust Results Framework with relevant indicators for measuring progress towards the project’s objective (see section 10 a for more details). Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank conducted regular supervision missions twice per year to assess implementation progress and identify any issues. The in-country presence of some team members allowed for continuous engagement with the counterpart and for fast responsiveness to address any implementation bottlenecks. Also, the project benefited from having the same Task Team Leader throughout implementation. The Bank submitted Implementation Status Reports in a timely manner and reviewed the project’s compliance with social and environmental safeguards throughout implementation. The project’s mid-term evaluation found that the growing political crisis might have a negative impact on the creation of new Protected Areas. However, the Bank, the government and the donors decided to maintain the project’s objective and did not change any target indicators. The Bank team was successful in securing new Global Environment Facility funds to support the third phase of the ARPA program ensuring the sustainability of already achieved outcomes. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) a. M&E Design The theory of change and how key activities and outputs would lead to outcomes were reflected in the Results Framework. The objective of the project was clearly specified. The Results Framework included an adequate set of PDO and intermediate outcome indicators to measure the contribution of the project towards achieving the outcomes. The indicators were specific, measurable, relevant and time bound. All PDO indicators had a baseline. The targets were realistic based on ARPA Phase 1, however, due to the changing political context they became too optimistic. The M&E unit within the Ministry for Environment (MMA), which was already established under ARPA Phase 1, implemented M&E activities independent from the Project Coordination Unit. The Protected Areas Evaluation Tool was used to report on the status of the consolidation of each Protected Area on an annual basis and provide data for the development of the biennial Protected Areas operating plans. The M&E system is likely to be sustainable since a new Bank project, which will continue to finance the M&E system, was approved in December 2017. In addition, the M&E system was adopted by the Protected Areas Agencies (ICMBIO). b. M&E Implementation The indicators included in the Results Framework were measured on a regular basis by the Project Implementation Unit. An M&E specialist within the PIU was responsible for data collection, tracking and monitoring progress. The project developed a systematic reporting tool which provided regular data on all project activities. A technical financial report on the progress of each of the project’s component was produced every two years. Bank missions produced regular progress reports on the selected indicators on a biannual basis. Additional data was also collected through the World Wide Fund’s (WWF) Rapid Assessment and Prioritization of Protected Area Management (RAPPAM) tool at a system-wide level and the Global Environmental Facility Management’s Effectiveness Tracking Tool at the Protected Area level. The Bank team stated that M&E data was of good quality and reliable. In addition to the Bank’s close supervision during project implementation, the ARPA program’s M&E system was also assessed by different partners such as WWF and KFW Development Bank and considered robust and of good quality c. M&E Utilization The Bank team stated that M&E findings were used to track progress towards the project’s objective and to inform decision making such as the funding made by the Transition Fund Board which was based on the achievement of established consolidation benchmarks for different types of Protected Areas. Furthermore, the project established a comprehensive system of biodiversity monitoring that is currently implemented in 45 Amazon Region Protected Areas. This monitoring system has already produced a large amount of data which is being used by researchers and provides new information about the biodiversity of the Amazon Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as category B and triggered the Bank’s safeguard policies Environmental Assessment (OP/BP 4.01), Natural Habitats (OP/BP 4.04), Forests (OP/BP 4.36), Physical Cultural Resources (OP/BP 4.11), Indigenous People (OP/BP 4.10), and Involuntary Resettlement (OP/BP 4.12). During project preparation, an Environmental Assessment was conducted, and an Environmental Management Framework was developed as well as an Indigenous People Policy Framework and Resettlement Policy Framework. No involuntary resettlement took place because of the project. Also, issues regarding some access restrictions to natural resources by neighboring communities were addressed by the management councils of the Protected Areas. The ICR (p. 16) states that safeguard compliance was satisfactory throughout project implementation. b. Fiduciary Compliance Financial Management: The project complied with the Bank’s policies, guidelines and financial covenants. The team at the Brazilian Fund for Biodiversity (FUNIBO) maintained detailed accounts throughout project implementation. Also, FUNIBO conducted a financial management assessment of the project which was found to be Satisfactory. The Bank conducted regular expenditure reviews. Any issues that were identified were clarified and addressed properly. All Interim Financial Reports were submitted on a timely basis and all external financial audits were found to be satisfactory. Procurement: The Bank conducted ex-post procurement reviews on a regular basis. During the 11th supervision some weaknesses in procurement capacity were identified. The Bank addressed them by providing additional assistance and training and procurement was rated Satisfactory at project closing. The Transition Fund adopted FUNIBO’s procurement rules as described in its procurement manual. Procurement was rated Satisfactory at project closing. c. Unintended impacts (Positive or Negative) NA d. Other Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Modest --- 12. Lessons The most notable lessons learned are taken from the ICR (p. 18 to 20) with some modification of language: • Cost modelling of PA management and of conservation fund dynamics are essential tools for long- term conservation planning: This project conducted extensive cost modelling on the Protected Areas to estimate a more accurate overall budget for the program and for a long-term conservation fund. This modelling provided the necessary information for the establishment of the Transition Fund such as the capitalization requirements, time limit for the sinking fund, and the government’s Protected Area budget required to achieve the total incorporation of the ARPA program. • Traditional income-generating activities for Protected Areas such as eco-tourism, concession agreements, end entrance fees might not work everywhere. In this project, as well as in Phase 1, such income generating activities were unsuccessful in the Protected areas due to their remoteness, lack of transportation infrastructure, health and sanitation concerns, and high costs. Therefore, other revenue generating mechanisms such as procuring compensation money from large-scale development projects, were identified. • Allowing the management approach to be adapted is critical for the management of Protected Areas over a long-term period since it allows for flexibility. Over a 15 year implementation period the ARPA program developed different methodological tools to address budgetary, institutional and political changes. This had a positive impact on the implementation of activities. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR GEF Amazon Region Prot Areas Phase 2 (P114810) The ICR provides a good overview of key factors affecting project preparation and implementation. The ICR is concise and internally consistent. However, the ICR does not provide a traditional Economic analysis and does not list any outputs the project produced. Also, the ICR does not provide sufficient information on critical areas such as M&E and does not state the date of the second restructuring. a. Quality of ICR Rating Modest Page 12 of 12