70513 JUNE 2011 PROFOR Evaluation Series Impact Note 2 An Evaluation of the Impacts of Selected Activities Supported by the Program on Forests in Central America Michael P. Wells Claudia Alderman Stephanie Altman A PROFOR WORKING PAPER A PROFOR WORKING PAPER ACKNOWLEDGMENTS This work was funded by the Program on Forests (PROFOR), a multi-donor partnership managed by a Secretariat at the World Bank. PROFOR finances in-depth forestry research and processes that support the following goals: improving people‟s livelihoods; enhancing forest governance and law enforcement; financing sustainable forest management; and coordinating forest policy with other sectors. Learn more at www.profor.info. This paper was prepared by Michael P. Wells, Claudia Alderman and Stephanie Altman. Many individuals provided suggestions and information: the list of people interviewed for this report is provided in Annex 2. DISCLAIMER All omissions and inaccuracies in this document are the responsibility of the authors. The views expressed do not necessarily represent those of the institutions involved, nor do they necessarily represent official policies of PROFOR or the World Bank. Suggested citation: Wells, Michael P., Claudia Alderman and Stephanie Altman. 2011. An Evaluation of the Impacts of Selected Activities Supported by the Program on Forests in Central America. Evaluation Series: Impact Note 2. Washington DC: Program on Forests (PROFOR). Published in June 2011 For a full list of publications please contact: Program on Forests (PROFOR) 1818 H Street, NW Washington, DC 20433, USA profor@worldbank.org www.profor.info/knowledge Profor is a multi-donor partnership supported by: Learn more at www.profor.info Evaluation Series ii TABLE OF CONTENTS List of Acronyms _________________________________________________________________________ iv 1.Introduction ___________________________________________________________________________ 1 2.Main Messages _________________________________________________________________________ 2 3.Evaluation Approach, Methodology and Constraints _________________________________________ 5 4. Overview of Findings ___________________________________________________________________ 10 5. Evaluation Reports on Individual Activities _________________________________________________ 19 C01: Knowledge Sharing for REDD Activities in Latin America and the Caribbean __________________ 19 F03: Strengthening the Value Chain for Indigenous and Community Forestry Operations Through Increased Investment and Use of Technical Assistance _______________________________________ 21 G01: Supporting Forest Stakeholders‟ Participation in Forest Consultative Councils _________________ 24 G03: Technical Assistance for the Development of the National Timber Yield Tables for Mahogany Standing Volume & Export Grade Sawn Wood ______________________________________________ 27 G05: Strategy to Combat Illegal Forest Activities in Guatemala _________________________________ 30 G23: Honduras: Forest Sector in Transition_________________________________________________ 36 G27: Linking Land Tenure Regularization and Forest Management in Honduras ____________________ 36 Annex 1 – Evaluation Questions ____________________________________________________________ 40 Annex 2. People Interviewed _______________________________________________________________ 42 Annex 3. Activities Eligible for PROFOR Support ______________________________________________ 44 iii Impact Note 2 LIST OF ACRONYMS CATIE Centro Agronomico Tropical de Investigacion y Enseñanza CBD Convention on Biological Diversity CIFOR Center for International Forestry Research COHDEFOR Honduras former National Forest Agency CONADEH Human Rights Commission of Honduras CONAP Protected Areas Council, Guatemala COP Conference of the Parties CITES COP Conference of the Parties EU European Union FAO Food and Agriculture Organization of the United Nations FCPF Forest Carbon Partnership Facility FCPF GEF Global Environment Facility GoH Government of Honduras IIED International Institute for Environment and Development INAB National Institute of Forests, Guatemala ITTO International Tropical Timber Organization IUCN World Conservation Union LAC Latin America and the Caribbean LCSAR Latin America and the Caribbean Region – Agriculture & Rural Development MAO Olancho Environmentalist Movement NFP National Forest Programs NGO Nongovernmental Organization ODI Overseas Development Institute OSINFOR Office of Supervision of the Forest Resource and Wildlife, Peru PBPR Forests and Rural Productivity Project POA Annual Operating Plan POA PROFOR Program on Forests, World Bank REDD Reduced Emissions from Deforestation and Forest Degradation TREES Rainforest Alliance Training, Extension, Enterprises and Sourcing Program UNFCCC United Nations Framework Convention on Climate Change Evaluation Series iv 1. INTRODUCTION The Program on Forests (PROFOR ) is a multi-donor collaborative partnership. Its goal is to strengthen forests' contribution to poverty reduction, sustainable economic development and the protection of global and local environmental values. The purpose of PROFOR is to contribute to the capacity of institutions and stakeholders in forest policy processes to address more effectively poverty alleviation, national economic development, climate change mitigation and adaptation, and sustainable forest management. PROFOR describes the initiatives it supports as Activities. By early 2011 more than 80 Activities had received financial – and in some cases technical – support since PROFOR was launched in 2003. This report describes the results of an evaluation/ex post review of 9 selected PROFOR Activities from Central America, carried out to help develop a better understanding of PROFOR‟s impacts. Conducted by a team of consultants, the evaluation was commissioned by the PROFOR Secretariat at the request of its Advisory Board as a contribution to strengthening performance monitoring. The specific objective of this evaluation was to assess the policy and other impacts of selected PROFOR Activities that had been completed or were close to completion. The evaluation included brief field visits to Guatemala and Honduras during January 2011. To streamline the text, the Activities are referred to in this report by the numerical code assigned to them for this evaluation (see Table 1). A parallel evaluation of 12 additional PROFOR Activities focused on „forests‟ roles in mitigating poverty in landscape preservation and rehabilitation‟ was carried out by the same consulting team. The results have been reported separately. In total, the 21 Activities reviewed by these two evaluations comprise 40% of the 51 closed Activities and 25% of PROFOR‟s 84 Activities (closed and open). These two evaluations were planned and implemented specifically to provide an independent view of the impacts of selected PROFOR Activities. This should be distinguished from an independent evaluation (as defined by international best practice) of PROFOR as a whole. 1 Impact Note 2 2. MAIN MESSAGES Generally high quality and effective Activities have achieved impressive results and impacts in Central America, especially given the relatively modest level of financial resources provided. Positive impacts were most evident as a result of PROFOR support for forest-related analyses (G16, L13, G05); mainstreaming sustainable forestry (G05, G23); testing innovative tools and approaches to better governance (G01, G03); and networks and partnerships (C01). In common with other Central American countries outside Costa Rica, Guatemala and Honduras – the two countries visited – have very limited resources for their forest sectors and remaining few international donor partners. These conditions have helped PROFOR play a relatively significant role in Central American forest management, largely through Activities that originated with FLEG. Major efforts to combat illegal forestry in Guatemala, although supported by a strong professional forestry authority and a committed government, were heavily dependent on the sustained involvement of FLEG and, more recently, PROFOR. PROFOR‟s involvement has helped leverage additional support from other donors, notably including the FINNFOR project supporting sustainable forest activities in seven Central American countries. In addition, the EU plans to provide support that will sustain PROFOR‟s investment in Activity G01 in Honduras. The PROFOR Secretariat team working in Central America, as well as their selected partners within the World Bank and beyond, display a deep knowledge and understanding of the forestry sector as well as a solid appreciation of shifting priorities and opportunities, all key contributing factors to PROFOR‟s effectiveness. While not the principal focus of our work, we detected a clear consensus among Central American stakeholders both within and beyond the World Bank that PROFOR‟s management, operations and procedures have evolved and improved significantly, especially during the last two years. PROFOR has maintained a productive, active presence in Central America, building on earlier work supported by FLEG. PROFOR‟s staff have been a consistent source of reliable and committed technical expertise. World Bank managers, especially in the LAC Region, have often been unwilling to invest staff time and resources in forestry. Until recently forestry investments were generally perceived as offering high risks, controversy and low returns. Evaluation Series 2 This situation now shows signs of changing, with PROFOR deserving at least some of the credit for having highlighted forest sector opportunities in LAC. This evaluation identified some areas where PROFOR‟s work could be strengthened and made more effective. These reflect our sample of completed Activities and in some cases PROFOR has already taken steps to improve these areas for ongoing projects. Baseline for proposed Activities: It is not easy to establish the impact of a given Activity in the absence of a clear statement of (a) the baseline situation and (b) how the Activity was expected to change the current situation. Beyond contributing to impact evaluation, such information would also help PROFOR make better decisions and thereby improve the likelihood of more significant impacts. Recommendation: Incorporate into PROFOR‟s Concept Notes/proposals concise, yet rigorous analysis of the baseline surrounding each proposed Activity. Monitoring of ongoing activities: The implementation of the Activities reviewed was not consistently monitored, although it is important to note that the Secretariat had a very small staff until recently 1. Activities contracted to organizations outside World Bank in particular require supervision during all phases from design through implementation and the final dissemination of products, to assure quality control as well as optimal linkages to Bank operations where applicable. Recommendation: Consider setting up an “alert� or “projects under watch� system to proactively review and decide on course of action in cases where there are changes in the political situation or other circumstances in the beneficiary country or where the party responsible for implementation may be underperforming. Documentation: Some Activity documentation was hard to locate and not systematically organized, sometimes with key information residing only on staff members‟ personal computers. Recommendation: Set up standardized and well organized management information system and assign responsibility for ensuring that the key information related each Activity is consistently archived. Assessment and incorporation of lessons into future work: Activity completion reports, online write-ups and equivalent documents, while factually correct, often make little attempt to realistically assess impacts or effectiveness or to extract lessons. Given that Activity reports are prepared by the proponent, there is a tendency to gloss over problems and eschew open discussion of performance. This evaluation appears to represent the first systematic attempt by PROFOR to reflect on and learn from its experiences. 1 The Secretariat staff numbers increased as a result of the merger with FLEG and an increase in the volume of Activities supported. 3 Impact Note 2 Recommendation: Establish routines within PROFOR for candid and analytical assessments of the impacts of each Activity, developing a culture where occasional failures are recognized as learning opportunities. Consider carrying out “exit interviews� with Activity proponents to elicit open discussions of the Activity beyond what is politically correct and comfortable to document in writing. This should support true learning while adding credibility to genuine achievements and impacts. Dissemination: dissemination of PROFOR‟s high quality products could be substantially improved. Knowledge dissemination was in most cases ad hoc, and treated as a separate component after project implementation. Consequently, these products did not always reach key decision makers. In some cases where governments had changed, incoming staff with new administrations were not aware of previous PROFOR products even when these were very relevant to topical issues. Recommendation: Ensure that effective and proactive dissemination of results and products is embedded into the individual Activity proposals and, where appropriate, incorporate dissemination of specific products in the program as part of a systematic effort. Dissemination should receive particular attention after changes of key staff and/or governments. In the case of LAC it is imperative that reports be translated into Spanish, and in Brazil into Portuguese in order to effectively reach key policy makers and other stakeholders. Evaluation Series 4 3. EVALUATION APPROACH, METHODOLOGY AND CONSTRAINTS APPROACH Starting in late 2010, the evaluation team carried out a preliminary review of the 84 Activities supported by PROFOR since its inception2. The 51 Activities that were closed or close to completion were considered for this evaluation, while the 33 currently ongoing Activities were excluded. The team also reviewed the reports on other recent PROFOR evaluations. In consultation with the PROFOR Secretariat, the evaluation team identified two clusters of Activities for detailed review. The objective here was to group Activities that could be expected to share common characteristics, with the intention of facilitating impact assessments that were as specific and comparable as possible. The four PROFOR „themes‟ (i.e., livelihoods, financing sustainable forest management, cross sectoral and governance) each contained ranges of Activities that were considered too diverse for this purpose. Certain “obvious� potential clusters of Activities – e.g., those focused on payments for ecosystem services – were considered to require more in-depth work than was logistically feasible in the time available. After considering several geographic and thematic options, the following two clusters of Activities were identified: Thematic: Activities focusing on forests’ roles in mitigating poverty in landscape preservation and rehabilitation (12 activities, presented under a separate report). Geographic: Activities centered in Central America (9 activities, Table 1). While the evaluation team consulted with the PROFOR Secretariat in selecting the clusters and Activities to be reviewed, the assessments of the individual Activities were carried out independently. The remainder of this report concentrates on the 9 Activities in the second cluster. 2 See Annex 4 of Impact Note 1: Wells, Michael P., Alderman, Claudia and Altman, Stephanie. 2011. An Evaluation of the Impacts of Selected Activities Supported by the Program on Forests. Evaluation Series: Impact Note 1. Washington DC: Program on Forests (PROFOR). 5 Impact Note 2 METHODOLOGY A set of evaluation questions was developed to guide interviews and document reviews (Annex 1). The evaluation team also explored how each of the selected Activities had been identified and implemented as a basis for assessing outcomes and impacts, based on (a) an extensive desk review of relevant documentation, (b) interviews, in person or by phone, with a selection of key informants and stakeholders, including the task teams that implemented the work, relevant PROFOR Secretariat staff, and national and international clients or intended beneficiaries of each Activity, and (c) brief field visits to Guatemala and Honduras in January 2011. The people interviewed are listed in Annex 2. The funding eligibility criteria outlined in the current Operational Guidelines (dated May 2010, see Annex 3) provide a clear statement of the areas where PROFOR expects to have impacts, by supporting collaborations that aim to: - Provide analysis - Mainstream sustainable forest management - Test innovative instruments, approaches, and processes leading to better governance - Develop knowledge products and dissemination - Build and strengthen networks, partnerships, processes and stakeholder dialogue These are the areas that our interviews and document reviews focused on. We also developed a simple scoring system to represent in relative terms the extent to which individual Activities had achieved impacts. The results, shown in Table 2 below, should not be taken too literally, and should not distract attention from the productive learning process offered by the study of a sample of Activities as presented in this report. CONSTRAINTS TO EVALUATION APPROACH AND METHODOLOGY Assessing the impacts of policy studies can be particularly challenging. Policies, and certainly developing country forest policies, rarely change or become implemented more effectively as a direct result of a single, relatively small initiative supported by an international agency. Impacts become even harder to pin down where PROFOR has sought to contribute to an emerging body of knowledge or help move along a debate. PROFOR‟s thematic and geographic interests are shared by a variety of organizations with overlapping and broadly compatible objectives (including CIFOR, FAO, ICRAF, IIED and WRI at the international level; other sections of the World Bank; bilateral and other multilateral agencies; and a host of national government agencies and NGOs). This complicates the attribution issue, in other words how to assess the importance of PROFOR‟s Evaluation Series 6 contribution in areas where other actors – including PROFOR‟s own partners – have been working towards comparable goals. In practice this required the evaluation to go beyond answering „what have we done?‟ for each PROFOR Activity and try to assess how the field has moved, what other key players are working on, and whether PROFOR identified and filled important and productive niches at particular points in time (contributions to knowledge, for example, are often time sensitive, with relatively short periods of time distinguishing genuine new insights from less valuable repetition and reinforcement of conventional wisdom). Establishing cause and effect is often elusive. Identifying whether PROFOR and its partners actually caused something to happen, whether it would have happened anyway, or something in between – the fundamental evaluation questions – can rarely be confirmed definitively. While causation can only rarely be proved, however, a plausible assessment can usually be based on: - Timing: Did the change happen after the Activity? - Logic: Is it reasonable to expect that these inputs would have contributed to the change? - Expert Judgment: Do knowledgeable people – including those involved – agree with the contribution claimed? - Alternative Explanations: What other factors could explain the change? Overall assessments of PROFOR‟s impact require aggregation of the results of multiple diverse Activities, each of which use different indicators to measure progress. The most meaningful of these indicators will often be qualitative. Even within clusters of comparable Activities, such aggregation will be challenging. Simply put, there is no way of doing this without drawing on the judgment of experienced practitioners able to look across thematic and geographic boundaries to assess the value of work done in a broad context, which is what has been attempted here. The evaluation has focused on Activities that have closed. This offers the important advantage of being able to assess longer-term effects based on actual experience, an advantage denied to more recent Activities. Offsetting disadvantages include difficulty in tracking down the key people who participated in the earlier Activities and the challenge of reconstructing the situation in which these were undertaken. Activity (completion) reports and web site write-ups, while usually accurate, are often relatively bland and uninformative reports, often failing to distinguish between Activities that were genuine successes from worthy efforts that did not lead to much or even outright failures. This tendency of being unable to document anything other than success – due, of course, to diplomatic sensibilities and the need for political discretion – is certainly not limited to PROFOR, or even the World Bank, but it does frustrate efforts to distinguish the more from the less effective initiatives and, critically, to learn from experience. Unfortunately, despite many agencies‟ rhetorical commitment to learn, there are few institutional incentives to allow initiatives to be described as less than a success. Fortunately, our experience during this evaluation was that the PROFOR team as well as their World Bank counterparts and the very diverse group of stakeholders who participated in or observed the Activities 7 Impact Note 2 assessed here were generally prepared to be refreshingly open about the results and impacts of their work, which we attribute to their genuine personal commitments to learn and move forward. One somewhat surprising constraint to our work was the lack of systematic organization of the key documents for PROFOR Activities especially, although not entirely, in the case of earlier Activities that had been closed. We found that individual PROFOR staff members often had key documents (proposals, progress reports, terms of reference, completion reports, etc.) on their own computers that were not consistently copied or retained in central files in a way that could easily be accessed, and in some cases were not recorded or logged as existing. Certain key documents for this evaluation, including Activity completion reports, could not be located, although equivalent information was eventually located in most cases. This not only slowed down the initial stages of the evaluation, as considerable time was spent looking for documents within disorganized computer records, but risks a loss of institutional memory were key PROFOR Secretariat staff members to leave. . Evaluation Series 8 TABLE 1. CLUSTER 2 ACTIVITIES SELECTED FOR THIS EVALUATION WITH IMPLEMENTATION PERIOD AND PROFOR CONTRIBUTION PROFOR Selected PROFOR Activities 2003 2004 2005 2006 2007 2008 2009 2010 Support (with codes assigned for this evaluation) ($000) Cross-sectoral and Macroeconomic Measures (C) C01 Knowledge Sharing for REDD Activities in Latin America and the Caribbean 150 Enhanced Financing Options (F) F03 Strengthening the Value Chain for Indigenous and Community Forestry Operations 150 Governance (G) G01 Supporting Forest Stakeholders' Participation in Consultative Councils 86 G03 National Timber Yield Tables for Mahogany 65 G05 Strategy to Combat Illegal Forest Activities in Guatemala 143 G16 Informal Institutions and Forest Resource Governance 119 G23 Honduras: Forest Sector in Transition 75 G27 Linking Land Tenure Regularization and Forest Management in Honduras 153 Sustainable Livelihoods (L) L13 Forest Resource Access and Local Livelihoods 119 9 Impact Note 2 4. OVERVIEW OF FINDINGS This section highlights key findings and broad trends from our review of 9 PROFOR Activities for cluster 2. These reviews are documented individually in the next section. To streamline the text, the Activities are referred to in this section by the numerical code assigned to them for this evaluation (see Table 1 above). THE CENTRAL AMERICAN CONTEXT According to FAO data, although the Mesoamerican region is continuing to lose forests, significant changes have taken place in the past 20 years. Compared to the 726,000 ha. forests lost annually in the 1990s, annual losses declined to 395,000 ha. between 2005 and 2010. However, the progress made has not been uniform. Successes in Mexico, Panama and Costa Rica have offset continued high rates of deforestation in Guatemala, Honduras and Nicaragua. On the positive side, there is some evidence of forest regeneration in Mesoamerica, particularly in indigenous territories3. The forest sector in Central America has been characterized by poor governance and a history of corruption, with problems reaching extreme levels in some countries. Violence and conflicts related to forest control and access have escalated in Honduras amidst growing community concerns about the environmental damage caused by timber extraction, especially the threats posed to water supplies. The forestry authorities have recently been reorganized in both Guatemala and Honduras in attempts to address entrenched corruption and inefficiency. Several of the Activities reviewed derive from FLEG‟s early support for forest sector governance through the Central American Commission of Environment and Development (CCAD), which included preparation of national forest governance case studies. In comparison to PROFOR, FLEG‟s activities have tended to include more hands-on, technical-assistance to national forestry authorities. In Central America in particular, the FLEG/PROFOR role was amplified by the small size of the countries and their limited resources for the forest sector. Major efforts to combat illegal forestry in Guatemala, although supported by a strong professional forestry authority and a committed government, were heavily dependent on the sustained involvement of FLEG and, more recently, PROFOR. 3 Designing a REDD+ Program that Benefits Forestry Communities in Central America, Fundacion Prisma, Grupo Cabal, 2010. Evaluation Series 10 ACTIVITY RESULTS AND IMPACTS In general, compared to the wide range of other international natural resource sector programs we have evaluated on different scales, we have been very impressed with the quality and effectiveness of the PROFOR Activities, and the results and impacts that have been achieved with relatively modest financial resources. High degrees of leverage have been achieved in influencing policies, deepening knowledge and understanding, developing new tools and methods, and strengthening networks. Table 2 provides a “snapshot� view of the impacts of the Activities examined, using the 5 types of impacts outlined in PROFOR‟s Operational Guidelines. The rating is based on the best judgment of the evaluators, after having examined all the documentation available for each Activity and interviewing a range of individuals associated with the given Activity. Taken together, the 9 Activities in Cluster 2 encompass the five areas where PROFOR aims to have impacts. Specifically, - Provide analysis: Via support to analytic work, PROFOR helped improve understanding of complex forest-related subjects. Specifically, Activity G16 examined how institutional and governance relations in forestry constrain or promote improved livelihoods for the poor, by looking at key variables affecting access to forest resources and their benefits, and „mapping tenure rights‟ and how rights are used. Activity L13 examined the way informal rules shape the use of forest resources by various groups such as communities and small holders. Finally, Activity F03 documented that increasing the capacity of community-based forest concessions to adopt better business practices and to engage in sustainable forest management resulted in significant economic gains. The results of these studies should be of crucial interest to several actors, particularly with the advent of new policies and projects related to REDD and other climate change related efforts. - Mainstream various aspects of sustainable forest management and governance: Building-on work initially financed under the FLEG Trust Fund, in 2008 PROFOR supported several in-depth analyses of the institutional, socioeconomic and environmental drivers of illegal logging in Guatemala (Activity G05). These analyses were used in the preparation of Guatemala‟s 2010 comprehensive Action Plan to Prevent and Reduce Illegal Logging. The Action Plan includes legislation and regulatory changes to reduce illegal activities, a review of forest fees, a design for improved forest control and supervision, the development of forest audit protocols, and a plan for capacity building and dissemination. A formal Inter-Institutional Agreement to Prevent and Reduce the Illegal Logging in Guatemala signed by the highest officials of eleven government institutions was a landmark achievement resulting directly from this Activity. In the case of Honduras, when the forest sector was in transition in 2003, PROFOR began to assist sector leaders to engage effectively in the reform process (Activity G23). In this context, PROFOR sponsored a policy workshop in Tegucigalpa to disseminate international experiences of forest sector reforms to policy- makers, legislators, academics, NGOs, private companies and rural forest producers. Unfortunately, this promising work was subsequently undermined by the unstable political situation in the country. Later attempts to continue supporting forestry reform and good governance in Honduras under Activity G27 did not gain traction mainly due to the change of government and their priorities. 11 Impact Note 2 - Test innovative instruments and approaches, and promote processes leading to better governance outcomes in forestry: PROFOR‟s timely support of the Consultative Councils (Activity G01), mandated under Honduras 2007 Forestry Law, offered a unique opportunity to both help foster a nascent democratic process and to test the impact of civil society participation on transparency and accountability in the forestry sector. - Develop knowledge products and dissemination: PROFOR's support for the development of country- specific mahogany yield tables (Activity G03) will help law enforcement agencies and private processing units in Guatemala and, and potentially Peru, to stop the trade of illegally harvested mahogany, much of which originates from indigenous territories and protected areas. This seminal work, endorsed by CITES can be replicated in other producing countries in Latin America (Nicaragua, Colombia, Ecuador, Bolivia) and Africa (Cameroon, Ghana). Brazil, with support from ITTO, has already adopted the original methodology with adjustments to Brazilian circumstances. - Build and strengthen networks, partnerships, processes, and stakeholder dialogue: Under Activity C01 PROFOR supported the sharing of experiences among countries in LAC through two workshops as a contribution to “REDD Readiness�. The event catalyzed a broad exchange on technical matters among countries in LAC and provided an opportunity to share Mexico‟s extensive community forestry experience. A second workshop in Cartagena, Colombia allowed countries to exchange experiences and lessons on formulating sound Readiness Plans, measuring, reporting, and verification, incorporating environmental and social issues into the readiness process, consultation and participation processes, and linking current mechanisms for payment for environmental services programs to REDD. The workshop strengthened regional cooperation and identified technical leaders on specific issues. Dialogue and access to information has continued after the workshops through websites associated with this activity. The true value of many of PROFOR‟s interventions can only become fully apparent several years after Activities have been completed – a key finding of this evaluation. Only then can PROFOR‟s contribution be fully assessed relative to the all-important context in which the Activities were undertaken. By context, we mean the overall set of opportunities and challenges that existed at the point in time when the Activity was initiated. We have tried to bring out the nature of the context of each Activity in the individual assessments in the next section of this report. These kinds of impacts can only be achieved by program staff – in this case the PROFOR Secretariat and their forestry collaborators – with a deep knowledge and understanding of the forestry sector in a wide range of countries as well as a solid appreciation of shifting priorities and opportunities. People interviewed during field visits to Guatemala and Honduras consistently provided highly positive feedback on the expertise and productive engagement of PROFOR staff. Evaluation Series 12 TABLE 2. TYPOLOGY OF ASSESSED ACTUAL IMPACTS OF SELECTED PROFOR ACTIVITIES Mainstream Test innovative Build/ strengthen Provide analysis instruments/ Develop PROFOR Activities sustainable networks, knowledge forest approaches/ partnerships, products management processes processes C01 Knowledge Sharing for REDD Activities in Latin America and the Caribbean *** F03 Strengthening the Value Chain for Indigenous and Community Forestry * * Operations G01 Supporting Forest Stakeholders' Participation in Consultative Councils *** *** G03 National Timber Yield Tables for Mahogany *** *** *** *** G05 Strategy to Combat Illegal Forest Activities in Guatemala *** *** G16 Informal Institutions and Forest Resource Governance ** G23 Honduras: Forest Sector in Transition * G27 Linking Land Tenure Regularization and Forest Management in Honduras L13 Forest Resource Access and Local Livelihoods ** Key: *** Relatively significant impacts; ** Moderate impacts; * Relatively minor impacts 13 Impact Note 2 ACTIVITY SELECTION Eight of the 9 Activities examined fit PROFOR‟s current selection criteria. Activity G27 is the only one that could potentially have been supported by the World Bank as part of its operational responsibilities. ACTIVITY TIMING The timing of several Activities appears to have been important, helping to leverage considerable impacts from modest levels of financial support. The clearest examples are C01, G01, G03 and G05. Another timing consideration is PROFOR‟s flexibility and ability to respond rapidly to opportunities that require immediate action with a minimum of bureaucracy – this was clearly the case for Activity G01. This characteristic is not shared by any other comparable mechanisms for natural resource sectors within the World Bank and very few we are aware of outside. It is also important to recognize that changes in political situations can undermine even the best planned Activities – for example the political upheaval in Honduras in 2009-2010, well beyond PROFOR‟s or the World Bank‟s sphere of influence, adversely affected the anticipated impacts of Activity G23 and G27. ACTIVITY COORDINATION AND PROFILE PROFOR has demonstrated the capacity to play a strong coordinating role within its own Activities by mobilizing and bringing together donor partners, national and local governments, NGOs and research institutions. In particular, PROFOR has enabled the World Bank to team up with forest sector leaders from the NGO and research communities. This effectiveness in catalyzing collaboration seems due both to the World Bank‟s convening power and to the high degree of respect which the PROFOR team members command in international forestry. PROFOR does appear to influence research agendas and donor partners more than government priorities and policies. The collaborative networks linked to PROFOR Activities offer significant benefits, not only facilitating the sharing of knowledge but by increasing the likelihood that new and innovative approaches will be implemented and widely disseminated. These benefits are evident in all of the Activities examined, even though the extent to which PROFOR staff, other Bank staff or contractors play the most active coordination role varies. There are signs that PROFOR has steadily increased its own convening capacity in ways that benefit the credibility and profile of both the World Bank and other PROFOR partners. This Evaluation Series 14 appears mainly to have been achieved through (a) a more systematic and effective approach to the publication and distribution of study reports and other outputs, and (b) a more prominent and active role at key international events, notably including the recent UNFCCC COP16 in Cancun. IMPACT AND PERFORMANCE ASSESSMENT While intended impacts and outcomes are required to be documented before PROFOR Activities begin, and then actual results noted after completion, these tend to be expressed more as outputs (e.g., to produce a report, hold a workshop, develop a new tool or method) or to be expressed in general terms (e.g., inform policy development or advance understanding). While perhaps counter to the increasing emphasis on results-based management, this approach does seem appropriate in most cases given the types of interventions that PROFOR supports, and we would not advocate any attempt to make PROFOR approach to impact and outcome definitions substantially more “rigorous�. The more systematic use of indicators may be productive but great care should be taken not to create incentives to start counting variables that provide little strategic insight. Most PROFOR Activity assessments will continue to require individual ex-post consideration that takes work by other partners into account. OTHER OBSERVATIONS PROFOR appears to occupy a unique funding niche, characterized by its close relationship with World Bank operational departments combined with flexibility, nimbleness and relative autonomy. In some ways PROFOR operates with many of the advantages of a small foundation and, like a foundation, is very heavily dependent on the capabilities of its staff and leadership. It would be hard to imagine how most, if not all, of these Activities considered here could have been undertaken had PROFOR‟s support not been made available. We detected a clear consensus among stakeholders both within and beyond the World Bank that PROFOR‟s management, operations and procedures have evolved and improved significantly, especially during the last two years4. Increased rigor, objectivity and efficiency of the application process as well as the high quality of technical support by PROFOR staff and other World Bank Forestry team members are all widely recognized. The monitoring of the 9 selected PROFOR Activities seems inconsistent. In some cases PROFOR staff are actively involved throughout or a World Bank task manager takes 4 Major recent procedural changes include: (i) the introduction (in 2007) and more structured use (from 2008) of Activity concept notes; (ii) the discontinuation of open calls for proposals and sole-source contracts; and (iii) the Secretariat‟s authority to approve proposals up to $150,000. 15 Impact Note 2 responsibility, both of which seem to operate reasonably well. But PROFOR staff have very limited current information on the status or even the substantive content of a surprisingly large number of Activities that were implemented by organizations outside World Bank. We recognize that this observation may not apply to open Activities currently under implementation as these were excluded from our sample. Evaluation Series 16 WHAT NEXT? In the LAC Region, PROFOR appears well positioned to continue supporting and informing the rapidly-expanding debate on how to address opportunities and challenges both in forestry and at the intersection of forestry, agriculture and climate change, especially in relation to food security, livelihoods and landscape-scale management. This is a debate that PROFOR has already helped shape both within the World Bank and beyond. In order to address the root causes of deforestation and degradation it will be vital to strategically integrate the aspirations and needs of local communities and indigenous peoples. It will also be a time to delve deeper into the inherent complexities of governance, participation, consultation, social justice, as well as rights of ownership, use and access. Yet, the REDD/REDD+ preparation and strategy development process that tends to dominate current forest policy debates appears to lack perspectives grounded in the realities of tropical forest management and how this connects with country-specific aspects of poverty, corruption, tenure and access, and the whole spectrum of relationships between local communities, commercial firms and the state. This deeper level of understanding of the role of forests in society seem essential as REDD+ expands to link with even more sectors: agriculture, energy, etc. PROFOR seems particularly well positioned to play a key role in clarifying the key issues and opportunities here, both through the personal effort of its team members as well as its financial support for related Activities. PROFOR‟s institutional location within World Bank and strong working relationships with both forestry, agriculture and climate change teams should facilitate such efforts. In the Amazon region, PROFOR may consider supporting Activities aimed at developing models and tools to mitigate impacts on forest ecosystems of ongoing large-scale infrastructure projects, in particular those associated with the Initiative for the Integration of the Regional Infrastructure of South America (IIRSA). This initiative, conceived during the 2000 Meeting of South American presidents, aims to connect all the South American countries by integrating transportation, energy, and telecommunications. While IIRSA addresses the real need to stimulate economic growth and reduce poverty among its member nations, it also has the potential of causing irreversible environmental damage to key Amazon ecosystems. In this context, PROFOR may want to explore potential interventions where it can leverage its “convening power� and knowledge base to bring together multiple high-level actors including other donors, multilateral organizations and authorities around common themes. Despite the clear quality of various analytic Activities undertaken under PROFOR auspices and which generated highly relevant and useful knowledge, PROFOR‟s dissemination of its high quality products could be improved. Knowledge dissemination was in most cases ad hoc, and treated as a separate component after project implementation, instead of being embedded as part of the individual project proposals or incorporated in the program as part of a systematic global effort. PROFOR outputs need to be better targeted at specific decision makers, and, to maximize its effectiveness, dissemination work must be ongoing, particularly after changes of key staff/governments in the various countries. For example, while there was some purposeful 17 Impact Note 2 dissemination effort associated with Activities G16 and L13 at the time the studies were completed, current government forestry officials in Guatemala had no knowledge of this work. While far from conclusive, this does indicate that, at least in Guatemala, the results are not widely known two years after the work was concluded. More surprising, however, was that these publications were not translated into Spanish, which substantially limited their dissemination to key policy makers and stakeholders in the LAC region. Looking inward, PROFOR should develop specific approaches to monitoring and reporting the impacts of its Activities on a continuing basis. If impacts are to be monitored effectively both during and after a PROFOR Activity, performance reporting should be introduced that encourages and captures a frank analysis of successes and failures as well as lessons. This could be attempted by modifying the guidance for completion and progress reports or by introducing additional reporting, perhaps even a newly-designed performance dashboard for each Activity5. Such information might be periodically prepared and updated by a designated PROFOR Secretariat staff member and then discussed and commented on by the remainder of the team, drawing on other expertise as needed. Batches of such performance dashboards could be reviewed by the PROFOR Board. 5 The purpose of a monitoring dashboard is to present topical, relevant summary information on performance in a highly-accessible and easily-updateable form, in contrast to lengthy written reports. Multimedia tools, including graphics, are often employed. Other inputs could include key performance-related quotes from Activity proponents, intended beneficiaries or other stakeholders. Evaluation Series 18 5. EVALUATION REPORTS ON INDIVIDUAL ACTIVITIES C01: KNOWLEDGE SHARING FOR REDD ACTIVITIES IN LATIN AMERICA AND THE CARIBBEAN6 Contractors Latin America and the Caribbean Region – Agriculture and Rural Development Unit (LCSAR) Beneficiaries Development practitioners and civil society organizations working in REDD Issues in Latin America and the Caribbean and Nepal Duration February 2010 – February 2011 Total Cost $150,000 Scale Regional WB Task Team Robert Ragland Davis Leader Outputs Forest Communities and Climate Change: Contributions from Mexico to the REDD Initiative (26-27April 2010): Workshop presentations and videos 7. Proceedings from the Cartagena meeting (May 2010)8. PROFOR supported the sharing of experiences among countries in LAC through two workshops as a contribution to “REDD Readiness�, with the work undertaken by LCSAR (a third workshop described on PROFOR‟s web site was not, we understand, funded by PROFOR). 6 http://www.profor.info/profor/knowledge/knowledge-sharing-redd-activities-latin-america-and-caribbean 7 http://www.forestandclimate.info/Mexico/Inicio.html 8 http://web.me.com/mchoyos/Cartagena2010/Inicio.html 19 Impact Note 2 The premise of the first workshop was that efforts to mitigate climate change through REDD+ schemes in Latin America will need to build on local community efforts. The keynote speaker, Nobel Laureate Elinor Ostrom, argued that the single greatest factor determining the fate of forests is the degree of involvement of local communities, largely because people with forest management and harvesting rights are more likely to monitor their own resources and alert regulatory officials when rules are broken. The workshop also included presentations on promising experiences with community forestry from Mexico, where an estimated 80% of forests are in community hands and “Community Forest Enterprises� harvest timber. This 2010 workshop attracted significant attention both within the World Bank and beyond. In a massive event, more than 90 people participated via teleconference from eight countries in Latin America. This included government officials – including REDD preparation teams, locally-based Bank staff, NGOs, journalists, researchers and students. Numerous additional attendees in Washington included Bank and UNDP staff as well as NGO representatives. While showcasing the importance of the substantive issues under discussion, this workshop also demonstrated PROFOR‟s considerable convening power and significantly enhanced PROFOR‟s visibility. Also under this Activity, the Bank‟s LAC Region organized a second, smaller workshop in Cartagena in May 2010, for Latin American countries participating in the Forest Carbon Partnership Facility (FCPF), to exchange technical knowledge on REDD Readiness. Country representatives identified priorities, exchanged experiences and discussed challenges in the preparation of their national REDD strategies. As is often the case with any workshop, detecting specific impacts is hard, although the first workshop certainly generated a lot of enthusiasm and added to PROFOR‟s credibility on REDD issues. The second workshop helped develop professional networks and among individuals working on REDD, with the agenda, choice of topics and actual presentations largely made by government officials from the region. Although well received in the region and appreciated by the FCPF team, it is not completely clear why the Bank‟s LAC Region sought PROFOR support for this second workshop among so many other REDD events that were taking place both in Latin America and elsewhere. Evaluation Series 20 F03: STRENGTHENING THE VALUE CHAIN FOR INDIGENOUS AND COMMUNITY FORESTRY OPERATIONS THROUGH INCREASED INVESTMENT AND USE OF TECHNICAL ASSISTANCE9 Contractors Rainforest Alliance Beneficiaries Forest communities needing to increase their operation‟s efficiency Target Audience International Donor Community potentially interested in providing technical assistance and financing to local community forestry enterprises. Duration May 2008 – July 2010 Total Cost $150,000 Scale Guatemala, Honduras and Mexico WB Task Team Leader Diji Chandrasekharan Outputs Case Study Reports in both Spanish and English (2010). The Impacts of Technical Assistance on a Community Forest Enterprise: The Case of San Bernardino de Milpillas Chico, Mexico; Lumber Cooperatives in the Platano River Biosphere in Honduras; Community Forest Enterprises in the Mayan Biosphere Reserve of Guatemala. In late 2007 the Rainforest Alliance responded to a Call for Proposals issued by PROFOR under the “sustainable livelihoods� umbrella. The topics of the Call for Proposals were (a) Improving market access for forest dependent households� and (b) “Democratizing� forest markets. The Rainforest Alliance, contracted as the result of this Call for Proposals, prepared three case studies based on their Training, Extension, Enterprises and Sourcing (TREES) program. TREES had been working with community and indigenous forestry operations in Mexico, Guatemala, and Honduras since 2002 to increase employment opportunities for women, augment community incomes, and stimulate investments in community forestry operations. The objective of this Activity was to build the case for (a) more investment in community/indigenous small- and medium-sized forestry enterprises and (b) related technical 9 http://www.profor.info/profor/knowledge/strengthening-value-chain-indigenous-and-community-forestry-operations 21 Impact Note 2 assistance for value-added processing in the certified forest product supply chain. PROFOR was interested in supporting an effort to progress beyond the anecdotal information available on these topics towards a more systematic quantification of the impacts of technical assistance and certification on community enterprises‟ incomes and investment returns. Donors interested in supporting local community-level sustainable forestry enterprises were the intended main audience. The case studies supported by this Activity concluded that even small levels of technical assistance resulted in substantial improvements in incomes and productivity in all three countries. The Rainforest Alliance‟s main findings were as follows: Mexico: the $1.1 million, three-year operation increased sawmilling efficiencies and lowered production costs by 43% without jobs losses, helping turn a $560,000 annual deficit into a $1.7 million profit. Guatemala: FSC-certified community concessions doubled their annual revenue to $5.8 million, due to sawmill efficiencies, higher grades of mahogany and higher price increases certified wood. Donor investments in training and technical assistance exceeded $10 million. Honduras: cooperatives collaborated to produce semi-processed mahogany for export to certified markets, changing their production methods and adopting sustainable forest management practices. With only a 19% increase in volume harvested, annual revenues increased by 128% to $580,000. The Rainforest Alliance presented these case studies in 2009 at the World Forestry Congress and at the Forestry Expo in Mexico. The Alliance reported that the methodology used will be replicated in a CONAFOR/UNDP/GEF project in Mexico to measure community level impacts on wood harvesting, usage and income, and will be adapted for community forestry projects in Ghana and Cameroon. PROFOR staff provided extensive comments on the earlier drafts of the studies, having identified serious flaws with the methodology, results and conclusions, arguing that the case studies lacked reliable data and credible analysis, and seemed mainly to promote Rainforest Alliance‟s work. Reviewers found that, while many organizations and donors had been working in the area, all positive outcomes tended to be attributed to the Alliance. Although Rainforest Alliance responded to many of the comments and completely rewrote one of the case studies, the revised products were still not fully accepted by PROFOR which did not allow its logo to appear on the published case studies. It is worth noting that the case studies are found on PROFOR‟s web site, which gives the case studies the appearance of having been endorsed by PROFOR. Subsequent PROFOR Secretariat field visits (for Activity G03) confirmed that the communities involved had indeed benefited from the technical assistance/capacity building they had received, at least in Guatemala. Evaluation Series 22 While this Activity had the potential of being an interesting and useful piece of work, and in the end was consistent with its terms of reference, some of its credibility was undermined by the inherent conflict of interest. Having the same organization which provided the technical assistance to the communities, and even the certifications of sustainability evaluate the merit of these processes appears ill-advised in retrospect. While there was some value in documenting the Rainforest Alliance‟s successes and these could arguably serve to motivate others who are considering investments in technical assistance for community forestry, PROFOR‟s own goals would probably have been better served by commissioning an independent review from a neutral party. 23 Impact Note 2 G01: SUPPORTING FOREST STAKEHOLDERS’ PARTICIPATION IN FOREST CONSULTATIVE COUNCILS10 Contractors Human Rights Commission of Honduras (CONADEH) Beneficiaries Forest Stakeholders via participation in Forest Consultative Councils. Duration September 2009 - May 2010 Total Cost $86,000 Scale Honduras WB Task Team Leader Gerhard Dieterle/ Edgardo Maravi Conflict and violence over forest control and access have escalated in Honduras amidst growing community concerns about the environmental damage caused by timber extraction, especially threats posed to water supplies. The highly vocal Olancho Environmentalist Movement (MAO), led by a Catholic priest, has routinely blocked highways and bridges to logging trucks and has taken over city halls in protests. Loggers have responded with escalating violence and at least nine MAO organizers have been assassinated since 2001. Following a 10-year struggle, pressure from social movements led to a new forest law in 2007 allowing greater community participation in forest management and use, regularizing land tenure, and attempting to control pervasive institutional corruption and mafia-scale illegal logging operations. The forestry law specifically mandated the establishment of Forest Consultative Councils at community, municipal and national levels to promote civil participation and transparency in the management of the country‟s forest resources. PROFOR supported the start-up costs of organizing the Consultative Councils in partnership with the Honduran Human Rights Commission (CONADEH). This Activity responded to a request from the Bank‟s LAC Region and is clearly consistent with PROFOR‟s goal of promoting better governance in the forestry sector. Work under this Activity took place in the same geographical area covered by the Bank‟s $30 million Forests and Rural Productivity Project (PBPR) between 2004-2010. When asked why this Activity was not financed under that project, the response to the evaluation team was that by the time this task was identified, the loan proceeds of the (near completed) PBPR Project had been fully allocated for other activities. 10 http://www.profor.info/profor/knowledge/supporting-forest-stakeholders-participation-forest-consultative-councils Evaluation Series 24 Under this Activity CONADEH organized 88 Forestry Communal Councils and 24 Municipal Councils. About 500 people from 41 Councils received basic training on human rights, on “social audits,� and on forest management oversight. CONADEH‟s capacity had been developed through earlier work with Global Witness piloting independent monitoring of commercial forestry concessions in Honduras11. According to CONADEH‟s Completion Report for this Activity, the newly organized Consultative Councils were beginning to show the benefits of this approach. Specifically, CONADEH documented the outcomes of five complaints of illegal logging brought forth by Consultative Councils. In one case CONADEH‟s inspection confirmed that logging was taking place at the water source for the community, and that the approved Annual Operating Plan (POA) had not identified this as an area where logging should not be permitted. The logging company agreed to correct the mistake, resulting in a peaceful resolution to the conflict. In the remaining four cases CONADEH facilitated meetings between the various parties to discuss the issues under contention and has referred the cases to the relevant authorities. Despite the good intentions enshrined in the forestry law to foster civil society participation in forestry oversight, however, Honduras is among the poorest countries in the hemisphere and the sustainability of these governance models will depend on external funding at least in the foreseeable future. While CONADEH exceeded its goals of establishing a large number of Consultative Councils, it lacked a strategy to develop a realistic plan of action. As a result, once PROFOR‟s funding ran out, CONADEH had to suspend the operations supported by this Activity, leading to unmet expectations among some communities. To be fair, the political situation in Honduras in 2009 and 2010 caused unforeseen delays in funding by donors (i.e. ODA and EU) and many projects were forced to slow down. Nevertheless, the two communities visited as part of this evaluation had each established a Council. Neither community had subsequently received the promised capacity building nor any guidance from CONADEH on how to address their urgent concerns that logging was adversely affecting the local water supply. More positively, the importance of this Activity has been recognized by the EU, which is in the process of approving funding for continuing the work of CONADEH under their FOCUENCAS project. Additional funding is also expected under a “Modernization of the Forest Sector� Project currently under preparation by the EU. Despite the limitations noted here, PROFOR‟s timely support offered a unique opportunity to both help foster a nascent democratic process and to test the impact of civil society participation on transparency and accountability in the forestry sector. This activity responded to an urgent need in Honduras to deescalate a tense and increasingly violent situation by finding a mechanism to give the poorest rural inhabitants access to the country‟s forest resources. The choice of partner for this activity was also noteworthy – governmental human rights institutions 11 According to Tierramerica (February 2010), of the 86 reports made in the last three years by the independent forest monitoring unit, 67 involved irregularities such as abuse of power, adulterated technical reports, pressuring local groups in order to obtain logging permits, logging in protected areas, tax fraud and complicity of authorities. http://www.tierramerica.info/nota.php?lang=eng&idnews=3318 25 Impact Note 2 have not been the traditional interlocutor for forestry activities, yet CONADEH represents an interesting ally in a country where forestry authorities have been plagued by inefficiency, corruption, conflicts of interest, abuse of authority and lack of enforcement of laws and regulations. For PROFOR to fully capitalize in this investment it will be important to follow-up and document the outcomes and impacts of the Consultative Councils in a few years as a means to further the knowledge base that PROFOR seeks to enrich. Evaluation Series 26 G03: TECHNICAL ASSISTANCE FOR THE DEVELOPMENT OF THE NATIONAL TIMBER YIELD TABLES FOR MAHOGANY STANDING VOLUME & EXPORT GRADE SAWN WOOD12 Contractors Centro Agronomico Tropical de Investigacion y Enseñanza (CATIE), Intercooperation. Beneficiaries Guatemala: National Institute of Forests (INAB); Protected Areas Council (CONAP). Peru: Office of Supervision of the Forest Resource and Wildlife (OSINFOR) Duration August 2009 - August 2010 Total Cost $65,000 Scale Guatemala and Peru WB Task Team Leader Gerhard Dieterle/ Edgardo Maravi Outputs Guatemala‟s National Tables for Volumetric Conversion of Mahogany Yields. Mahogany (Swietenia macrophylla) is the single most valuable tropical timber species, commanding more than double the price of other tropical wood. Over-harvesting and illegal logging has contributed to commercial extinction in many areas, including Bolivia and possibly Peru in the LAC region, and mahogany is listed in CITES Appendix II. The species is further threatened by a persistent practice of some producer countries that deliberately adhere to the implausible assumption that 100% of the standing volume is transformed to export grade timber, while other countries apply more realistic conversion rates of 50-60%. Using inaccurate conversion factors allows illegal traders to overstate the volumes produced from legally harvested trees, and fraudulently obtain the CITES licenses required for mahogany export. This loophole facilitates the pervasive large-scale “laundering� of illegally harvested mahogany, much of which originates from indigenous territories and protected areas. A key response for exporting countries is to both develop and adopt accurate yield tables based on realistic conversion factors. To address this need, in 2007 the Forest Law Enforcement and Governance (FLEG) Trust Fund (later to be merged with PROFOR) had financed the development of a “Methodology for Developing National Volume Conversion Tables - Standing Volume & Export Grade Sawnwood� 13. This methodology was later endorsed by CITES, with the 12 http://www.profor.info/profor/knowledge/national-timber-yield-tables-mahogany 13 Note that FLEG/PROFOR Secretariat staff member Edgardo Maravi was a co-author of the Methodology. 27 Impact Note 2 caveat that each country retains the right to develop their own yield tables, either by using the FLEG methodology or designing their own. In this context, the current Activity responded to requests from Guatemala and Peru for technical assistance from PROFOR/FLEG to develop national yield tables to meet their commitments under CITES. In Guatemala this Activity was carried out with technical assistance from CATIE and Intercooperation, a Swiss-based NGO. The work involved an impressive collaboration between the National Institute of Forests (INAB), the Protected Areas Council (CONAP) and a large number of people from local communities who did extensive and painstaking field work to measure and track the volume of wood produced by more than 200 trees (the value of the labor donated by the communities was estimated at over $30,000). INAB and CONAP contributed US$10,000 and US$3,500 respectively. When this evaluation enquired about the motivation behind the voluntary participation of communities in this exercise, the response was that as wood producers they understood the usefulness of the effort and were interested in improving the regulation of mahogany wood production as they face unfair competition from illegal harvesters. The importance of developing accurate volumetric yield tables was reinforced by Guatemala‟s findings, which showed that an average of only 47% of the original volume of the tree trunk yields export-quality wood. An unexpected finding was that as much as 49% of the branches could be turned into wood suitable for export, a finding that encourages greater use of the entire tree, as branches were often wasted or underutilized. The study also provided data on the quality of expected wood yields, which is useful as a business planning tool. As a next step, Guatemala‟s draft yield tables will be subject to a participatory multi-stakeholder vetting process before being adopted nationally, which INAB and CONAP expect in the near future, making Guatemala the first country to comply with this obligation under CITES. In contrast to these achievements in Guatemala, Peru has made little progress on the development of their yield tables. The activity has been stymied by national and local politics and industry‟s lack of commitment to the process. Deeply-embedded interests in lucrative illegal harvesting have likely delayed the project. The Bank tried to move the Activity forward by arranging a meeting between Bank senior staff and the relevant Minister, but no substantive progress resulted. In March 2010 the CITES Standing Committee (the body that governs CITES between COP meetings) gave Peru a six-month ultimatum to address critical issues following Peru's repeated failure to control illegal logging and trade of mahogany. Failure to meet these requirements could result in the Standing Committee voting on a suspension of mahogany exports from Peru. In addition, the United States has threatened to revoke its free trade agreement with Peru, since this specifically requires the Peruvian government to enforce its international treaty obligations and to increase monitoring and enforcement of illegal logging. Recognizing the importance of this Activity for the sustainable management of mahogany, PROFOR gave Peru a six-month extension move forward with this Activity. Evaluation Series 28 In addition to supporting the development of a key tool for improving the sustainable management of mahogany and financing the yield tables for Guatemala, FLEG/PROFOR staff have provided substantial technical leadership in the field and have participated actively in international discussions on how to curtail the illegal harvest of mahogany. The work under this Activity is replicable if additional exporting countries decide to take their CITES responsibilities seriously. The very different results to date from the two countries are a clear illustration of how political will can determine the success or failure of a given Activity regardless of its inherent merit. In the case of Guatemala, the positive outcomes can be attributed to government‟s support and ownership, good institutional capacity, and the fact that this Activity fits within a broader strategy to control illegal logging (See “G05 Strategy to Combat Illegal Forest Activities in Guatemala�). 29 Impact Note 2 G05: STRATEGY TO COMBAT ILLEGAL FOREST ACTIVITIES IN GUATEMALA14 Contractors Centro Agronomico Tropical de Investigacion y Enseñanza (CATIE) Beneficiaries National Institute of Forests (INAB) Duration April 2008 - June 2010 Total Cost $143,000 Scale Guatemala WB Task Team Leader Gerhard Dieterle/ Edgardo Maravi Outputs A proposal for policy and legislative changes and adjustments, and a proposal for institutional changes to improve the implementation of the Strategy to Combat Illegal Forest Activities; A technical proposal to improve forest control and supervision; A financial and cost effectiveness analysis of forest fees, royalties and other revenues and taxes; An information system containing baseline data related to illegal activities in the forest sector; A training program targeting staff from INAB and other actors involved in the implementation of the Strategy to Combat Illegal Forestry; and A national communications and awareness program of the damage caused by illegal practices in the forest sector targeting the general public The Bank began supporting forest sector governance through the Central American Commission of Environment and Development (CCAD) with funding from the Forest Law Enforcement and Governance (FLEG) Trust Fund (later to be merged with PROFOR). This effort included the preparation of national forest governance case studies, one of which led to a 2004 Strategy for Combating Illegal Forestry in Guatemala in a context where the National Institute of Forests (INAB) had estimated over 95% of forest products (equivalent to 31 million m3 of wood) were being harvested illegally. 14 http://www.profor.info/profor/knowledge/strategy-combat-illegal-forest-activities-guatemala Evaluation Series 30 Little progress was made on implementing this Strategy, and in 2007 INAB received further FLEG support to revisit the Strategy and identify the implementation barriers. The resulting analysis concluded that combating illegal forestry required consistent inter-institutional coordination and buy-in from several government agencies, institutional capacity building, analytical work on law enforcement and illegal forestry, and broader dissemination of the measures taken. In 2008 FLEG (which merged with PROFOR that year) approved the Activity Strategy to Combat Illegal Forest Activities in Guatemala. This Activity supported a more in-depth analysis of the institutional, socioeconomic and environmental drivers of illegal logging, resulting in six high- quality products (listed above under outputs) prepared with technical assistance from CATIE and the Institute for the Environment and Natural Resources of the Rafael Landivar University. On the basis of the analytical work and measures proposed, INAB‟s Board of Directors approved an Action Plan to Prevent and Reduce Illegal Logging in February 2010. The Action Plan includes legislation and regulatory changes to reduce illegal activities, a review of forest fees, a design for improved forest control and supervision, the development of forest audit protocols, and a plan for capacity building and dissemination. A formal Inter-Institutional Agreement to Prevent and Reduce the Illegal Logging in Guatemala signed by the highest officials of eleven government institutions was a landmark achievement resulting directly from this Activity 15. The Coordinating Roundtable established to oversee this agreement is led by the Vice President, giving significant high-level support to this broad effort involving all key ministries. INAB and CATIE officials credit FLEG and later PROFOR support with having launched a process and supported the recent attempts to combat illegal logging in Guatemala. It seems unlikely that INAB would have made significant progress without FLEG/PROFOR‟s steady support over the past six years, a period when neither World Bank projects of other donors have been active in the forest sector. FLEG/PROFOR has also been credited with catalyzing additional funding for the FINNFOR project supporting sustainable forest activities in seven Central American countries. The progress made so far in establishing stronger institutions, high level political support and financing, all achieved largely with FLEG and PROFOR support, is to be continued by INAB under a $560,000 ITTO project Support for the Application of Guatemala’s Forestry Law to Promote Governance. A further follow-up Activity supported by PROFOR, Auditing Timber Supply to the Forest Industry in Guatemala 16, is ongoing and was not included in this evaluation.17 15 The parties to the Agreement included: the office of the Vice Presidency, the Judicial Power, the Ministry of Agriculture, the Ministry of Environment, the Ministry of Government, the Ministry of National Defense, the National Association of Municipalities, the National Council of Protected Areas, The Office of Taxation, the Attorney General, the Disaster Coordination Office and INAB. 16 http://www.profor.info/profor/knowledge/auditing-timber-supply-forest-industry-guatemala 17 The objective of this Activity is to provide technical assistance and capacity building to INAB and other government agencies to design and implement a Forest Audit System to ensure the control and verification of timber supply in the forest industry 31 Impact Note 2 located in selected regional administrations in Guatemala, and to ensure transparency, accountability and participation of forest stakeholders in law enforcement. Evaluation Series 32 G16: FOREST RESOURCE ACCESS AND LOCAL LIVELIHOODS 18 L13: INFORMAL INSTITUTIONS AND FOREST RESOURCE GOVERNANCE 19 Contractors Center for International Forestry Research (CIFOR) Beneficiaries/Target Audience Government institutions, policy makers, local communities, donor community Duration May 2007 - June 2008 Total Cost $237,000 (PROFOR); $472,000 (all sources); Scale Regional – Latin America and the Caribbean WB Task Team Leader Jill Blockhus, Diji Chandrasekharan Outputs The Role of Informal Institutions in the Use of Forest Resources in Latin America (2008). Tenure Rights and Beyond: Community Access to Forest Resources in Latin America (2008). These two studies, approved by PROFOR at roughly the same time, were undertaken simultaneously by CIFOR. Over the course of implementation, CIFOR suggested that these activities be amended and linked given their complementarity. From that point on, the two were treated as a single Activity by both institutions. The “Informal Institutions� study examined issues affecting forest resource use by smallholders and communities based on case studies in Bolivia, Brazil, Honduras and Nicaragua. Findings indicated that while many governments have introduced progressive policies intended to benefit rural populations and their forest use, the extent to which such policies have actually brought about any real change to benefit communities is questionable. The cases suggested that although formal rules are becoming increasingly important for influencing forest resource use in the context of expanding markets, mainly for timber products, their outcomes depend on their interactions with existing informal rules. The study concluded that understanding the informal arenas becomes extremely important for shaping state efforts for the formalization of property rights and regulation of forest resource use. 18 http://www.profor.info/profor/knowledge/forest-resource-access-and-local-livelihoods 19 http://www.profor.info/profor/knowledge/informal-institutions-and-forest-resource-governance-latin-america 33 Impact Note 2 The “Forest Resource Access and Local Livelihoods� study examined key variables affecting access to forest resources and their benefits, with case studies in Bolivia, Brazil, Guatemala, Honduras, and Nicaragua that included conservation settlement communities, indigenous territories and agro-extractive communities. The main findings of this study were as follows: Winning legal rights on paper are only a first step. Virtually all of the communities studied faced practical challenges, which included disputes with other resource users, failures of the state to define or defend rights, conflicts with local authorities, difficulties with old institutions taking on new roles, barriers to reaching markets, and lack of forest management capacity. The legal reforms brought new rights but also new responsibilities and restrictions. State authorities gained new decision-making roles. New institutional arrangements gave power to persons lacking experience or accountability. Governments gave more effort to regulating than to defending community rights; communities spent resources defending their rights that could have gone to building governance and forest management capacity. Policy frameworks failed to support community-based management opportunities. The role of the state, in the granting, implementation and protection of rights, is decisive in shaping outcomes. The rights-based approaches necessarily demand duties and accountability – particularly the duty of the state to protect human rights and to be accountable for the implementation of policy. In none of the cases studied had the state played a particularly effective role in fulfilling its duty to defend community rights or the perimeter of the territory. Policy frameworks had generally failed to establish an enabling environment for the development of integral and innovative management of community forests. Community forest enterprises have been promoted based on blueprints for organizations and resource extraction that require heavy external support and fail to build on the self-governance capabilities of smallholders and communities. Forest enterprises are established from outside with little understanding of deeper cultural issues such as ancestral rights to cultural reproduction. In the search for market solutions and the development of viable enterprises, it is important to put the social and cultural considerations of local governance institutions at center stage. Despite all this, some communities did gain better livelihoods. Understanding the successes and failures in these cases may help future reformers anticipate problems, prepare for them, and improve the chances of success. The conceptual framework and methods for the LAC case studies funded under this Activity were developed within a larger CIFOR project that included work in Africa and Asia with funding Evaluation Series 34 from other sources. PROFOR supported CIFOR extending this work to Latin America and bringing in more experienced researchers to strengthen the overall research team 20. Local workshops were conducted at all of the sites to present findings to the communities and partner organizations involved. Larger dissemination and policy meetings were held in Bolivia and Brazil, and four papers based on this research were presented at an international conference in 2008. IDRC and the Ford Foundation supported a related series of policy events in Bolivia, Guatemala and Nicaragua, while results were presented to Bank, FAO and USAID staff in Nicaragua and Guatemala. Despite CIFOR‟s efforts at disseminating the results of the study, it is unclear whether or how the work served to influence policy or change behavior. Government forestry officials in Guatemala had no knowledge of this work. While far from conclusive, this does indicate that, at least in Guatemala, the results are not widely known two years after the work was concluded. Surprisingly, these publications were not translated into Spanish, which substantially limited their dissemination to key policy makers and stakeholders in the LAC region. From a research perspective the two reports produced by CIFOR are of high quality and the topics are highly relevant. The value of the work rests on the methodologies developed and the experiences obtained in the case studies for „mapping tenure rights‟ and examining how rights are used. This information should be of crucial interest to several actors, particularly with the advent of new policies and projects related to REDD and other Climate Change related efforts. The impact of this work will depend on the ability of PROFOR and/or the other participants to transmit the research findings into decision-making arenas, which does not appear to have happened so far. 20CIFOR is a member of the Rights and Resources coalition, which initiated a two-year project in 2007, funded by IDRC and Ford Foundation, to examine rights and resources with a “modified action research approach� in 10 countries (Bolivia, Brazil, Burkina Faso, Cameroon, Ghana, Guatemala, India, Nepal, Nicaragua and Philippines). Seven of these are also countries where ACM originally worked. http://www.cifor.cgiar.org/acm/beyond/rights-resources.htm 35 Impact Note 2 G23: HONDURAS: FOREST SECTOR IN TRANSITION G27: LINKING LAND TENURE REGULARIZATION AND FOREST MANAGEMENT IN HONDURAS21 Contractors LAC‟s Agriculture and Rural Development Unit (LCSAR); Forest Trends (2004 policy workshop only) ; The Overseas Development Institute (ODI). Beneficiaries Key Honduran Forest Sector Leaders (Policy-makers, legislators, academics, NGOs, private companies, and forest producers and their associations); The Government of Honduras (GoH) in connection to the implementation of two Bank-supported projects: the Forests and Rural Productivity Project (PBPR) and the Second Land Administration Project. Duration May 2004 – June 2007 Total Cost $228,600: $75,000 (2004); $77,000 (2006); $76,6000 (2007) Scale Honduras WB Task Team Leader Jim Smiley; Laura Ivers; Diji Chandrasekharan Outputs “Forests: A Resource for Development� workshop presentations and companion case studies: http://www.forest-trends.org/event.php?id=163 (English and Spanish). In 2003, the forest sector in Honduras was in transition: proposed changes included decentralization and devolution of decision-making powers; recognition of forest land rights of forest populations and indigenous peoples; and the creation of incentives for local governments to take lead roles in the conservation and management of public forest lands. The Activity “Honduras: Forest Sector in Transition,� started in 2003 with $75,000 from PROFOR, seeking to assist Honduran forest sector leaders to engage effectively in the reform process. PROFOR sponsored a policy workshop in Tegucigalpa in 2004 to disseminate international experiences of forest sector reforms to policy-makers, legislators, academics, 21 Neither of these two Activities are listed on PROFOR‟s website. Evaluation Series 36 NGOs, private companies and rural forest producers. Forest Trends coordinated the selection, preparation, and presentation of case studies and lessons, and assisted the Ministry of Agriculture and Livestock and COHDEFOR (the National Forest Agency) in convening the workshop. Discussions focused on: (a) global market trends and implications, including new markets for ecosystem services; (b) community forestry enterprise experiences; (c) lessons from decentralization; and (d) designing more optimal policy and regulatory frameworks. A participant in the 2004 policy workshop reported that the presentations made by the staff of Forest Trends were of high quality and were well received 22. But there appears to have been no follow-up to the workshop, and potential impacts were either not realized or not documented. While Forest Trends‟ responsibility was limited to the policy workshop, PROFOR continued to support this Activity as it evolved over the next three years. However, the changes in approach over time and the outcomes and impacts were not well documented and have proven hard to reconstruct. According to PROFOR‟s 2004 Annual Report “Government plans to use these [workshop] materials for ongoing capacity building and training..…while follow-up work on community tenure and the role of communities and municipal governments in forest conservation and management is under preparation�. The 2005 Annual Report made only minimal reference to the Activity status, with no funds reported as disbursed. Despite this apparent lack of progress, the 2006 Annual Report lists a second phase with a budget of $77,000. The anticipated outputs were: (a) cadastral and land-cover overlays, forest land tenure simulations and land-use analysis; and (b) stakeholder analysis, identification of alternative forest management regimes and simulation of possible outcomes in the selected sites. It is unclear whether the outputs expected from Phase II were produced or even who was responsible for their preparation. There appears to have been a loss of institutional memory with the retirement of the original TTL who could not be interviewed. It has not proven possible so far to identify impacts of the “Honduras: Forest Sector in Transition� Activity. Continuing the reconstruction, the 2006 PROFOR Annual Report does not report any results from this Activity. Instead, it appears that the Activity was renamed “Linking Land Tenure Regularization and Forest Management in Honduras: A Practical Approach�, as a reorganized version of the Activity discussed above. This is described as follows in PROFOR‟s 2006 Annual Report (pages 7 and 39-40): “Building on outcomes from phase 1…… the second phase of the project has developed criteria for determining appropriate land titling regimes based on the new legal framework and GIS information.……..This approach takes into account local livelihood systems with safeguards for the forest dependent poor with weak derived rights. Next steps include: promotion or strengthening of appropriate local organizations (e.g., cooperatives, private owners associations, 22 The workshop materials are on PROFOR‟s website under “Forests: a Resource for Development�. http://www.profor.info/profor/knowledge/forests-resource-development 37 Impact Note 2 producer groups, etc.), regulatory reform to adapt legal regulations to small-scale management and reduce transaction costs, training and skill building activities, and promotion of links with industries and markets (vertical integration of small- and large-scale forest management)�. This “new� Activity (which we have coded G27) retains some of the same outputs listed in Phase II of the earlier Activity, but with somewhat different objectives. However, neither an explanation for the change in approach nor any documentation of outcomes from the earlier Activity could be located. PROFOR‟s 2007 Annual Report lists Activity G27 as completed but with the report “not available at the time the Annual Report was finalized�. The 2008 Annual Report does not include any mention of this Activity or its outcomes. Interviews with LCSAR‟s TTL for this Activity, with a government official formerly in charge of the Bank‟s Forest and Rural Productivity (PBPR) Project and with a consultant who worked in Honduras at the time of the Activity helped clarify the background and reconstruct the complicated history of the two Activities. As best as could be pieced together, this project suffered from lack of buy-in and continuity resulting from changes of administration in the Honduran Government, changes of staff within the Bank, and the departure from Honduras of consultants working on the Activity. PROFOR‟s initial workshop and Phase II tasks proposed in 2005 were developed in partnership with the National Forest Agency (CODEHFOR). Due to rampant corruption and inefficiency, CODEHFOR was placed under interdiction and management by a special commission in 2006, and in early 2008 was replaced by the Instituto de Conservacion Forestal. Implementation of the Activity was further stalled as the new administration was reluctant to take-on any of the projects associated with the discredited CODEHFOR. Given the new political situation, the Bank began discussions with the new administration on how PROFOR‟s funds could be best used. After multiple delays, the Government of Honduras (GoH) asked that the funds be directed to preparing a policy brief on the types of conflicts that can arise in the process of land tenure regularization. The Activity was delegated to ODI, a credible partner, and the Bank took a secondary role; but unfortunately key ODI staff left the country leading to further delays. The Final Report for this Activity, dated February 4th, 2008 (not included in PROFOR‟s Annual Reports) states that the results of the Activity were highly valuable to the Government and the World Bank. It also states that under the Bank-financed PBPR Project, the Government was initiating a formal pilot project of land regularization to which many of the Activity‟s recommendations would be applied. But according to an official working on the PBPR project at the time, the report prepared using PROFOR funding was not incorporated into their work due to delays in its preparation. Apparently supporting this view, the Final Report of the PBPR Project does not make any reference to the work produced with PROFOR‟s funding. Evaluation Series 38 From the perspective of this evaluation, the Final Report for this Activity gives an overly- optimistic view of a highly complex situation, and does not describe the problems encountered in implementing this Activity as originally proposed. The findings and recommendations listed in the Final Report are overly generic principles and hypotheses and, at least as presented, do not seem to meet the expected outputs. At the time of this evaluation, the final report for the earlier Activity could not be located. The two Activities, with perhaps the exception of the initial Policy Workshop, seem to be “funding in search of a project�, that is, supply and not demand-driven. There is also a question of whether the 2006 Activity should have been financed by PROFOR at all, or whether it should have been funded under one or both of the closely associated Bank-financed projects. Finally, the two Activities did not appear to have been a high priority to the LAC Region, and as such, appear to have received insufficient supervision on the part of the Bank. Based on the documentation available and interviews conducted as part of this evaluation, we were unable to identify any lasting impacts of these two Activities. 39 Impact Note 2 ANNEX 1 – EVALUATION QUESTIONS RELEVANCE Where did idea for the activity originate? Were the target audience and/or beneficiaries of the activity clearly defined, and how well did the outcomes /outputs maintain these in focus? Was the high priority/potential of the activity clearly demonstrated and documented? How innovative and/or timely was this activity? How relevant was the activity to PROFOR‟s objectives and criteria? How well did the activity „fit‟ with or complement related activities of other actors? SELECTION How rigorous was the proposal development process? Was there peer review? Were comments received incorporated in the final proposal/Concept Note? IMPLEMENTATION PERFORMANCE Which components worked best and which worked the least well? Were any problems encountered during implementation? How effectively were these addressed? What were the actual and budgeted expenditures? Were any significant changes made in the approach during implementation? Compare activities planned with those actually completed What role did PROFOR management and staff play? Could PROFOR or World Bank have helped/acted differently to increase impacts? ACHIEVEMENTS AND IMPACT Were the outputs proposed consistent with those achieved? Evaluation Series 40 Assess the quality of the outputs Were the outputs peer reviewed? To what extent were the outputs taken up as inputs into policy processes or catalytic in stimulating other related processes? Where they any unintended impacts (positive or negative)? Have the impacts been strengthened or weakened by subsequent developments in the forest policy field? Can synergies be identified with other PROFOR studies? FOLLOW-UP AND SUSTAINABILITY What has been the longer term impacts and relevance of the activities supported? Are there indications that activities been followed up and that outcomes have been mainstreamed? 41 Impact Note 2 ANNEX 2. PEOPLE INTERVIEWED Alejandro Santos, Rainforest Alliance Guatemala André Fache, European Union, Tegucigalpa Andy White, Forest Trends Association Augusta Molnar, Forest Trends Association Carlos Escobel, Municipal Consultative Council Carlos Estrada, , National Institute of Forests-INAB Guatemala Deanna Newsom, Rainforest Alliance Deborah Barry, CIFOR Diji Chandrasekharan , ARD TTL Dinosco Garcia, Community Consultative Council Edgardo Maravi, ARD TTL Fausto Mejía Zelaya, CONADEH Filippo del Gatto, Consultant Gerardo Segura, , TTL LCSAR Gregory Frey, LCSAR Hugo Cabrera, Rainforest Alliance Guatemala Hugo Flores, National Protected Areas Council-CONAP Ivan Salazar, National Protected Areas Council-CONAP Jeannette Ramirez, LCSAR Jorge Lainez, Coordinaator, Forests and Rural Productivity Project (PBPR) Jorge Luis Galindo, National Protected Areas Council-CONAP Jorge Sosa, Institute for Forest Conservation (ICB) Josue Morales, Manager, National Institute of Forests-INAB Juan Carlos Raudales, Municipal Environment Coordinator Julio Lopez, CATIE Guatemala Juventino Galvez, University Rafael Landivar Laura Ivers, Previous ARD TTL Luis Alejandro Argueta, , National Institute of Forests-INAB Guatemala Maria F. Miguel, National Institute of Forests-INAB Noe Lopez, Prosecutor, Public Ministry Guatemala Evaluation Series 42 Odilio Flores, Institute for Forest Conservation (ICB) Pedro Pineda, University Rafael Landivar Ramon Custodio, President, CONADEH Robert Ragland Davis, TTL/LCSAR Roberto Caceres, Association of Environmental NGOs ASOREMA Roberto Kometter, Intercooperacion-Peru Wyllson A. Martinez, National Institute of Forests-INAB Guatemala 43 Impact Note 2 ANNEX 3. ACTIVITIES ELIGIBLE FOR PROFOR SUPPORT PROFOR will support global, regional, national, and subnational level collaboration among Governments, the World Bank, donors, the private sector, regional and non-governmental organizations, and civil society to: - provide analysis with a focus on (a) the role of forest resources in poverty alleviation, sustainable economic growth, addressing climate change adaptation and mitigation, and in protecting and valuing environmental services, (b) forest law enforcement and governance (FLEG), including issues related to tenure, community rights, benefit sharing, trade in timber and wood products, etc., (c) sustainable forest management, including biodiversity conservation; and (d) reducing emissions from deforestation and forest degradation (REDD+); - mainstream various aspects of sustainable forest management (SFM) and forest governance within international agreements, national development strategies, policy dialogue, and other relevant policy and technical instruments; - test innovative instruments and approaches, and promote processes leading to better governance outcomes in forestry; - develop knowledge products and dissemination to a targeted audience; and to - build and strengthen networks, partnerships, processes, and stakeholder dialogue. Source: PROFOR Operational Guidelines (May 2010) Evaluation Series 44