Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) Report Number : ICRR0020136 1. Project Data Project ID Project Name P066051 VN - Forest Sector Development Project Country Practice Area(Lead) Additional Financing Vietnam Environment & Natural Resources P124040,P126542 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-39530,IDA-50700,TF- 31-Mar-2011 74,550,000.00 50865,TF-54122,TF-54523,TF- 54524 Bank Approval Date Closing Date (Actual) 08-Jul-2004 31-Mar-2015 IBRD/IDA (USD) Grants (USD) Original Commitment 39,500,000.00 11,686,539.20 Revised Commitment 65,216,463.08 9,690,546.61 Actual 66,403,029.80 10,481,378.70 Prepared by Reviewed by ICR Review Coordinator Group Houqi Hong J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan PHPROJECTDATATBL Project ID Project Name VN - GEF Forest Sector Development P074414 Proj ( P074414 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) Page 1 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) TF-53397 31-Mar-2011 15,950,000.00 Bank Approval Date Closing Date (Actual) 08-Jul-2004 30-Mar-2013 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 9,000,000.00 Revised Commitment 0.00 8,002,682.89 Actual 0.00 8,002,682.89 2. Project Objectives and Components a. Objectives The project development objective (PDO) for the Forest Sector Development Project (FSDP) as stated in the 2004 project appraisal document (PAD) was “to achieve sustainable management of plantation forests and the conservation of biodiversity in special use forests." The PDO stated in the 2005 Development Credit Agreement (DCA) for the project was “to assist the Borrower to enhance the contribution of forestry to: (a) rural poverty reduction and (b) global environmental protection, through the sustainable management of plantation forests and the conservation of biodiversity in special use forests.” The 2012 Financing Agreement (FA) which provided additional financing for the project amended the PDO to read “The objective of the Project is to achieve sustainable management of plantation forests and the conservation of biodiversity in special use forests.” The project was defined as comprising the original project plus the added activities funded by the additional financing. Amending the PDO had the effect of making it identical to the PDO in the PAD. The Global Environment Objective (GEO) of the GEF component of the project as stated in the PAD was “to improve conservation of biodiversity of international importance in up to 50 Special Use Forests”. The GEO stated in the GEF Grant Agreement (GA) was the same as the original PDO in the DCA: “to assist the Recipient to enhance the contribution of forestry to: (a) rural poverty reduction and (b) global environmental protection, through the sustainable management of plantation forests and the conservation of biodiversity in special use forests”. The Guidelines state that irrespective of the GEO, the project’s overall achievements will be assessed on the basis of the articulation of the PDO in the project’s legal agreement and hence any changes in the PDO in subsequent legal agreements. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Page 2 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) Yes Date of Board Approval 22-Mar-2012 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The project had the following four components, based on Section C and Annex 2 in the PAD. Component 1: Institutional Development (appraisal cost US$1.20 million; actual cost US$4.14 million) Assist the Government of Vietnam to improve the country’s enabling environment for sustainable forest management and biodiversity conservation, including: (a) Revising selected policies and regulations based on field implementation experiences, such as guidelines on forest land allocation, regulations on management of special use forests (SUFs) enabling management boards to enter into co-management agreements with local stakeholders, and incentives and tax systems for promoting plantation forestry; (b) Establishing farm forestry groups to stimulate the development of small holder forestry; and (c) Promoting plantation forest certification in selected areas to ensure environmental sustainability, premium prices of products, and better market access of participating households. One reason for the dramatic increase in the actual cost of this component was the significant expansion of institutional development activities supported by the additional financing. Component 2: Smallholder Plantation Forest (appraisal cost US$52.56 million; actual cost US$52.50 million) Support in the four provinces of Quang Nam, Quang Ngai, Binh Dinh, and Thua Thien Hue to establish forest plantations of different cropping systems, including mixed agroforestry crops, fast-growing plantations, and non-timber trees; promote tree growing by smallholders in rural communities, many of whom were poor; and improve productivity of existing poorly performing plantations. Activities included: (a) participatory site selection based on village consultation and technical and environmental assessment of proposed sites, (b) land allocation and issuances of land use right certificates as eligibility criteria for investment credits, (c) extension and other services delivery to assist smallholders, (d) design and management of plantations, and (e) credits to eligible households and some eligible state forest enterprises (SFEs) to support plantation investments. By contrast, the ICR didn't mention promotion of tree growing by smallholders and improvement of productivity of existing poorly performing plantations as part of the project components. Component 3: Special Use Forest (appraisal cost US$15.97 million; actual cost US$14.14 million) Improve biodiversity conservation and sustainable use of biological resources in priority special use forests (SUFs) and increase SUF funding reliability through: (a) establishment and operations of Vietnam Conservation Fund (VCF) for SUFs, including fund management structure and procedures, a competitive small grants program, and monitoring, reporting, lessons dissemination; and (b) strengthening of SUF planning and implementation, focusing on site-specific activities, including conservation needs assessment, development of operational management plans, strengthening of management boards capacity to reach co-management agreements with local stakeholders, strengthening of field implementation capacity, and operationalization of a site-specific M&E system. Up to 50 priority SUFs Page 3 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) would benefit from the grant packages. The VCF would be open-ended and could be replenished by other donors at project completion. Component 4: Project Management and Monitoring and Evaluation (appraisal cost US$4.86 million; actual cost US$19.42 million) This component would strengthen institutional capacity necessary to plan, coordinate and manage the project implementation, including coordination of government agencies at the central;, provincial, and district levels, project specific monitoring to track project’s technical and financial progress and performance at various administrative levels, and collaboration with other partners in the national Forest Sector Support Program (FSSP), a sector-wide program with 22 signatories comprising Government, donors including the Bank, and non-governmental organizations (NGOs). One reason for the dramatic increase in the actual cost was that significant new activities on institutional strengthening and capacity building were added during the additional financing. Implications of Additional Financing for Components Although the additional financing did not change the components of the project, it expanded the scope of Components 1, 2 and 4, namely: • In Component 1 further support to strengthen the enabling environment for smallholder plantation forestry through additional studies, additional capacity building and institutional development, and piloting of independent certification of forest management; • In Component 2 additional support in the following areas: forest land allocation to smallholders as incentives to promote productive investments; technical assistance in the areas of plantation planning and design, nursery development, and extension; financing of additional smallholders for plantation investments. The program was both expanded in the original four provinces and scaled up to two additional provinces; and • In Component 4 further institutional strengthening and capacity building on planning, coordination, and management of project implementation, covering the additional project areas in the original and new provinces (Project Paper for the Additional Financing). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost The project’s total cost was estimated at US$74.59 million at appraisal. Actual total project cost increased to US$90.20 million at completion, mainly due to the additional financing which expanded the scope of the project. Financing The project was financed by US$39.50 million from the original IDA credit, US$30.00 million from the additional financing IDA credit, US$9.00 million from the GEF grant, US$12.70 million in grants from the Vietnam Trust Fund for Forests (TFF) under the Forest Sector Support Partnership (FSSP)--a multi-donor trust fund for forests in Vietnam (of which US$6.90 million came from a Netherlands trust fund and Page 4 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) US$5.80 million from a Finland trust fund), US$2.56 million from a European Commission grant, and US$4.32 million of counterpart funding from the Government. All grants were channeled through the Bank. The original IDA credit, the additional financing IDA credit, and the GEF grant were 99%, 91%, and 89% disbursed, and the grants from Finland, Netherlands, European Commission were (respectively) 88%, 78%, and 62% disbursed. The Government counterpart funding was paid fully. Key dates The project was approved on July 8, 2004. The original closing date was Mar. 31, 2011. It was extended three times by a total of four years, to March 31, 2015, making the actual implementation period ten years and eight months, compared to the originally planned six years and eight months. The reasons for the extensions were to compensate for project implementation delays and the need for time to process and implement the additional financing (ICR, Section 1.9). Effectiveness date was August 4, 2005, thirteen months after approval. The Mid-term Review was January 23, 2007, about four months ahead of the schedule set at appraisal. In January 2009 a second mid-term review was conducted to assess the project's responses to recommendations made in the first mid-term review and to re-adjust the focus of the project. Restructurings There was one level I restructuring and two level II restructurings. • Level I restructuring (March 22, 2012): An additional financing of SDR19.00 million (about US$30.00 million) in IDA credit was approved, to provide support in both the original four provinces and two additional provinces for the following activities areas: forest land allocation to smallholders as incentives to promote productive investments; technical assistance in the areas of plantation planning and design, nursery development, and extension; financing of additional smallholders for plantation investments, and further strengthening of the enabling environment, institutions, and project management capacity. The Additional Financing also amended the PDO as mentioned in Section 2a above. Its closing date was established as March 31, 2015. • Level II restructurings (March 28, 2011): The Project Design Summary in the PAD (described by the ICR as a Log-Frame) was replaced by a results framework to improve the basis for results measurement; the consultants' services category was added in Schedule 1 of the GEF Grant Agreement; reallocation of project proceeds was approved to meet changed funding needs for project categories; procurement provisions in the DCA were revised for the selection of individual consultants; and the GEF grant closing date and the IDA Credit closing date were extended to March 30, 2013 and March 31, 2012, respectively. • Level II restructuring (March 28, 2012): This restructuring extended the IDA Credit closing date from March 31, 2012 to February 27, 2013. Because of the change to the PDO as part of the Additional Financing (AF) Agreement, this Review will assess the extent to which this project achieved its original and revised objectives and estimating an average of them weighted by disbursements before and after the AF. Page 5 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) 3. Relevance of Objectives & Design a. Relevance of Objectives As noted already, the original PDO in the Development Credit Agreement (DCA) was amended in the context of the Additional Financing (AF) Agreement. At project approval, the original project objectives were highly aligned with Bank Country Assistance Strategy (CAS) for Vietnam. Enhancing environmental sustainability and reducing poverty in rural areas were among the key priorities of Theme 2 of the CAS for FY2002-2006. The project objectives were also fully aligned with the government strategy. Two of the Vietnam Government’s Development Goals in the 2002 Comprehensive Poverty Reduction and Growth Strategy (CPRGS, equivalent to the Poverty Reduction Strategy Paper for Vietnam) were to reduce the percentage of poor and hungry households and ensure environmental sustainability, with two of the specific objectives for FY2003-2005 being to develop agriculture and the rural economy to widely reduce poverty and to strengthen the sustainability of natural resource use in the rural areas (PAD, Annex D1). After the additional financing the project's objectives were still aligned with the Bank's Country Partnership Strategy (CPS) for Vietnam but less ambitious than the original objectives. In brief the objectives of "rural poverty reduction" and "global environmental protection" were revised to "achieve sustainable management of plantation forests" and "conservation of biodiversity in special use forests". At the project's close in 2015 Pillar 2 of the CPS for FY2012-2016 had a strong emphasis on support for sustainable management of plantation forests and conservation of biodiversity, while Pillar 3 had a strong focus on poverty reduction, particularly in rural areas (CPS for FY2012-2016, Annex 1: Results Matrix). The revised project objectives remained consistent with Government priorities and goals such as addressing environmental and natural resource degradation, promoting environmental sustainability, and enhancing social equity, as laid out in the Vietnam's Socio-Economic Development Strategy (SEDS) 2011-2020 and Socio-Economic Development Plan (SEDP) 2011-2015. Summary: The original PDO was highly relevant to both Government and Bank development strategies and aimed at important objectives such as poverty reduction and global environment protection. While the revised objectives after the Additional Financing were also relevant to Government and Bank strategies, they were focused only on sustainable management of plantation forests and conservation of biodiversity in special use forests and were hence far less ambitious and therefore this Review assessed the relevance of the revised objective substantially relevant to Government and Bank strategies because one of the core strategic country development objectives for both the Bank and the Government (rural poverty reduction) was dropped. Rating Revised Rating High Substantial b. Relevance of Design The project’s activities were substantially relevant to both the original and amended project objectives. There were a number of reasons for this. First, the project supported the establishment of forest plantations with different production systems in Vietnam through integrated activities ranging from the issuance of land use rights certificates, design and management of plantations, provision of low-interest investment credits, delivery Page 6 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) of technical assistance and technology extension services, to certification of plantation forests. These activities, complemented by participatory site selection and the technical and management requirements for smallholders to access the low-interest credits, would contribute to improved financial, social and environmental sustainability of smallholder plantation forests. The resulting increase in wood supply would reduce pressure to over-exploit scarce natural forests, contributing further to environmental biodiversity. Overall, these activities were relevant to the achievement of sustainable management of plantation forests and global environmental protection. Although the project did not target poor households specifically, it was designed to contribute to rural poverty reduction because households below the poverty line were expected to benefit either directly or indirectly, from land allocation and related forestry development, in a project area where 73 percent of project communes had over 50 percent of households in poverty (ICR, para. 24). Second, the project provided integrated support for (a) the establishment and operations of the Vietnam Conservation Fund (VCF) for Special Use Forests (SUFs) as a sustainable funding channel to finance biodiversity conservation planning and management activities in priority SUFs, and (b) strengthening of SUF planning and management, focusing on development and implementation of key procedures and tools such as Social Screening Report (SSR) for identifying livelihood needs of local communities, Conservation Needs Assessment (CNA) to identify threats to biodiversity, OMP (Operational Management Plan) to guide actual conservation activities, and Benefit Sharing Mechanisms (BSM) to promote co-management of SUFs between communities and the Management Boards of SUFs. And the VCF funding was conditioned on the adoption of the key management procedures and tools. These activities were designed to improve management effectiveness of SUFs and reduced threats to areas of international conservation importance and would, according to the PAD, complement large-scale donor and government investments which were mostly focused on physical infrastructure, thus relevant to strengthening biodiversity conservation and global environmental protection. The project's support for development of a national enabling environment for both plantation forests management and biodiversity conservation in SUFs would further strengthen the linkage between the project activities and the achievement of project objectives. The original project design in 2004 was based on the Project Design Summary in the PAD (a log-frame approach according to the ICR) instead of a results framework approach which at appraisal had not yet been introduced as standard practice in the Bank (ICR, page 15, para. 16). The project formally adopted a results framework in 2011, about 7 years after project effectiveness, although the results framework had already been formulated during the first midterm review in 2007 and had since been used to track project implementation. The causal chain in the Project Design Summary/Log-Frame in the PAD was clear and realistic and the link between funding, output and outcome was convincing although the Project Design Summary/Log-Frame misclassified outcome and output in some cases and included some irrelevant results indicators. A moderate shortcoming was that the results framework revised in 2011 didn't specifically measure the real effects of biodiversity conservation, such as changes of the condition of biodiversity values or threat reduction to areas of international conservation importance. Instead it measured the outcome of biodiversity conservation through the WWF\World Bank Management Effectiveness Tracking Tool scores (METT scores). The METT scores are calculated based on a number of parameters with real effects of biodiversity conservation being only one of the parameters. As a result, the link between the outcome, which is measured by the METT scores, and the objective of conservation of biodiversity was not very straight forward in the results framework. Page 7 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) Summary: The relevance of the project’s design before and after the Additional Financing had minor shortcomings and were therefore both rated substantial. Rating Revised Rating Substantial Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective As noted earlier the achievements of this project will be assessed against the project’s objectives defined before and after the Additional Financing (AF) Agreement in March 2012. The original objective (Objective 1) was defined in Development Credit Agreement (DCA) as “to assist the Borrower to enhance the contribution of forestry to: (a) rural poverty reduction and (b) global environmental protection, through two sub-objectives namely, (i) sustainable management of plantation forests and (ii) conservation of biodiversity in special use forests.” However, the AF revised the PDO to "achieve the sustainable management of plantation forests and the conservation of biodiversity in special use forests." Hence the sub-objectives of the original objectives became the project’s revised objectives - namely Objective 1 Revision 1 and Objective 2 Revision 1 after the AF. The Global Environment Objectives (GEO) in the GEF Grant Agreement (GA) was “to assist the Recipient to enhance the contribution of forestry to: (i) rural poverty reduction and (ii) global environmental protection, through the sustainable management of plantation forests and the conservation of biodiversity in special use forests”. The GEO was the same as the original PDO and was not changed during project implementation. Nevertheless, according to the Guidelines, the project’s achievements with respect to the GEO will be based on the achievement of the objectives in the original or the revised credit agreement. Rationale Objective 1: To assist the Borrower to "enhance the contribution of forestry to rural poverty reduction" Outputs: • Policies, procedures and guidelines were developed for forest land measurement and allocation, plantation design, investment procedures, and credit and project implementation and were applied in all 6 provinces, as planned (ICR, Data Sheet, Section F). • 41,545 households received Land-use Rights Certificates (LURCs) issued under the project for 75,559 ha of smallholder plantations (ICR, page 33, Table 1). The PAD targets for the original project were 19,000 households for 53,000 ha of smallholder plantations (PAD, Annex 1, page 41). No target for the additional financing part. • 700 billion VND (US$32 million and US$45 million based on the exchange rates at project completion and appraisal, respectively) in low interest loans issued to smallholder plantation forest investors by Vietnam Bank for Social Policy (VBSP) (no target) (ICR, page 33, Table 1). Page 8 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) • 100% of prescribed yields of short rotation plantations were met or exceeded (target was 100%) (ICR, Data Sheet, Section F). • A 20-30 percent price premium and improved access to global forest product markets were achieved based on pilot certification for over 850 ha of plantations owned by 354 households (no target) (ICR, page 33, para. 90). • 20,152 ethnic minority farmers participated in 140 Ethnic Minority Development Plans (no target) (ICR, page 60). Intermediate Outcomes: • 73.2% of the smallholder plantation area established under the project (56,050 ha out of 76,571 ha of smallholder plantation forests) was of certifiable international standards for sustainable forestry (including financial, social, and environmental sustainability), according to independent audits conducted using Forest Stewardship Council (FSC) guidance which is internationally accepted. This is substantially higher than the target for each province which was 50% (ICR, page 33, Table 1), although there is no information on achievement in each specific province. • 43,743 households involved in / benefited from the plantation forest component (target was 43,743). They established 76,571 ha of plantation forests, of which 56,050 ha were of certifiable standards (ICR, Table 1 and ICR, page 63, Table A2.2). Overall Outcome Rural Poverty Reduction. The ICR shows that poverty reduction was achieved in the project provinces (Annex 2 - Output by Component). However, the ICR does not provide any evidence on whether the decline in poverty was attributable to the project and therefore whether the project enhanced rural poverty reduction. Based on these shortcomings in the achievements of the original objective its outcome is rated modest. Rating Modest PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective As a result of the Additional Financing in March 2012, Objective 1 was changed to: "achieve the sustainable management of plantation forests". Revised Rationale Objective 1 Revision 1: To "achieve the sustainable management of plantation forests" Outputs: The outputs for Objective 1 Revision 1 are the same as the outputs reported in the discussion of achievements of "sustainable management of plantation forests" for Objective 1 above. Outcomes: The outcomes for Objective 1 Revision 1 are the same as the intermediate outcomes achieved with respect to sustainable management of plantation forests described in the section on Objective 1 above. • 73.2% of the smallholder plantation area established under the project (56,050 ha out of 76,571 ha of smallholder plantation forests) was of certifiable international standards for sustainable forestry (including financial, social, and environmental sustainability), according to independent audits conducted using Forest Page 9 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) Stewardship Council (FSC) guidance which is internationally accepted. This is substantially higher than the target for each province which was 50% (ICR, page 33, Table 1), although there is no information on achievement in each specific province. • 43,743 households were involved in / benefited from the plantation forest component (compared with a target was 43,743). They established 76,571 ha of plantation forests, of which 56,050 ha were of certifiable standards (ICR, Table 1 and ICR, page 63, Table A2.2). Summary: There was considerable evidence that plantation forests were being sustainably managed and thus, the efficacy of this objective is rated as substantial. Revised Rating Substantial PHEFFICACYTBL Objective 2 Objective To assist the Borrower to "enhance the contribution of forestry to global environmental protection". Rationale Objective 2: To assist the Borrower to "enhance the contribution of forestry to global environmental protection". Outputs: • Key procedures and tools on Special Use Forest (SUF) planning and management, including Social Screening Report (SSR), Conservation Needs Assessment (CNA), OMP (Operational Management Plan), and Benefit Sharing Mechanisms (BSM), were standardized, as planned (ICR, page 34, para. 92 and page 52). • The Vietnam Conservation Fund (VCF), a competitive small grants program for SUFs, was established as a sustainable funding channel for biodiversity conservation and for planning and management of protected areas and SUFs, as planned (ICR, page 34, para. 92). • The VCF issued 100 grants for a total amount of US$7.7 million to 69 local SUF Management Boards (SUF MBs) of the Ministry of Agriculture and Rural Development (MARD) to implement improved OMPs (Operational Management Plans). About half of the funding from the grants was spent on capacity building for SUF MBs to improve management effectiveness (no targets were available). • 30 SUFs have Operational Management Plans that meet international standards based on the WWF/World Bank Management Effectiveness Tracking Tool (METT) scores and are managed in accordance with Benefit Sharing Mechanisms (BSMs) with local communities (target was 30) (ICR, Data Sheet, Section F). • 40 biodiversity inventories, conservation needs assessments, special screening reports, and Management Effectiveness Tracking Tool (METT) surveys identified threats, priorities and endangered species that were incorporated into Operational Management Plan (OMP) and Benefit Sharing Mechanisms (BSM) (no target). • 396 villages entered into 63 BSMs with SUF MBs that resulted in a three-fold increase in livelihoods support funding 2009-2012 and the issuance of a Government of Vietnam Decree on promoting BSMs (no target). Intermediate Outcomes: Page 10 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) • Management effectiveness of SUFs improved by 19% (in the Central Region) to 39% (in the North Region), as measured by the METT scores produced using the WWF/World Bank Management Effectiveness Tracking Tool (METT). Project areas include four WWF's Globally Important Ecoregions and 67 Important Bird Areas or Endemic Bird Areas identified by Birdlife International (no target) ((ICR, Data Sheet, Section F and page 34, para. 91). As an environmental management rating system, the METT assesses legal status, operational plans, availability of resources, management systems, and the effects of management in relation to conservation, among others. However, there is no information on how the last dimension was rated in the METT. • VCF secured additional funding for 11 grants beyond the project from the multi-donor Trust Fund for Forests of Vietnam (TFF), and integrated into the new Vietnam Fund for Forests (VNFF) which is a mechanism for longer-term financing for environmental services. The government is providing funding to maintain the momentum established under VCF (ICR, pages 42 and 69). • VCF policy, procedures, and tools incorporated into formal government policies and regulations. For example, Decree 117 on SUF organization and management, effective March 1, 2011, incorporated the tools of Operational Management Plan, Social Screen Report, and Conservation Needs Assessment. The Prime Minister Decision 126 of 2012 allowed the piloting of Benefit Sharing Mechanisms in three SUFs as an initial step to introduce participatory co-management across all protected areas in the country (ICR, Page 34, Para. 92). • Stronger SUFs conducted biodiversity patrols and recorded data on presence of key endangered species and existing biodiversity inventories. However, the data were not consolidated across SUFs, hence there is no evidence on whether biodiversity was maintained during the project implementation period (target was that biodiversity in 30 SUFs would be maintained as per the Project Design Summary/Log-Frame in PAD (ICR, page 53). • No evidence in the ICR on whether or not reduction of threats in up to 30 SUFs (a target specified in the Project Design Summary/Log-Frame in the PAD) was achieved. Overall Outcome Global Environmental Protection. There is no evidence in the ICR regarding the achievement of the objective of global environmental protection (i.e. conservation of biodiversity). Although management effectiveness of SUFs was improved, there was no evidence on the effects of improved forest management on conservation of biodiversity. The Project Team provided IEG with additional evidence arguing that the project achieved positive effects of improved forest management on the conservation of biodiversity: “By completion, over US$1 million in VCF financed biodiversity monitoring was conducted for 40 target SUFs to capture and analyze changes in indicator species. The monitoring effort concluded that 30 SUFs showed an increase of these species, five had no changes, four showed both gains and losses of species; while only one showed an overall loss of species. Critically endangered and endangered species inventoried and monitored in the project SUFs, include the Asian Elephant, tiger, leopard, Huede’s pig, Saola, Large-antlered Muntjac, Pyygmy Annamite Muntjac, and Sitka Deer, Gaur Buffalo, Banteng Cattle, Chinese Serow, Eld’s Deer, Hog Deer, and Clouded Leopard. Many SUFs provide habitats to 50 or more vulnerable or near threatened species. The project’s target SUFs had been identified during preparation as harbouring biodiversity of global importance.” However, the Project Team did not provide any information to confirm that the changes in the incidence of endangered species were attributable to the project. Indeed, the ICR does not present the methodology used to estimate the changes in the incidence of endangered species at the project’s close other than the Page 11 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) reference to “Stronger SUFs undertook biodiversity patrols that recorded GPS referenced data on the presence of key endangered species and updated existing biodiversity inventories” (page 53). On the basis that the improvements in SUF management as measured by the METT scores could (despite the many persistent threats to endangered species) have achieved the conservation of biodiversity in special use forests and hence enhanced the contribution of forestry to global environmental protection, the efficacy of this objective is rated Substantial. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Revised Objective As a result of the Additional Financing in March 2012, Objective 2 was changed to: "the conservation of biodiversity in special use forests." Revised Rationale Objective 2 Revision 1: "The conservation of biodiversity in special use forests". Outputs: The outputs for Objective 2 Revision 1 are the same as those described regarding the achievements of conservation of biodiversity in special use forests for Objective 2 above. Intermediate Outcomes: The outcomes for Objective 2 Revision 1 are the same as the intermediate outcomes described in the section on Objective 2 above, namely: • Management effectiveness of SUFs improved by 19% (in the Central Region) to 39% (in the North Region), as measured by the METT scores produced using the WWF/World Bank Management Effectiveness Tracking Tool (METT). Project areas include four WWF's Globally Important Ecoregions and 67 Important Bird Areas or Endemic Bird Areas identified by Birdlife International (no target) ((ICR, Data Sheet, Section F and page 34, para. 91). As an environmental management rating system, the METT assesses legal status, operational plans, availability of resources, management systems, and the effects of management in relation to conservation, among others. However, there is no information on how the last dimension was rated in the METT. • VCF secured additional funding for 11 grants beyond the project from the multi-donor Trust Fund for Forests of Vietnam (TFF), and integrated into the new Vietnam Fund for Forests (VNFF) which is a mechanism for longer-term financing for environmental services. The government is providing funding to maintain the momentum established under VCF (ICR, pages 42 and 69). • VCF policy, procedures, and tools incorporated into formal government policies and regulations. For example, Decree 117 on SUF organization and management, effective March 1, 2011, incorporated the tools of the Operational Management Plan, Social Screening Report, and the Conservation Needs Assessment. The Prime Minister Decision 126 of 2012 allowed the piloting of Benefit Sharing Mechanisms in three SUFs as an initial step to introduce participatory co-management across all protected areas in the country (ICR, page 34, para. 92). Page 12 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) • Stronger SUFs conducted biodiversity patrols and recorded data on presence of key endangered species and existing biodiversity inventories. The data were not consolidated across SUFs, hence there is no evidence on whether biodiversity was maintained during the project implementation period (target was that biodiversity in 30 SUFs was maintained as per the Project Design Summary/Log-Frame in PAD) (ICR, page 53). • No evidence in the ICR on whether or not reduction of threats in up to 30 SUFs, a target specified in the Project Design Summary/Log-Frame in the PAD, was achieved. Overall Outcome Despite the improvement of SUF management, there is little evidence in the ICR of conservation of biodiversity in special use forests. However, as stated in the assessment of the efficacy of the original Objective 2, on the basis of the additional evidence the Project Team provided to show an increase in indicator species in targeted special use forests, the improvements in SUF management as measured by the METT scores could (despite the many persistent threats to endangered species) have achieved the conservation of biodiversity in the special use forests. The efficacy of this objective is therefore rated Substantial. Revised Rating Substantial PHREVISEDTBL 5. Efficiency The weighted average of financial rates of return to the project's investments and of financial net present values of plantations, based on the percentage of land area of each plantation in the total land area for all plantations, were 23.3% and 65 million VDN/ha respectively at project completion; both compared favorably to the estimates at appraisal. The economic rate of return to the total investment in all plantations was 17.4% based on assumptions comparable to those at appraisal, slightly higher than it was at appraisal (17%). However, the ERR would be reduced to 13.2% assuming positive opportunity cost of land, higher labor cost, and value of the land use rights certificate (LURC) to better reflect the situation at completion. It still exceeds the threshold discount rate used in the ICR of 10%. But the ICR did not adequately explain the assumptions and the basis for the estimates especially regarding the value of the land use rights certificate (LURC) nor did it prepare any sensitivity analysis. There is thus uncertainty as to the actual level of efficiency of the plantation forests component which accounts for 58.2% of the total project cost. The Special Use Forests (SUFs) component was financed by GEF and focused primarily on biodiversity conservation management and capacity building. A cost-benefit analysis or cost effectiveness analysis was not conducted for this activity. Benefits included significant improvement in planning and management in SUFs, many of which are globally important protected areas; the establishment of the Vietnam Conservation Fund (VCF) as a sustainable funding mechanism for SUF management activities; and the incorporation of VCF policy, procedures, and tools, such as Operational Management Plan, Social Screen Report, Conservation Needs Assessment, and Benefit Sharing Mechanism, into formal government policies and regulations. The ICR Page 13 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) states that the efficiency of this component improved over time, but provides no evidence to allow for an assessment of improved efficiency. For example, it is difficult to assess whether the benefits justify a cost of US$14.14 million for the component. The ICR provided no evidence of real positive effects of improved planning and management of SUFs on biodiversity. As noted already the Project Team later provided evidence that thirty out of the forty monitored SUFs achieved an increase of indicator species, but without a quantitative assessment of the significance of the increases, nor evidence on whether the increases were attributable to improved planning and management of the SUFs. Similarly, there is no evidence on the efficiency of the institutional development component, the cost of which almost quadrupled, from US$1.2 million at appraisal to $4.14 million at completion. Administrative efficiency was negligible. There were delays in implementation and disbursements. Project implementation was extended three times by a total of 4 years, from about 6 years to 10 years to a large extent due to the additional financing. The project management costs quadrupled, from $4.86 million at appraisal (6.5%) to the actual cost of US$19.42 million (21.5% of total project cost). This large increase in project management cost could not have been due to the additional financing and was not explained in the ICR. In summary, there are strong grounds for questioning the project’s efficiency. The project's efficiency is therefore rated modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 70.50 Appraisal  17.00 Not Applicable 58.20 ICR Estimate  13.20 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the project's original and revised objectives was, respectively, rated high and substantial. The relevance of design was rated substantial for both the original and revised objectives. There was inadequate evidence in the ICR to establish that the project's original first objective namely “to assist the Government to enhance the contribution of forestry to rural poverty reduction” was achieved and thus its efficacy was rated modest. On the basis of the additional information provided by the project team the original Page 14 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) second objective “to enhance the contribution of forestry to global environmental protection” was substantially achieved. The efficacy of the revised first and second objectives were both rated substantial. There were strong grounds on which to question the efficiency with which both the first and second objectives were achieved and hence efficiency has been rated modest. In summary, the disbursements of project funds before and after restructuring in March 2012 were almost exactly the same and hence the project's outcomes before and after restructuring were weighted equally for this "split" evaluation. Based on substantial relevance of both objectives and substantial relevance of design before and after restructuring in March 2012, as well as modest efficiency of the project under both sets of objectives, the overall outcome for the project depended on the weighted average of the project’s efficacy before and after restructuring. It is apparent that the weighted average efficacy is substantial and therefore the project had moderate shortcomings and thus its overall outcome is rated Moderately Satisfactory. . a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating The plantations established under the project had high technical standards. And there are strong incentives for the smallholders to keep these high technical standards as they will bring high returns and enhanced access to international markets. Smallholders will also have the capacity to keep high standards as the project helped improve relevant technical and management capacity. The regional market demand for the timber from the plantations established under the project is strong and is expected to continue to be strong in the foreseeable future. Also, the Government of Vietnam places a strong emphasis on developing plantation forests so as to reduce its current heavy reliance on timber imports. The analysis of efficiency of project plantations indicates that their financial sustainability is high. The weighted average financial rate of return and financial net present value were 23.3% and 65 million VDN/ha respectively at project completion. According to the ICR, upon harvesting most smallholders paid back their loans and replanted without having to re-borrow (page 47, para 146). There are some risks from pests, which can be effectively mitigated through diversification of species in different rotations and through small harvesting coupes, complemented by careful pruning. This mitigation method has been recommended to the Ministry of Agriculture and Rural Development (MARD). Typhoons, which are frequent in Vietnam, are another type of risk but even wind damaged wood can still be sold for salvage. The project also provided support for upgrading of access tracks, construction of fire towers, and introduction of community based fire management approaches to mitigate the risks of forest fires (ICR, page 43, para 129). The project’s revolving fund managed by Vietnam Bank for Social Policy (VBSP) would continue to finance plantations according to the same standards as adopted under this project through year 2036, based on the on-lending agreement between Vietnam's Ministry of Finance and VBSP (ICR, page 26, para 61). On conservation of biodiversity in Special Use Forests (SUFs), the Government is committed to maintaining the management effectiveness of the project SUFs. In fact, the Government is committed to expanding the adoption of key planning and management tools for SUFs and key policies of the sustainable funding Page 15 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) channel Vietnam Conservation Fund (VCF) established under this project, both of which are being incorporated into formal government policies and regulations pertinent to biodiversity conservation and management of SUFs. The VCF secured additional funding from donors for activities beyond the project and the Government is providing funding to keep the momentum of the VCF activities (ICR, pages 42 and 69). There are clearly significant challenges to sustainability. However, on the basis of the evidence in the ICR and the Government's confidence that there will be future improvements in the conservation of biodiversity, the risk to the project's development outcome is rated modest. a. Risk to Development Outcome Rating Modest 8. Assessment of Bank Performance a. Quality-at-Entry The project was prepared by a team which had sound and appropriate experience. The project objectives were substantially relevant to both country and Bank strategies throughout the project’s implementation period. Project activities were well aligned with the achievement of the project objectives. The causal chain in the Project Design Summary/Log-Frame was problematic in that it suffered from shortcomings such as a PDO that was incomplete and not the same as the PDO in the main text of the PAD, indicators that did not measure whether the PDO in the PAD was achieved, and no results chain showing how inputs would be converted to outputs and outcomes. It is also noted that the PDO in the Development Credit Agreement was far too ambitious and needed to be revised. Technical, financial and economic assessments were generally sound but the assessment of project risks was cursory with the risk of not conserving biodiversity not mentioned. The fiduciary and safeguards arrangements as well as implementation arrangements were adequate. However, monitoring and evaluation design and arrangements at appraisal were clearly not adequate because they were subject to a significant overhaul at the Mid-Term Review (ICR, para 62). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Bank supervision proactively identified and addressed issues that affected achievement of the PDO and the Bank's fiduciary role. The supervision missions were conducted about 2-3 times a year and were timely, most of which involved extensive field visits, and focused on key issues such as the new mechanisms and tools the project supported and compliance with safeguards and fiduciary measures. Supervision teams comprised the right mix of expertise and members from multiple relevant fields, and were joined by donor representatives. However, it took almost eight years to achieve a change in the PDO and revisions to the results framework. Also, the revised results framework had a significant shortcoming in that it failed Page 16 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) to provide for the measurement of the impact of improved SUF management on biodiversity conservation. The Project Team provided IEG with additional information in an explanatory note stating that the project monitored 40 SUFs to assess changes of indicator species. However, the methodology for assessing the incidence of endangered species at the project’s close was not clear in either the ICR or the explanatory note. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance The relevant central government agencies collaborated among themselves and with the Bank on project preparation and implementation, demonstrated strong leadership, and provided counterpart funding on time. Government agencies also had a clear division of labor among themselves which contributed to the achievement of the financial and technical deliverables. The Government provided a platform for successful implementation of policy and institutional development; it supported reviews of relevant policies, and approval and implementation of relevant decrees, decisions, and regulations, such as those related to land allocation and land certificates, plantation certification, technical accreditation, and a series of plantation and conservation funding mechanisms and tools. On the other hand, there were shortcomings including the Government's slow approval of procurement at the start of the implementation which caused delays of the implementation for up to 2 years, and the acts of fraud and corruption the Bank's Integrity Vice Presidency identified, which affected the procurement of two consultancy contracts (ICR, para. 79). Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The Project Steering Committee was effective in providing guidance on policy, annual work plans, and high-level coordination with relevant agencies. The separate Management Committee for the Special Use Forests component effectively oversaw the policy, operations, and grand disbursement. The Central Project Coordination Unit (CPCU) within the Ministry of Agriculture and Rural Development was staffed with qualified people and worked effectively with the provincial- and district-level project management units to carry out the implementation of the plantation forests component and related institutional development activities. However, no consolidated evidence was collected on real conservation effects (ICR, Page 53). The ICR made no reference to the performance of the implementing agency in monitoring and evaluation of the project outcome. Page 17 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The project management units at various administrative levels as well as the final beneficiaries, including smallholders and Management Boards of Special Use Forests had strong ownership of using the M&E system to collect data and monitor results as the results were set as precondition for getting financing under the project. However, the causal chain in the Project Design Summary/Log-Frame suffered from shortcomings such as a PDO that was incomplete and not the same as the PDO in the main text of the PAD, indicators that did not measure whether the PDO in the PAD was achieved, and no results chain showing how inputs would be converted to outputs and outcomes. It is also noted that the PDO in the Development Credit Agreement was far too ambitious and needed to be revised. In the results framework introduced during the Additional Financing, the data collection and analysis were based on generally mature methods embedded in the internationally accepted Forest Stewardship Council (FSC) certification for plantation forests and the WWF/World Bank Management Effectiveness Tracking Tool (METT) for Special Use Forests. However, the results framework had a relatively weak outcome indicator for conservation of biodiversity in special use forests. For example, the overall METT scores are not designed to measure conservation of biodiversity but rather to measure management effectiveness. b. M&E Implementation The M&E process for the small plantation forests was anchored in a computer based M&E system established following the 2009 midterm review. The system was effective in integrating data on a variety of outputs and outcomes from a large number of beneficiaries and project communities in six provinces. The system also facilitated the consolidation and sharing of data among project management units at the different administrative levels. Field data collection was conducted by technical service providers through surveying of smallholders, with data consolidated in project management units at the commune, district, province, and the national levels. Final outcome data was obtained through an independent international forest certification process using the Forest Stewardship Council (FSC) guidance, conducted by international and national professional forest certification specialists recruited by the Central Project Coordination Unit. Key baseline data were collected. Data on output and intermediate outcome for the Special Use Forests and Institutional Development components were also collected effectively in general. Bank supervision in 2006 found the weakness of the original Project Design Summary/Log-Frame which led to its conversion into a results framework with a simplified indicator specification. But this conversion was not made official until 2011 and the results framework itself had significant shortcomings. Also, the project’s impact on rural poverty reduction and on biodiversity changes were not measured in the ICR. Again, the Project Team provided IEG with additional information to that available in the ICR stating that the project monitored 40 SUFs to assess Page 18 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) changes of indicator species. However, the additional information provided no evidence on the magnitude of the changes nor the extent to which these changes were attributable to the project. c. M&E Utilization The ICR reports that monitoring of the performance of plantation forests informed decision to redeploy technical support to those areas that required more support (ICR, para. 63). The ICR also reports without elaboration that monitoring of the performance of Special Use Forests (SUFs) helped Management Boards of SUFs to identify key issues and threats to biodiversity and take actions (ICR, para. 64). The Mid-Term Review process helped the project strengthen its focus on plantation forests' compliance with standards, technical specifications, environmental guidelines and management plans, as well as its focus on commune- level planning, site selection and plantation design (ICR, para. 55). M&E Quality Rating Modest 11. Other Issues a. Safeguards The project was classified environmentally as “Category B” and triggered safeguards policies on Environmental Assessment (OP/BP 4.01), Natural Habitat (OP 4.04), Forestry (OP 4.36), Indigenous People (OP 4.20), and Involuntary Resettlement (OP 4.12). The ICR reports that compliance with these social and environmental safeguards was monitored regularly during the implementation process but "no significant issues were observed" (ICR, para. 67). The ICR made no specific statement on whether or not there was compliance. b. Fiduciary Compliance Financial Management: The project experienced start-up delays in disbursement by nearly two years due to delay in government approval of procurement packages. This issue was resolved and disbursement since accelerated. Another issue was that an external audit identified an ineligible expenditure of US$800,000 disbursed to civil servants due to a misunderstanding about the eligibility criteria. But the issue was corrected with the amount refunded to the Bank. Overall the financial management performance was considered moderately satisfactory, according to the ICR (ICR, para. 76). Procurement: In general, procurement of goods and services complied with relevant provisions of the project's legal documents and the Bank’s Procurement Guidelines and was effective. An issue was that the Bank's Integrity Vice Presidency identified acts of fraud and corruption in the procurement of two consultancy contracts. The ICR reports that the findings were shared in accordance with a standard protocol, Page 19 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) without mentioning with whom they were shared (ICR, para. 79). c. Unintended impacts (Positive or Negative) The plantation forests component had some good demonstration effects among some smallholders outside the project areas, who borrowed from outside the project to replicate the project activities by investing in plantation forest development using the same approach, design and standards as adopted in the project. The plantation forests component also demonstrated to communities and local authorities that investment in smallholder plantation forests can catalyze wider rural development by creating new business opportunities. For example, the project stimulated the development of small businesses in bee keeping in places close to acacia plantations and near good roads and tracks, as well as in nursery development, site preparation, planting, thinning, harvesting, trading, and transport and processing of increased volumes of wood. d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment The reduction in the project's Outcome Satisfactory Moderately Satisfactory Outcome rating was due to a modest rating for efficiency. Risk to Development Modest Modest --- Outcome The quality of supervision was rated moderately unsatisfactory and consequently overall Bank Bank Performance Satisfactory Moderately Satisfactory performance was rated Moderately Satisfactory because Outcome was rated in the satisfactory range. Moderately Borrower Performance Moderately Satisfactory --- Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as Page 20 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) appropriate. 13. Lessons The ICR listed 12 lessons learned from the project, based on which this Review highlights the following which have the potential for being useful beyond this project: 1. An integrated, multi-sector, and participatory approach can be effective for promoting sustainable management of plantation forests. The project provided comprehensive support in areas ranging from regulations, technical design and extension services, access to markets and credits, to issuance of Land-use Rights Certificates, complemented by a participatory plantation site selection. All of these were pillars of successful smallholder plantation investments and proved to be essential for the achievement of sustainable management of plantation forests in the project. Such investments under the project transformed barren hills into productive landscapes and substantial livelihood improvement. 2. Co-management of Special Use Forests and participatory approaches are useful tools for enhancing the effectiveness of biodiversity conservation management. The project financed a range of activities on co- management of Special Use Forests (SUFs) with local communities based on a participatory approach, resulting in improved communication between the Management Boards of SUFs and communities, stronger community ownership of and awareness about biodiversity conservation, better understanding about the socioeconomic needs of local communities, more sustainable use of forest resources, and better monitoring of threats. This contributed significantly to the enhancement of management effectiveness of SUFs under the project. 14. Assessment Recommended? Yes Please explain This Review has raised questions about the extent to which the project’s contributions to the conservation of biodiversity stated by the project team were attributable to the project and if they were significant contributions. IEG decided to conduct a Project Performance Audit Report (PPAR) for this project which will, amongst other things, provide an opportunity to examine the available evidence on the project’s impact on the conservation of biodiversity and the project's efficiency. 15. Comments on Quality of ICR Page 21 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN - Forest Sector Development Project (P066051) The ICR was thoughtful and made a lot of effort to assess the outcome of the project. Its analysis of results was outcome-driven. However comprehensive evidence on core project outcomes was limited in the ICR, but subsequently supplemented by the Project Team. The ICR could have been improved if the evidence on some key PDO indicators had been presented in a more structured and verifiable manner. Finally, the lessons learned from the project were too narrowly focused on the project and hence lacked general application. a. Quality of ICR Rating Substantial Page 22 of 22