Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) Report Number : ICRR0020100 1. Project Data Project ID Project Name P100620 DRC- Forest and Nature Conserv. - PFCN Country Practice Area(Lead) Congo, Democratic Republic of Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H4570,TF-92910 30-Jun-2015 149,000,000.00 Bank Approval Date Closing Date (Actual) 02-Apr-2009 30-Jun-2015 IBRD/IDA (USD) Grants (USD) Original Commitment 64,000,000.00 7,000,000.00 Revised Commitment 64,000,000.00 6,966,124.96 Actual 65,187,479.36 6,415,722.96 Sector(s) Public administration- Agriculture, fishing and forestry(52%):Forestry(27%):General agriculture, fishing and forestry sector(21%) Theme(s) Environmental policies and institutions(56%):Other environment and natural resources management(17%):Participation and civic engagement(12%):Indigenous peoples(10%):Biodiversity(5%) Prepared by Reviewed by ICR Review Coordinator Group John Redwood George T. K. Pitman Christopher David Nelson IEGSD (Unit 4) PHPROJECTDATATBL Project ID Project Name P111621 DRC: PFCN GEF GRANT ( P111621 ) Country Practice Area(Lead) Congo, Democratic Republic of Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-94135 30-Jun-2015 70,000,000.00 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) Bank Approval Date Closing Date (Actual) 02-Apr-2009 30-Jun-2015 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 6,000,000.00 Revised Commitment 0.00 4,763,759.74 Actual 0.00 4,763,759.74 Sector(s) Forestry(32%):General agriculture, fishing and forestry sector(52%):Public administration- Agriculture, fishing and forestry(16%) Theme(s) Indigenous peoples(5%):Biodiversity(53%):Environmental policies and institutions(16%):Other environment and natural resources management(26%) 2. Project Objectives and Components a. Objectives The statement of development objectives in the Project Appraisal Document (PAD, pg. 8) and Financial Agreement (FA, Schedule 1) and the Global Environmental Objectives (GEO, PAD pg. ix)) are identical: "To increase the capacity of the Ministry of Environment, Nature Conservation, and Tourism (MECNT) and the Congolese Nature Conservation Institute (ICCN) and increase collaboration among government institutions, civil society, and other stakeholders in order to manage forests sustainably and equitably for multiple uses in Pilot Provinces. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Components 1 Institutional Strengthening of Ministry of Environment, Nature Conservation, and Tourism (MECNT), which was later renamed the Ministry of Environment and Sustainable Development - MEDD] (Appraisal cost: US$ 37.18 million; Actual cost: US$ 48.79 million). This component aimed at restoring essential institutional functions in MECNT, both at the central level and in Equateur, Bandundu, and Orientale Provinces. More specifically, it was expected to re-establish institutional capacity at the central and decentralized field levels, where 12 district offices and 53 sub- district offices were to be re-constructed and re-equipped and their staff was to be trained. Support would also be extended to some additional areas in other provinces to be agreed upon with IDA in order to carry out forest control operations and governance enhancement. There were four subcomponents: 1 Infrastructure, Equipment, and Managerial Functions (Appraisal cost: US$ 18.88 million; Actual cost: US$ 10.74 million). This subcomponent would: improve institutional capacity of MECNT and Provincial Ministries by: (i) reinforcing MECNT s general administrative functions at the central level, including financial management, human resources, and monitoring and evaluation, and in MECNTs Coordination Agencies in the pilot provinces; (ii) restoring MECNTs essential forestry and conservation Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) administrative functions in the pilot provinces; (iii) designing and implementing training programs in forest management and environmental protection; (iv) carrying out studies for implementation of the decentralization policy in the forestry and nature conservation sector and strengthening forestry and conservation management capacities within Provincial Ministries in the pilot provinces, and strengthening coordination with MECNT; (v) building strategic planning capacity within the Ministers cabinet and the general secretariat of MECNT in charge of forestry and nature conservation; (vi) creating a division in charge of community forestry within MECNT and designing legal texts, a methodology and tools for the implementation of participatory community forestry management and its promotion; and (vii) construction of offices for MECNT in Kinshasa. 2 Establishment of Specialized Forest Management Systems (Appraisal cost: US$ 8.11 million; Actual cost: US$ 19.06 million). This subcomponent would strengthen MECNT s forest management technical capacity by: (i) establishing an integrated forest management information system, including a geospatial information management system, a forest accounting and fiscal and para-fiscal data system and a log tracking system; (ii) increasing the Government's capacity to evaluate and oversee forest management plans; and (iii) strengthening forest regulation and implementation capacity, including mechanisms to support data collection, administrative processes, field visits, public certification, and cost recovery. 3 Institutional Readiness and Transformation (Appraisal cost: US$ 5.00 million; Actual cost: US$ 8.99 million). This subcomponent would help carry out institutional reform within MECNT through the preparation and implementation of a Human Resources Plan and reorganization of MECNT to implement its new institutional framework, which had been approved by the Government in February 2009, and for the financing and verification of staff retirement payments. 4 Support to Project Management (Appraisal cost: US$ 5.19 million; Actual cost: US$ 10.00 million). This subcomponent would help carry out fiduciary and procurement activities, technical and financial audits, and monitoring and evaluation activities by financing the costs that the Coordination Unit (CU) of MECNT incurred during project implementation. The CU would be integrated within the MECNT as part of the projects capacity building plan, and the Government would pay the salaries of the civil servant appointed to it 2 Community Participation in Forest Management (Appraisal cost: US$ 20.87 million, including US$ 1.87 million in GEF grant funding; Actual cost: US$ 14.35 million, including US$ 1.11 million GEF). This component would be implemented in four heavily forested landscapes located in the three pilot project provinces -- (i) Lac Tumba, (ii) Maringa-Lopori-Wamba, (iii) Salonga, and (iv) Ituri which were inhabited by a range of ethnic groups of Bantu origin as well as by Pygmy minorities and were a subset of 12 priority landscapes across Central Africa in the framework of the Congo Basin Forest Partnership, which had benefited from support of the Central African Program for the Environment (CARPE) since 2004. More specifically, it would focus on land management and socioeconomic development initiatives determined and/or implemented in collaboration with local communities. It would have three subcomponents: 1 Participation of Local Communities and Civil Society (Appraisal cost: US$ 5.53 million; Actual cost: US$ 4.25 million). This subcomponent would help increase local community and civil society participation in forest management through: (i) preparation of participatory zoning of forest areas, including the clarification of their use, sensitization of local communities, and use of conflict management mechanisms; (ii) local consultations on the management of forests for various environmentally sustainable uses; and (iii) organization of consultative fora among stakeholders with the aim of improving transparency, management and protection of forest resources. 2 Development Activities for Forest Communities (Appraisal cost: US$ 11.8 million, including US$ 1.9 million from GEF; Actual cost: US$ 9.77 million, including US$ 1.11 million GEF). This subcomponent would support: increased use of environmental services and community development initiatives and diversification of economic activities. Increased use of environmental services would be supported through: (i) establishment of an environmental services unit (USE) within MECNT; (ii) identification of financing sources for environmental services available to the Recipient; and (iii) design and implementation of development activities in connection with environmental services. The USE of MECNT was expected to be strengthened and enabled to work with communities in order to identify environment deals and launch pilot initiatives such as conservation concessions, afforestation under the Clean Development Mechanism (CDM), and other schemes identified at a workshop on Alternative Models and Finance Mechanisms for Sustainable Forest Use in DRC. 3 Social, Environmental, and Cultural Safeguard Activities (Appraisal cost: US$ 3.50 million; Actual cost: US$ 0.34 million). This subcomponent would assist with implementation of the measures included in the projects environmental and social studies and safeguard plans. 3 Management of Protected Areas and Strengthening of ICCN [Congolese Institute for Nature Conservation] (Appraisal Cost: US$ 4.13 million GEF; Actual cost: US$ 3.65 million GEF). This component would strengthen ICCN and help to rehabilitate Maiko National Park. It had two subcomponents: 1 Institutional Strengthening of ICCN (Appraisal cost: US$ 1.02 million; Actual cost: US$ 1.38 million) This subcomponent would increase ICCNs capacity through: (i) improvement of its infrastructure and management capacity; (ii) collection and processing Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) of data; and (iii) updating, publication, dissemination, and implementation of the Conservation Law and public education and awareness initiatives. Specific activities would include: (i) dissemination of the new Nature Conservation Law and preparation of implementation decrees; (ii) enhancement of the Protected Area Information Management System (SyGIAP), staff training, analysis and entry into SyGIAP of data from 11 priority conservation sites not covered by UNESCO. 2 Rehabilitation of Maiko National Park (Appraisal cost: US$ 3.11 million; Actual cost: US$ 2.26 million). This subcomponent would help rehabilitate the Maiko National Park by: (i) enhancing park management capacity; (ii) developing and improving its infrastructure; (iii) upgrading offices, communications, and field equipment; (iv) design and implementation of specific community development projects to benefit communities in buffer zones; and (v) providing assistance to communities established within the Maiko National Park in order to allow for their voluntary resettlement. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Costs Total project costs were more or less as estimated at appraisal. However, there were significant differences between appraisal and actual costs for various components and subcomponents, as is indicated in Section (c) above. Actual costs for Component 1 (Institutional Strengthening of MECNT), for example, were 31 percent higher than anticipated, while those for Component 2 (Community Participation in Forest Management) were only 70 percent of the appraisal estimate. There was also significant variation within Component 1, as the Subcomponents for Establishment of Specialized Forest Management Systems, Support for Project Management, and Institutional Readiness and Transformation exceeded their appraisal estimates by 135 percent, 93 percent, and 80 percent, respectively, while that for Infrastructure, Equipment and Managerial Functions was only 57 percent of the appraisal estimate. Financing This project was co-financed by an IDA Credit of Special Drawing Rights (SDR) 42.3 million (US$ 64.0 million equivalent) and a Global Environment Facility (GEF) Grant of US$ 6.0 million. Actual IDA financing of US$ 65.9 million was slightly greater than the anticipated amount due to appreciation of the SDR against the US$, while only US$ 4.76 million of the GEF Grant was utilized during project implementation. The last TTL informed IEG that the Project Implementation Unit (PIU) was unable to produce an adequate procurement plan to utilize the remainder of the GEF resources. Borrower The Borrower was expected to provide US$ 15.0 million for in kind counterpart funding for the IDA credit (and none for the GEF grant), but, in practice, either this did not occur or the equivalent amount of the actual contribution was not calculated by the ICR team. Dates The Key Performance Indicators (KPIs) were revised at the mid-term review in November 2012. However, these changes were not formalized until a corrective Level 2 restructuring was processed in October 2014, eight months before project closing as originally scheduled on June 30, 2015. The reason for this nearly two year delay is not described in the ICR. 3. Relevance of Objectives & Design a. Relevance of Objectives The project had one general objective -- to manage forests sustainably and equitably for multiple uses in selected pilot areas of the DRC -- and two intermediate ones -- (i) to increase the capacity of MECNT and ICCN and (ii) to increase collaboration among government institutions, civil society, and other stakeholders -- in order to achieve this overall one. These objectives were highly relevant given that the DRC contains one of the largest remaining tropical forest areas in the world -- 145 million hectares, covering more than three-fifths of the national territory -- and that there had been considerable problems with forest governance in the past due largely to institutional weakness in the public sector, poor management of protected areas, and an associated lackluster enforcement of the Forest Code approved in 2002. In 2005, there had even been an Inspection Panel investigation request by Pygmy communities that believed they had been harmed by World Bank-supported forest reforms under the earlier Emergency Recovery Support Project and the Transitional Support Economic Recovery Operation. According to the PAD (Box 2, pg. 7), in order to address these weaknesses, the government formulated the National Forest and Conservation Program (PNFoCo) to mobilize donor assistance and the present project would be the cornerstone of this multi-donor initiative whose objectives were to: (i) build institutional capacity; (ii) safeguard the forests social, economic, and environmental functions, (iii) ensure that they contributed meaningfully to the livelihoods of rural and forest people; and (iv) expand the countrys protected area network. In addition, the project grounded other Bank-supported interventions, especially the Multi-Donor Trust Fund for Forest Governance financed by Belgium, the European Union (EU), France, Germany, Luxembourg, and the Netherlands, a GEF project to protect National Parks, and projects for carbon storage and deforestation offsets. It also complemented other donor projects to support the PNFoCo, particularly those financed or soon to be Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) financed by the EU, the USA, the UK, Germany, Belgium, France, Norway, Sweden, Germany, the African Development Bank, GEF/UNDP, and UNEP as well as with efforts financed through voluntary contribution to national and international NGOs. Thus, the project was viewed as a critical part of an intensive international effort to support improved forest governance in the DRC and the Congo Basin more generally. The Country Assistance Strategy (CAS) for the DRC for FY 2008-2011 pointed out the sensitive nature of the forest and mining sectors in the country and indicated that, among other activities to support efforts to achieve high, sustained, and shared economic growth, the Bank would assist government policies and agencies for enforcement of strong environmental standards for logging and the involvement of communities in monitoring forestry activities. It would also manage the Multi-Donor Trust Fund to support forest governance and support Government in its implementation of the Forest and Nature Conservation Sector Program in order to bring more transparency and public participation to forest management and ensure more equitable sharing of benefits. The most recent CAS for the DRC, for FY 2013-2016, also highlights the importance of the country's forests, as well as the Government's recent measures to manage natural resources and protect the environment, including its intention to rehabilitate and expand its entire protected area network. It likewise mentions the present project in this context, under the general strategic CAS objective of increasing state effectiveness and improving good governance, together with several other Bank- administered forest-related GEF and other Trust Fund-financed operations, and affirms that reduced deforestation would significantly help to reduce world CO2 emissions. Rating High b. Relevance of Design Project design was based on that of the multi-donor PNFoCo program, while reportedly concentrating on the most urgent sectoral needs and specific geographical locations, and sought to set the stage for interventions financed by other development partners. While the project was designed to address many of the identified governance shortcomings in the forestry sector, it was only partially adequate. The Results Framework was weak and incomplete. How increasing collaboration among government institutions, civil society, and other stakeholders in order to manage forests sustainably and equitably for multiple uses would be measured was unclear as there were no specific results indicators to gauge project achievements in relation to this objective. In addition, project design failed to address significant problems, such as illegal logging, and underestimated persisting security risks in parts of the project area, especially Maiko. Nor did the project seek to address the strong vested interests in the forest sector, which were responsible for many of the observed governance problems. Some outputs, such as management plans depended on activities not directly supported by the project. The proposed risk mitigation measures were insufficient and, in some cases, depended on elements that were outside the project's control. Finally, the large number of subcomponents and the geographic dispersion of project activities made supervision difficult even in the absence of the continuing security problems. The results expected to be achieved were not well specified. There were different versions of the key performance indicators (KPIs) in the PAD and the Financial Agreement. Even the latter set, which was more numerous than the former, had to be revised at the time of the mid-term review. More importantly, it is not clear that these revised indicators would have been able to measure achievements in relation to project objectives while they also failed to improve the alignment between and measurability of the Project Development Objective (PDO) and Global Environmental Objective (GEO) outcomes. Rating Modest 4. Achievement of Objectives (Efficacy) PHREVISEDTBL Objective 1 Objective Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) To manage forests sustainably and equitably for multiple uses in selected pilot areas of DRC. Modest Rationale Outputs: These will be listed in the following order: (i) outputs retained from the original list; (ii) outputs added following restructuring; and (iii) initial outputs dropped at the time of restructuring. Outputs retained from the original list: • Percentage of forest users in sample areas, including indigenous communities, aware of their legal and customary rights and obligations under the forest code; this target was substantially underachieved, as the final end-of-project percentage was only 5 percent compared with a target of 30 percent. • Percentage of households in targeted forest communities reporting additional sources of regular income this target was exceeded by 214 percent, but the ICR observes that this result refers only to micro-project intervention zones. While the magnitude of this increase was generally higher in intervention areas than non-intervention ones, there was no data to show that the percentage of households reporting additional income was higher than in control areas • Number of water sanitation, feeder roads, and other small participatory infrastructure projects implemented with project financing; this target was exceeded by 268 percent (67 compared with a target of 25). • Number of households utilizing small participatory infrastructure projects implemented with project financing; while this indicator was not formally dropped, it was not monitored due to security concerns and, thus, no data is available regarding actual results. • Number of indigenous population's households benefitting from project-financed initiatives; this target was reportedly exceeded by 160 percent, 480 as compared with the initial target of 300. Outputs added after restructuring: • Households benefiting from at least one micro-project in conservation landscapes; this indicator was reportedly 98 percent achieved, i.e., 34,321 compared with 35,000 households at the time the project closed in June 2015. • Volume of wood seized as a share of total illegal logging volume marked for seizure in government control posts; the target was reportedly exceeded, 99 percent compared with the target of 90 percent, but this figure is questionable given the highly uncertain extent to which illegal logging was actually monitored and controlled. • Direct project beneficiaries exceeded by 2 percent, 170,360 as compared with the target of 166,000. • Direct project beneficiaries, of which female; target partially achieved (84 percent -- 42 percent versus a target of 50 percent). Outputs dropped at the time of restructuring: • Kilometers of illegal industrial logging tracks outside legal logging concessions in pilot provinces; this indicator was dropped because it was reportedly beyond the scope of the project and no monitoring had taken place. • Reduced ratio of violations of Maiko park regulations (i.e. poachers arrested, fire arms seized, traps found, endangered species confiscated) to number of patrols conducted; this indicator was dropped because monitoring proved impossible due to security concerns. Outcomes These are listed in the same order as above: Outcomes retained from the original list: • Forest fees and taxes collected by DGRAD (Directorate General of Administrative Judicial Land and Participation Receipts) as a percentage of the total billed by MECNT at the national level; the target was exceeded (115%, or 86.5 percent compared with the target of 70 percent) by the end of 2012, but, according to the ICR, this indicator subsequently became non-pertinent when collection responsibility for the surface area tax was transferred from DGRAD to the provinces. • Number of logging infractions revealed by Independent Observers in pilot provinces; the non-specific target for this indicator had reportedly attained a positive trend at end of the Independent Observers contract in late 2013, but no further data was collected after this contract was closed, so actual end-of-project performance is unknown. Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) Outcomes added at the time of restructuring: • Protected Area Management Effectiveness Tracking Tool (METT) score for Maiko National Park (MNP) target was 100 percent achieved (both target and actual score were 58). According to the ICR, the target for the area brought under enhanced biodiversity protection was misstated in the PAD. The ICR (pg. xiv) observed that this World Bank core indicator requires that, to consider MNP to have been brought under enhanced protection, its initial METT value that was 49 (i.e., within the 35-70 range) would have to have improved to the 75-100 range. That would have been unrealistic and was never anticipated, so the target should have been 0 [as opposed to 250,000] ha. • Absolute number of elephants in Maiko National Park; data unavailable at time of project closing because of security problems. • Forest concessions with social responsibility contracts signed and publicly available; this was target exceeded by 15 percent, 75 compared with 65 percent although the total area in, rather than the number of, such concessions would have been a more appropriate indicator. Outcomes dropped at the time of restructuring: • Area covered by various types of forest land management plans agreed upon by the MECNT and other stakeholders (adopted and under implementation) in the three pilot provinces; while this indicator was dropped, its value at the time of restructuring was 2 million ha, or just 44 percent, of the initial target of 4.5 million ha, • Trend in abundance of key bio-indicator species within and outside protected areas in project landscapes indicator was dropped because data were unavailable for Maiko National Park because of security concerns and because it was considered irrelevant for protected areas not covered by the project. • Percentage of logging infractions discovered that are prosecuted in pilot provinces; this indicator was dropped as prosecutions were considered to be beyond the project's control. • Area of additional forest areas zoned using a participatory approach in the pilot provinces; this indicator was likewise dropped because the related project activities were canceled; the value achieved at the time of the MTR was 17.2 million ha, compared with the end-of-project target of 22 million ha, or 79.5 percent. Based on the indicators considered above, the project's outcome in relation to this objective appears to have been achieved to a modest extent. Even though the revised results indicators were generally met or exceeded, as they were formally introduced late in the project implementation period -- in October 2014, eight months before the project closing date, at a time when 93.6 percent of the IDA credit and 81.1 percent of the GEF grant, or an average of 92.7 percent of the total financing, had already been disbursed -- their relative importance in terms of the project's overall performance is minimal. While some of the initial indicator targets were partially achieved, many others were dropped because they could not be measured or were not properly monitored for security-related or other reasons. The ICR's assessment was similar (paras. 54-56, pp. 15-16) as it rates project results with respect to sustainable forest management as "modest." This was particularly the case with respect to its conservation activities in Maiko National Park due to the presence of some 5,000 rebels in its territory, which caused a two and a half year delay in the initiation of activities there. Rating Modest PHREVISEDTBL Objective 2 Objective To increase the capacity of MECNT and ICCN to manage forests sustainably and equitably for multiple uses in selected pilot areas of DRC. Modest Rationale Outputs Outputs dropped at the time of restructuring: Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) • Number of field supervision reports made by Provincial and District MECNT teams in the pilot provinces; this Indicator was dropped as it was not considered linked to project outcomes; however, the value at the mid-term review (MTR) in November 2012 was 25, compared with an end-of- project target of 150, or just 16.7 percent. • Number of reports made by MECNT teams on the work of contractors and NGOs in pilot provinces; this indicator was dropped as it was allegedly part of standard management functions of the PIU: however, the value at the mid-term review (MTR) was 5 compared with a end-of-project target of 20, or 25 percent. • Number of people trained in safeguards-related measures this indicator was dropped as it was not considered relevant to the results chain and no monitoring took place. • Number of responses developed to mitigate impacts; this indicator was dropped because no small infrastructure projects had been delivered and development and implementation of appropriate safeguard measures was an obligation of the government. • Number of annual evaluations by qualified assessors on quality of responses; the indicator was dropped as no monitoring was documented, and the activity described was deemed to fall under the regular project management responsibilities of the PIU. There were no specific outcome indicators with respect to this objective either before or after restructuring and, as indicated above, all of the possible output indicators were dropped at that time. The ICR (para. 47, pg. 14) acknowledges that strengthening of the institutional capacity of MECNT/MEDD and ICCN was achieved only modestly both before and after restructuring, even though installation of the specialized forest management system was considered successful. However, it does not assess the extent to which this capacity was specifically strengthened to manage forests sustainably and equitably for multiple uses in pilot provinces. Instead, it affirms that the most successful contribution to MEDDs institutional strengthening, in spite of initial political resistance, was the provision of compensation to allow the retirement of 1,702 staff and 972 managers who had passed the retirement age (para. 48, pg. 14). But the linkage between this alleged achievement and improved capacity to manage forests is unclear. Elsewhere, moreover, it stated that capacity building activities for ICCN were largely unsatisfactory and that no evidence was presented that the planned Protected Areas Management System (SyGIAP) was ever created (para. 50, pg. 14). Rating Modest PHREVISEDTBL Objective 3 Objective To increase the collaboration among government institutions, civil society and other stakeholders to manage forests sustainably and equitably for multiple uses in selected pilot areas of DRC. Modest Rationale No output or outcome indicators were specifically identified in the Results Framework to assess achievements with respect to this objective either before or after the mid-term review (MTR) and restructuring. Thus, no associated concrete outputs were recorded. Outcomes While the ICR does report that three Consultative Provincial Forest Councils were established and equipped for policy planning and monitoring purposes (but the proposed National Council was not created) and that the project supported the negotiation of 75 social responsibility contracts between concessionaires and local communities in 57 forest concessions, it does not explicitly assess project performance with respect to its aim to increase collaboration among government institutions, civil society and other stakeholders to sustainably manage forests. Furthermore, while the provincial consultative committees reportedly provided a basis for policy engagement and consultation, a lack of funding limited their effectiveness. Thus, even some of the positive results with respect to increased collaboration appear to have broken down during the course of implementation and were, thus, unsustainable. The ICR argues that this objective was "substantially achieved" both before and after restructuring since "the PDO and activities did not change at restructuring." However, the logic behind this statement is questionable. And, as noted immediately above, this judgment is not based on any specific results indicators. So it was unclear from the outset how the project was intended to be held accountable in this regard and even what exactly was meant in operational terms by increased collaboration among the different stakeholders cited. This situation did not change after the Mid-term Review (MTR) and restructuring as none of the added results indicators referred to increased collaboration among the various government and non-government actors involved. Accordingly, there is insufficient evidence to justify a higher than Modest rating with respect to the achievement of this objective. Global Environmental Objective The GEO objectives were identical to the Bank's development objectives, but one of its three outcome indicators ("stable trend in Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) abundance of key bio-indicator species within and outside protected areas in project landscapes") was also dropped at the time of the MTR because data was unavailable due to security concerns and because it was not considered relevant for protected areas not covered by the project. A second indicator -- protected area management effectiveness tracking tool (METT) score for Maiko National Park (MNP) -- was met however, while a third one, absolute number of elephants in this part, which was added at the time if restructuring, could not be assessed due to the same security concerns mentioned above. In any event, the initiation of conservation activities in the MNP was delayed for two and a half years due to the presence of some 5,000 rebels in the park. The project nonetheless completed a number of buildings for park administration and provided equipment and training for its staff. Rating Modest 5. Efficiency Economic and Financial Efficiency Ex-ante The PAD estimated an internal rate of return for the project of 17 percent based on ongoing initiatives in the project area. Direct benefits to local communities were assumed to include support to local infrastructure, enterprise and crop production initiatives which would support community participation in natural resource management. They were estimated to be on the order of US$ 2.6 million annually at the end of the project and were also expected to be substantially greater than the project's annual incremental operating costs. Ex-post The ex-post economic analysis sought to replicate that undertaken during appraisal, but that, as much of the project's expenditures were for institutional capacity building, their benefits could not be readily quantified. Based on the results of household surveys carried out in the landscapes in which the micro-projects were implemented, the recalculated ERR was 12 percent, but this was based on just 14 percent of actual project costs. Expected project fiscal benefits (PAD, para. 98, pg. 27) would come from increased collection of tax revenues from sustainably managed forests. Actual fiscal benefits due to the project, however, could not be quantified. This was, according to the ICR Annex, primarily because attribution is difficult, but also as the devolution of a number of forest taxes combined with poor data management means that the authorities are unaware of the amount of tax revenue collected between the provinces and the central government. Operational and Administrative Efficiency Failure to create the proposed timber production control marketing program adversely affected overall project efficiency not only because it delivered few results, including foregone timber revenues, but also because the contract may have been overpriced, as suggested by the substantial downward correction of the monthly fees as part of the effort to save the system. There were substantial budget overruns under project management. In addition, numerous qualified audit reports and procurement delays do not suggest efficiency in project management (see Section 11b below) and there is mention of corruption in the ICR and unaccounted-for money. . Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  17.00 Not Applicable Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) 14.00 ICR Estimate  12.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of project objectives was High given the globally strategic importance of the DRCs extensive tropical forests, but relevance of design, efficacy (all three objectives), and efficiency are all rated Modest. The project suffered from significant design shortcomings, making it nearly impossible to measure achievements in relation to two of its three declared objectives. This flaw was never corrected during implementation, even after the mid-term review clearly recognized the inadequacy of the performance indicators contained in the original project Results Framework, dropping many of them and adding several new ones at that time. Thus, there were major shortcomings in the operation's achievement of its objectives, relevance of design, and efficiency. a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating The institutional capacity required to sustain the project's limited achievements does not appear to have been established, therefore calling into question their likely sustainability. The main exception perhaps is for the more positive results of the community micro-projects, which, however, accounted for only a minor share of total project expenditures. A combination of poor budget execution by the MEDD, continued low capacity, and fluctuating political leadership, moreover, means that the ministry is not yet in a position to systematically uphold the few project outcomes. According to the ICR (pg. 18), the project's retirement process was effective, and is being replicated by five other ministries. However, the project's inability to develop a recruitment process before project closure meant that no recruitment system based on objective criteria exists. Finally, as ICCNs capacity building activities under the project were likewise unsatisfactory and the security situation in parts of the project area remain uncertain, risks to the development outcome are high. a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance a. Quality-at-Entry Even though the Bank undertook a comprehensive study of the forest sector in the country as part of project preparation, its design had significant quality at entry shortcomings, particularly with respect to the statement of objectives and the associated Results Framework and project monitoring and evaluation arrangements. More specifically, performance indicators were lacking altogether for one of the project objectives (i.e., increased collaboration among government institutions, civil society, and other stakeholders for sustainable forest management) and so inadequate for the other two that many had to be subsequently dropped and replaced. Some risks, particularly as concerned continuing security problems in and around Maiko National Park, while recognized, were underestimated. In addition, as the ICR (para. 68, pp. 18-19) points out, in an operating environment with high corruption risk, more stringent financial management and procurement supervision arrangements would have been advisable in design. Quality-at-Entry Rating Unsatisfactory Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) b. Quality of supervision Bank supervision failed to adequately address the significant quality at entry problems and this may have been due to several changes in Task Team Leaders (TTLs) over the life of the project. In addition, until close to the end of project implementation the IDA and GEF components were supervised in parallel by separate TTLs. The availability of budgetary resources, however, does not appear to have been a constraint, although the persisting poor security situation clearly was and made it impossible to supervise activities in part of the project area. The number of supervision missions was insufficient, and even though they reportedly correctly identified numerous implementation bottlenecks and provided detailed action plans to address them, Aide-memoires suggest that follow-up was spotty. A number of Implementation Status Reports (ISRs) were filed with delays and supervision of safeguards application, especially on the social side prior to the MTR, as well as of procurement and financial management aspects, was inadequate (see Section 11 below). The initial lack of clarity and measurability and the absence of one or more results indicators for one of the project's declared objectives should have been corrected at the time of the mid-term review, but was not. In addition, changes in the Results Framework introduced during the mid-term review were not formalized in terms of an actual restructuring until nearly two years after they reportedly occurred. This was a clear omission on the part of the Bank. No clear reasons for this very extensive processing delay were provided. The last TTL subsequently informed IEG that this delay was, in fact, largely due a very long period of inactivity following the MTR on the part of the then TTL, who subsequently left the Bank, together with the time needed for the new TTL to familiarize himself exacerbated by further delays on the part of the Government in formally requesting the restructuring. Even though the Government sought an extension of the closing date, Management did not agree due to the project's unsatisfactory performance. Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance Initially strong Government commitment to key reforms upon which the project was predicated declined over time due in part to a change of Ministers, adversely affecting establishment of the planned timber production control and marketing program in particular. Furthermore, while the Government reportedly met its legal covenants, in several instances, this involved delays, especially with respect to recruitment of a monitoring and evaluation specialist. It likewise failed to provide agreed counterpart funding for construction of a new headquarters building for MEDD, following a pattern in which the release of the MEDDs budget voted by the national assembly was consistently problematic. Government Performance Rating Unsatisfactory b. Implementing Agency Performance Implementation delays reportedly led to some incomplete activities and safeguard and fiduciary performance was problematic, especially prior to the MTR but also subsequently. There was poor coordination between the PIU and both MEDD and ICCN. While IDA resources were fully used, this was due in part to cost escalations for the ultimately unsuccessful timber production control and marketing program, and part of the GEF grant was undisbursed at the time of project closing due to ICCN's incapacity to use these resources. Two staff members of the project coordination unit were fired for corruption and other misuse of project funds occurred. Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) 10. M&E Design, Implementation, & Utilization a. M&E Design Project M&E design was weak. The Results Framework, as noted above, was inadequate, the initial specification of performance indicators was poor, and they only partly reflected the projects development objectives. As a consequence, many of the initial indicators had to be dropped at the mid-term review and were replaced by others, which, however, still failed to reflect one of the projects two declared objectives, thus making it difficult to monitor and assess actual progress. Some of the project performance indicators, moreover, were related to the application of safeguards, thus, referring to standard Bank policy requirements, and, according to the ICR, should not have also been used in this way, although the fact that two previous forest-related Bank operations had been subject to an Inspection Panel request for investigation may have explained why they were initially proposed as such. There was also a difference in the key performance indicators (KPIs) listed in the Results Framework in the PAD [(i) areas covered by various types of forest land management plans agreed upon by the MEDD [Ministry of Environment and Sustainable Development] and other stakeholders (adopted and under implementation) in the three pilot provinces; and (ii) stable trend in abundance of key bio-indicator species within and outside protected areas in project landscapes] and in the Financing Agreement [(i) forest land management plans, including community forests (simplified management plan), protected areas, timber concessions, conservation concessions, community hunting zones (simplified management plan) and others, have been agreed upon by all concerned stakeholders, and subsequently adopted and are being implemented; (ii) population of key bio-indicator species in targeted forest landscapes; (iii) percentage of cases of illegal forest exploitation for commercial purposes which are detected are prosecuted; (iv) forest areas covered by a participatory zoning plan; and (v) development initiatives completed with community participation.] There was no explanation in the ICR as to why these differences occurred. b. M&E Implementation M&E implementation was substantially delayed because of the initial inability by the Government to contract qualified national staff, such that no reporting against project indicators occurred for the first two years after Credit effectiveness. Twelve of the initial 21 indicators, including the two that referred specifically to the development objectives, were dropped at the time of the MTR, and the five new indicators introduced at that time were not formalized until the restructuring which occurred nearly two years later. Some indicators subsequently also proved impossible to monitor because of persisting security risks in part of the project area. The proposal to strengthen MEDDs M&E capacity was likewise unimplemented. c. M&E Utilization The ICR was unable to assess M&E utilization, observing that while the M&E system worked quite well during the tenure of an international M&E expert hired for this purpose, the degree to which data was evaluated and used to inform decision-making and resource allocation cannot be impartially assessed post-project due to a lack of documentary evidence. This suggests that M&E results were unlikely to have been used effectively, especially for that part of the project area where no monitoring was possible as the result of security problems. There appears to have been no significant evaluation activity either prior to or following project completion. M&E Quality Rating Negligible 11. Other Issues a. Safeguards Because the environmental and social sensitivity of forest-related activities, the project was classified as in Category 'A' under OP/BP 4.01 (Environmental Assessment) and triggered five other environmental and social safeguard policies. As a result, it was subject to an up-front Strategic Environmental Assessment (SEA). An Environmental and Social Management Framework, a Resettlement Policy Framework, a Process Framework, an Indigenous Peoples Policy Framework, and a Physical Cultural Resources Framework were also adopted and disclosed in late 2008, and the project contained a US$ 3.5 million subcomponent for safeguards implementation. However, less than 10 percent of the originally anticipated resources (i.e., some US$ 340,000) were actually used for this purpose and the ICR does not explain why such a small share of the resources originally allocated for safeguard implementation was used. Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) The ICR (para. 35, pg. 11) rated project safeguard compliance, including both implementation and supervision -- as having been only Moderately Satisfactory. This appears to have especially been the case on the social side prior to the mid-term review, although shortcomings in the application and supervision of environmental safeguards were also reported. On the more positive side, however, the project succeeded in developing nine indigenous peoples development plans, whose associated investments reportedly reached nearly 500 households. b. Fiduciary Compliance Procurement was subject to delays and was rated Moderately Satisfactory by the supervision team throughout implementation (ICR, para. 38, pg. 12). While this activity had initially been outsourced to a multi-donor agency with experience in World Bank procurement procedures, it was transferred to the PIU after the multi-donor agency was dissolved. Additional training by Bank staff and others did not substantially improve performance over time, as local procurement staff continued to be weak. Project financial management (FM) performance was unsatisfactory. The majority of the interim financial reports were not submitted on time and numerous audit reports were qualified, primarily due to the existence of ineligible expenditures, which, according to the ICR, took a long time to resolve. Actions recommended by supervision missions were not consistently implemented. Two local FM staff were fired because of financial improprieties and, at the time the ICR was issued (December 2015), the Bank's Integrity Vice Presidency (INT) had an open investigation involving allegations of fraud and corruption on the part of project staff, some of whom had already been removed from the project team during effectiveness. In addition, unjustified outstanding expenditures of more than US$ 500,000 were declared ineligible at the time of closing and there were a number of unpaid bills. INT informed the last project TTL in March 2016 that the case had been closed because it had been determined that no Bank staff or consultants were involved. However, it has taken the Bank considerable time to determine the full extent of the ineligible expenditures that will need to be reimbursed by the Government, which are now (i.e., May 2016) estimated to be over US$ 1 million. The last TTL also indicated to IEG that local corruption is a pervasive problem that affects much of the Bank's portfolio in the country. c. Unintended impacts (Positive or Negative) --- d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Despite the High relevance of project objectives, performance on all other aspects, including the relevance of project design and Outcome Moderately Unsatisfactory Unsatisfactory efficacy in relation to its objectives was Modest. Thus, the project had major shortcomings in its relevance of design, efficacy, and efficiency. Risk to Development Outcome High High --- Bank Performance Unsatisfactory Unsatisfactory --- Borrower Performance Unsatisfactory Unsatisfactory --- Quality of ICR Substantial --- Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC- Forest and Nature Conserv. - PFCN (P100620) Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR presents a number of valuable lessons including: (i) Avoid reliance on policy or regulatory pre-conditions as Government commitment with respect to them can change over time. In this project the promised government decrees for the timber production control and monitoring program and community forestry, were not met; (ii) Governance reforms in settings with a complex political economy should consider phased approaches with incentives for delivery, such as performance-related disbursements; and (iii) Reliance on co-financing is inherently risky in countries with constrained fiscal space. In the case of this project, unwillingness to set a budget led to cancellation of the construction of MEDD's headquarters. IEG draws the following additional lessons: (iv) Substantial up-front training and other institutional capacity building by the Bank is needed when borrower capacity for procurement, financial management, environmental and/or social safeguards compliance, and/or monitoring and evaluation, is weak, together with frequent and proactive hands-on supervision; and (v). Formal restructuring should be administratively processed in a timely way following mid-term or other critical project performance reviews which identify significant implementation problems that can be traced back directly to poor project design. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR is well-written and of generally good quality. It identifies the project's, the Bank's, and the Borrower's principal deficiencies and failures, both in terms of design and during implementation/supervision, in a clear and forthright manner. However, it could have been more explicit as to the reason or reasons for the Bank's close-to-two-year omission in consolidating the substantial changes introduced in the project's Results Framework at the time of the mid-term review by means of a formal restructuring. The ICR could also have indicated the reasons for the high turnover in TTLs during the life of the project and provided additional information concerning the considerable differences between appraisal and actual project costs at the subcomponent level. Despite these shortcomings, the overall quality of the ICR, including its annex on economic and financial aspects, is substantial. a. Quality of ICR Rating Substantial