Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) Report Number: ICRR0021794 1. Project Data Project ID Project Name P129688 BF-APL 3 Com Based Rur Dev III Country Practice Area(Lead) Burkina Faso Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H8220 31-May-2018 71,413,070.70 Bank Approval Date Closing Date (Actual) 20-Dec-2012 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 70,000,000.00 0.00 Revised Commitment 70,000,000.00 0.00 Actual 64,872,861.15 0.00 Prepared by Reviewed by ICR Review Coordinator Group Songporne J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Tongruksawattana Pellekaan P130568_TBL Project ID Project Name P130568 BF-APL3 Com Based Rur Dev III(GEF) (PSG) ( P130568 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-13637 7293235.13 Bank Approval Date Closing Date (Actual) Page 1 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) 20-Dec-2012 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 7,407,408.00 Revised Commitment 0.00 7,293,235.13 Actual 0.00 7,293,235.13 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Community-Based Rural Development Program (CBRD) in Burkina Faso was designed with the objective "to reduce poverty and promote sustainable development in the rural areas". The program was implemented over a 15-year period as a three-phase Adaptable Program Loan 2001-2015. This project represented the third and last phase (2012-2018) of the CBRD Program. The Project Development Objectives (PDO) of the Third Phase Community Based Rural Development Project (CBRD 3) as stated in the Financing Agreement dated February 25, 2013 (p.5) were "to enhance the capacity of rural communities and decentralized institutions for the implementation of local development plans that promote sustainable land and natural resource management, and productive investments at commune level". The PDO formulation was identical in the PAD (para.23). The same PDO was formulated in the GEF Grant Agreement dated February 25, 2013 (p.5) which also financed a component in this project. The Global Environmental Objective is subsumed in the PDO (PAD, p.7). The PDO remained unchanged throughout the project period. This Review will assess this project’s achievements against the PDO as stated in the Financing Agreement. For the purpose of this review the PDO will be parsed into three sub-objectives referred to as Objectives 1, 2 and 3 in Section 4. They will be: Objective 1: Enhance the capacity of rural communities and decentralized institutions for the implementation of local development plans Objective 2: Enhance implementation of local development plans that promote sustainable land and natural resource management at commune level Objective 3: Enhance implementation of local development plans that promote productive investments at commune level b. Were the project objectives/key associated outcome targets revised during implementation? Yes Page 2 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components Component 1 – Strengthening Capacity for Decentralized Rural Development (Estimated cost at appraisal US$13.39 million, actual cost US$8.85 million.) Building on the achievements from the first two CBRD phases, this component aimed to enhance local governance and improve local development planning and management by deepening technical and capacity building for local governments at all levels. Under this component, local governments were expected to develop Local Development Plans (LDP) which is a five-year referencing instrument for communal activities, prepare Annual Investment Plans (AIP) to implement the LDP, and identify local development projects in strategic, holistic and multi-sectoral manner in collaboration with local communities, rural communes, deconcentrated technical services, private sector and civil society. The capacity building activities aimed to provide local actors, including municipal councils (CM), Village Development Council (CVD), and deconcentrated technical services with the necessary tools to implement, monitor and maintain local development investments. This component was comprised of four sub-components: 1-A Capacity Building for Rural Local Governments targeting CM and CVD to consolidate the capacity of local governments to prepare and manage LDPs taking into account economic, social and environmental sustainability; 1-B Capacity Building for Regional Governments targeting 13 regional councils; 1-C Capacity Building for Local Communities targeting private contractors, NGOs and decentralized government institutions; and 1-D Collaboration and Communication to strengthen communication flows and collaboration strategies on project activities and results at regional and communal levels. Component 2 – Implementation of the Rural Land Legislation and Enhancement of Local Dispute Resolution Mechanisms (Estimated cost at appraisal US$5.91 million, actual cost US$2.23 million.) This component aimed to implement the Rural Land Tenure Law adopted in June 2009 which placed the central role on local actors to ensure land security and tenure supported by earlier phases. Component activities included: (a) establishment and operation of Rural Land Tenure Services at commune level through Village Land Tenure Commissions, and Village Conciliation Commissions; (b) land titling operations; (c) strengthening capacity of local actors to carry out land titling processes efficiently and transparently; and (d) dissemination of the law to increase men, women and illiterates’ awareness about their rights, and promoting gender-balanced access to land. Following the socio-political unrest during 2014-2015, the President of the transitional regime declared a nationwide suspension of land tenure operations in November 2014. Not until June 2015 were land tenure activities authorized in peri-urban and rural areas (MTR Aide-Mémoire p.7). In addition, municipal councils were dissolved and replaced by interim special delegates in February 2015. Given socio-political and institutional changes, it was proposed at the MTR to significantly scale down the activities and lower the Page 3 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) target indicators resulting in the partial transfer of this component’s resources to component 3 (ICR para.44). At the request of IEG, the Bank project team clarified that the transfer decisions were made during the project steering committee sessions with the Bank’s representation and during GoBF-World Bank joint implementation support missions (ISMs). Specifically, a formal restructuring was not necessary since the project had only one expenditure category, and targets were approved instead of direct transfer amounts. This component also planned work on securing community investments and/or resolving conflict between farmers and herders, and piloting mobile courts, but due to the socio-political unrest during this period, the project was able to implement these activities in only 50 out of the planned 100 communities. Component 3 – Local and Regional Investments (Estimated cost at appraisal US$54.24 million, actual cost US$56.31 million. The increase in component cost was due to the reallocation of resources from Component 2 and Component 4.) With more than 65 percent of the of the total project funds, this component aimed to support communes in promoting socio-economic growth and sustainable development through the financing of local development investments based on a demand-driven approach as proposed in the LDPs for all the 302 rural communes in the 13 regions. The project co-financed micro projects approved in the AIPs at the commune level, and in some cases co-financed larger infrastructure investments at regional level to benefit several communes. Building on the structural investments support in phase I (CBRD 1) of the three-phase program and the social investments support in phase II (CBRD 2), this component in phase III focused on productive investments to promote local economic growth and revenue generation as well as natural resource management (NRM) and environmental investments. The GEF co-financing in this component focused on four ecological zones to enhance environmental impact and complement activities in Component 4. Component 4 – Sustainable Land and Forestry Management (Estimated cost at appraisal US$2.00 million, actual cost US$0.53 million.) Funded entirely by GEF, this component aimed to support restoration and protection of natural resources, forest and biodiversity in the larger ecosystem landscape affected by agricultural expansion, beyond the community-based investments. The project identified four target zones for intervention: the PONASI (Pô- Nazinga-Sissili) ecological complex in the Centre-Sud and Centre-Ouest regions, the Nazinon (Red Volta) watershed north of PONASI, the provinces of Kouritenga (Central East) and Gnagna (Eastern Region), and Plateau Central Region in Kourwéogo Province. This component would also support the National Office of Protected Areas (OFINAP) to develop and/or update management actions in the management plans of the four protected areas and provide technical and financial support to local authorities for the management of their natural resources. However, faced by numerous challenges that rendered some NRM investments and activities ineffective, close to 75 percent of the initial funds was reallocated to NRM activities in Component 3 (ICR para.47), through a similar transfer process as described under Component 2. Component 5 – Project Management, Monitoring and Evaluation (Estimated cost at appraisal US$17.87 million, actual cost US$15.20 million.) This component aimed to ensure the effective implementation, coordination and regular monitoring and evaluation of the project’s activities. It was also intended to involve civil society in project supervision and monitoring, create a project website, and introduce a formal mechanism to handle project complaints. Page 4 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost The estimated total cost at appraisal was US$93.41 million. The actual cost reported in the ICR (Annex 3) was US$83.12 million, equivalent to 89 percent of the appraisal cost. Financing The IDA grant for the project was US$70 million, of which US$64.9 million (93 percent) was disbursed. The grant was complemented by a fully blended grant of US$7.41 million from the GEF Trust Fund providing parallel financing for targeted activities under Components 3 and 4. US$7.4 million was disbursed, equivalent to 99 percent of the GEF allocation. Borrower Contribution At appraisal, the government was expected to contribute US$8 million to support the activities in components 1, 2 and 5 (PAD para.58). The project’s beneficiaries were also expected to contribute US$8 million to finance the activities in component 3. The actual amount contributed was US$4.8 million from government and US$3.0 million from local communities (ICR p.ii). Dates The CBRD 3 was approved on December 20, 2012. The project became effective on June 21, 2013, almost three months after the original start date proposed in the PAD (p.vi) on April 1, 2013. The closing date at appraisal was May 31, 2018. The MTR on April 12-29, 2016 reported satisfactory implementation progress despite the nationwide socio- political and institutional challenges during 2014-2015. Nonetheless, activities in Components 2 and 3 faced substantial delay mainly due to the dissolution of all municipal councils, the slow transition to install the special delegates as interim councils, and the weak capacity of the new administration at the commune level. In these circumstances the MTR recognized the need to review some of the project’s targets and proposed adjustments in the MTR Aide-Mémoire (April 2016). However, no formal restructuring was undertaken to revise the targets as, according to advice received from the Bank project team, it was deemed unnecessary. Although the socio-political situation returned to normal in May 2016 with the election of new municipal councils, the project's progress lagged two years behind. In order to ensure a successful completion, the project was approved for a Level 2 Restructuring on November 16, 2017 to extend the closing date by seven months from May 31, 2018 to December 31, 2018 without implications for the PDO. 3. Relevance of Objectives Page 5 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) Rationale With 80 percent of its population living in rural areas and less than 18 percent of land cultivable due to poor soil quality, soil erosion and drought, rural development in Burkina Faso has relied heavily on area expansion and deforestation for farming and exploitation of forests. As the country population is expected to exceed 20 million in 2019 based on an annual growth rate of 3 percent, sustainable rural development is pivotal to poverty reduction, national economic growth, food security and conservation of natural resources. Since 1986, the GoBF has embraced a community-driven development (CDD) approach to involve local communities in the identification, planning and realization of local development investments with support from the World Bank and other international development partner initiatives. In response to increasing land conflicts, the GoBF adopted a new Rural Land Tenure Law in 2009 to ensure equitable access to rural land, promote investments and sustainable natural resource management, and provide a mechanism for participatory management of local natural resources through the local land tenure charter. Throughout the project period from appraisal to closing, the CBRD 3 PDO remained highly relevant to the national development strategies and policies of Burkina Faso. The CBRD 3 supported the Strategy for Accelerated Growth and Sustainable Development (SCADD, 2011-2015) to link poverty reduction and human development with economic growth. To operationalize the SCADD, the GoBF adopted a National Program for the Rural Sector for the same period, focusing on food and nutritional security, agricultural productivity, rural poverty reduction, sustainable access to drinking water and sanitation, and protection against the degradation of vegetation cover. These elements were integrated in CBRD 3 design. The World Bank appraised the Community-Based Rural Development program (CBRD) in 2000 to support the GoBF’s Deuxième Programme National de Gestion des Terroirs (PNGT 2) (2000-2015) which aimed at promoting decentralized rural development and reinforcing the capacity of local institutions to sustainably deliver services selected by and for their population (ICR p.56). At appraisal, the PDO was aligned with the Country Assistance Strategy (CAS) FY10-12 on both strategic themes; (a) minimizing economic vulnerability and promoting growth through economic transformation; and (b) promoting shared growth through effective social service delivery. The PDO was also consistent with the World Bank’s Strategy for Africa (2011) based on two main pillars; (a) competitiveness and employment covering all sectors including agriculture; (b) vulnerability and resilience to macroeconomic and idiosyncratic shocks; and (c) an overarching foundation of the strategy related to governance and public sector capacity building from the demand and the supply side. In addition, the project was part of the Sahel and West Africa World Bank/GEF Program (SAWAP) in support of the Great Green Wall for the Sahara and Sahel Initiative (GGWSSI) approved by the GEF Council in 2011. During project implementation, the PDO was also consistent with the Country Partnership Framework (CPF) FY13-16 objectives including to "Reduce economic, social, and environmental vulnerabilities". At completion, however, the PDO remained only tenuously aligned with the CPF FY18-23 focus areas including "Accelerate sustainable private-sector led growth for job creation" as the share of micro project investments aimed at supporting future growth and employment were below expectations while social public investments were above expectations. This review further assessed the relevance of the PDO for CBRD 3 – the last in the three-phases – against the higher-level objectives of the CBRD Program aiming "to reduce poverty and promote sustainable development in the rural areas". Since the PDO of the first two CBRD phases focused primarily on capacity building support and investments in micro projects in rural communes to implement LDPs, the reiteration of the emphasis in the PDO for CBRD 3 on local and institutional capacity building and development and implementation of micro projects – rather than on poverty reduction and sustainable development – diluted its relevance to the PDO for the CBRD Program. Following clarification to IEG by the Bank project team, the Page 6 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) reinforcement of the PDO for CBRD 3 with respect to capacity building and implementation of micro projects was essential to deepen the achievements in previous phases and to update and enhance knowledge and skills of local actors along with the decentralization and local development process. However, this review regarded the objective for CBRD 3 as almost a replica of the objectives for the previous two phases and the relevance of CBRD 3 PDO to the overall objective of the CBRD Program remained weak. This review also noted the lack of attention in CBRD 3 to "establish local institutions to effectively implement the new Rural Land Tenure Law at village and commune level" which should have been included more explicitly in the PDO. The significance of this issue was not only integrated in the key objectives laid out by CBRD 1 and 2 on functioning land tenure institutions, but the PAD CBRD 3 (para.37) emphasized its intention that Component 2 support the implementation of the new Rural Land Tenure policy and law achieved in earlier phases. However, in the correspondence with IEG, the Bank project team assessed the establishment of local institutions to effectively implement the new Rural Land Tenure at village and commune level as “an ideal scenario but not a prerequisite to the implementation of project activities”. The team further noted that “other securing land instruments such as the “Procès Verbaux de Cession” are available at commune level to secure project investments even when the commune did not have the institutions required in the Land tenure law.”, and concluded that “the establishment of local institutions to effectively implement the new Rural Land Tenure is a process”. As elaborated in the Efficacy analysis (see Section 4. below), the consequence of this approach by the Bank not only hampered the process facilitation, it eventually impacted the implementation of all activities and the achievements of the PDO negatively. Conclusion. The PDO was highly relevant to the government's national development strategies and substantially relevant to the World Bank/GEF development objectives and strategies from CBRD 3 appraisal until completion. Nevertheless, it overlooked the necessary underlying requirement of the effective implementation of the new Rural Land Tenure Law for the ultimate achievement of the project’s objective. Another shortcoming was that the PDO for CBRD 3 made no reference to the PDO for the CBRD three phase Program, namely to achieve "sustainable development and poverty reduction in rural areas". Overall, however, the objective's main weakness was that its core aim of enhancement of the capacity of rural communities and institutions, the enhancement of local development plans that promote sustainable land and natural resource management, and the enhancement of local development plans that promote productive investments was undefined, vague and not sufficiently monitorable. But the PDO also lacked the ambition one would expect in the final phase of a three-phase program which so far had delivered a satisfactory performance. This review has therefore rated the relevance of the PDO as Modest. Rating Relevance TBL Rating Modest 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Page 7 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) Objective Objective 1 – “Enhance the capacity of rural communities and decentralized institutions for the implementation of local development plans” Rationale The evidence on outputs and outcomes below for all objectives is drawn from the ICR Section II.B. and ICR Annexes 1A-1B, 4, 5, 6 and 7. Since the project did not carry out a final impact evaluation as planned (no explanation for this was provided in the ICR), the evidence in the ICR came from several sources including the government CBRD 3 completion report and an economic and financial analysis prepared by the PCU (December 2018), field visit by authors of the ICR to a sample of micro projects (no information on methodology provided in the ICR), stakeholder workshop organized as part of the last Bank supervision mission (November 2018 – see ICR Annex 7), and analyses prepared for CBRD 1-3 (2001-2018) without any information on the basis for the data. Finally, this review noted that the baseline for indicators related to activities in Components 1-4 were set to zero in the CBRD 3 newly developed Results Framework instead of building on the achievements of previous phases. Theory of change. The ICR indicated that the combination of capacity building activities, training programs and financial resources for local and regional investments will allow "learning by doing" for rural communities and local/regional governments, in a participative and decentralized manner and in conjunction with support from non-state actors. Following the capacity building objectives of the two previous phases, CBRD 3 evolved their content and activities in response to the needs of beneficiary communes in accompanying the decentralization and local development process and updated key knowledge necessary for new and existing members of the municipal councils to exercise their mandate. To assume their roles and responsibilities in developing and implementing their land development plans (LDPs) including the operation and maintenance of new infrastructure, it would be necessary to operationalize the new Rural Land Law (ICR, Figure 2). Outputs: This review acknowledges numerous outputs accomplished by the project’s intervention (see ICR Annex 1 for a complete record); the most important ones were as follows:  43,752 beneficiaries participated in capacity training activities (baseline = 0; original target 500; proposed target at MTR 8000), of which 35% were female (baseline = 0%; no original nor MTR proposed target). The capacity training activities consisted of training, awareness-raising, local planning, communication and consultation actions.  1,362 institutions had at least one representative who benefited from one or more trainings with the support of the project (baseline = 0; original target 650; proposed target at MTR 500).  49 training topics delivered (baseline 0; original target 30), of which 24% focused on NRM (baseline 0%; original target 20%),  302 LDPs finalized by the stakeholders and submitted to the relevant village development council (CVD) for review/approval in all 302 rural communes (baseline 0; original target 302). However, the ICR (page 31) noted that 190 communes had already finalized their LDPs by the end of CBRD 2, hence the original target for CBRD 3 should have been set at 112 for the remaining communes. In addition, out of 112 achieved LDPs, 95 LDPs were updated with support from CBRD 3 whereas the other 17 LDPs were supported by other development partners which could not be attributed to the project’s intervention. Page 8 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688)  As part of the implementation of the Rural Land Tenure Law, 50 Rural Land Tenure Services operational at commune level (baseline 0; original target 150; proposed target at MTR 25), 785 certificates of rural land ownership delivered in 50 communes (baseline 0; original target 7,000; proposed target at MTR 700), and 10 local land charters approved by the municipal council (baseline 0; original target 50; proposed target at MTR 10). The ICR (para.39) however noted that these activities only reached half of 100 planned communes. Due to socio-political situation in 2014/2015, activities on conflict resolution between farmers and herders, and the mobile court piloting could not be fully implemented. As a result, land tenure security remained an important issue (ICR Annex 5 para.13), including lack of discipline of the actors resulting in recurrent conflicts, transboundary transhumance at the time of animal movement, anarchic occupation of livestock routes despite demarcation, non-respect of consent for land use for NRM activities. Despite the effort to strengthen the communication and collaboration across all level of administrations and development activities, the collaboration between decentralized administration and locally elected representatives was sometimes non-existent, timid, or conflicted (ICR, Annex 5 para.15). Lack of enforcement for uniform collaboration protocols between CBRD 3 and the decentralized government services among service types and regions, made implementation and supervision difficult (ICR, Annex 5, para.14). Outcomes:  PDO core indicator [IDA/GEF]: 4,046,760 people have benefited directly from the different capacity building activities or infrastructures built with the support of the project during the first five years of their existence (baseline 0; original target 2,000,000; proposed target at MTR 1,200,000), 51% of project beneficiaries were women (baseline 0%, no original nor MTR proposed target). The original target was an estimate due to the demand-driven nature of a CDD project with no advance information on the type of micro projects to be financed (ICR, Annex 1, p.25).  PDO indicator [IDA]: 80% of approved AIPs executed more than 50% of the planned activities (baseline 0%; original target 80%).  96% of total micro projects successfully completed (baseline 0%; original target 85%).  Despite the socio-political and institutional instability in Burkina Faso (2014-2015), the interim special delegations and the newly elected municipal councils – after having benefitted from capacity building from the project to fill the experience gap – were able to assume and continue their local development activities. During this period, the execution rate of Component 3 increased from 12% in Q1 to 64% in Q2 (ICR, para.30). An additional proxy for the enhanced capacity was the emergence of local and national political leaders from the pool of project’s beneficiaries (ICR, para.30). Conclusion. Despite challenging socio-political situations in 2014-2015, shortcomings in the indicators measuring the project’s achievements (to be discussed in Section 9a), and the lack of a counterfactual analysis to validate attribution, the available evidence demonstrated a substantial level of improvement in the capacity of rural communities and decentralized institutions to implement local development plans. Rating Substantial Page 9 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) OBJECTIVE 2 Objective “Enhance the implementation of local development plans that promote sustainable land and natural resource management at commune level” Rationale Theory of change. Building on the strengthened capacity at the community and regional level to develop and implement their local development plans (LDPs) over the previous two phases, CBRD 3 was expected to shift the LDPs orientation from social public investments towards natural resource management/sustainable land and forestry management (NRM/SLFM), in particular with the additional fully blended GEF grants available in this last phase. The ICR indicated that the specific capacity building activities and training in natural resource management/sustainable land and forestry management (NRM/SLFM) for rural communities and local/regional governments, coupled with financial resources earmarked for NRM/SLFM micro project investments, and the operationalization of the Rural Land Tenure Law related to identification of land areas (agriculture, pastoral, forests, protected areas) and demarcation of the four ecozones for restoration, would enable and encourage the communes to incorporate NRM/SLFM components into their local development plans (LDPs) and implement more micro projects focusing on NRM/SLFM (ICR, Figure 2). Outputs:  7 sustainable land and forest management (SLFM) practices disseminated in targeted zones (baseline 0; original target range between 5 and 10; proposed target at MTR 8), related to reforestation, enrichment, protection, assisted natural regeneration, water and soil conservation/soil defense and restoration, forest and/or wildlife management, and capacity building in NRM.  4 management plans for selected protected areas developed/updated (baseline 0; original target 5; proposed target at MTR 3). However, as CBRD 3 was targeting a total of 4 ecozones, the initial target of 5 was incorrect, and it was proposed to be reduced to 3 at MTR (ICR, page 36).  34 participatory forest management plans developed and/or implemented in targeted communities shared forests in targeted zones (baseline 0; original target 25).  Sustainable land and NRM investments: 13,759 compost pits and 774 ha stone bunds at farm level; 144 ha of degraded land rehabilitated; 14,730 ml of riverbanks delimited; 7 springs protected; 1,041 kg of catalyzer for compost provided; 4,069 improved stoves in clay and 222 metal stoves distributed. But no targets.  Reforestation: 332,921 trees (55% survival rate) and 17,134 multipurpose trees planted which covered only 1,300 ha; delimitation of 397 km of forest; and 70 km of forest roads constructed. But no targets. Outcomes:  PDO indicator [GEF]: 213,320 hectares additional land area under SLFM practices increased (baseline 0; original target 15,000 ha; proposed target at MTR 200,000 ha). The ICR (page 27) noted that the original target was set as a "placeholder" and following detailed work with OFINAP it was realized at project effectiveness that the initial area of the four ecozones already amounted to 200,000 ha. The actual achievement was therefore only 13,320 ha.  In terms of local and regional investments), at project appraisal, the objective was to provide 20% of funding for NRM activities (PAD, para.27). The actual proportion of financing to implement NRM micro projects was 12% but this expenditure was for activities such as training, control of bushfires, studies, Page 10 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) training and environmental education and only 2.90% was for actual micro projects to address environmental challenges, of which only 0.69 percent was for reforestation. Considering the GEF co- funding for this component and the project's title this low percentage of investments and sparse implementation for NRM investments, particularly reforestation, indicated a compelling underachievement of this objective.  Due to the delay in Rural Land Tenure reform following the socio-political unrest in 2014-2015 and the subsequent reallocation of 75% of initial resources from Component 4 to Component 3, the activities focusing on NRM/SLFM were substantially reduced. In addition, NRM activities were mainly implemented at regional level which discouraged the communes to engage, and the regional implementation faced lengthy procurement processes and a lack of capacity to carry out large infrastructural work. Conclusion. Although many output indicators were achieved or exceeded, outcomes were considerably underachieved for both SLFM land area covered and NRM micro project investments. The theory of change was not achieved and there was only a negligible enhancement of local development plans that promoted sustainable land and natural resource management at the commune level. The project’s achievements towards Objective 2 were therefore rated Negligible. Rating Negligible OBJECTIVE 3 Objective “Enhance implementation of local development plans that promote productive investments at commune level” Rationale Theory of change. As an important step beyond the micro project investments undertaken over the previous two phases, CBRD 3 was expected to advance their activities in transforming the existing infrastructures into measurable improvements in livelihoods of rural communities and ensuring their financial sustainability in preparation for the APL exit, through the focus of productive investments in the local development plans (LDPs). The ICR indicated that, with enhanced local development capacity, functioning institutions to manage local land titling thereby increasing land tenure security, and funding available for local development plans (LDPs), the communes would be able and motivated to undertake more productive micro projects that would boost local sustained economic growth through income generation and employment, and hence raise fiscal revenue through the collection of taxes and rent (ICR, Figure 2). Outputs:  35% of funding was allocated to economic productive investments in the annual investment plans (AIPs) (baseline 0%; original target 15%), such as lowland development, horticulture schemes, retail shops, storage facilities, input stores and beekeepers. Page 11 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688)  Only 2% of funding was allocated to economic public goods that may generate income compared to the original target of 40%. These included, for example, access corridors and boreholes for livestock, grassland improvement, market facilities, rural roads and river crossings.  In response to strong revealed community demand, 48% of the total available funds was allocated to social public investments (baseline 0%; original target 25%), such as boreholes for drinking water, classroom and school construction, classroom equipment, maternity clinics, hospital rooms and housing for hospital staffs, medical vehicles and equipment, community centers, and public latrines. Outcomes:  Based on the growth of commercial micro projects communes were able to raise fiscal revenue from taxation and rental income. The ICR (para.37) cited the average monthly rental costs at 2,856 FCFA per shed; 4,969 FCFA per shop; 28,500 FCFA per house; and monthly income of 300,000 FCFA from cattle markets. An example of fiscal impact were the street shops in Matiacoali’s commune which generated an additional 10% annual revenue for the municipality budget (ICR Annex 6 para.9). However, the lack baselines and targets made it difficult to evaluate the context of these fiscal revenues and how much to attribute to the investments made in the context of this objective.  The relatively large allocations to socio-economic infrastructures in local communities were demanded by those communities and arguably resulted in substantial social benefits. The ICR mission reported feedback from the beneficiaries during the stakeholder workshop that "the activities under the objective greatly contributed to the well‐being of the communities resulting in an improvement in the conditions of access to basic social services (schools, health infrastructures, increase of the schooling rate)" (ICR, Annex 6, para 9).  Since a smaller proportion of the project's funding was actually allocated to productive and economic micro projects than initially planned, the generation of local wealth, municipal revenues and human capital development were arguably also less than expected. On the other hand, the benefits from socio-economic infrastructures were undoubtedly larger than expected.  Some unfinished or abandoned micro projects were observed by the ICR mission due to bad site selection, lack of supporting infrastructures, and poor quality of existing infrastructure as a result of limited capacity of local contractors (ICR Annex 5 para.18). Conclusion. Despite the project design to promote investments in productive and economic assets and income-generating activities, in practice the communities expressed a clear preference for social public investments such as classrooms and boreholes for drinking water. This review found fundamental shortcomings in the adequacy of output and outcome indicators for "productive" investments (e.g. employment) which thwarted the measurement of the project's achievements towards this objective (see Section 9a). Nevertheless, overall there was an enhancement of local development plans that promoted productive investments and there were successful social public investments which were also productive in terms of their broad positive impact on community welfare. This review therefore concluded that there was a substantial enhancement of local development plans that promoted productive investments in many sectors in response to community demand. Rating Substantial Page 12 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The ICR’s efficacy analysis focused entirely on the project’s achievements compared to targets established at appraisal and hence the attribution of the achievements to the project’s interventions was unclear. The assessment of achievements was further obscured by the non-formalization of target revisions proposed at the MTR. Considering the context for this project in underdeveloped and remote rural areas in Burkina Faso, there was nevertheless sufficient evidence to acknowledge the project’s substantial contribution to enhancing the capacity of rural communities and decentralized institutions for the implementation of local development plans (Objective 1). However, available evidence could only indicate a negligible achievement for Objective 2 that aimed to enhance sustainable land and natural resource management and productive investments to stimulate growth at the commune level. The reason for this was that implementation of micro projects in NRM/SLFM was hampered by a number of external and internal factors. Notably these factors included the delay in the implementation of the Rural Land Tenure Law and shortcomings in local dispute resolution mechanisms caused by socio-political unrest in 2014/2015. Therefore, the achievement of Objective 2 was rated negligible. With respect to Objective 3, although there was a preference by communities for investments in social public goods over commercial investments which the project design had not anticipated, this review considered that both types of investments were productive in the sense that they resulted in either increased welfare or increased income. Communities decided that increased welfare was more important than increased income. Hence the achievement of Objective 3 was rated substantial because there had been an enhancement of a range of productive investments at the commune level. Overall, however, the project's efficacy remains weak in terms of transforming the previous achievements of capacity building, Rural Land Tenure reform, and LDPs development and implementation in CBRD 1 and 2 into tangible progress towards the 15-year APL program objectives of poverty reduction and sustainable development. The overall efficacy of the project's achievements was therefore rated Modest. Overall Efficacy Rating Primary Reason Modest Low achievement 5. Efficiency At Appraisal The potential economic and financial viability of this project was estimated at appraisal (PAD, Annex 9). paragraph 4 of Annex 9 noted the difficulties associated with an economic analysis of institutional development and capacity building and also drew attention to the demand-driven nature of CBRD 3 micro project investments which precluded specification of future investments. The ex-ante analysis used data from a sample of the results from the previous CBRD 2 project. The randomly selected sample covered 6 regions, 13 provinces, 22 rural communes, 58 villages and 66% of the total number of micro projects financed across 10 different types. The analysis used direct and indirect financial costs, and direct benefits in monetary and quantifiable non-monetary terms. The analysis also assumed a discount rate of 10% for the benefit/cost analysis, and an inflation rate of 3.9%. Table 3 in the PAD summarized the results which indicated that the average expected estimated rate of return for boreholes, animal slaughtering, animal vaccination, compost pits, Page 13 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) stone fences and lowland agriculture (all potential activities for CBRD 3) was 59.8%. All of them except lowland agriculture showed rates of return well above the discount rate which remained so in the face of either a 20% increase in costs or a 20% reduction in revenues. Since most of the micro projects were still to be selected and hence had non-quantifiable benefits, a cost- effectiveness analysis was also performed. According to the PAD for CBRD 3 (Table 22), on average, CBRD 2 micro projects and infrastructures were built with lower unit costs than by other public ministries and government agencies in the country. It was assumed at appraisal that this relationship would continue in CBRD 3. At Project Close The ICR (Annex 4) summarized the ex-post economic and financial analyses (EFA), using a combination of data collected during the field visit by the ICR mission to a sample of micro projects in November 2018, information from the CBRD 3 completion report, and a micro project EFA report in December 2018 prepared by the project coordination unit, as well as available analyses prepared by CBRD 1-3 over the implementation period for those projects (2001-2018). CBRD 3 did not conduct an overall impact study as planned but, in its place, adopted the results of an impact study prepared by the University of Ouagadougou which had been arranged to assess the impact of the CBRD Program on rural household living conditions between 2004 and 2017. Given this collection of information the ICR provided a number of perspectives of this project’s efficiency. Financial Analysis. A financial analysis was completed for 13 types of micro projects assuming typical models for such projects (without describing the assumption on discount rate and expected lifespan for each type). The models represented environmental protection, productive investments, economic public goods, and social public goods. This analysis concluded that most of the models were financially sound; exceptions were retail shops (due to very low rent earnings) and bus stations (due to low fee collection rates). On the other hand, financial performance of social public goods such as the model for a borehole used for drinking water was very weak as the users saw access to water as a social right and mobilized contributions only to cover operations and maintenance. The ICR (Annex 4 Table 4.4) presents the results of the financial performance measures for each type of micro project. While the results presented were interesting, they provided a very thin analysis of the efficiency of micro projects (e.g. 1 ha models are very unlikely to be representative) and they provided no assessment of the results in a counterfactual (e.g. what were the financial rates of return without the environmental intervention of compost pits and stone bunds). Economic Analysis. For an analysis of the project’s ex-post economic return, the ICR explained that investments in productive goods in the private and public sectors generated benefits in the form of livelihoods, and social public goods (education, health, drinking water) for farm households. A multiplier factor of 1.6 was used (as a “best guess” without a clear rationale provided by the Bank task team) to estimate the indirect spillover benefits (secondary benefits such as income generation created by the project in local communities) for the public investments. For productive private investments and economic public goods, economic or shadow prices of investments, and labor inputs were used to calculate the economic incremental cash flows. The Bank task team provided IEG with the conversion factors used to obtain the shadow prices. For social public goods, proxies were used to quantify the derived economic benefits. For example, the ICR noted the extent to which the population was provided with access to potable drinking water had an impact on human health and hence on the number of productive working days in a year. For education, the difference between the opportunity cost of skilled and unskilled labor was used as a proxy for education benefits, as it was assumed that education gives people more opportunities in life and consequently higher revenues (ICR, para.40). These were general assumptions for the methodology but the actual data used were not stated in the ICR. While the original ICR provided no overall estimate of the project’s rate of return, following questions by IEG the estimated rates of Page 14 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) return and net present values for the base case and for scenarios assuming a 20% increase and decrease in benefits were provided by the Bank task team. The base case EIRR was 22% and for the two scenarios 25.2% for a 20% increase in benefits and 18.4% for a 20% reduction in benefits. Environmental Benefits. The benefits of climate change mitigation were calculated based on the same assumptions and project boundaries as the economic analysis. Using the Ex-Ante Carbon Balance Tool (EX- ACT) tool and Green House Gas (GHG) calculation was made to account for additional in GHG emission or its reduction resulting from the project’s investments of micro projects (Component 3). According to the ICR the results from the EX‐ACT analysis showed that the CBRD 3 contributed to a net carbon sink of ‐683,367 tons CO2 equivalent over 20 years, equal to −34,168 tons CO2 equivalent annually (ICR, para.51). While this review cannot validate these results, the sequestration of significant amounts of carbon attributable to this project is highly questionable because only 350,055 trees were planted and sustained on 1,300 ha and there was no reference in the ICR to the carbon sequestration on the areas where trees were not planted where the situation without the project would probably have been open grasslands with their own capacity for carbon sequestration. Following questions by IEG the Bank task team added that “The GHG calculation is based on the following five elements: (a) reforestation of 1,300 ha and recuperation of 144 ha of degraded land; (b) compost pits for 13,759 ha and stone bunds on 744 ha; (c) productivity increases in 486 ha of lowlands and 96.5 ha of vegetables, with a shift from traditional cultivation to improved agronomic practices; (d) additional fuel consumption due to marketing and transport of the additional production and development of local economic activities; (e) increased use of fertilizer and agro-chemicals in lowlands and horticulture; (e) a reduction of wood consumption by 30% due to the use of improved stoves by 4,300 households.” Nevertheless it was still not possible for IEG to validate the estimated total sequestration of 683,367 tons of CO2 by the project and it was also not clear that this sequestration was all attributable to the project. Cost-Effectiveness. The ICR (Annex 4, para.52) concluded that the unit costs of CBRD 3 investments when compared to those of other developed partners’ projects were competitive, and that the actual cost of reaching each beneficiary was low due to the mobilization of the communes. Nonetheless, the ICR did not provide any information on the basis for the unit cost analysis to support the cost comparisons. At the request of IEG, the Bank task team provided additional information on the unit cost comparison between ex-post CBRD 3 and the Agricultural Value Chains Promotion Project (PAPFA-Extension) in Burkina Faso developed by IFAD in 2019. The results showed that CBRD 3 incurred up to 50% lower cost per unit for investments in irrigation equipment with solar pump, 80% lower unit cost for lowland development, and 15% lower unit cost for storage facility of onions, than PAPFA-Extension. Again the comparability of costs of "irrigation equipment with solar pump" and "lowland development" were not evident in the information provided to IEG by the Bank task team. Conclusion. The analysis of the project’s rate of return at closing differed from the analysis at appraisal because it was more comprehensive. The estimate of financial and economic rate of return for mainly agricultural development micro projects showed positive results at appraisal. While there were positive economic and financial results which emerged from the project, there were significant shortcomings in the ex-post economic and financial analysis (EFA) in the original ICR. Following some questions by IEG the Bank task team provided IEG with an amended version of the EFA which provided additional information on assumptions made in the analysis and more detail on the results. Based on the information in the ICR and the additional information provided to IEG by the Bank task team the project’s efficiency for micro project investments (important elements of both Objectives 2 and 3) is rated Substantial. Efficiency Rating Page 15 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 66.00 Appraisal  59.80  Not Applicable 67.74 ICR Estimate  22.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of this project's objectives was rated modest because its aim of enhancing the capacity of rural communities and institutions, enhancing the promotion of land and natural resource management, and enhancing productive investments was undefined, vague, not sufficiently monitorable and of low ambition. In addition, the objective excluded the need for effective implementation of the Rural Land Tenure Law. The low ambition of the objective and its focus on “enhancement” without any explanation of how enhancement should be calibrated meant that the assessment of the efficacy of the project’s achievements required a judgement about the thresholds for a rating such as substantial enhancement. As explained in Section 4 this review assessed the efficacy of capacity building that enabled rural communities and decentralized institutions to implement their respective local development plans, namely Objective 1, as substantial. The efficacy of Objective 2, the enhancement of sustainable land and natural resource management and productive investments at the commune level, was rated negligible. On the other hand, the enhancement of productive commercial investments at the commune level together with the community-driven social public investments added up to a substantial achievement of Objective 3 despite the fact that investments were skewed towards the social investments. This was the result of community preference. The project’s overall efficacy was rated modest because of the major shortfall in efficacy of improved land and forest management and inadequate progress towards realizing the higher-level APL objectives over the course of a 15-year operation. The project's efficiency was rated substantial based on evidence validating positive economic and financial performance of the project at closing, notwithstanding a questionable calculation of the environmental benefits. In summary, the project recorded achievements in capacity building, institutional development, and social public investments. More crucially, however, the project failed to address or achieve in any substantive way its core challenge, namely the promotion of enhanced land and natural resource management which was a unifying thread for the project’s planned activities and the PDO. The project also failed to demonstrate any progress towards the APL program objectives of poverty reduction and sustainable rural development. Therefore, based on significant shortcomings in this project's relevance and achievements, this review concluded that the overall outcome of this project was Moderately Unsatisfactory. a. Outcome Rating Page 16 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) Moderately Unsatisfactory 7. Risk to Development Outcome The ICR (paras 91-93) identified three risks to development outcome as follows: Institutional Capacity Risk. Previous experiences of high turnover rate of the political appointees and government agents continue to be relevant in the versatile political dynamics of Burkina Faso that is expected to lead to frequent staff transfer. There is currently, however, no formal arrangement to ensure the sustainability of the technical capacity building activities after CBRD 3 (ICR, para.91). Nonetheless, the ICR noted recent efforts at the municipal level to increase staff stability and academic qualifications, and the enactment of new laws by the GoBF in January 2017 to facilitate the transfer of state personnel to territorial communities and to improve the salary/compensation structures to increase the attractiveness of the territorial positions. Infrastructural Risk. Given the low technical and financial capacity of the rural communities there is a substantial risk for the degradation of the constructed infrastructure due to lack of adequate maintenance (ICR para.92), and there are no provisions in the project for future maintenance. Financial Risk. As the micro project investments in productive assets and economic public goods were underachieved under CBRD 3, there is a concern over limited fiscal capacities of the communes and the insufficiency of resources for the financing of the next generation of LDPs/AIPs (ICR para.93). The following two additional risks are suggested by IEG: Institutional Support Risk. Although one of the implicit objectives of CBRD 3 was to prepare an exit strategy for the CBRD program, there is no plan to ensure the continuity of the collaboration between local communities and the municipal/regional planning and budgeting institutions. Considering that the implementation of the Rural Land Tenure Law and the NRM/SLFM activities was limited, there is a risk that these activities would not have been pursued in the remaining areas after the project closed which would further weaken the structural framework necessary to implement sustainable local development plans. Political stability and general security risk. With the resignation of the Prime Minister and the cabinet on January 19, 2019, the GoBF was dissolved. Although a new Prime Minister was appointed by presidential decree, a new cabinet was not appointed. Due to increasing Islamist insurgency and terrorist threats, a state of emergency had already been declared at the end of 2018 and the constitutional referendum scheduled for 24 March, 2019 was postponed indefinitely. The political and security situation remains fragile throughout the country with the next presidential election anticipated in October 2020. Conclusion. Given the reasonable concerns over the above-mentioned risks, the sustainability of the CBRD 3 institutional capacity and infrastructural investments after project closing is critically under threat. More importantly, considering that CBRD 3 is the last phase of the CBRD Program, it is highly unlikely that the overall higher-level Program objectives “to reduce poverty and promote sustainable development in the rural areas” will be achieved. Page 17 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) 8. Assessment of Bank Performance a. Quality-at-Entry  The preparation of CBRD 3 design was built on similar component structure, identical geographical coverage, known beneficiaries and implementing institutions, and functioning operational arrangement from CBRD 2 (with an exception for the SLFM activities in Component 4 which were delegated to OFINAP). This advantage facilitated and accelerated the project appraisal and effectiveness of CBRD 3. Nevertheless, as noted in Section 3 of this review, the objective for CBRD 3 was almost a replica of the objectives for the previous two phases and as a result was not designed to reach the overall higher level PDO for the three-phase program.  Preparation for this project considered previous experience with the Sahel Integrated Lowland Ecosystem Management (SILEM) project (2004-2010) which was funded by the GEF. The CBRD 3 served as follow-up to the SILEM project by extending the experience in Integrated Ecosystem Management to communes where capacity building for ecosystem management was essential and in expanding SLFM activities to additional areas formerly supported by SILEM.  The quality of entry was substantially affected by the newly developed Results Framework for CBRD 3 which contained new sets of indicators, as opposed to building on the RF for CBRD 2. In particular, a PDO-level indicator for assessing the extent of local development plans implementation that promote productive investments at commune level was missing, and several target indicators were incorrect, incomplete, and vague. In addition, the baselines for almost all indicators were (as noted in para 82 of the ICR) set to zero although they were supposed to be build on the achievements of earlier phases.  Despite the PDO focus on promoting NRM/SLFM and productive investments, the project design did not incorporate any incentive or control on the financing of micro projects that would guide the community choice of AIPs/LDPs towards these activities. Although the project team referred to the orientation and guidance in available project documents to emphasize community investments in productive and economic activities and NRM/SLFM, no formal binding condition was available in the Financing Agreement.  The allocation of small financial envelopes at the commune level was in practice considered inadequate to implement large infrastructure projects laid out in AIPs/LDPs (Stakeholder workshop 2018).  Despite apparent signs of political instability starting in 2011/2012, the project appraisal assessed country level risk as moderate, and the capacity of implementing agency risk was rated low although the weak capacity of the accounting and administrative staff was a concern at appraisal (PAD Table 2 p.17 and Table 6 p.44). Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision a. Quality of supervision Page 18 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688)  CBRD 3 was supervised by the same Task Team Leader as in CBRD 2 for the first 3.5 years before his retire close proximity to the implementing partners and beneficiaries.  The Project Steering Committee held all 10 bi-annual meetings to validate project reports and annual work pl  After the disruption of project implementation caused by the socio-political unrest, efforts were reinforced to q elected municipal councils, to assume their responsibility in the implementation of LDPs.  According to the project team, the Bank was extensively involved in encouraging the stakeholders to prioritize influence was limited and the considerable swing of micro projects towards public social projects was a reflec  The supervision was briefly interrupted due to the political unrest in 2014-2015 when the municipal councils w out, and the usual practice of twice a year was respected for the remaining project period.  With regard to supervision reporting, the results in the Aide Mémoires and the Implementation Status and Re or at times improbably ahead of the ISMs (ICR para.87). The ISRs successively rated the progress towards a project since the MTR. In addition, the rating of implementation and risk did not reflect the prevailing situation addressed relevant attention in a timely manner. The ICR (para.87) further noted that the ISR reporting did no ISR no.5. Most of the ISRs were primarily focused on disbursements with little attention to other implementa  The lack of proactivity was considered by the ICR (paras.67 and 88-89) as one of the major shortcomings of substantial implementation disruption. No clear efforts were made to adapt to the prevailing situation and the although initiating this action was clearly a responsibility of the supervision team. In reply to an IEG query on made after the MTR. However, the project’s performances had provided hope for the attainment of many of th MTR to extend the closing date but the team did not find it necessary to revise the results framework since th all other indicators were met and even exceeded some of the initial targets”. This justification from the Bank p essentially dismissed the MTR recommendations to realistically adapt the Results Framework to prevailing si activities, especially those related to the implementation of the new Rural Land Tenure law (Component 2). Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design  From the beginning, the project benefited from the operational and robust M&E system established under two earlier phases. Information was collected from two sources: M&E units at the commune level and PCU at the national level.  Instead of building on the existing Results Framework in CBRD 2, this project developed a new RF which proved to be weak with missing PDO level and output indicators, and incomplete, inaccurate and vague targeting. Shortfalls in the indicators of the project’s achievements for each of the three objectives defined in Section 2a of this review were as follows: o Objective 1 – The core outcome indicator target of beneficiaries was poorly set as it was overachieved by 200% despite difficult implementation conditions during 2014-2015; no gender-specific target was set for the percentage of female beneficiaries; no indicator was defined to evaluate the collaboration and communication sub-component. Page 19 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) o Objective 2 - Initial target setting for NRM was vague (as a range) or incorrect (overcounting of ecozones). o Objective 3 - In addition to the percentage of micro project investments, the project should have defined indicators that measured the improvement in income, employment, and fiscal revenue as a result of the project support in micro project investments.  Most of the indicators did not provide gender-disaggregated targets and the mechanisms to monitor and evaluate these indicators were limited to activities in Component 1.  The absence of baseline data was reported in the ICR (para.81, Annex 1 pp.29-30). In response to an IEG query, the Bank project team indicated the decision to develop and use the new Results Framework (RF) as the main reason for the absence of baseline data, particularly for activities in Components 1-4 which were given new sets of target indicators. However, despite the availability of data and relevance of the RF from CBRD 2, the project did not assess the prevailing level of each indicator and build on previous achievements at appraisal, instead the baselines were set at zero. This lack of rigor in establishing a realistic baseline consequently hindered the effective monitoring and evaluation of project progress and achievement of project objectives. b. M&E Implementation  The implementation at national and local level appeared to be efficient and the M&E was rated satisfactory throughout the project period. Implementation progress were regularly documented as quarterly and semi-annual reports, quarterly activity planning reports, and annual reports.  As one of the outputs in Component 5, 5 GEF tracking tools were updated (baseline 0; original target 4) in all four operational areas including land degradation, biodiversity, climate change, and sustainable forest management.  No CBRD 3 final impact study was carried out as initially planned (ICR para.25) due to the growing insecurity on the ground. As an alternative, the project team saw an opportunity to engage the University of Ouagadougou to conduct the impact study for CBRD 3 within the existing arrangement previously commissioned by the CBRD Program since 2003 to assess the impact of the CBRD Program on rural household living conditions between 2004 and 2017. However, the rural household survey took place in 2017, one year in advance of the project closing. In addition, as it was implemented under an umbrella of the CBRD Program, the impact study framework and objectives were not specifically tailored to CBRD 3 and the sampling methodology entailed shortcomings in aggregating panel rural household data across 6 surveys (2004, 2005, 2006, 2010, 2011, and 2017) which was a major limitation for impact evaluation of the entire CBRD Program intervention across the period 2004-2017 and undermined the attribution of CBRD 3 intervention.  Overall M&E implementation capacity was weak. At commune level, there were no designated M&E positions and financing for M&E activities was inadequate (ICR Annex 5 para.16), whereas shortage of M&E staff was also observed at the national level (ICR Annex 5 para.17). These shortcomings resulted in limited capacity to carry out M&E activities and field visits, and overburdened of existing staff to take on additional M&E tasks. c. M&E Utilization Page 20 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688)  In practice, information on the project progress including signals of concern from the M&E system was not utilized effectively to guide timely restructuring or making strategic decisions related to the project. Although signs of delay were evident from the project onset, no action was taken to review the RF until the MTR, and the RF revision at MTR was not followed up with a formal restructuring. M&E Quality Rating Modest 10. Other Issues a. Safeguards CBRD 3 was classified category B for environmental safeguards. Given the relatively small financial allocation to each rural commune, the diversity of micro projects and their distribution throughout the national territory, the risk of significant cumulative impacts was considered negligible (PAD para. 91). Four World Bank safeguard policies were triggered: (a) Environmental Assessment (OP 4.01); (b) Natural Habitats (OP 4.04); (c) Forests (OP 4.36); and Involuntary Resettlement (OP 4.12). In consultation with relevant groups of stakeholder, environmental and social impacts and mitigation measures were addressed in an Environmental and Social Management Framework (ESMF), and a Resettlement Policy Framework (RPF) including a Process Framework (PF) was prepared to provide guidance on resettlement matter including compensation and restoration of livelihoods. The ICR (para. 76) stated that mitigation actions and measures were adequately implemented and that the project was in compliance with all environmental and social safeguards. b. Fiduciary Compliance Financial management was satisfactory. The accounting system was in place and functioning during implementation, the financial reports were considered reliable, and all of financial audit reports were unqualified (100% target achievement of output Component 5). A procurement plan was prepared, updated, and approved annually and the procurement administrations received regular training to become proficient with the Bank’s procurement guidelines. Procurement challenges included inadequate specialized staff at the commune level, lengthy procurement procedures, non-compliance with procedures, low technical qualifications among local service providers, and long delays in processing payment for service providers. At project completion procurement practices had reached a satisfactory standard. c. Unintended impacts (Positive or Negative) Local Development. As mentioned above, the CBRD Program, in collaboration with the University of Ouagadougou, conducted a total of 6 rural household surveys (2004, 2005, 2006, 2010, 2011 and 2017) to assess the overall impact of the CBRD program on their living conditions. The temporal evaluation of key indicators shows a general improvement in economic and social conditions towards poverty reduction and Page 21 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) sustainable development as a result of the Program between 2004 and 2017 (ICR, Annex 5, Appendix 5.1) and the CBRD 3 between 2011-2017 (ICR, Annex 6, para.17-26). However, the survey design and sampling methodology revealed shortcomings in aggregating panel rural household data across 6 surveys (2004, 2005, 2006, 2010, 2011, and 2017) which undermined the attribution of CBRD 3 and Program interventions to the project/Program impact. d. Other Gender. In supporting the implementation of the National Gender Policy (2009), the project built on the achievements of CBRD 2 and integrated gender aspects in all of the project’s components. The ICR (para.58-59) reported positive results of the gender mainstreaming effort. For example, 274 certificates of rural land ownership were issued to women farmers of lowland plots developed for rice production in Douna in the Cascades region; 3,859 women were trained in the context of SLFM representing 79% of the total training beneficiaries. The micro project financing activities contributed to the improvement of women’s lives and their empowerment by reducing the burden of chores with access to equipment (5,455 improved stoves, 7,950 manure pits); giving access to improved basic social services (442 drinking water points, 24 maternity wards); increasing employment and income-generating capacity (27 vegetable gardens). Nonetheless, the project’s impact on gender empowerment was only partially observed due to inadequate M&E design (ICR, para.60). 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment The ICR (para.54-56) concluded that the combination of high relevance of objectives, modest efficacy and substantial efficiency amounted to an overall outcome rating of Moderately Satisfactory. This conclusion was not consistent with OPCS guidelines; the rating should have been Moderately Moderately Moderately Outcome Unsatisfactory. This review, Satisfactory Unsatisfactory having rated relevance of objectives, efficacy and efficiency respectively modest, modest and substantial concluded that the overall outcome of this project was Moderately Unsatisfactory because it failed to achieve the project's core challenge of enhanced land and natural Page 22 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) resource management and its program objective of sustainable rural development and poverty reduction in rural areas. Weak Results Framework at entry and lack of project mechanism to generate incentives to invest in NRM/SLFM and productive Moderately Bank Performance Unsatisfactory commercial investments. Unsatisfactory Supervision was undermined by inadequate ISRs and lack of proactivity especially with regard to formal restructuring following MTR decisions. Significant shortfalls of RF indicators led to disputable assessments of project Quality of M&E Substantial Modest achievements. Weak M&E implementation capacity and ineffective utilization of M&E. Quality of ICR --- Modest 12. Lessons The ICR proposed four lessons. The following two lessons were drawn from those in the ICR (para.94-97), with some changes in presentation: A demand-driven community development without a clear targeting strategy at entry and an effective incentive mechanism to attract participation runs a high risk of underachieving its objectives. In this project the example was that, despite a strong focus at project design on financing natural resource management and local development of income generating enterprises using a community driven development (CDD) model, in practice the communities preferred investing available funds in social public goods with immediate benefits rather than long term development programs. The lesson is that without a mechanism in the Financing Agreement that frames the direction of the demand for funding and establishes a threshold for each potential type of micro project, strong operational guidance, and effective communication to the beneficiaries, a CDD type project is likely to underachieve its development objectives. Sustainability and continuous enhancement of decentralized support and autonomous financing are essential to achieve sustainable development. In this project satisfactory results were achieved in building technical and institutional capacity for the implementation of local development plans, but sustaining that capacity in the longer term to achieve higher order objectives such as poverty reduction after the project closed could not be assured since there was no plan to continue collaboration between communities and municipal/regional planning. There is therefore a considerable risk that technical and institutional capacity and budgeting institutions cannot be maintained due to inadequate financing and that development projects already constructed will not Page 23 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) be maintained, infrastructure will deteriorate. Consequently, new development programs may divert local financing from maintenance to new operations, and hence the original projects will finally be abandoned. The lesson from this project is that the momentum of sustainable rural development can only be maintained with strenuous efforts to ensure long term local technical and financial resources and strong institutions to overcome the cycle of funding of new projects replacing previous projects. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain The concerns over sustainability in terms of institutional support, institutional capacity, infrastructure maintenance, and financial risk, could be critically reviewed with a field evaluation. Additional assessment is recommended to fill the evidence gap considering the long incubation horizon to measure the effects of institutional reforms and decentralization, including the implementation of Rural Land Tenure Law, on rural socioeconomic growth, poverty reduction, and the progress of NRM/SLFM in reducing vulnerability to climate change. As several investments came to completion only towards the end of CBRD 3, the merit of a CDD approach on local development and NRM/SLFM could be assessed more precisely. In addition, a comprehensive field evaluation of the CBRD Program (2000-2018) as a whole would contribute to evidence- based lessons for the Great Green Wall for the Sahara and Sahel Initiative (GGWSSI) and the World Bank/GEF Sahel and West Africa Program (SAWAP). 14. Comments on Quality of ICR The ICR presented key features of three phases in the CBRD Program and their cumulative achievements to provide context for CBRD. It also provided a logical overview of the project, a useful context at appraisal, and a sound summary of circumstances during implementation that affected the project's performance. The Theory of Change was well crafted and clearly explained for the CBRD Program and for CBRD 3. Actual achievements and challenges as a result of shortcomings in project design and implementation were assessed with candor and conciseness. Despite the absence of a final impact study, the ICR triangulated information from various sources including field visits of a micro project sample and a stakeholder workshop. Nonetheless, this review noted significant shortcomings in the ICR:  The ICR did not follow the OPCS Guideline for the overall outcome rating which concluded a Moderately Satisfactory from a combination of high relevance of the PDO, modest efficacy and substantial efficiency (ICR para.56), when the resultant rating should have been Moderately Unsatisfactory (OPCS 2018, Appendix H, p.38).  The ICR parsing of PDO modified the focus and scope of the formal PDO. Considering the PDO statement "…for the implementation of local development plans that promote sustainable land and natural resource management, and productive investments at commune level", the parsing in the ICR Page 24 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-APL 3 Com Based Rur Dev III (P129688) became "Rural communities fiscal income increased" and "Natural resources, forest & biodiversity protected/ restored". This interpretation led to a distorted assessment of efficacy of the original PDO.  The efficacy analysis primarily focused on "before-after" the project rather than "with and without" the project, making the attribution of outcomes to project interventions questionable.  The presentation of ex-post economic and financial analyses was very brief, leaving out the assessment at appraisal available in the PAD (para.78-81 and Annex 9), omitting key assumptions used, presenting doubtful estimates, and withholding sensitivity and cost-effectiveness analyses, leading to ambiguous interpretation of the project’s efficiency results. Although the presentation of achievements and economic and financial analysis (EFA) is based on data from all available sources, it lacked appropriate references throughout the document, undermining the validity and reliability of evidence. It is acknowledged that many of the shortcomings in the ICR such as the EFA were filled through an exchange of a considerable amount of information between IEG and the Bank task team - which contributed to improving the coverage substance of this review. This review rates the quality of the original ICR as Modest. a. Quality of ICR Rating Modest Page 25 of 25