Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) Report Number: ICRR0022814 1. Project Data Project ID Project Name P125669 Niger CAPCR Country Practice Area(Lead) Niger Environment, Natural Resources & the Blue Economy L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-63990,TF-11338,TF-11426,TF- 31-May-2021 74,684,532.58 A9610 Bank Approval Date Closing Date (Actual) 19-Jan-2012 31-May-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 72,597,000.00 72,597,000.00 Revised Commitment 72,597,000.00 70,384,799.87 Actual 70,384,799.87 70,384,799.87 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD) (p.vi) and the Financing Agreement of May 8, 2019 (p. 5) the objective of the project was “to improve the resilience of the populations and of production systems to climate change and variability in targeted Communes”. Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) According to the PAD (page 11) the selection of targeted communes was based on the following criteria: i) extent of poverty, ii) degree of exposure to climate risks, and iii) synergy with on-going or planned Bank projects. For the purpose of assessing this project’s achievements in Section 4 of this review the PDO will be parsed into two parts which will be referred to as Objectives 1 and 2, as follows: Objective 1: Improve the resilience of the populations to climate change and climate variability in targeted communes Objective 2: Improve the resilience of production systems to climate change and climate variability in targeted communes b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project included three components: Component.1: Mainstreaming climate resilience into development strategies at national and local levels (appraisal estimate US$5.0 million, AF US$1.56 million, actual US$2.85 million): This component included two sub-components: Sub-component 1.1: Mainstreaming climate variability and change into sectoral policies and local development planning: This sub-component was to provide technical assistance, goods and services to make the social and economic development policies in the health, water and road sectors more responsive to climate change. This was to include at the central level: a) the establishment of a multidisciplinary task force to ensure a participatory process of the policy formulation, b) the implementation of in-depth studies and assessments in order to identify potential climate scenarios over the next decade, c) the establishment of major tools for planning, monitoring and evaluation of climate-sensitive investments, and d) the organization of large public consultations. Activities on the local level were to include: i) the review, revision in order to incorporate climate sensitive issues, and validation of the guidelines for local development planning in collaboration with the General Directorate of Territorial Management and Community Development (DGAT/DC), ii) the provision of support to the targeted communes in the preparation or revision of their local development plans and their annual budgets in order to better integrate climate- sensitive initiatives, and iii) the provision of institutional support to the recipient’s National Agency for the Investments of Local Municipalities. Sub-component 1.2: Communication strategy and knowledge management: This sub-component was to provide technical assistance, goods and services for the purpose of defining and implementing a comprehensive communication strategy and a system of effective knowledge management, including Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) the organization of general sensitization, social mobilization and information initiatives, as well as specialized training sessions addressed to key stakeholders. When the project received Additional Financing in May 2019, a third sub-component was added: Sub-component 1.3: Integration of new technologies to the knowledge management system: this sub- component was to provide technical assistance, and services to integrate new technologies to the knowledge management system developed under sub-component 1.2 such as satellite imagery and other state-of-the-art technologies to help inform national and regional decision making. Component 2: Integrating climate resilience practices into agro-sylvo-pastoral systems and local populations’ social protection measures (appraisal estimate US$53.0 million, AF US$11.4 million actual US$66.37 million): This component included two sub-components: Sub-component 2.1: Improving the resilience of agro-sylvo-pastoral systems: The resilience of agro-sylvo- pastoral systems was to be improved by : a) scaling up sustainable land and water management practices to reduce climate change impacts through capacity building activities in the areas of agriculture, agro- forestry, agro-pastoralism and pastoralism, selected with a gender-sensitive approach, and with the aim of improving agricultural productivity, sustainable management of forest resources, and productivity of grazing areas; and b) providing Matching Grants to targeted communes to finance activities to improve agricultural productivity, sustainable management of forest resources and productivity of grazing areas, included in targeted communes’ approved local development plans and budgets. The AF was to continue supporting this sub-component and to leverage the successful implementation of SLM management interventions in the parent project, across the 38 target communes. Sub-component 2.2: Integrating innovative measures for protecting the most poor and vulnerable households through Adaptive Social Protection (ASP) actions: This sub-component was to finance: a) designing and establishing a system aimed at integrating into local planning innovative measures for the protection of the poorest and the most vulnerable households through flexible social protection actions; and b) providing matching Grants to Targeted Communes to finance social protection actions as part of the implementation of the system designed and established including: i) the protection and rehabilitation of socio-economic community facilities that would be vulnerable to the main effects of climate risks, ii) the implementation of a simple, realistic and well performing cash transfer system for the most vulnerable households, iii) the organization of remunerated seasonal labor intensive activities for the members of the poor households, and iv) the distribution of food stamps to chronically poor households. When the project received Additional Financing in May 2019, a third sub-component was added: Sub-component 2.3: Establishing and operationalizing integrated multi-services platforms in targeted communes (“maisons du paysan”): This sub-component was to establish and operate as integrated multi- service platforms providing agricultural advisory services to local communities. Component 3: Ensuring coordination of all the activities of the project, including monitoring and evaluation activities, and PSCR overall strategic coordination (appraisal estimate US$5.0 million, AF US$1.63 million actual US$5.46 million): Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) Sub-component 3.1: Ensuring coordination of the project activities: This was to include an efficient system of monitoring and evaluation aimed at helping decision making. Sub-component 3.2: Ensuring strategic coordination to the SPCR: This sub-component was to ensure strategic coordination to the Strategic Program for Climate Resistance (SPCR) as well as knowledge sharing concerning the project’s and SPCR’s approach, results, challenges and impacts at the national level with key stakeholders and at the international level with other countries participating in the pilot program for climate resilience. Under the AF, the activities under Component 3 were a continuation of the parent project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$77.59 million. Actual cost was US$74.68 million. Financing: The project was financed by a Bank Trust Fund (TF) in the amount of US$28.0 million (of which US$27.9 million was disbursed), a TF in the amount of US$35.0 million (of which US$34.9 million was disbursed), a TF in the amount of US$9.6 million (of which US$7.4 million was disbursed) and an IDA credit of US$5.0 million (of which US$4.29 million was disbursed). Borrower Contribution: The Borrower did not make a direct contribution to project financing. . Restructuring and Dates: The project was restructured four times:  On December 11, 2014 the project was restructured (level II) to: a) modify the two PDO indicators to allow for a better measurement of project activities; b) reallocate budgets between the disbursement categories; and c) to modify the institutional arrangements to establish a Project Coordination Unit (PCU) within the implementing agency. The PDO indicators were modified as follows: i) PDO indicator 1 was revised from “reduced cereal deficit in targeted agro systems within the area covered by the project” to “increased cereal production in targeted agro systems within the area covered by the project”; and ii) PDO indicator 2 was revised from “reduced forage deficit in targeted agro- pastoral systems within the area covered by the project” to “increased forage production in targeted agro-pastoral systems within the area covered by the project”.  On June 20, 2017 the project was restructured (level II) to extend the parent project’s closing date from June 30, 2017 to December 31, 2018 to allow for the preparation for the project's closing, prepare the Government's implementation completion report as well as a report on the progress of the overall Niger Strategic Program for Climate Resilience.  On December 10, 2018 the project was restructured (level III) to: a) reallocate budgets between the disbursement categories; b) amend the disbursement percentages for some disbursement categories; and c) extend the closing date of the parent project from December 31, 2018 to May 31, 2019 to allow for the implementation of activities under the Additional Financing.  On May 8, 2019 the project received Additional Financing in the amount of US$14.6 million to scale up ongoing activities and add some new activities that supported the consolidation of results of the parent project. Also, the project was restructured to: a) amend the disbursement percentages of some disbursement categories; b) extend the closing date of the parent project and associated Trust Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) Funds from May 31, 2019 to May 31, 2021 to allow for the implementation of the AF; and c) modify the Results Framework consistent with the AF’s increased scope. The project closed 47 months after the original closing date on its revised closing date of May 31, 2021. Split Rating. This ICRR does not perform a split validation since the level of ambition for the project was increased when the project received AF.. 3. Relevance of Objectives Rationale Context. Niger is one of the poorest and least developed countries in the world. According to the PAD (p. 2) in 2010, at the time of project appraisal, average Gross Domestic Product (GDP) per capita was US$360 and average life expectancy was 58 years. Agricultural production accounted for 40 percent of the GDP. Due to the degradation of natural resources, rapid population growth, and climatic changes in precipitation over 50 percent of the population suffered from food security. About 84 percent of Niger’s population were dependent on natural resources for their local livelihoods. Poor households, especially female headed households faced an increased risk to shocks and seasonal variations in agricultural production. Relevance to government policy. According to the PAD (p. 1) Niger was one of nine countries selected for the Pilot Program for Climate Resilience (PPCR) which was funded by the Climate Investment Funds and implemented by Multilateral Development Banks (MDBs). In 2010, the government of Niger developed the Strategic Program for Climate Resilience (SPCR) which aimed to increase food security by improving resilience of populations and production systems to climate change. This project was one of four complementary investment projects included in the SPCR. The project’s objective was in line with Niger’s most recent National Socio-Economic Development Plan (2017-2021) especially with the following priorities: i) sustainable management of land, water, and biodiversity, and ii) improvement of livelihoods of populations and climate change resilience. Furthermore, the project supported the implementation of the Nigeriens Nourish Nigeriens Initiative (3NI), which aimed to address chronic food shortages by introducing modern farming techniques and creating better access to production value chains. Relevance to World Bank assistance strategy. At project appraisal the objective of the project was also in line with the Bank’s Country Assistance Strategy (CAS) (2008-2011) which emphasized the importance of strengthening the role of decentralization and democratization of the society by empowering local institutions and communities. At project closing the objective of the project was consistent with the Bank’s most recent Country Partnership Framework (CPF) (FY18-FY22) which aims to increase rural production with diversified output in the agriculture and livestock sectors. Also, the project’s community driven approach to service delivery and development of community-based platforms was in line with the CPF’s citizen engagement approach. Finally, while the objective of the project was sufficiently clear, two PDO indicators “reduced cereal deficit in targeted agro systems within the area covered by the project” and “reduced forage deficit in targeted agro- Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) pastoral systems within the area covered by the project” were revised to better align them with the objective. The PAD did not include a definition for “resilience”. Given the above, the project’s relevance of objective is rated Substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Improving resilience of the populations to climate change and variability in targeted communes Rationale Theory of Change: The project’s theory of change envisioned that project outputs such as integrating climate change into sectoral strategies, strengthening institutions, and raising awareness, and providing information and specialized training were to result in the medium-term outcome building institutional capacity, which was to result in the overall objective of the project. Also, protecting and restoring of socio-economic facilities and establishing safety nets, cash transfers, food stamps, and wake works were to result in the medium-term outcome of adapting social protection for the most vulnerable households. Building institutional capacity Outputs (based on the ICR, Annex 1):  Three sectoral strategies that incorporated resilience to climate change were developed. The transport, water and health sectors were identified as key sectors to be adapted, achieving the target of three sector policies fully incorporating climate resilience. The ICR did not specify which policies were identified as providing key contributions to achieve resilience to climate change in each of these three sectors.  Resilience to climate change was integrated into Communal Development Plans (CDPs) and Annual Investment Plans (AIPs) for each of the 38 communes in the project area, achieving the target of 38 Local Development Plans (LDPs) and AIPs. According to the ICR (paragraph 48) CDPs were supported through allocating funds in the associated AIPs. At the national level, climate resilience indicators were integrated into the public investment system and the Strategic Coordination Unit within the Ministry of Planning, Territorial Development and Community Planning (MPATDC) which actively participates in national strategic planning activities.  60 knowledge products on climate change issues such as newsletters, brochures, publications, and other media were produced and disseminated, achieving the target of 60 knowledge products being disseminated. Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669)  Channels and/or institutionalized mechanisms allowing citizens to engage in actions on resilience to climate change through government agencies and other stakeholders were established, achieving the target. According to the ICR (p. 34) the institutional structure of the “Maison du Paysan” was proving to be a suitable platform for community engagement. A citizen engagement plan consolidated the existing grievance redress mechanism and community monitoring system into an engagement channel. Intermediate-Outcomes (based on the ICR, Annex 1):  3.5 million people were informed on climate issues through outreach activities, surpassing the original target of 800,000 people and achieving the revised target of 3.5 million people.  86,358 households benefitted from accumulated adaptive social protection (ASP) activities, surpassing the original target of 50,000 households and the revised target of 35,000 households. Key activities included: i) protection and rehabilitation of socio-economic infrastructure vulnerable to the main effects of the climate; and ii) organization of remunerated labor-intensive work for members of poor households. Improving resilience of local households: Outputs (based on the ICR, Annex 1):  Farmers adopted sustainable land and water management (SLWM) practices on 6,061 hectares of agricultural land, surpassing the original target of 3,000 hectares and the revised target of 5,000 hectares.  46,991 hectares of silvo-pastoral areas were improved with SLM, surpassing the original target of 10,000 hectares and the revised target of 38,000.  235 community facilities vulnerable to the main effects of climate change were protected or rehabilitated, surpassing the original target of 210 facilities and the revised target of 175 facilities. The ICR (p. 39) stated that the reduction of target facilities that was made when the project received AF might have been a mistake since at the time 221 facilities had already been protected.  Eight integrated services platforms known as “Maisons du Paysan” (MPs) were established and operational, not achieving the target of 15 platforms. When the project closed an additional five MPs had been constructed but were not yet operational. Furthermore, two MPs were under construction with completion rated of 77.5 percent (Gangara) and 81 percent (Chetimari). The MPS provide services including four sectoral elements (storage, agricultural inputs, fodder supplements, equipment rental and repair) and four transversal elements (training, communal radio, microfinance, administrative unit). Each MP developed a strategy to improve women’s access to its services. Outcomes (based on the ICR, Annex 1):  A 2021 beneficiary survey including 20 households showed that 77 percent of rural producers had fully adopted at least one SLWM technique introduced by the project, surpassing the original target of 60 percent and the revised target of 75 percent.  In total, the project benefitted 493,236 people, surpassing the original target of 180,000 and the revised target of 280,000. 48 percent of the beneficiaries were women, not achieving the original target of 60 percent nor the revised target of 40 percent. Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) Rating Substantial OBJECTIVE 2 Objective Improving resilience of production systems to climate change and climate variability in targeted communes Rationale Theory of Change: The project’s theory of change envisioned that project outputs such as i) implementing vegetative measures, ii) establishing passageways for livestock and iii) constructing water catchments, and managing watershed would result in the medium-term outcome of improving the resilience of agro-silvo pastoral activities. Outputs According to the Bank team in an exchange with IEG on January 31, 2022) quantitative outputs for this dimension were not defined at the PAD stage, but rather defined by community members during implementation. The ICR recorded the following outputs.  2.6 million plants planted  125,494 linear meters of irrigation network installed  90,426 linear meters of fencing constructed  21,765 kilometers of livestock corridors/tracks established  17,045 tons of improved seeds provided  15,882 tons of livestock feed produced  13,566 tons of fertilizer used  1,529 pumps acquired  1,196 boreholes for domestic water supply drilled Outcomes (based on the ICR, Annex 1)  Crop yields were 56 percent higher in the project’s areas of intervention than in control sites, surpassing the original target of 30 percent and the revised target of 48 percent.  Sampling of dried biomass among 625 farmers in 41 project and control sites showed that average forage yields in project sites were 32 percent higher than average forage yields in control sites, surpassing the original target of 20 percent and the revised target of 30 percent.  42,105 tons of animal food of various kinds per year were produced by the project, surpassing the target of 40,000 tons per year. Rating Substantial Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The overall achievement of the PDO is rated Substantial due to the project’s achievement regarding both parts of its objective. Under Objective 1 “improving the resilience of the populations” the project achieved that 77 percent of rural producers had fully adopted at least one SLWM technique introduced by the project. Under Objective 2 “improving the resilience of production systems” the project was able to achieve that crop yields were 56 percent higher in the project’s areas of intervention than in control sites as well as that average forage yields in project sites were 32 percent higher than average forage yields in control sites. Overall Efficacy Rating Substantial 5. Efficiency The PAD, the AF and the ICR each included a traditional economic analysis. The cost-benefit analysis in the PAD (p. 15) made the following assumptions: i) about 30 percent of the households of the project area would be targeted by the project activities in year 2, with an annual growth rate of 10 percent in the following years. ii) revenues from agriculture and livestock would initially increase by 15 percent, with a subsequent annual increase of 5 percent (over a period of 10 years); iii) revenues from outside agriculture and livestock would increase by 3 percent for all the households in the area covered by the project. The target was to be an annual increase of 3 percent each year over a period of 10 years. However, the analysis noted that the project’s results would be subject to various risks caused throughout the implementation phase (falling yield levels, technological adoption rates below expectation, and unfavorable changes in product prices, and the like). Applying a discount rate of 10 percent, the net present value of the project was estimated at US$5.8 million and its estimated Economic Internal Rate of Return (EIRR) was 14.20 percent. The cost-benefit ratio was 0.64. These measures indicated that the project was a worthwhile investment. When the project received Additional Financing (AF) another cost-benefit analysis of establishing the MP was conducted in the authorizing Project Paper assuming a project life of 20 years. The analysis resulted in an EIRR of 20 percent and a Net Present Value (NPV) of US$12.2 million assuming a discount rate of 6 percent.. The ICR (paragraphs 50-56 and Annex 4) also conducted a cost-benefit analysis resulting in a NPV of US$26.0 million (using a discount rate of 6 percent) and an EIRR of 14 percent. All these analyses indicated that the project was an economically efficient investment. The ICR noted that the results of the economic analysis in the PAD and the ICR were not comparable due to differences in the time horizons, discount rates, and types of costs and benefits included. The ICR states that in the appraisal report “The analysis considered the investment costs, without including any annual operation and maintenance (O&M) costs, and benefits in terms of expected increase in revenues from agriculture” (paragraph 51). On the other hand the ICR also states that, while the Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) general approach to the economic analysis of the project ex-ante and ex-post were the same, in the ex-post analysis in the ICR all investment costs associated with the project, annual costs related to SLWM investments and O&M costs of the MPs were included. It captures economic benefits of increased agricultural yields due to technical assistance, improved seeds, fertilizers. The ex-post analysis also takes account of additional forage yields as a result of project interventions in silvo-pastoral areas. For example, establishing pastoral half-moons, planting fodder crops, and controlling invasive species; and forest benefits due to reforestation are included as benefits. On the other hand, erosion control due to enhanced pasture could not be estimated in monetary terms due to lack of information. Hence the ex-post economic analysis “underestimates of the project’s economic net benefits” (paragraph 52). Of course the ex-ante analysis in the PAD missed including many benefits and it was therefore also an underestimate. A sensitivity analysis in the ICR showed that the project remains economically attractive even with higher discount rates, and lower adoption rates of conservation practices after the project’s close to the end of its assumed lifetime (Table 5). Operational efficiency: The project disbursed 95 percent of allocated resources. Actual disbursement for Component 1 was only at 58 percent of the appraised cost due to lack of capacity in the implementing agency. Actual expenditure for Components 2 and 3 were in line with the appraisal estimate. The project’s original implementation period of five years was extended by 47 months to allow for the scaling up of ongoing activities and adding some new activities which were financed through the AF as well as delays resulting from insecurity in several of the project intervention zones. According to the ICR (paragraph 77) the project’s coaching system in the Project Coordination Unit also improved the project’s implementation efficiency when operationalization of the concept of the MP was initially slow and resulted in some inefficiencies. Some aspects of the project’s design were overly complex to implement (such as establishing a cash transfer system and distributing food vouchers to vulnerable households as a means to build climate resilience) and was therefore abandoned at the first restructuring in 2014. Taking everything together, this review rates the project’s overall efficiency as Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  14.20  Not Applicable 100.00 ICR Estimate  14.00  Not Applicable Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the objective was rated Substantial given its alignment with the Bank’s Country Partnership Framework (CPF) (FY18-FY22) which aims to increase rural production with diversified output in the agriculture and livestock sectors. Both, efficacy and efficiency were rated Substantial. This review concludes that there were minor shortcomings in this project’s achievements and its overall outcome is therefore rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The risks to the development outcome were assessed in the ICR in paragraphs 94 to 99 and can be classified into the following broad categories: Government commitment: According to the Bank team (January 31, 2022) the government continues to be committed to the project’s objective. Sustainable land management remains the key priority for Niger which is enshrined in the country’s main development policy document [the Economic and Social Development Plan, (PDES)]. By international standards, sustainable land management and restoration of degraded lands (3.2 million hectares by 2030) Niger also has ambitious targets. Similarly, the establishment of the integrated service platforms for farmers (MPs) are enshrined in the PDES and the government continues to pursue its objective of establishing an MP in every commune in Niger. The government and the Bank continue their collaboration through a new Bank project, the Niger Integrated Landscape Management Project (P177043), which is currently being prepared. The project aims to support integrated landscape management in targeted areas in Niger and provides proposed financing of US$150.0 million. Economic: Even though the project achieved a high adoption of SLWM practices among farmers, there is a risk that some farmers might return to old techniques in case of low returns on investments or labor and capital constraints that cannot be mitigated. Financial: The project made physical improvements to community facilities which will require Operation and Maintenance (O&M) to ensure their sustainability and the delivery of services to the community. According to the advice to IEG by the Bank project team (January 31, 2022) the local communes are responsible for covering O&M for community infrastructures. As these are priority investments for the communes, fulfilling critical functions for community members, it is highly likely that the communes will provide a budget for O&M to the best of their ability. The project also supported the mainstreaming of climate resilience into local development plans and budgets. Technical capacity: While the project was able to build capacity at the national and communal level, it will be critical to continue to do so in order to ensure the sustainability of project outcomes. Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) 8. Assessment of Bank Performance a. Quality-at-Entry The project’s objective was well aligned with the government’s strategy and the Bank’s engagement strategy. The ICR noted that the project’s design was simple and built on lessons learned from previous Bank engagement in Niger such as: i) the importance of empowering local governments in development planning and management through decentralized support and mobilization of rural communities, ii) the attention to measures that aim at not only supporting agricultural production, but also focusing on fostering diversity of production and at reducing dependency on agriculture employment, and iii) the need to improve access to appropriate rural services such as for food storage as well as health and education services (paragraphs 67 to 69). According to the PAD (p. 68) the Bank team identified potential risks, which were assessed to be low to moderate since the project used existing structures of the Community Action Project (CAP) 2 for financial management, procurement, safeguards, and monitoring. However, the Bank team did not sufficiently mitigate the risk of weak coordination capacity within the Ministry of Planning due to an understaffed SPCR Coordination Unit. The ICR (paragraph 75) stated that this might have resulted in missed opportunities of collaboration taking advantage of synergies between the four SPCR projects. The project’s Results Framework in the PAD had several shortcomings such as the selection of PDO indicators and the setting of baselines (see section 9a for more details). Also, the M&E arrangements were complex and included monitoring activities at the central, regional, municipal, and community levels. Finally, while the project design was simple, the operational details at the sub-component level resulted in implementation challenges such as the plans to establish a cash transfer system and distribute food vouchers to vulnerable households as a means to build climate resilience. Also, the planned micro-scale of the community projects lacked an overall coordinated approach (ICR, paragraph 91). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (paragraph 92) the Bank team conducted 16 supervision mission during the project implementation period. In order to allow for a better measurement of project activities, the Bank restructured the project and the two PDO indicators in 2014. Also, the Bank restructured the project in 2019 and obtained AF to scale up ongoing activities and add new ones. The ICR (paragraph 78) also stated that the Bank team identified and addressed implementation bottlenecks fast and proactively. For example, when disbursement was low in 2015, the Bank team consolidated project activities and accelerated proposed community projects. Also, in 2016, after the rollout Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) of project activities across the intervention zone, the Bank intensified technical support for monthly field monitoring of implementation progress. According to the ICR (paragraph 80) the Bank team ensured that adequate transition arrangements were in place when the project closed. The seven MPs, which had not become operational by the time of project closure, were asked to develop a contingency plan in which they specified activities that could be supported by the commune, state, and technical and financial partners. Finally, the Bank team used the project’s M&E to inform decision making such as the need for project closing date extensions. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s theory of change and how key activities and outputs were to lead to the intended outcomes - in this case food security - was sound. The project’s objective, while complex was clearly specified However, the project’s Results Framework had shortcomings such as the original PDO indicators lacking set baselines on crop and forage deficits which did not allow for the assessment of outcomes. As a result of the revision of the PDO indicators, those indicators became sufficiently specific, relevant, measurable, and achievable. According to the PAD (paragraph 46) the M&E expert in the SPCR coordination unit was to coordinate the M&E for all the investment projects while the various agencies in charge of implementing specific components of the CAPCR were to ensure the timely monitoring of the implementation of each component of the project against the annual work plan cleared prior to November 30 of each year for the following year. During project preparation a detailed guide was developed which identified the monitoring and evaluation role of different stakeholders. b. M&E Implementation According to the ICR (paragraph 83) the project’s M&E included several stakeholders at different levels increasing transparency and accountability and promoting local ownership. Data on crop and forage yields were collected systematically on an annual basis in partnership with the National Center of Ecologic and Environmental Monitoring. The project collected data from sites that benefitted from project activities and control sites, which shared the same edaphic characteristics and had the same amount of rainfall. While farmers at the project sites received improved drought-resistant Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) seeds, mineral fertilization, Sustainable Land and Water Management (SLWM) techniques and agricultural extension, farmers at the control sites did not. This approach allowed for a yearly comparison between project and control sites. The Bank project team advised IEG (January 31, 2022) that the indicators included in the Results Framework were measured in a timely manner and reported regularly through comprehensive and multilevel implementation arrangements involving a broad range of stakeholders from the community to the central level. implementation arrangements involving a broad range of stakeholders from the community to the central level. According to the Bank project team (January 31, 2022) integrated M&E measures to run the MPs in the communes are likely to be sustained after project closing as means of daily operational management. Other aspects of M&E were specifically related to tracking project progress and in themselves did not offer further benefits to local, communal, and national stakeholders. These were built from national processes (i.e., the national agricultural survey) and reinforced the capacity of national actors to continue such activities in future. The Bank project team (January 31, 2022) also advised IEG that the PCU was dedicated to the sound implementation, collection, and reporting of M&E data. The Ministry of Agriculture and Livestock and the National Center of Ecologic and Environmental Monitoring were responsible for monitoring SLWM-related activities (and later yield sampling) and paid careful attention to sampling methods and selection of control sites. The National Institute for Statistics was responsible for monitoring household level project benefits and conducted carefully designed targeted surveys to measure the impact. Day-to-day tracking of implementation progress at the local level was, as noted earlier, supported by an innovative coaching system in the PCU that ensured effective M&E implementation, follow-up, and course correction of key issues for project outcomes. c. M&E Utilization According to the ICR (paragraph 85) M&E data were regularly used to inform decision making such as modifying the Results Framework and to extend the project’s closing dates. The implementation of a coaching system also resulted in the strengthening of utilization capacities. The ICR further stated paragraph 86) that M&E data were used to inform the evaluation of the SPCR and PPCR. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as category B and triggered the Bank’s safeguard OP/BP 4.01 (Environmental Assessment), OP/BP 4.36 (Forests), OP/BP 4.11 (Physical Cultural Resources), and OP/BP 4.12 Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) (Involuntary Resettlement). According to the ICR (p. 27) the project prepared all safeguard instruments and timely disclosed them. The ICR further stated that the project complied with all safeguard policies and implemented mitigation activities as planned. The Bank team (January 31, 2022) stated that the developed and implemented an Environmental and Social Management Framework and Resettlement Policy Framework. Activities were adequately reported and the project closed with a Satisfactory safeguards rating. Main activities that were implemented included: i) consultations with key stakeholders, citizen engagement, proactively engaging women; ii) screening and review of eligible sub-projects in accordance with the Niger Environmental Impact Assessment (EIA) regulation and categorization and the Bank policies, a check-list of mitigation measures for each sub-project type, EIA for each sub-project; and iii) a functional grievance address mechanism. However, in May 2019, the designated safeguard coordinator resigned which resulted in health and safety issues, issues related to the functioning of the Grievance Redress Mechanism (GRM) and relative lack of participation by women. The Bank addressed these issues by providing recommendations such as urgent recruitment of a safeguard specialist, re-dynamization of the GRM through training and budget allocation and strengthening of the participation of women especially in MP related activities. In January 2020, a new safeguard specialist revised the GRM manual and developed an action plan. The ICR (paragraph 89) stated that at project closure the project complied with all the safeguard policies. b. Fiduciary Compliance Financial Management: According to the Bank project team (January 31, 2022) the project complied with the Bank’s financial covenants. Furthermore, the review of financial reports consistently concluded that financial management was overall acceptable. Annual audit reports carried out in accordance with international standards were duly submitted with unqualified clean opinions every year. The PCU at the MPATDC level was responsible for the project’s financial management. According to the ICR (paragraph 90) the external auditor’s annual opinions were unqualified. Procurement: The project’s procurement functions were performed at the communal level. According to the ICR (paragraph 90) the PCU did not systematically monitor contracts awarded at the community level resulting in a high risk for noncompliant procurement. According to the Bank project team’s advice to IEG (January 31, 2022) the project complied with Bank procurement guidelines. The annual procurement assessment highlighted the risk of non-compliance due to Bank ex post-review of contracts granted, but no instances of non-compliance were recorded. While the project experienced slight implementation delays due to the complex national procurement process, the Bank team was able to forestall delays through supervision. Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) c. Unintended impacts (Positive or Negative) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Quality at Entry was rated Bank Performance Satisfactory Moderately Satisfactory moderately satisfactory by this review Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR (p. 29-30) included several lessons learned:  Building ownership among local stakeholders through a bottom-up and results-based management approach can positively impact project implementation and sustainability of project outcomes. This project allowed each commune to develop the integrated service platforms for farmers (Maisons du Paysan or MPs) to the specific needs of the community, which resulted in building ownership and engagement. During the Covid-19 pandemic this approach placed communes at the center of project implementation and helped to ensure implementation progress.  Introducing a coaching system designed such that a team of trained coaches work directly with staff responsible for implementation at the sub-component level can positively impact implementation. Through the introduction of a coaching system this project was able to strengthen implementation capacities while at the same time building a culture of results, transparency, accountability, collaboration and efficiency 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger CAPCR (P125669) The ICR provided an adequate overview of project preparation and implementation consistent with OPCS guidelines and included a simple but appropriate economic analysis. Furthermore, the ICR was sufficiently clear and outcome driven, internally consistent and provided useful lessons learned that could be applied to similar future Bank projects. However, the ICR provided very limited information in a number of critical areas including Financial Management and Procurement. For the purpose of completing this review these gaps in information were filled through an exchange between IEG and the Bank project team. Despite some shortcomings, the quality of the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 17 of 17