ICRR 14274 Report Number : ICRR14274 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/19/2014 Country : Mali Project ID : P052402 Appraisal Actual Project Name : Gourma Biodiversity US$M ): Project Costs (US$M): 9.08 4.3 Conservation Project L/C Number : Loan/ US$M ): Loan /Credit (US$M): 5.5 5.36 Sector Board : Environment US$M): Cofinancing (US$M ): 1.8 0.66 Cofinanciers : French GEF - FFEM Board Approval Date : 09/02/2004 Closing Date : 06/30/2011 12/31/2012 Sector (s): General agriculture fishing and forestry sector (40%); Central government administration (25%); Other social services (20%); Sub-national government administration (15%) Theme (s): Biodiversity (40% - P); Environmental policies and institutions (20% - S); Rural non-farm income generation (20% - S); Participation and civic engagement (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : John Redwood Roy Gilbert Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: To assist the Recipient in implementing environmental protection and biodiversity conservation activities in the Sahelian priority area of Gourma, through improved sustainable management of natural resource use, improved returns for the local population (Grant Agreement, Schedule 2, pg. 24). In the project appraisal document (PAD) the objective formulation uses different language, namely : Biodiversity degradation trends are stopped and, in some cases, reversed in key conservation areas and project sites in the Gourma. (PAD, pg. 2) The ICR (para. 7, pg. 14) observes that the difference between these two versions "was a potential source of confusion" and that the project team -- and the ICR itself -- used the Grant Agreement version "as it deemed that this definition better captured the project development objective ." As per usual IEG practice for all ICR Reviews, the present review is based on the objective formulated in the project's legal document, in this case the Grant Agreement .. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components: 1. Support to Inter communal Management of Conservation Areas (appraisal cost - US$ 1.97 million; actual cost -- US$ 1.04 million). This component was intended to finance : (i) local capacity building, including : (a) communication; (b) creation of Conservation Area (CA) Management Organizations (OGACs); and (c) training; (ii) studies, including ecological and diagnostic and applied research; (iii) CA creation and management, including : (a) negotiation, delineation and gazetting of CAs and core zones; and (b) preparation of CA management plans; (iv) implementation of CA management plans, including: (a) construction and maintenance of small infrastructure (deepening existing ponds, watchtowers, dirt road repairs ); (b) surveillance of CAs; (c) use of CAs; and (d) local wildlife surveys. (PAD, pg. 11) 2. Support to Community-based Biodiversity Initiatives (appraisal cost - US$ 1.49 million; actual cost -- US$ 1.14 million). This component was intended to finance : (i) community awareness building; (ii) community training; (iii) complementary diagnostics on biological resources of communal land; (iv) specific studies to explore options for : (a) ecological restoration outside conservation areas; and (b) tourism development and private sector involvement; and (v) biodiversity-related local initiatives. Eligible local initiatives included: (a) protection of threatened biological resources; (b) improvement of natural pond, wetland, spring or watercourse management; (c) improvement of forest management; (d) improvement of pasture land management; (e) measures in favor of elephant conservation; (f) support to ecotourism development; (g) education on biodiversity; (h) support to conservation of cultural assets; (i) support to ecotourism development; and (j) support to sustainable use of secondary products . (PAD, pg. 11-12) 3. Capacity Building for Communal and Inter communal Biological Resources Management (appraisal cost -- US$ 3.12 million; actual cost -- 0.13 million). This component was intended to finance : (i) support to the Inter communal Orientation Committee (CIO), Communal Councils (CCs), and Centers for Communal Councils (CCCs) (animation, workshops, legal advice, travel ); (ii) contracting out design, creation and capacity building of the Inter communal Gourma Association (AIG); and (iii) studies and workshops, including : (a) study of rangeland users and tenure; (b) regional and local workshops on conservation and range use; (c) consolidation of communal development plans for territorial coherence as well as design and adoption of a territorial chart or equivalent; and (d) designing and testing an incentive-based mechanism for financing CAs in Gourma . 4. Capacity Building for Public Support to Decentralized Biodiversity Management (appraisal cost -- US$ 2.50 million; actual cost -- US$ 1.99 million) 4.1 Support to Project Operations in Gourma . This subcomponent was intended to finance : (i) support to the Nature Conservation Services (SCNs) operating in Gourma; (ii) support to other public deconcentrated institutions or NGOs; (iii) monitoring of the Environmental and Social Analysis as well as the Process Framework; (iv) contracting the Institute for Rural Economy (IER) to: (a) conduct external ecological monitoring; (b) monitor vegetative cover and land use; and (c) set up a local Geographical Information System (GIS) to monitor habitat and land use and produce maps; and (v) transfrontier coordination with Burkina Faso . 4.2 Support for National Supervision and Stocktaking . This subcomponent was intended to finance : (i) incremental support to the National Directorate for Nature Protection (DNCN), including: (a) operational support; (b) training and awareness building; and (c) documentation center and website; (ii) studies and workshops, including : (a) national reforms of biodiversity management; (b) transfer of natural resource management to Communes; (c) wildlife inventories; and (d) feasibility studies for establishment of, and fundraising for, a Malian foundation . (PAD, pg. 12) 4.3 Project Management. This subcomponent was intended to finance : (i) incremental support to steering and advising by the Steering Committee; (ii) support to the Project Management Unit (UGP) for procurement, financial management, planning and supervision; and (iii) audits. (PAD, pg. 12) d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost . Both the PAD and the ICR report project costs and financing inconsistently, leading to important errors. Annex 3 of the PAD indicates a total project cost of US$ 7.8 million, while the table on page 13 states that it is US$ 9.08 million and the ICR (Annex 1) uses the US$ 7.8 million figure. In addition, the paragraph on project economic analysis in the PAD (pg. 20) states that "the total project cost was "estimated at US$ 11.62 million, with a baseline of US$ 5.32 million co-financed by the Government of Mali, the 18 Communes and associated communities of Gourma, IDA (through implementation of other operations, AFD [the French bilateral aid agency ] -- actually resources from the French GEF, FFEM [Fonds Francais pour l'Environnement Mondial ] -- and UNCDF [the United Nations Capital Development Fund]." In contrast, the Financing Plan table at the front of the PAD, cites the US$ 9.08 total, including US$ 5.5 million from GEF, US$ 1.8 million from the Government of France, US$ 1.72 million from "Borrower/Recipient", and US$ 0.06 million from local communities. Financing . Actual financing according to the ICR (Annex 1) consisted of: (i) US$ 5.36 million from GEF (or 97.5 percent of the appraisal estimate ); (ii) US$ 1.72 million from the Borrower, (iii) US$ 0,66 million (36.7 percent of the appraisal estimate) from France; and (iv) US$ 35,000) from local communities. The low actual disbursement of FFE ) resources was due to its decision to suspend its contributions to the project in July 2009 following disagreement with the Bank/GEF over project implementation arrangements . This had three adverse effects : (i) creation of a financing gap that required the project team to re -prioritize activities; (ii) postponement of the closing date; and (iii) the intended synergies between the project and other GEF operations could not be harnessed . As concerns the latter, for example, "parallel and complementary GEF projects that were designed to construct the infrastructure identified by the Gourma project to deliver biodiversity benefits had either closed or were out of funding by the time the Gourma project identified activities. Faced with a long-term drought, the Gourma project was obligated to re -prioritize its activities to include construction of critical small -scale infrastructure [not originally included in the project ] to sustain its elephant population." (ICR, para. 19, pg. 18) Neither in the PAD nor the ICR does the financing table include other IDA operations or UNCDF, which, according to the economic analysis in the former (pg. 52), were expected to jointly account for the US$ 2.54 million difference between the US$ 11.62 million and US$ 9.08 million figures cited in different parts of the PAD (pg. 56). Furthermore, the ICR (para. 26, pg. 19) states that in April 2007, UNDP signed a contract with the Government of Mali, committing a grant of US$ 444,000 for the Gourma project, covering four communes in addition to the original 18 supported under the GEF grant. However, the ICR does not consider this additional source of funding to be part of the project, nor does this amount appear in the financing table in Annex 1 (pg. 37) Restructuring . A level two restructuring of the project occurred on March 22, 2011, with 12.2 percent of the GEF grant -- US$ 682,000 -- still to be disbursed. The project objective was not changed, but their were several modifications to the components and the KPI targets, as summarized below : Component 1. The scope was reduced to mainly focus on creation of conservation areas and the target number of such areas was reduced from seven to four, for which baseline data already existed and management plans were developed and implemented. Component 2 .The scope was reduced by removing activities planned to be financed by FFEM funds, specifically : (i) complementary diagnostics on biological resources of communal lands; (ii) specific studies to explore options for ecological restoration outside conservation areas; and (iii) tourism development and private sector involvement . Component 3. The scope was reduced to building the capacity of the International Orientation Committee (CIO), Community Council (CC).and the Center for Communal Councils (CCC) and to support the 18 project communes for the inclusion of biodiversity activities in communal development plans . Component 4. The scope of subcomponent 4.2 was reduced by removing activities planned to be financed by FFEM funds, specifically: (i) contract with the l'Institut d'Economie Rural (IER) to conduct external ecological monitoring, monitoring of vegetative cover and land use, and to set up and manage a local Geographic Information System (GIS) to monitor habitat and land use and produce maps; (ii) studies and workshops on national reforms of biodiversity management; (iii) wildlife inventories, and (iv) a study for the establishment and fundraising for a Malian foundation for biodiversity conservation Borrower Contribution . Upon completion, the Government's contribution was estimated to be the same amount (US$ 1.72 million) as anticipated at appraisal, but that by local communities was just 58 percent of the original estimate (US$ 35,000 in practice as compared with US$ 60,000 at appraisal). Dates. Dates The project closing date was extended by 18 months, from June 30, 2011 to December 31, 2012 in order to complete activities stalled by implementation delays caused by disagreements over project management arrangements between GEF and FFEM. 3. Relevance of Objectives & Design: a. Relevance of Objectives: The Gourma Region is one of the five priority areas for conservation highlighted in Mali's National Biodiversity Strategy and Action Plan (NBSAP), submitted to the United Nations Convention on Biodiversity and which committed to place 15 percent of the country's territory under protection by 2010. Project objectives were aligned with the most recent CAS (April 2011), which indicates that "the Bank will continue to help strengthen the resilience of agricultural and natural resource sectors with the preparation of a regional Bank /GEF project in support of the Great Green Wall initiative." (para. 16, pg. 4) The project team later clarified to IEG that this Initiative refers to a regional GEF project for that part of Africa south of the Sahara desert and north of the Sahel, extending from Ethiopia in the east to Senegal in the west, including parts of Mali in general and the project area in particular . Thus, the Gourma region continues to be a priority area for biodiversity conservation in the country, spreading over 3 million hectares between the Niger River and the border with Burkina Faso and possessing what the ICR describes (para. 2, pg. 13) as "diverse landscape features -- lakes, dunes, lowland forests and inselbergs [isolated hills that stand well above surrounding plains] -- and unique biological features such as the continent's northe rn most 350-strong elephant population." In addition, the area continues to be subject to serious environmental degradation, including extinction of local animal and plant populations and growing desertification . Relevance of project objectives is, thus, considered Substantial . b. Relevance of Design: The ICR (para. 49, pg 25) concludes that project design "appeared to be too optimistic" and that the Bank "overlooked the lack of technical capacity, the political risk, and regional security ." This will be considered in further detail in the section on Quality at Entry below . However, focusing here specifically on the likelihood that the project, as designed, would be able to achieve its development objectives (as stated in the Grant Agreement), in principle its components and subcomponents, had they been implemented as originally anticipated, could well have generated the project's intended outputs and outcomes as indicted in the "Project Design Summary" in Annex 1 of the PAD, which, was its de facto Results Framework. The expected component outcomes -- e.g., Conservation Areas (CAs) are created and their management is efficient, municipalities have improved their capacities to plan for and manage biological resources in their own development programs, the Gourma communes are organized themselves to better plan and organize management of their land and biological resources, and public institutions are able to advise and help communes and communities in the management of biological resources -- if achieved, seem likely to lead to achievement of the project's development objective with respect to the implementation of environmental protection and biodiversity conservation and improved sustainable natural resource management in the project area . However, it is less specific as to how the project would achieve its objective of generating improved returns for the local beneficiary populations. Relevance of project design is nonetheless rated Substantial . 4. Achievement of Objectives (Efficacy): Assist the Recipient in implementing environmental protection and biodiversity conservation activities in the Sahelian priority area in Gourma through (i) improved sustainable management of natural resources . 2,751 km2 of the Gourma region was formally demarcated for biodiversity conservation, compared with a target of 2,200 km2, four communal and intercommunal conservation area organizations (OGAGs) were created (of a target of 4) and an equivalent number of management plans was developed and implemented . In addition, 23 communes (including those financed by the UNDP grant ) included biodiversity actions in their economic, social, and cultural development plans (versus a target of 20) and the Inter-communal Gourma Association (AIG) was established and made operational. There were 7,335 direct beneficiaries (of which 47 percent were female) of biodiversity micro-enterprises, as opposed to a target of 881 (of which 13 percent were to be female).The one intermediate result indicator that was not met concerns the number of development and management plans for state implementation developed, for which only one of the four targeted plans were achieved by the end of 2012. Most of the above results had been achieved before the project was formally restructured . However, the ICR states (Annex 2, Table 5, pg. 44) that, due to the lack of monitoring data, the actual percentage of achievement for most of the original project indicators is unavailable . In particular, information is only available for one of the seven KPIs contained in the PAD (pg. 3 and Annex 1) -- four conservation areas have acquired legal status and possess a management plan by year 6 (achieved). The PAD (pg. 4) suggested that the appraisal team intended the two first original KPIs to constitute the primary outcome indicators, by stating that "achieving outcomes set by the Project Development Objectives will be verified by (1) the percentage improvement of key wildlife -based abundance indicators within conservation areas : red-fronted gazelles, dorcas gazelles, Guinea fowl in terrestrial areas, raptors in inselbergs and crown cranes and /or brown pelicans in wetlands; (2) the percentage improvement of an indicator of vegetation cover and composition in conservation areas (e.g., incidence of perennial grass ); (3) the weighted average of percentage improvement of bio -indicators at local initiatives sites outside conservation areas; and (4) the reduction of human-elephant conflict events. The fact that none of this information was collected means that it is not possible to determine whether or not these outcomes were achieved, but given the declining security situation in the project area, it is unlikely that they were . If anything human-elephant conflict events have most likely increased, and other wildlife has become increasingly threatened as well, thus offsetting any positive results of the project in this regard. In addition, the reduction in scope of the project at the time of restructuring, and especially the decrease in the number of protected areas to be supported from 7 to 4, among other revisions (as described in section 2d above) also needs to be taken into account . The final project cost table indicates that only 4 percent of the resources allocated at appraisal for component 3 (capacity building for communal and intercommunal biological resources management) and just 52 percent of those for component 1 (support to intercommunal management of conservation areas) were actually used for these purposes during project implementation . Accordingly, project achievements in relation to the original scope and KPIs are rated Modest . The ICR (para. 54, pp. 26-27) reports improvements in natural resource management under the project: all of the 22 rural communes in the Gourma area [i.e., the 18 supported by the project and the 4 additional ones subsequently supported by UNDP] integrated biodiversity and natural resources management in their local and social development plans. In addition, communes have displayed the ability to seek and evaluate various partners to support natural resources in their communes." The ICR also observes (para. 55),however, that "due to the insecurity it was not possible to verify the above benefits in the field ." With only limited demonstrated project results in terms of the "sustainable" management of natural resource use in the project area efficacy in relation to the first objective, even after restructuring, is rated Modest . Assist the Recipient in implementing environmental protection and biodiversity conservation activities in the Sahelian priority area in Gourma through (ii) ii) improved returns for the local Population . Here too the ICR provides some evidence for a positive result, affirming (paras. 56-57, pg. 27) but without providing specifics, that "many households have improved their incomes through micro -projects on biodiversity development and management ." It also argues that improved incomes could be expected from small -scale irrigation, gardening, cattle and sheep fattening, food product processing, fuelwood, seedling production and fishing, but does provide any data in this regard. Given the serious drought and insecurity situations in the project region, however, it is unclear how substantial (and/or sustainable) any of these improved returns really are . Likely results with respect to this objective are, therefore, also rated Modest . 5. Efficiency: The PAD (pg. 20 and Annex 4) presented an incremental cost analysis, which considers additional benefits -- such as land use planning and biodiversity conservation from the earliest stage of municipal development in the present case -- generated by GEF-financed activities over a non -GEF baseline that does not include such actions, which is required for all GEF projects. However, the discussion of Efficiency in the ICR focuses only on project outcomes -- i.e., what it refers to as "social benefits" in terms of improved natural resource management, improved well -being of the population, and enhanced local capacity -- and not on the use of project resources per se . The corresponding annex (No. 3) consists only of a two page, two column table listing what are identified as domestic and global benefits after stating economic and financial analysis "not applicable." Thus, there is no available information regarding how efficiently project resources were utilized in practice . Furthermore, the cost data reported in the ICR are inconsistent with those presented in the PAD , leaving uncertainty about inputs for the efficiency assessment . Thus, project Efficiency is rated Low. Low ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Considering that the relevance of project objectives and design are rated Substantial , but efficiency is rated Low and efficacy of all three of its objectives Modest , and the ICR fails to provide information that would justify a higher rating, overall, project outcome is rated Moderately Unsatisfactory . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Given the precarious security situation in the project region, the risk to the project's development outcomes are, indeed, considerable. According to the ICR (para. 67, pp. 29030), as a result of this situation, "all project activities had to be suspended in the field, the project area was occupied by military groups, [and] associations, staff and community members could no longer function, as local officials had fled the project area to southern cities in Mali or to neighboring countries such as Burkina Faso, Mauritania, and Niger ." It also states that "it was impossible to predict the fate of project achievements such as equipment, hydraulic infrastructure, and the protected elephants in the areas occupied by rebels ." In all likelihood, these, together with local populations, have all been placed at risk even thought the ICR also indicates, more positively (paras. 45-46, pp. 24-25) that "the Government of Mali intends to continue with the Gourma biodiversity program " and that "in addition to UNDP, Japan and Denmark have expressed an interest to bilaterally contribute to future operations ." However, as long as the insecurity continues in the region, it is hard to imagine effective actions being implement to safeguard local biodiversity, especially the "continent's northernmost 350-strong elephant population." a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: In retrospect, with the breakdown of relations between the GEF and FFEM, project design proved to be too ambitious and subsequently had to be scaled back . The ICR (paras. 48-49, pg 25) recognizes this, observing that, even though project design was consistent with all the pertinent Biodiversity and Desertification Convention priorities and guidelines for biodiversity conservation operations in semi -arid regions, it was "too optimistic." It also observed that the Bank underestimated the lack of technical capacity, the political risk, and the regional insecurity situation. Nor did project design foresee that the crucial water infrastructure that success of the Gourma project would rely on would not be built by other GEF operations if the Gourma project did not identify the need for such infrastructure on time . This disconnect, in fact, occurred although it probably could not have been anticipated during preparation and appraisal, but had to be corrected at the time of the MTR by adding such investments to component 1 at the time the project was restructured . The ICR does not clarify what it meant specifically by lack of technical capacity, political risk and regional insecurity . The project team later clarified to IEG that the latter referred to a rebellion in the project area that had occurred in 2002, when the project was under preparation, and, in retrospect, was a clear forewarning of the more serious political and security problems that would interrupt project implementation nearly a decade later, while the former referred to the weak operational capacity in the country, whose implications for subsequent project implementation were also seriously underestimated by the appraisal team . The ICR (para. 68, pg. 30) also reported that, even though project design had been widely discussed with key stakeholders, including the communes, and both international and local NGOs, its proposed coverage of 18 communes, was "too optimistic." This reportedly occurred at the insistence of the French Embassy and not the Bank, which would have preferred limiting the project area to 9 communes The Bank nonetheless went along with this broader scope despite recognizing that it "overstretched" the design, ultimately also affecting both implementation and supervision quality . Elsewhere, the ICR (para 77, pg. 35) states that "risks to project outcome like (a) lack of government capacity; (b) lack of counterpart funding; and (c) complications in financial arrangements with FFEM could have been anticipated during the project design stage, " suggesting that this was also a quality at entry shortcoming on the Bank's part . at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: Supervision was carried out on a regular and timely basis, although due to the initial implementation delays the mid-term review was postponed by two years from March 2007 to March 2009. The ICR attributes at least part of this to "a misunderstanding" between the Bank and FFEM that caused a delay in the recruitment of Field Implementing Unit (FIU) staff," which also affected the recruitment of disbursement, procurement, and financial management specialists. There were, in fact, serious problems during supervision with FFEM, including a disagreement over the NGO to be responsible for implementation in the field, which delayed the project for more than a year. The Bank ultimately had its way in this regard, but the result was that FFEM subsequently "demonstrated a lack of interest and commitment by abstaining from supervision missions and delays of up to six months in providing No Objections to TORs, until finally suspending contributions to the Gourma project " (ICR, para. 18, pp. 17-18). The Bank nevertheless bears some responsibility for the breakdown in relations between the two key partners, which ultimately forced the need for the project restructuring . The project team was reportedly in "constant" contact with Bank management, FFEM, UNDP and the International Union for Nature Conservation (IUCN), the Government and the PIU and the supervision team allegedly took "decisive drastic actions, and carefully handled the financial issue with FFEM by working with Bank management and the Government to reallocate GEF funds to components that directly contributed to project objectives." In addition, during the period of insecurity, project supervision continued even though the Bank suspended non-emergency operations in the country, although it is not clear if this included supervision visits to the project area, as elsewhere (ICR, para. 35, pp. 22-23),as the ICR states that, "due to the lack of security in the project area, the project team was relocated to Bamako in March 2012 [and] all project activities were suspended [with] field staff, associates, and community members [fleeing] to southern cities in Mali or took refuge in neighboring Burkina Faso, Mauritania, and Niger ." On balance, the Bank could have done more to avoid and /or reduce implementation delays and the breakdown in relations with FFEM that ultimately required the project to be restructured and reduced in scope . The Bank also should not have taken two years following the mid -term review to have formally restructured the project, which only occurred in March 2011 when the GEF grant was already 88 percent disbursed. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government's performance was considered "moderately satisfactory" by the ICR (para. 72, pg. 32) because of delays in: (i) recruitment of financial management and procurement specialists; (ii) conducting feasibility studies and training; and (iii) delays in providing counterpart funds, although some of these delays were reportedly partly due to the complications in the project's financial arrangements between the three parties -- the Government, GEF [i.e., the Bank], and FFEM. As concerns the counterpart funding delays more specifically, security expenditures apparently deviated funds from the project for other uses . While the deteriorating security situation can not be attributed to the Government, it did play a role in the other negative performance factors, which, in turn, contributed to the implementation delays . Collectively, these shortcomings are sufficiently serious to justify a less positive rating regarding Government performance Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: Despite the deteriorating security situation, the ICR reports (para. 75, pg. 34) that the PIU did an excellent job in coordinating the project, with the project staff even putting "their own lives at risk during project implementation and especially during field missions ." However, the ICR does not provide specifics about the actual quality of IA performance other than to say that "the delays in the recruitment of specialists and UNDP coming on board in 2007 (two years into implementation) led to an overload [of] work for the project technical staff " and "despite the above constraints and the acute rise of extreme insecurity at the project ending period, the PIU did the best it could to coordinate the project, to evacuate field staff and promptly re -locate (sic) its base to Bamako following the political instability." Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The l'Institut d'Economie Rural (IER) was contracted to carry out M&E for the project, including ecological monitoring, tracking vegetative cover and land use, and setting up a local GIS to monitor habitat and land use and produce relevant maps. However, the original outcome and output indicators later proved to be impossible to monitor and needed to be revised at the time of project restructuring . The ICR (para. 9, pg. 15) states, more specifically, that this occurred because there was "no M&E capacity to track project performance against the original key performance indicators" with less than two years remaining for project implementation . Rating Modest . b. M&E Implementation: Lack of commitment from IER resulted in no M&E data being collected until 2010, when the project team disengaged with this Institute . In part as a result most of the Key Performance Indicators (KPIs) had to be revised at the time the project was restructured, also reflecting the reduced scope of the project . Rating Negligible c. M&E Utilization: The ICR (para. 39, pg. 23) does not report on M&E utilization, observing only that the team "was unable to evaluate the performance of the M&E system due to technical complications at the PIU, and had to rely on a limited number of interviews to verify the project's results as reported by the project team since it was unable to visit the project site for a comprehensive evaluation due to a lack of security ." Given the lack of any evidence to the contrary, it is likely, however, that this was Negligible . M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: For safeguards purposes, the project was classified in Category B and the environmental assessment (EA), natural habitats and involuntary resettlement policies were triggered . In addition, because small-scale water infrastructure works were added to the project when it was restructured, an Environmental Impact Assessment (EIA) for the development of bore holes and a water pond was carried out in July 2011. The Bank's EA policy had been triggered initially, according to the ICR (para. 39, pp. 23-24), because "although the project sought to conserve and restore biodiversity that had been greatly threatened by overhunting and overgrazing, and expected to yield (i) increased conservation area with associated rise in vegetation cover and wildlife populations, including the Gourma elephants; and (ii) improved management of grazing areas with consequential decrease in land degradation, there was a possibility that an increase in quality of grazing land could increase the number of livestock and attract herders from other areas which would (i) reverse positive trends established by the project, and (ii) increase degradation of local vegetation in lowland acacia forests from long sojourns of elephant populations ." These potential risks reportedly did not materialize during implementation. However, the ICR does not report on actual safeguard compliance . The involuntary resettlement policy was triggered even though resettlement was part of project design because of the possibility that collective management of communal lands could restrict access of certain users to these resources and, more specifically, "could cause a discrepancy between the benefits reaped by farmers and herders and the will of the powerful traditional elite" (ICR., para. 42, pg. 24). The ICR does not indicate, however, whether this risk occurred in practice, stating only that "a framework process was prepared and cleared [and] local biodiversity micro-projects outside the protected areas did not require land acquisition, and consequently no Resettlement Plan or Policy Framework was required or prepared ." b. Fiduciary Compliance: The ICR does not specifically report on fiduciary compliance, including the status of financial reporting and audits, even though it contains a section entitled "Safeguard and Fiduciary Compliance ." This is a key shortcoming of the document. Elsewhere (para. 72, pg. 32), however, the ICR states that there were delays in hiring financial management and procurement specialists, and that some of the implementation delays were partly due to complications in the project's financial arrangements between the Government, GEF and FEEM . c. Unintended Impacts (positive or negative): No unintended impacts were reported in the ICR . d. Other: One noteworthy, although presumably intended, outcome of the project, according to the ICR (para. 61, pg. 28) was improved trans-border cooperation for natural resource management between Mali and Burkina Faso, which has reportedly also "facilitated integration of Malian refugees into local communities in Burkina Faso ." The ICR likewise observes (para. 62, pg. 28) that the Malian Government had "officially integrated biodiversity and conservation of natural resources into its legislation and authorized local governments to create and manage their own protected areas," but it is not clear how the project led to this positive result although in a subsequent conversation with IEG, the project team stated that because of the project's good reputation in Bamako, the Government decided to strengthen its national legislation regarding natural resource management and conservation . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately As the GEF grant was 88 percent Satisfactory Unsatisfactory disbursed at the time the project was formally restructured (March 2011), project achievements must be measured primarily in relation to the original KPIs, only one of which was clearly achieved. The ICR did not provide sufficient information to judge project Efficiency. Risk to Development High High Outcome : Bank Performance : Moderately Moderately There were clear quality at entry Satisfactory Unsatisfactory problems as well as shortcoming during supervision, both in terms of safeguarding its relations with a key co-financier, FFEM, and its delays in responding to project implementation delays and other problems, as well as by only formally restructuring the project two years after the mid-term review, which was also delayed by two years because of the initial implementation delays. Borrower Performance : Moderately Moderately The Government was partly Satisfactory Unsatisfactory responsible for the initial implementation delays and delayed releasing counterpart funds because of the deteriorating security situation . Procurement and financial management capacity of the PIU also suffered, at least initially, as a result of the Government's delay in recruiting the appropriate specialists. Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Several key lessons are of relevance for other operations : 1. A project should not be designed in such a way that achievement of its development objectives depends on the outputs of other operations -- in this specific case the provision of essential water infrastructure in the semi -arid project region -- a risk that should be anticipated and addressed in project design . 2. Although it could have been difficult to predict exogenous risks to project outcomes such as the emerging insecurity in the project area, risks to project outcome such like lack of government institutional capacity and complications in project financial arrangements among partners -- in this specific case between the Bank /GEF and FFEM -- could have been anticipated during the design stage . The associated lesson is the need to pay much more careful attention to these critical preconditions for smooth project implementation and likelihood of achieving its development objectives during preparation, appraisal, and supervision . 3. In a situation where a project is co -financed, prior agreement regarding and formalization of the responsibilities of each donor is essential to avoid ambiguities and delays -- not to mention a total breakdown in relations, as occurred in the present project -- during implementation. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR lacks a specific discussion of project fiduciary aspects and compliance of the project . In addition, its project cost data are inconsistent with those in the PAD, and the ICR has weaknesses both in terms of its reporting on M&E (due in part to security concerns which did not allow the team to visit the project area ) and safeguards (i.e., by not discussing whether potential risks in terms of restricting access of certain local stakeholders to resources in project conservation areas, as well as the possibility of "elite capture" of certain project benefits, materialized in practice or not, although in a subsequent discussion with IEG, the project team affirmed that these risks did not materialize ). Beyond this, the Efficiency section of the ICR does not discuss how project resources were used, but instead provides information that is more relevant to the discussion of Efficacy (e.g., improved natural resource management, improved well-being of the population, and enhanced local capacity ). Even though the ICR was hampered by the inability of the team to visit the project area as a result of the continuing poor security situation, it could have done a better job in covering the various topics to be discussed in an ex -post assessment of this sort, as suggested in the observations immediately above . a.Quality of ICR Rating : Unsatisfactory