Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Report Number: ICRR0021659 1. Project Data Project ID Project Name P124648 BF:Mineral Development Support Project Country Practice Area(Lead) Burkina Faso Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H6930 30-Sep-2016 29,767,464.19 Bank Approval Date Closing Date (Actual) 21-Jun-2011 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 33,000,000.00 0.00 Revised Commitment 32,869,377.55 0.00 Actual 29,767,464.19 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hiroyuki Yokoi Victoria Alexeeva Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objective of the project is to strengthen the capacity of key institutions of the Recipient to manage its industrial mineral sector (Financing Agreement, page 5; Project Appraisal Document, page 7). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 02-Aug-2016 c. Will a split evaluation be undertaken? No d. Components Component A: Fostering Beneficial Mineral Resources Development (Appraisal cost: US$15.0 million, Actual cost: US$15.5 million) This component included three activities: (i) development and updating of policy, legal, and regulatory framework governing the mineral sector; (ii) development of geo-data infrastructure; (iii) development of mining cadaster through capacity building of the Office of Mineral Cadaster and Geology. Component B: Strengthening Institutional Sector Management, Coordination, and Accountability (Appraisal cost: US$15.7 million, Actual cost: US$10.62 million) This component involved four capacity building activities: (i) institutional strengthening of the key ministries, i.e. Ministry of Mines, Quarrying, and Energy (MMQE); Ministry of Environment and Sustainable Development (MESD); and Ministry of Finance and Economy (MoFE); (ii) tertiary education in Mineral Resource Management; (iii) Extractive Industries Transparency Initiative (EITI) - a global standard to promote the open and accountable management of oil, gas and mineral resources – post validation and mining information management system; (iv) assessments on economic integration, facilitation and knowledge sharing. Component C: Project Management (Appraisal cost: USD$2.3 million, Actual cost: US$2.8 million) This component included project coordination, fiduciary management, monitoring and evaluation, project reporting and audits. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The estimated project cost was US$33.00 million. The cost was revised to US$32.87 million, and the actual cost was US$29.77 million (ICR, p. 6). Financing: An IDA Technical Assistance (TA) grant covered all the project cost. Borrower Contribution: No recipient contribution was planned, and none materialized. Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Dates: The project was approved on August 2, 2011, and became effective on March 7, 2012. The project was restructured on August 2, 2016 (i) to extend the loan closing date for 21 months and (ii) to reallocate loan proceeds between disbursement (ICR para. 28). Upon restructuring, one PDO indicator, the average number of days to process a mineral exploration license application, was changed from 30 to 60 days in accordance with the reassessed baseline value during the project (ICR para 25 and 31). A split rating will not be applied as the target was updated in line with the improved data. The project closed on June 30, 2018. 3. Relevance of Objectives Rationale Country context: The objective of the project was in line with the government strategy. The mining sector has become a major contributor to the export commodities in Burkina Faso since 2007. The amount of gold produced tripled between 2008 and 2011, creating over 3,000 job opportunities, accounting for 2.8 percent of GDP. The government realized that sustaining the mining sector growth would require a new set of policies and strategies to keep attracting private sector investments and improved governance to enable shared prosperity through the mining industries to the general socio-economy. The national development strategy, the Accelerated Growth and Sustainable Development Strategy (SCADD) 2011-2015, emphasized the importance of the mining sector as a potential growth pole as well as of strengthening environmental oversight of the mining sector. The project was central to address the development challenge that the country faced. Alignment with strategy: The objective of the project remains aligned with the current Bank’s strategy. The Country Partnership Framework (CPF) (FY18-23) describes three focus areas: (i) accelerate sustainable private sector led-growth for job creation; (ii) invest in human capital and social protection systems; (iii) strengthen governance and support citizen engagement. The project was identified as supporting the first area, in which one of the five sub-objectives is given as addressing management of extractives and sustainability of natural resources. Previous sector experience: The Bank has executed a series of capacity building and advisory services in the mining sector since the 1990s in Burkina Faso. The Bank financed the Mining Sector Capacity Building and Environmental Management Project (1997-2004), which primarily aimed to promote private investment in the mining sector through capacity building. Competitiveness & Enterprise Development Project (2003-2013) supported capacity and policy issues in the mineral sector development. EITI MDTF also supported EITI implementation. The project was built upon the earlier engagements to develop the broader capacity of institutions around the mining sector. Rating Relevance TBL Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen the capacity of key institutions of the Recipient to manage its industrial mineral sector. Rationale Theory of Change The PDO was to strengthen the capacity of key institutions to manage its industrial mineral sector, which was underpinned by three areas: (i) transparent and sustained licensing, (ii) improved mining operations, and (iii) increased governance, economic integration and sustainability for the mining sector. The first areas entailed the development and revision of the policy, regulatory and legal framework; geo-data infrastructure; and mining cadaster. The second area involved institutional capacity building; tertiary education in mineral sector management. The third area covered transparency and accountability through EITI standard; and knowledge engagement on economic integration analysis. Outputs:  The project updated and modernized a mineral cadaster and made it publicly available as planned.  Twelve regulatory frameworks were developed and adopted against the target of six (Although ICR stated the project developed nine frameworks, the project team clarified that it is twelve). The frameworks included three laws, which are the revised mining code; the organization of the sale of gold and other precious metals; and the usage of explosives for civilian use), and nine decrees on managing organization, operation, and financing of the mining sector.  Neither manuals for mining inspection nor manuals for environmental inspections was drafted due to the delay in procurement of the consultants. National guidelines on mining community development were developed.  Geo-data database, which integrated geological, geophysical and geochemical information, was developed and made accessible to the public.  The number of exploration sites where Geology Office inspected and commissioned increased from 20 to 211, exceeding the target of 200.  The project developed (i) a national framework for local content to increase local supply to the sector and (ii) Ministry of Mines’ communication strategy, which was an outreach strategy to the general public on the mining industry.  While the total number of civil servants trained exceeded the target (1261 people against the target of 815), the number of female beneficiaries was only 56 percent against the target (184 women against the target of 326). The failure in reaching the target of the number of female beneficiaries was due to Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) (i) a small ratio of women to men in the hierarchy of the education system and (ii) the competition for well-paying jobs in the private sector.  The number of public oversight institutions, which monitored legal compliance of the key accountable governmental institutions, reached 285, exceeding the target of 250.  The number of people trained in the implementation of environmental legislation on mining operations reached 997, significantly exceeded the target of 500.  The number of advanced projects controlled financially (tax audits of mining companies) increased from 6 to 25, far beyond the target of 12.  The number of community members trained and consulted in the development of the Mining Community Development Framework reached 650, significantly exceeding the target of 450.  The annual graduation rate at the School of Mines of Fada surged from 5 percent to 99 percent, nearly reaching the target of 100 percent.  Artisanal and Small-Scale Mining (ASM) and mining communities sensitization and training were implemented 8,050 times, as compared to the target of 9,200.  Six EITI reports were produced. Outcome:  The targets for the average number of days to process mineral exploration and exploitation license applications were achieved. For the exploration license application, the average number of days reduced from 118 days to 57 days, exceeding the target of 60 days. For the exploitation license application, it reduced from 120 days to 90 days, in line with the target. These reductions in the number of days were attributed to the project activities such as the strengthened institutional and mining supervisory capacities, the modernization of the cadaster through the establishment of new procedures, and equipment including the software.  Inspections on industrial mining operations increased significantly but slightly below the targets. As for the environmental inspection, industrial mining operations covered by environmental inspection increased from 30 percent to 97.6 percent, slightly below the target of 100 percent. For the annual technical inspection, the percentage was increased from 30 percent to 82.14 percent, not reaching the target of 100 percent. It is noted that the project failed to develop the manuals on environmental and technical inspections; however, the project used its older manual while partially applying another manual that the World Bank supported in Guinea.  Industrial mines covered by annual tax controls was successfully increased from 30 percent to 100 percent. Because of the relevant policy measures such as the adoption of regulatory frameworks, cadaster information, geo-data infrastructure, the outcome of the objective was substantially achieved with minor shortcomings. Before the project, there were pending exploration licenses that lasted over 1,000 days and no environmental inspections of industrial mines were executed. Rating Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Substantial OVERALL EFF TBL OLD Rationale The overall efficacy rating is substantial given the substantial achievement of the PDO. Overall Efficacy Rating Substantial 5. Efficiency No attempt was made at appraisal or closure to measure efficiency for this technical assistance operation. The ICR discussed the efficiency of implementation, in terms of (i) operating costs that amounted to 10 percent of total expenditures of the project and were reported reasonable for similar mining TAs of this size and length; (ii) the rate of completion of procurements of goods and services; (iii) and disbursement that was delayed in the initial three years of implementation but then picked up after the change of a project coordinator. Economic and financial analysis was not applied to the project since the project aimed to develop capacities of the institutions which were not measurable in economic and financial values. Nevertheless, the project was considered to have brought economic and financial benefits such as through (i) the decreased opportunity cost for mineral exploration and exploitation license applications; (ii) the decreased opportunity cost for the private sector to access mineral cadaster and geo-data database; and (iii) increased access to capital through the Mining Fund for Local Development and a national framework for local content (ICR, para 60 and 72). Efficiency in the designs and implementation of the project was hindered by various factors. The delay in the first seven months was caused by the inability of the government to fulfill the covenants (ICR, para 64). The project had continued to observe the delay in the first three years, which was attributed to less qualified project coordinator and PIU; the lack of coordination between the beneficiary ministries; dual use of national and World Bank procurement procedures; as well as political uncertainties (ICR para 65, Restructuring Paper, page 2). While the project observed efficient operations in administering procurement and financial issues upon the change of a new project coordinator, the project closing date was extended for 21 months. On balance, the efficiency of the project is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the objective is high because the project was central to addressing challenges of the mining sector in Burkina Faso and tightly aligned with the Bank strategy. The project substantially strengthened the capacity of the country's key institutions to manage its industrial mineral sector as evidenced by the achievement of the outcomes. Efficiency is assessed as substantial. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Technical and Operational Risks: Given the importance of the mining sector to the national economy and export and its improved polity and institutional capacity, the regulations and infrastructure are likely to sustain. The follow-up works by the Geological Survey Office demonstrated the operationalization of the geological survey and inspection after the project (ICR, para 100). Some capacity building activities were well integrated into the business practice exemplified in the Revenue IT system of the ministry of finance, which would be the platform for the relevant officials to periodically use and improve trained skills. But given the variability of the capacity of the intuitions involved in the capacity building, the evidence such as budget commitment and continuity of training in each institution was not provided in the ICR and they could turn into potential risks to sustain the outcome and outputs of the project. Financial Risks: While there was no financial input by the client, the project team informed IEG of the fact that the Government contributed to project activities during implementation through an annual allocation of 140 million CFA ($300,000). After the closure of the project, the government maintained funding to sustain the activities on technical, environmental and fiscal inspections. Institutional Risks: The roles and responsibilities in the responsible ministries and institutions were clear enough to engage with the activities supported by the project, however the historical record of the project suggested that the operation in the mining sector was susceptible to the political environment. 8. Assessment of Bank Performance Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) a. Quality-at-Entry The project design was well informed by global and country experience. The project gained lessons from the Bank’s experience in the mining sector in the African region as well as to incorporate global lessons such as mining cadasters and guidelines for community development in the mining sector. Also, the EITI++ approach was instrumental in strengthening links between access to resources and the collection of taxes and royalties. The project partly underestimated the institutional capacity at the initiation of the project. The PAD described that the Project Implementation Unit was well functioned in the earlier engagement and assessed the likelihood of risks was low (PAD para 39). However, the ICR noted that the main problem at entry was the PIU’s staff capacity (ICR para 90). Although the project meant to improve institutional capacity of the concerned stakeholder including PIU, the risk mitigation measures could have been taken based on the earlier experience. Quality-at-Entry Rating Satisfactory b. Quality of supervision Bank’s supervision was considered to be satisfactory performed as reported by the ICR. The Bank supervised the project with detailed supervision Aide Memoirs and a total of eleven ISRs (ICR para 93). The Bank team identified the lack of coordination between the PIU and other relevant institutions was the cause of the malfunctioned project implementation (ICR para 95), which was not easily turned around despite the Bank’s continuous engagement with the PIU and consultations to higher authority. In 2016 upon the change of the political system, the project experienced a positive shift to get the project implemented with a new project coordinator (ICR, para 95 and 96). Although the exogenous factor may have caused the change, Bank’s continuous advice and supervision were catalytic to achieve the stated objective. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was detailed and comprehensive but lacked a systematic link between activities, indicators, and outcome. While most of the project activities were well documented and demonstrated a clear link to the outcome, some activities were not clearly documented in the results framework. For Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) example, the ICR explained that the project had association with the Mining Fund for Local Development, which was not documented in the results chain. Several dropped intermediate indicators could have been well linked to the results framework since the reasons to drop them had been apparent before the project initiation. For instance, the number of students graduating from a master’s degree was replaced into the annual graduation rate, which should have been informed at the preparation stage (ICR para 32). b. M&E Implementation M&E was satisfactorily implemented. The regular supervision mission allowed the stakeholder to recognize the performance of the project (para 69). The project also reassessed the accuracy of the baseline data, revising the PDO indicator on the average number to process and exploration permit application to reflect the actual processing days. c. M&E Utilization As documented in the ICR (para 67 and 69), M&E data was well utilized during the supervision mission, particularly at the earlier stage. The evidence-based discussion such as disbursement ratio and procurement was used to accelerate the project implementation. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Environmental Category B (partial assessment) and triggered the Bank’s safeguard policies on Environmental Assessment (OP/BP/GP 4.01). Given the nature of the technical assistance, the project did not directly cause environmental and social impact, while cumulative impacts after the project closure were considered and the Sectoral Environmental and Social Assessment (SESA) was conducted (para. 78). Out of the 18 recommendations provided under SESA, the government addressed 11 recommendations. According to the project team, the remaining recommendations would be addressed by the new mining technical assistance project by the World Bank. b. Fiduciary Compliance Financial Management Project financial management was satisfactory, as reported by the ICR. All the eight legal covenants were complied with; submission of acceptable financial reports including audits was compliant with the Bank’s Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) standard; record keeping was well managed for the entire project life. The supervision missions periodically reviewed the financial reports, validating the adequacy of the financial reports. Procurement From the beginning of the project until early 2016, the Bank had observed two issues on procurement: a poor performance by the initial project coordinator and the double application of World Bank’s procurement guidelines and national procurement rules, which resulted in procurement delays. However, after the new project coordinator was assigned in 2016, the client improved its performance and quality of the documents such as terms of reference. Since then, the rating was improved from moderately unsatisfactory to satisfactory rating. c. Unintended impacts (Positive or Negative) NA d. Other NA 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons A multi-faceted approach is critical to improving the public sector's capacity to keep up with the well-advanced private sector. The mining sector in Burkina Faso and other countries is driven by the private sector, while the capacity of supplying relevant infrastructure including policy, information, and human capital is often lacking. In Burkina Faso, policies and regulations were outdated; transparent and scientific geo information was not maintained; governments were not coordinated; limited human resources for both private and public sectors were available. The project took an approach to opt in the concerned elements that were linked to private sector engagement, Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) spanning from policy and regulations to tertiary education and international EITI mechanism. Although there still needs an improvement in reducing skill gaps for the private sector, the project was successful in engaging with the private sector. The success of the multi-faceted approach in the mining sector hinges upon the inter- institutional coordination. The project applied the EITI++ approach, which focused on access to resources, monitoring of operations, and collection of taxes and royalties. This approach required various coordination, particularly between the ministry of mines, finance, and environment. The project observed the lessons to establish an enabling coordination mechanism in (i) the capable project management unit; (ii) frequent supervision missions; (iii) authorized focal-point from the respective ministries. Engagement of multiple institutions needs a structure to prioritize the activities and assignment of the specific role with some incentives. The project involved various institutions such as the ministry of finance, line ministries and agencies concerned with the mining sector, schools and communities. To encourage effective coordination among the stakeholders, the project established a mechanism to prioritize the activities with the allocation of funding strategically. The focal point assigned in each institution helped the process. To incentivize the focal point to engage with the project, the government spent extra costs for the focal points even though it was not defined in the financing agreement. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a rich context of the project implementation with adequate evidence. The relevance section was well explained in terms of environmental, social, and economic challenges in the mining sector, but it could have provided evidence of the alignment of the PDO against the national context. The efficacy section provided detailed evidence including the rationale of revising the indicators. Some of the important activities which were provided in the later section, such as the Mining Fund for Local Development, were not explained in the ICR. The efficiency section was poorly described. Given the anticipated economic benefits through the project, such as the decreased opportunity cost of the license applications, the ICR should have attempted an economic analysis. Other sections were concise and well informed. While the ICR came up with such lessons as the importance of coordination and management of the PIU, some more evidence on how the team overcame difficulties other than political transition could have been usefully provided. a. Quality of ICR Rating Substantial Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF:Mineral Development Support Project (P124648) Page 12 of 12