Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) Report Number : ICRR0021185 1. Project Data Project ID Project Name P081567 BF-Ag. Diversification & Market Dvt. Country Practice Area(Lead) Additional Financing Burkina Faso Agriculture P147978 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-41950,IDA-54750 31-Mar-2013 84,500,000.00 Bank Approval Date Closing Date (Actual) 20-Jun-2006 30-Jun-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 66,000,000.00 0.00 Revised Commitment 115,984,654.02 0.00 Actual 114,958,854.82 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ebru Karamete Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The project development objectives stated in the Financing Agreement (page 5) and Project Appraisal Document (page 4) is: "to increase the competitiveness of selected agricultural sub-sectors that target national, sub- regional and international markets thereby contributing to shared agricultural growth in Burkina Faso". b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project had three components: 1. Improvement of agro-sylvo-pastoral supply chains performance (Appraisal Estimate: US$38.1 million, Revised: US$64.1 million, Actual: US$70.4 million). This component included capacity building for professional and agricultural trade organizations so that they can respond to market opportunities and requirements. It also included investments to improve the productivity of targeted value chains (mango, onion, poultry, and livestock-meat). 2. Development of irrigation and market infrastructure (Appraisal Estimate: US$34.3 million, Revised: US$61.2 million, Actual: US$44.8 million). With the goal to improve productivity in the targeted supply chains, through availability of productive and marketing infrastructure, and implementation of private-led operation of market infrastructure, this component financed two sub-components: the development of an irrigation infrastructure and construction of market infrastructure. 3. Improvement of the business environment, regulatory framework and provision of advisory services. (Appraisal Estimate: US$13.0 million, Revised: US$24.3 million, Actual: US$29.6 million). This component was to improve the business environment to make it more attractive to local and foreign investors, through development of enabling legal and regulatory framework. It also included the capacity development activities for public and private service providers serving the targeted value chains. In addition, the component comprised financing of project coordination and management (equipment and other operating costs, monitoring and evaluation, communication). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Total project cost at appraisal was estimated at US$ 83.6 million, then it was revised to US$ 149.5 million with the Additional Financing in 2014, as project scope and coverage increased from 9 regions to all 13 regions in the country. Actual costs were slightly lower with US$ 144.8 million (97 % of the planned amount). Financing: The IDA grant (IDA-41950) of US$ 65.3 million disbursed US$ 65.2 million and IDA Additional Financing grant (IDA-54750) of US$ 46.0 million disbursed US$ 44.8 million and 1.2 million was canceled at closing. IMF provided US$ 9.9 million, all of which were disbursed. It was planned that beneficiaries would provide US$ 14.1 million, again all of which was disbursed. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) Borrower Contribution: At appraisal, the Borrower planned to contribute US$ 4 million; this was increased to US$9.5 million, but the actual contribution at closing was US$ 6.2 million. Dates: The project was approved on June 20th, 2006 and effective six months later, on December 11th, 2006. The original closing date was March 31, 2013 and the actual closing date was four years and three months later (a total of 51 months’ extension with the Additional Financing), on June 30th, 2017. The closing date of the original project was extended for 23 months to end-February 2015 via three Level II restructurings in January 2012, March 2014, and September 2014. The third extension was approved in September 2014 to allow the completion of the construction of the market infrastructure, which was delayed by the rainy season. Delays in establishing both the irrigation and market infrastructure were the main causes for the three extensions of closing date under the Initial Financing. Then the Additional Financing was approved on June 13, 2014 and became effective on October 9, 2014. The fourth restructuring (February 19, 2016) extended the closing of the Additional Financing for one year (June 30, 2016 to June 30, 2017) in order to facilitate the completion of remaining activities before the project’s closing. The ICR noted that (page 12) project activities were also delayed due to the socio political turmoil in the country that erupted in 2015, and negatively impacted the country for many months afterwards. Restructuring: The project went through four restructurings, and one additional financing. The first restructuring (approved January 13, 2012) extended the original closing date from March 31, 2013 to March 31, 2014; the second restructuring extended that closing date to September 30, 2014; and the third extension (September 24, 2014) extended the closing date to February 28, 2015. Then the Additional Financing was approved on June 13, 2014 and became effective on October 9, 2014. The fourth restructuring (February 19, 2016) extended the closing of the Additional Financing for one year (June 30, 2016 to June 30, 2017). Project scope was increased and outcome targets were increased via Additional Financing, however, no split rating is conducted, as PDO and outcome indicators were not revised, and the outcome targets increased in general. 3. Relevance of Objectives Rationale The original project development objectives were highly relevant to the country, region and sector strategies and needs. Agriculture is the prevailing sector, providing livelihood for 86% of the population, generating 80% of exports, and accounting for 40% of gross national product (GNP). Although, declining over 1998-2003, poverty remained high (46.4%), with 94% of the poor living in rural areas. The high economic growth (on average around 5 % annually) is mainly achieved through high annual growth in the agricultural sector (7.8 %) (ICR page 6). Therefore, improving the contribution of the private sector-led agricultural sector to economic growth and poverty reduction had become the focus of Government’s policies and strategies over the last few years (Policy Letter of Decentralized Rural Development -2002, Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) Rural development Strategy – 2003, Strategy for Accelerated Growth and Sustainable Development -2011- 2016). Given these policies and strategies, agricultural diversification, (moving from the dominant sub- sectors such as cotton, cereals, and livestock) was considered a growth opportunity to reduce the vulnerability of the agricultural sector making it sustainable, stable, and able to provide outreach to rural areas. The issues that needed to be addressed included an inadequate policy and institutional framework and poor services for supporting production and trade; poor infrastructure and high cost of public services and utilities; limited capacity in the public and private sectors; and weak producer associations. The objectives were aligned with Burkina Faso’s Rural Development Strategy objectives, which were to: promote sustainable agricultural development, improve food security, and increase rural incomes. The project was relevant to the World Bank strategies. The World Bank’s Country Assistance Strategy’s (CAS -2005) objectives included: increased regional integration, expanded and diversified export earnings, increased economic opportunities for women, and reduced risk and increased revenues for rural households, all of which were linked to the project. The project objectives were in line with the Country Partnership Strategy CPS (2013-2016) at closing, specifically outcome 1.3: reduced infrastructure deficits (particularly in large and small-scale irrigation) and more effective value chains. In addition, the 2017 Specific Country Diagnostic (CSD) set priorities that the project had made its own during implementation, which included reducing the gender bias against women; building inclusive and transparent institutions (for which the project contributed in establishing the IPOs); and promoting competition and private sector development to stimulate efficiency and innovation. The project objectives were also relevant to the objectives of the Bank’s Africa Action Plan: identifying the drivers of growth, developing the private sector in Africa, encouraging exports, promoting regional trade integration, expanding irrigation perimeters, and supporting sustainable land management. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective “To increase the competitiveness of selected agricultural sub-sectors that target national, sub-regional and international markets thereby contributing to shared agricultural growth in Burkina Faso ". Based on this statement, this Review considers two objectives (i) increase the competitiveness of selected agricultural sub-sectors that target national, sub-regional and international markets (ii) contributing to shared agricultural growth in Burkina Faso. (i) Increase the competitiveness of selected agricultural sub-sectors that target national, sub-regional and international markets. Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) Rationale (i) Increase the competitiveness of selected agricultural sub-sectors that target national, sub- regional and international markets Outputs: Outputs apply to both objectives: Infrastructure: • Hectares of irrigated land financed by the project was 2,647 ha (less than the revised target of 4,400 ha - 60 % of the target). • Market infrastructures were not completed by project closing. Only the Koudougou onion market platform (Comptoir in French) and 20 vaccination posts had been completed and were in operation at the close of the project. Other incomplete ones were: livestock (6 market platforms, at 75% completion); onion (4 market platforms, at 25% completion); poultry (2 market/dressing platform, at 60% and 85% completion); mango (3 market platforms, at 40%, 85%, and 95% completion (no targets were provided by the ICR). Capacity Building: • 5 Inter-professional organizations (IPO) were established, achieving the target, to deliver services to their members and help them better respond to market requirements in terms of product quality and quantity. • A pool of providers was established (three individual consultants for each region) to support the establishment of 3,350 (Micro Projects) and 133 (SMEs). They were also expected to provide advisory support (technical, financial and accounting management) to promoters. The ICR did not provide any targets on this. • The project entered into financial partnership agreements with the two partner banks, Ecobank and Coris Bank, that benefited from a credit line for technical assistance from Danish International • Development Agency (DANIDA), with the Guarantee Fund of the U.S. Agency for International Development (USAID and with Banque Régionale de Solidarité); while collaboration continued with Societé financière de guarantie bancaire (SOFIGIB). A partnership Agreement was signed with the Maison de l'entreprise du Burkina Faso (MEBF) as a specialized service for advising SMEs. MEBF acted like a business incubator, providing training and advice to microprojects, including in accounting, contracting, building partnerships. The project also promoted cluster enterprises to facilitate access to credit and markets. • The Project provided Technical Assistance (TA) to the Permanent Secretary of the Sectoral Coordination of Agricultural Policy (SP/CPSA) in formulating the framework for the implementation of the reform measures in the agricultural sector by: (i) establishing a fund for agricultural development, (ii) establishing a program to support the promotion of SMEs in the rural sector mechanism; (iii) establishing a comprehensive financial instrument that incorporates the National Climate and Environmental Response Fund; (iv) establishing an agricultural insurance Fund; (v) creating a marketing structure and stabilization of local products for export; (vi) developing and adopting a code for specific investments in the agricultural sector; (vii) developing and implementing a national system of extension and advisory support, taking into account the specificities of the different departments and the involvement of different stakeholder groups; (viii) developing a national policy on water projects; (ix) establishing a fund for the development of agricultural entrepreneurship; and (x) strengthening the monitoring and evaluation system of the PNSR, particularly on monitoring of reform measures in the rural sector . Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) Mango: • 1,200 mango orchards were planted using improved techniques (plowing, drip irrigation, high density planting). Mango producers adopted a wide range of improved agricultural practices (i.e., pruning, and grafting of varieties in demand). No targets were provided by the ICR. • 353 producers were certified. Timely treatment of orchards against fruit flies led to improved mango quality, helped by producers acquiring GLOBALGAP certification. No target was provided by the ICR. • Post-harvest technologies were implemented. These include: (i) upgrading of the pack houses for fresh mango export (Bobo Dioulasso fruit terminal and Ranch Koba), and improving dried mango processing technology through the use of tunnel drying technique (ii) application of phyto-sanitary treatments that reduced production losses. • The project strengthened the capacity of the mango inter-professional organizations (IPOs). No concrete examples were provided by the ICR. • The project facilitated critical partnerships between value chain operators and foreign investors/clients through networking at commercial fairs and tours. No details were given by the ICR. Sesame: • Capacity building activities for producers were provided to apply treatment against the salmonella bacteria; and improved coordination, achieved by establishing the sesame inter professional organization (IPO). • Onion (No targets were provided by the ICR on these outputs): • Good agricultural practices (irrigation techniques and improved varieties) were supported; • Farm-level storage units (ruudu) were financed; • A strong onion IPO was established and Koudougou (Centre West Region) onion market platform, coordinated by the IPO became operational. Livestock: • Improved technologies among project beneficiaries were disseminated. These included technical reference kits such as artificial insemination from introduced genitors to improve breeds and access to finances as well. No targets were provided by the ICR. • Increased production generated by the funding of (about 170) of micro-projects (over 60% of total micro- project funding) by stakeholders who applied these practices. Such project support originated from the change in course at the MTR, away from focusing on meat exports to promoting animal fattening for regional exports of live animals. • In total 2,899 micro projects (less than the target of 3,350 projects) were successfully implemented. The ICR did not give the product breakdown of these micro projects. Over 1,000 micro-projects had to be called off because they had failed to meet a trigger milestone for subsequent tranches of financing. • A total of 177 enterprises were converted to micro and small and medium enterprises through project support (exceeding the target of 150). Outcomes: The theory of change (TOC) presented in the ICR (page 7-8), had shortcomings. Firstly, the TOC did not sufficiently define the “competitiveness” concept, and identify the outputs and intermediate outcomes that Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) would serve achieving the objective: “to increase the competitiveness of selected agricultural sub-sectors”. We can infer the elements of ‘competitiveness’ from the outlined theory of change, but these are not easily aligned with the critical assumptions in how they align with the project’s outcomes. IEG believes that in addition to exports, improved yield, product quality, effective partnerships are all outcomes that would be useful in triangulating the project achievements and thus, be able to measure competitiveness of agricultural products. Another shortcoming is that although the TOC mentioned linking small farmers to markets as the core of the project, the ICR did not include any discussion or results regarding the size of the farms when presenting the achievements of the project. Therefore, it is not clear what the size of the farms were that were served by the project. This is an important factor given the ICRs explanation of the theory of change. The ICR reported (page 14) that improved agricultural practices and post-harvest technologies supported by the project played a role in improved mango quality and increased production amongst project beneficiaries. Also, the ICR reported (page 15 and 51) that as a result of improved techniques and adoption of improved agricultural practices, a yield increase by 113% was achieved (from 7 tons per ha at baseline in 2006 to 15.2 tons at closing in 2017 slightly exceeding the 15 tons target). However, it is important to note that the planned irrigation investment, that would also contribute to the yield increase was not fully completed (only 60% of the planned amount was finished by closing). “The project team added that for mangoes, irrigated orchards had not started producing yet. Although the actual productivity increases were mainly due to maintenance practices and crucial activities to fight fruit flies, these are expected to improve further once the irrigation facilities become fully operational”. For onion, the increased yield was 104% (17 tons per ha in 2006 to 22.08 tons in closing slightly exceeding the target of 22 tons), reportedly as a result of good agricultural practices as well as use of storages. The project team also added that although irrigation investments for onion were not complete at the time of project closing, that the yield increase had been mostly due to new varieties introduced as well as new cultural practices. Increased quality reportedly achieved from providing sorting/cleaning of onion for storage and at market platform; and improved coordination along the value chain improved, through establishment of a strong onion IPO; the operation of the Koudougou (Centre West Region) onion market platform. Increase in livestock productivity (cattle weight gain increased from 500 at baseline to 667 at closing exceeding the target of 600) achieved by the adoption of and dissemination improved technologies among project beneficiaries. In terms of exports: • International exports of selected products (mango and sesame) increased from the baseline of 6,000 tons to 226,130 tons at closing (exceeding the revised target of 106,500 tons), and 94 % of exports were sesame and 6 % was mango. International exports for mangoes increased from 3,500 tons in 2007 to 7,663 tons in 2016. . • Regional exports of sesame, onion, mango, livestock, and cowpeas increased from a total of 17,000 tons in 2006 to 121,993 tons at closing (exceeding the revised target of 96,000). The project team provided the disaggregated figures: regional exports for mangoes increased from 2,679 tons in 2007 to 3,105 tons in 2016, for onions from 3,000 tons to 8,105 tons, sesame increased from 11,354 to to 32,566 in 2016. On the other hand, exports of cowpeas and livestock declined from 12,722 to 4,233 tons and from 12,417 tons to 2,684 tons during this period. In addition, according to the disaggregated figures provided by the project team, regional exports for cowpeas and livestock in fact declined significantly. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) The ICR raised issues of attribution (page 15) mentioning that the evidence supporting the results chain of international exports is more compelling for mango than for sesame exports, yet sesame dominated the total international exports (94% of quantity of exports); in addition, other concurrent operations overlapped with the project and likely contributed to achieve some of the project’s results (the team noted eventually that only in 4 regions). A similar argument was made for regional exports. Evidence for the regional exports of mango, onion and livestock can be linked to the project’s outputs and activities. Regional exports were also dominated by sesame (64%), whereas the share in regional exports in tons of mango, onion and livestock were 6%, 16% and 5% respectively. Similarly, other concurrent operations overlapped with the project and likely contributed to the achievement of some of the project’s results. Based on the additional information provided by the project team, the project support on better agricultural practices and post-harvest technologies, contributed to increased competitiveness for mangoes, and onions as evidenced by increased yields and exports throughout project duration. However, the irrigation investments planned under the project that would contribute to even higher yields were not complete and 40 percent were incomplete by project closing. In addition, although for livestock the yields increased as targeted, regional exports declined substantially as well as for cowpeas suggesting that other factors were at play in preventing improvements in the competitiveness of the sector. The actual project achievements and the associated evidence as it aligns to the theory of change and to the PDO are insufficient to confidently argue that considerable progress was made in the improvement of the sector’s competitiveness. The achievement of this objective is therefore rated Modest. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective ii) Contributing to shared agricultural growth in Burkina Faso Rationale Outcomes: 78% of beneficiaries experienced an increased income by at least 50%, exceeding the target of 60% of beneficiaries. The increase in income resulted from several factors related to production, post-harvest handling, and marketing. The 2014 survey revealed that about 122,000 beneficiaries adopted project sponsored technologies (no targets were given), which yielded an increase of five-fold in the average value Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) of production of the onion and 1.6-fold in the livestock/meat value chain (no specific targets were provided). The use of project-financed farm-level storage units in the onion value chain clearly contributed to this increase. These units allowed farmers to store bags of onions for 6 months after harvest. They could then sell the stored onions at ten times the price they would have been able to get at harvest (i.e., 60,000 CFA vs. 6,000 CFA for a 100kg bag). As a result, the average value of production of onion per project beneficiary rose from 2.5 million XOF to 13 million XOF, with the help of yield increase from 6.5 tons/ha to 17 tons/ha. Similarly, the average value of production per beneficiary increased from 6.2 million XOF to 10 million XOF for farmers in the livestock/meat value chain who had access to micro-project financing. It is important to note that over 60% was allocated to the livestock/meat value chain. On the contrary, the ICR reported that (page17) the value of the production of mango and poultry did not increase as much as those of onions and livestock because the unavailability of market infrastructure for these chains. The project team reported that (May 21, 2018) a stratified random sample by product groups were chosen for the survey (612 beneficiaries). The ICR noted too that (page 17) the impact study used the double difference evaluation model, in which beneficiaries are paired with like-control groups and actual/baseline results were compared for “with” and “without project” groups. The project did not include any indicators to measure the “shared” aspect of agricultural growth. There were no indicators on shared prosperity / poverty, but the ICR provided qualitative information on this subject. The ICR argued that (page 23) the project’s help on establishing IPOs benefitted the poorest members of the value chain, namely producers as opposed to traders and processors, who are considered more prosperous. Including producers in the IPO, gave them power for better negotiation, which reportedly resulted in better price discovery, transparency in business transactions, and common decisions fair to all. So, the ICR considered all project beneficiaries as poor, which may not be the case, as the ICR did not provide information on the size of the farmers supported (although the theory of change of the ICR mentioned the beneficiaries were to be small farmers). In addition, the ICR argued (page 22) that “Of the 409,101 direct beneficiaries, 124,367 (i.e., 30.4 percent) were women. The project made a concerted effort to increase the participation of women by supporting activities in which the participation of women is important”. The project team reported that (June 14, 2018): “in 2014, enterprises financed through the project generated 3,752 new jobs, 58% went to women owing to the higher needs in mango drying and onion conservation. In 2016, enterprises supported by the project generated 4,964 jobs, of which 21% went to women. Based on a sample of project beneficiaries for the micro-project program (under the Impact Study 2017), direct job creation was estimated at 14 751 jobs in 2016 of which 17% is permanent wage labor and 36% is family labor. Also, value added from project beneficiaries were estimated at 0.17 percent of Agricultural GDP (using volumes of production, gross margins and costs from micro-projects financed in the four main value chains). Given the evidence, the report finds that even with considerable attribution issues, the result for this objective is marginally Substantial. Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale The overall efficacy of the project is rated Modest. Although there is yield increase for mangoes and onions, irrigation activities that would contribute to further increases were only 60% complete by project closing. In addition, disaggregated export figures showed mixed and inconclusive results (exports increased for mangoes and onions but declined for cowpeas and livestock.) The project provided some evidence on the increase in the overall contribution of project beneficiaries to agricultural GDP and specifically increased income for cattle and onion produces, but incomes did not increase or mango and poultry producers (as marketing infrastructure was not complete at closure). Thus, while there is reasonable evidence on “shared” growth, the limited information on granular project contributions in the various sectors results in an overall Efficacy rating of Modest. Overall Efficacy Rating Primary reason Modest Insufficient evidence PHREVISEDTBL 5. Efficiency An ex-ante economic and financial analysis was conducted for the original project and the Additional Financing. The analysis in both cases was carried out for investments on supply chain development; small-scale irrigation, and market development infrastructure. For the ex-ante analysis. the financial internal rate of return (IRR) at the enterprise level ranged from 30% (mango on 0.25 ha) to 45% (onion on 0.5 ha), while the economic internal rate of return (ERR) ranged from 43% (mango on 0.25 ha) to 90% (enterprise with 4 cows). The economic analysis at project level (not including capacity building activities) suggested an IRR of 25% and an ERR of 23%. A different approach than the one used for the original project was used for the AF ex-ante analysis. In the latter approach, the financial analysis was performed only at the enterprise level, and the economic analysis only at the project level. At the enterprise level the estimated IRRs for individual enterprises ranged from 32% (local poultries) to 70% (stored onions in a 12-ton infrastructure). At the project level, the net present value (NPV) at 10% was estimated at $26.3 million, and the ERR at 17.3%. This ERR was lower than the estimate of the original project (23%), but still adequate, with the assumed opportunity cost of capital of 10%. The ex-post economic and financial analysis conducted for the ICR relied on the same methodology used in the ex-ante FA analysis. However, the ICR did not provide any information on the assumptions used by the economic and financial analysis. The project team provided that (June 14, 2018) that the main assumptions were: “(i) economic benefits were estimated at 85 percent of the financial benefits, with retail prices computed as farm-level or factory-level product prices, without transport costs, taxes, or transfer between agents; (ii) Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) economic cost of labor was considered, corresponding to about 50 percent of labor financial costs, in view of the high unemployment rate and underemployment in Burkina Faso; (iii) the discount rate used was 10 percent; (iv) the period of analysis is 25 years; and (vi) benefits from capacity-building activities, were not included as it was not possible to quantify them. The IRRs at enterprise level ranged from 25% (cattle fattening) to 52% (onion growing, and mango processing), and the NPV at 10% from 1.6 million CFA (poultry production) to 84 million CFA (onion production). The sensitivity analysis indicated good robustness for most enterprises; however, dried mango processing was very sensitive to a decrease in overall benefits of about 23%, and an increase in overall cost of about 30%. The economic analysis at project level estimated the NPV (at 10%) at $72.5 million and the ERR at 15.6%. The lower return at the close of the project may be explained by the poor performance experienced in component 2 with the delays in establishing irrigation and market infrastructure. Administrative and Operational Efficiency: The project closing date was extended more than four years (including additional financing). Despite the extensions, as mentioned above, the overall efficiency of the project was negatively impacted by the untimeliness and incompleteness of several public irrigation and market infrastructures. In addition, more than 1,000 micro-projects had to be called off because they had failed to meet a trigger milestone for subsequent tranches of financing. Although the project managed to recycle the funds provided, there were significant inefficiencies in the program. Project efficiency is rated Modest, due to significant operational inefficiencies and not finishing all planned outcomes in line with project time and budget. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  23.00 Not Applicable 100.00 ICR Estimate  15.60 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The Relevance of Objectives is rated High. Efficacy of the objective “to increase the competitiveness of selected Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) agricultural sub-sectors that target national, sub-regional and international markets thereby contributing to shared agricultural growth in Burkina Faso” is rated Modest as evidence on yield increases as well as increases in international and regional exports cannot be fully attributed to the project. There is some evidence on shared agricultural growth, but there are considerable attribution issues that make it difficult to know what the project’s contribution was to this improvement. Efficiency is rated Modest as key project outputs were not completed on time and within budget even though the project closing date was extended for more than four years. The combined outcome rating is therefore Moderately Unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Institutional Risk: The inter-professional organizations met expectations by improving coordination in the value chains and providing services to their members (as highlighted earlier), even if these organizations are still maturing under the aegis of a law that is just putting in place its implementation measures. The risk of the IPO faltering is low, given that there are additional benefits in the future in terms of public support. Private actors are already capably managing the market infrastructures (e.g.Koudougou onion market platform) and stand ready to do so with the facilities to be completed—with this delayed completion posing a significant threat to the development outcome if not resolved in the near future. Capacities have been built within the Ministry of Agriculture, the agricultural technical school, etc., but the network of public and private service providers has not materialized—except for the MEBF, which is a network by itself in the promotion of entrepreneurship and coaching SME to access credit. However, the institutional sustainability of the project is questioned as it did not transform into a program led by a formal institution established under a PPP arrangement dedicated to promoting competitiveness of value chains. The project did not attempt to establish that structure. 8. Assessment of Bank Performance a. Quality-at-Entry The project team utilized lessons learned from projects in the region and in the country in the design of the project. They also held several consultations with key stakeholders, including civil societies and anticipated beneficiary groups, during project preparation. A clear and reasoned choice was made to focus on selected value chains and the economic and financial analysis supported that choice, except for sesame that was to be covered by other operations. However, the project tried to do too many activities for each selected value chain (infrastructure, institutional, advisory, financial), and this proved to be difficult to implement. Environmental and fiduciary aspects were addressed, and institutional arrangements identified at national Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) and regional levels. Similarly, risks were identified and Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis performed. There was, however, a significant shortcoming with the M&E Framework: With an operation covering only 9 out of 13 regions, the decision was made to use country-wide data to measure and suggest the operation’s achievement. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision Supervision missions were managed by country-based TTLs for most of the project’s lifespan. There were two changes in TTLs during implementation. The ICR (page29) claims, however, that these changes in TTLs had no significant negative impact on supervision. Overall, the Bank’s implementation support provided good on-site leadership, a balanced Bank team and level of expertise, frequent missions, and mostly rapid response to problematic situations when they arose. Supervision missions were conducted jointly with the Government and the Bank. The teams were candid in their findings, straightforward and proactive in their recommendations. Non-performing micro-projects were terminated promptly, and unused funds were recouped. The need for an internal auditor, who was overlooked in the design, was identified at the MTR and filled quickly afterward. The need for a dedicated environmental and safeguard specialist, however, was identified only in the AF. Changes to the targets and some indicators were made in the AF, but not in the way of collecting export data to address the problems with attribution. A shortcoming of the supervision was the acknowledgement that exports to international and regional markets were skewed by sesame exports, but nothing was done to address the issue. Similarly, the problem with using countrywide data to document the achievement of the PDO outcome indicators was not addressed. The project team mentioned that (June 14, 2016), the incompleteness of some infrastructure was not due to the lack and/or quality of supervision, but due to the weak technical and financial capacity of the local construction enterprises. Specific recommendations were made through supervision missions and Aide Memoires. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E system was designed to capture project inputs, outputs and processes, including through a Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) dedicated database to monitor and evaluate the micro-projects (put in place in the FA). The M&E manual provided clear definitions of the indicators and discussed the data collection procedures at various levels (at the project coordination unit and satellite locations). Although an increase in agricultural exports to international and regional markets, tracked by two PDO indicators, is an acceptable proxy for competitiveness, the M&E also included other indicators such as yield data and number of enterprises converted to micro enterprises and SMEs. The M&E did not provide a breakdown of the international and regional exports by commodities, this was introduced during AF, so there were no baselines.. In addition, the M&E design collected national-wide data to document the achievement of a project that did not cover the whole country (9 regions out of 13) in the original project. In addition, the yield data showed only the baseline and closing year, rather than a trend covering several years before and during implementation, to rule out weather impacts. The project did not include outcome indicators to measure “shared” growth aspects. Also, no intermediate outcome indicators were designed to measure and report on market infrastructure investments and partnerships. Project outputs on capacity development also were not monitored adequately as no relevant indicators were designed. b. M&E Implementation The project implemented the M&E as designed, with a core team at the national level and additional teams in all three local offices (West, Center and North). The M&E system integrated the various activities of the project and functions in the project coordination unit (PCU). The M&E system implemented the changes in the RF introduced by the FA. The project also worked with inter-professional organizations so that they could provide information to their members. In the FA, the project added a database of micro-projects, along with a database manager, to monitor and evaluate their activities and results. The initial difficulties encountered in establishing the database were resolved, and the M&E system was computerized. The M&E system did not allow for monitoring of exports of actual project products. Information on these agricultural exports gathered by IPO supplemented, rather than replaced data from the customs office. Two impact studies were conducted in 2014 and 2017.by IPO supplemented, rather than replaced data from the customs office. c. M&E Utilization The M&E information was used to produce timely reports and to make informed decisions. Documents included work plans, technical and financial reports (RSF), annual reports, etc. The M&E was used to inform decisions regarding restructurings, allocation of funds, and the AF. The micro-project database was used to closely monitor micro-projects and help weed out non-performing ones. The impact analyses rely on the M&E data to draw samples of micro-projects for their estimates. The impact analysis, however, did not explore the counterfactual, to really gauge the attribution of the agricultural exports to the project. Ample accounts of the project activities and results were provided to the public in the media (newspapers, radios, TV, YouTube, website, flyers and posters). This widely disseminated information was very effective in getting potential beneficiaries to approach the project and eventually participate in its activities. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as Category B and triggered three safeguards: (a) OP 4.01 – Environmental Assessment, (b) OP 4.09 – Pest Management and OP 4.12 Involuntary Resettlement. All safeguards instruments were prepared, consulted upon and disclosed. The Environment and Social Management Plan (ESMP) prepared/updated for the original projected and AF provided guidance on management and mitigation measures for environmental and social impacts of the micro-projects in the four major value chains, and irrigation and market infrastructure. A Resettlement Policy Framework (RPF) was prepared and recommended loss compensation measures and the capacity building of actors in handling resettlement. In practice, the option of reducing or avoiding land acquisition was restrained as availability of land documents was considered an eligibility criterion for all microprojects. Notwithstanding the application of this selection process, several micro-projects and infrastructure sites were abandoned or replaced to comply with provisions of the RPF. Implementation and monitoring of social safeguards did not provide quantified data to inform on achievement of performance indicators (e.g. land issues). No project affected people (PAP) has been identified in the implementation of physical investments. Important capacity building activities on social and environmental coordination monitoring and control have been conducted including but not limited to: a) hiring a consultant in charge of environmental evaluation; b) establishing a technical environmental monitoring committee; c) awareness programs on social and environmental issues. Challenges faced during implementation included limited staff capacity in the PCU, unclear formulation of measures in safeguard frameworks and insufficient reporting, documentation and traceability in the monitoring of safeguards measures. The ICR also noted that (page 28) compliance with safeguards was rated moderately satisfactory. b. Fiduciary Compliance Financial management. The ICR did not provide information on how the financial management of the project was handled, any issues encountered, the results of internal audits and any ratings of the overall financial management system. The project team reported that (May 21, 2018) the financial management was adequately handled and annual audits regularly carried and accounts certified with no qualified opinions. Procurement Overall, the project’s procurement plan implementation was satisfactory. The procurement plan put in place was regularly monitored. The project unit ensured that all contracts were duly closed by the grace period after the close of the project. There are no ineligible expenses. Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) c. Unintended impacts (Positive or Negative) No unintended impacts were reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Efficacy is rated Modest as there is insufficient achievement on increased Moderately Moderately competitiveness; Efficiency is Outcome Satisfactory Unsatisfactory also rated Modest, and this gives an overall Outcome rating of Moderately Unsatisfactory. The project results framework and M&E had significant weaknesses which were not Moderately Moderately completely resolved during Bank Performance Satisfactory Unsatisfactory implementation that led to some of the low achievements on outputs and attribution of project results. There were significant Quality of M&E Substantial Modest shortcomings in M&E design and Implementation. Quality of ICR Modest --- 12. Lessons The ICR provided comprehensive lessons. The most important follow with some modification of language: Projects trying to tackle almost all constraints for competitiveness at once (enabling environment, infrastructure, technology, markets and market information, access to credit etc.) may be slowed down during implementation. Therefore, it is important to start with more simple operations, use pilots and then scale up adding more complexity at a later stage. An intervention involving a multi-sector/multi-level approach covering wide range of geographic regions is difficult to implement particularly in fragile environments with weak institutional capacity. Designs initially should be kept simple for eventual scale up and with closer Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) supervision from authorities so that learning opportunities are utilized to attain stronger institutional and managerial capacity for similar operations. Meeting financing needs for infrastructure and equipment are key to addressing agro-procesing in the mango sector. Drying tunnel equipment promoted by the project is a key equipment for mango processors to ensure quality for mango exports. Only 10 dry mango processors out of the 76 in existence in in the country are equipped this equipment. In addition, in order to match the growing market demand for mango exports, there is a need for processors to increase production capacity, renew the equipment and construct new infrastructure for storage and processing. A lesson learned during implementation is that banks have difficulty granting medium and long term loans. Therefore, dedicated medium and long term credit lines have therefore to be put in place. Also, under the project tunnel dryers were subsidized up to 65 percent, the need to continue such an incentive has to be addressed. In future support to agro-processing, one of the possible options might be to continue having such matching grants to help smaller processing enterprises; matching grants could also be considered for innovating equipment, given that banks may be reluctant to fully finance due to the risk related to innovation. The project experience showed that entrepreneurship can be expanded in rural areas via business incubator organizations. Business incubator is a model to emulate by other countries that don’t yet have similar bodies. In its free advisory role, it is similar to agencies in developed countries, such as the Score association of business mentors supported by the US Small Business Administration (SBA). It helps with actors trying to start a business (e.g., with its one-stop window), or needing assistance with their existing business. It goes one step further, as they can take on projects turn them into formal SMEs. A formal SME enhances its chance to access credit and compete in public bids. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was comprehensive and included a good formulation of lessons. However, the ICR had weaknesses regarding quality of evidence and analysis in general. For example, (i) the ICR did not provide information on assumptions used on economic and financial analysis; (ii) results on exports were not disaggregated in the efficacy section. (iii) The analysis of Efficacy was weak, rather than rating the achievement of the two PDOs of the project, the ICR rated the outcome indicators, which is not in line with the ICR guidelines. (iv) The TOC did not discuss, increasing competitiveness of selected agricultural sub- sectors objective. In addition, the ICR did not include information on how the financial management of the project was handled, the results of internal audits and any ratings of the overall financial management system. Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Ag. Diversification & Market Dvt. (P081567) a. Quality of ICR Rating Modest Page 18 of 18