Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004683 IMPLEMENTATION COMPLETION AND RESULTS REPORT (CREDIT NUMBER 509-WSO) ON A CREDIT IN THE AMOUNT OF SDR 5.2 MILLION (US$ 8 MILLION EQUIVALENT) AND A FOOD PRICE CRISIS RESPONSE MULTIDONOR TRUST FUND GRANT IN THE AMOUNT OF US$ 5 MILLION TO THE INDEPENDENT STATE OF SAMOA FOR THE Samoa Agriculture Competitiveness Enhancement Project June 28, 2019 Agriculture Global Practice East Asia And Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective: Nov 01, 2018) Currency Unit = West Samoan Tala 2.24WST =US$1 1.55US$ = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Victoria Kwakwa Country Director: Michel Kerf Senior Global Practice Director: Juergen Voegele Practice Manager: Nathan M. Belete Task Team Leader(s): Stephane Forman ICR Main Contributor: Kara Mouyis ABBREVIATIONS AND ACRONYMS APHD Animal Production and Health Division ASCD Agriculture Sector Coordination Division CTSSU Centralized Technical Services and Support Unit DBS Development Bank of Samoa EFA Economic and Financial Analysis EIRR Economic Internal Rates of Return FI First Generation FM Financial Management GDP Gross Domestic Product ICR Implementation, Completion and Results IDA International Development Association M&E Monitoring and Evaluation MAF Ministry of Agriculture and Fisheries MGP Matching Grant Program MIS Management Information System MOF Ministry of Finance MSME Micro, Small and Medium Enterprise MSU Mobile Slaughter Unit NPV Net Present Value PCU Project Coordination Unit PDO Project Development Objective RPF Regional Partnership Framework SACEP Samoa Agriculture Competitiveness Enhancement Project SAFPROM Samoa Agriculture and Fisheries Productivity and Marketing project SAT (Western) Samoan Tala SBEC Small Business Enterprise Sector SSU Static Slaughter Unit UN United Nations USD United States Dollar WB World Bank WIBDI Women in Business Development Incorporated TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 5 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ..................................................... 10 A. CONTEXT AT APPRAISAL .......................................................................................................10 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .....................................14 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................15 B. ACHIEVEMENT OF PDOs (EFFICACY) ......................................................................................15 C. EFFICIENCY ...........................................................................................................................23 D. JUSTIFICATION OF OVERALL OUTCOME RATING ....................................................................24 E. OTHER OUTCOMES AND IMPACTS .........................................................................................25 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 28 A. KEY FACTORS DURING PREPARATION ...................................................................................28 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................30 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 32 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................32 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE .....................................................33 C. BANK PERFORMANCE ...........................................................................................................34 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................35 V. LESSONS AND RECOMMENDATIONS ............................................................................. 37 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 40 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 53 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 56 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 57 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 65 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 66 ANNEX 7. SPLIT RATING ....................................................................................................... 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P115351 Samoa Agriculture Competitiveness Enhancement Project Country Financing Instrument Samoa Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Independent State of Samoa Ministry of Agriculture and Fisheries Project Development Objective (PDO) Original PDO The development objective of the project is to support fruit and vegetable growers and livestock producers to improve their productivity and take greater advantage of market opportunities. Page 5 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 8,000,000 7,889,732 7,097,866 IDA-50890 5,000,000 4,999,585 4,999,585 TF-12179 Total 13,000,000 12,889,317 12,097,451 Non-World Bank Financing 0 0 0 Borrower/Recipient 260,000 260,000 220,000 Local Beneficiaries 2,900,000 2,900,000 2,200,000 Total 3,160,000 3,160,000 2,420,000 Total Project Cost 16,160,000 16,049,316 14,517,451 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 29-Mar-2012 04-May-2012 02-Mar-2015 28-Mar-2017 31-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 13-Feb-2017 8.00 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Disbursements Arrangements Change in Legal Covenants 21-Dec-2017 10.59 Change in Loan Closing Date(s) Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest Page 6 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 11-Jun-2012 Satisfactory Satisfactory 0 02 16-Feb-2013 Satisfactory Moderately Satisfactory .80 03 10-Nov-2013 Satisfactory Moderately Satisfactory 1.41 04 28-Jun-2014 Satisfactory Moderately Satisfactory 1.17 05 22-Dec-2014 Moderately Satisfactory Moderately Satisfactory 2.73 06 29-Jun-2015 Moderately Satisfactory Moderately Satisfactory 5.39 07 18-Dec-2015 Moderately Satisfactory Moderately Satisfactory 6.01 08 15-Jun-2016 Moderately Satisfactory Moderately Satisfactory 7.29 09 22-Dec-2016 Satisfactory Moderately Satisfactory 8.00 10 21-May-2017 Satisfactory Moderately Satisfactory 8.86 11 04-Dec-2017 Satisfactory Moderately Satisfactory 10.59 12 16-May-2018 Satisfactory Moderately Satisfactory 11.11 13 26-Nov-2018 Moderately Satisfactory Moderately Satisfactory 11.69 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 50 Crops 8 Public Administration - Agriculture, Fishing & Forestry 26 Livestock 16 Industry, Trade and Services 50 Agricultural markets, commercialization and agri- 50 business Page 7 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 107 Jobs 100 Enterprise Development 7 MSME Development 7 Finance 19 Financial Infrastructure and Access 7 MSME Finance 7 Finance for Development 12 Agriculture Finance 12 Urban and Rural Development 55 Rural Development 55 Rural Markets 12 Rural Infrastructure and service delivery 32 Land Administration and Management 11 Environment and Natural Resource Management 55 Climate change 33 Adaptation 33 Renewable Natural Resources Asset Management 22 Biodiversity 11 Landscape Management 11 ADM STAFF Role At Approval At ICR Regional Vice President: Pamela Cox Victoria Kwakwa Country Director: Ferid Belhaj Michel Kerf Senior Global Practice Director: John A. Roome Juergen Voegele Practice Manager: Charles M. Feinstein Nathan M. Belete Page 8 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Task Team Leader(s): Mona Sur Stephane Forman ICR Contributing Author: Kara Mouyis Page 9 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At appraisal, Samoa’s economy was growing steadily. Over the preceding 15 years the average Gross Domestic Product (GDP) per capita growth rate was 3 percent per annum and in 2012, GDP per capita was US$4,235 1. Samoa had one of the highest Human Development Indexes in the Pacific 2 and extreme poverty was low, with only 3 percent of households falling below the food poverty line. Despite these results, basic needs poverty was high (26.9 percent of the population) and many subsistence farming households fell into this category. This made them vulnerable to shocks, such as increased food and fuel prices, which many had already experienced after the tsunami of 2009. Due to its small size, limited natural resources, narrowly based economy and distance to markets, Samoa’s potential for growth was also constrained, and the economic outlook remained conservative. 2. In the agriculture sector, a variety of factors had led to a declining contribution to GDP; from 22 percent in 1994, to just 9 percent in 2012 3, despite agricultural growth being identified as a key development objective in the 2008- 2012 Strategy for the Development of Samoa and important for improving the quality of life for Samoans. Some key constraints identified at appraisal included (i) low public and private investment; (ii) rocky soil presenting a challenge for larger scale farming and mechanization; (iii) poor attention to nutrition, breeding management and animal health in the livestock sector; (iv) difficulty accessing agricultural credit; (v) a lack of extension services and; (vi) no abattoir, impacting public health and limiting local meat’s ability to compete with imports. 3. Cultural aspects including fa’alavelave slaughter also played a role in limiting formal meat production. Fa’alavelave; occasions such as weddings, funerals or matai (chief) accession, include the presentation of goods, including meat. Livestock for fa’alavelave were sold through an informal market and slaughtered unhygienically, undermining the formal system. Farmers were likely to get a higher price for the size and bulk of animals, rather than the quality of meat which was necessary in the formal market and, due to increasing obligations, there had been evidence of breeding cattle being slaughtered to maintain sale numbers and weight. 4 In 2015, fa’alavelave was estimated to account for 79 percent of all cattle slaughtered in Samoa, and was therefore likely to have contributed to the low numbers of farmers producing for commercial purposes. 4. In 2009, 84 percent of households were involved in some level of farming 5, but only 1 percent derived all their income from agriculture, representing a significant opportunity for growth. A challenge was that household production varied widely between crop and livestock types, size, use and the proportion of income derived from production. The most common types of crops included Taro, Banana, Coconut, Breadfruit, Ta’amu, Cocoa and Taro palagi. For livestock, apart from a few larger scale cattle operators, the sector was dominated by household production, with only a small percentage of what was raised being sold at the commercial market. 1 www.worldbank.org/data GDP per capita (current US$) 2 Seven percent, behind Palau, Fiji and Tonga: http://hdr.undp.org/en/countries 3 Agriculture, Forestry and Fisheries contribution to GDP is combined. 4 Impact of Fa’alavelave Slaughter on Retail Meat Marketing in Samoa; Farm Consultancy Services (2015) 5 2009 Agriculture Activity Census Page 10 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 5. At the same time, food was increasingly imported. At appraisal more than 51 percent of vegetables and 65 percent of meat was brought in from overseas. In the absence of suitable abattoir facilities or refrigerated transportation, livestock sellers faced large obstacles to reaching the quality standards of imported meats. Alongside this, health burdens from non-communicable diseases were also increasing, with particularly high levels of obesity recorded 6 and type two diabetes increasing from 1.7 to 19.5 percent of the population between 1978 and 2013. 6. Food security was a growing concern with climate change and natural disasters increasingly being recognized as a risk to the sector and farmers with predictions of more frequent and extreme rainfall events, longer drought events, extreme winds, and high air and water temperatures. In the preceding two decades, the agriculture sector had been affected by numerous tropical storms, earthquakes, pests and diseases. The Taro Leaf Blight epidemic in 1993 devastated the staple taro crop, and the incomes of farmers who grew it, and following the food and fuel crises of 2008, food insecurity was a major concern for the country. In 2009, Samoa was hit by a severe tsunami which killed 143 people, left over 5,000 homeless, and cost an estimated US$124 million, or more than a fifth of the country’s GDP. 7 It had a significant impact on farming household incomes with widespread damage and the loss of food gardens, crops, livestock and farming equipment. 8 7. As part of the push to strengthen and grow the agriculture sector, Samoa’s Ministry of Agriculture and Fisheries (MAF) had recently prepared their first ever Agriculture Sector Plan (2011 – 2015). The Plan was an output of the 2008-2012 Strategy for the Development of Samoa and was committed to agriculture-led growth and revitalizing the agriculture sector in the country. The project was aligned to all four objectives of the Plan through policy and regulatory support, food security activities, private sector involvement and climate-smart adaptation measures. The project was also consistent with the Government of Samoa’s 2012-2016 Strategy for the Development of Samoa which included dedicated interventions in the agriculture sector. 8. The project objectives were also consistent with the World Bank’s first Samoa-specific Country Partnership Strategy (FY2012-2016) (Report #67194). This came at a time of increased assistance from the Bank across the Pacific region. In particular, Priority 1 – Rebuilding Economic Resilience and Encouraging Inclusive Growth –sought to improve agriculture productivity and strengthen opportunities for small and medium scale farmers. At the time of appraisal, the World Bank had previously worked with the Government of Samoa on two agricultural development projects, however it had not engaged in the agriculture sector since 1998, when these projects closed 9. Theory of Change (Results Chain) 9. The below diagram illustrates the Theory of Change, on which the Samoa Agriculture Competitiveness Enhancement Project (SACEP) was built. Outputs, which were required to achieve the PDO, aligned well with project activities and reflect a holistic approach to the project. Several critical assumptions were required for the 6 WBG Samoa Country Partnership Framework (2012-2016) 7 WBG Samoa Post Tsunami Reconstruction Project (2012) 8 Analyzing villages affected by the 2009 tsunami using Samoa Labor Mobility Survey data, Gibson (2010) 9 IDA 9510 (US$8m) Agriculture Development Project closed in June 1998 and IDA 10800 (US$2m) Second Agriculture Development Project closed in December 1997. Page 11 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) project to achieve both PDO-level outcomes and those in the longer term which will contribute to achievement of Government of Samoa’s development objectives and the World Bank’s twin goals. Project Development Objectives (PDOs) The PDO was to support fruit and vegetable growers and livestock producers to improve their productivity and take greater advantage of market opportunities. Key Expected Outcomes and Outcome Indicators The four main outcomes against which the achievements of this project would be measured were: 1. Improved productivity of fruit and vegetable growers. This was measured using the following indicators: (i) Percentage increase in average fruit and vegetable yields of growers participating in the Matching Grant Program (MGP). (ii) Percentage increase in average fruit and vegetable yields of targeted subsistence farmers. 2. Improved productivity of livestock producers. This was measured using the following indicators 10: (iii) Increase in reproduction rates in response to improved husbandry – calving rate (cattle). (iv) Increase in reproduction rates in response to improved husbandry – average number of pigs weaned per litter. 10 Indicators for outcome #2 were drawn from the intermediate results indicators. Page 12 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 3. Greater advantage of market opportunities taken by fruit and vegetable growers. This was measured using the following indicators: (v) Percentage of locally produced fruits and vegetables sold by domestic retail and food service channels 11. (vi) Percentage increase in the value of sales of fruit and vegetable growers participating in the MGP. 4. Greater advantage of market opportunities taken by livestock producers. This was measured using the following indicators: (vii) Percentage of locally produced beef sold by domestic retail and food service channels 12. (viii) Percentage increase in the value of sales of livestock producers participating in the MGP. Components 10. At appraisal the total project cost was estimated to be USD16.16 million, including contingencies. This cost was financed by an International Development Association (IDA) Credit of SDR 5.2 million (USD8 million equivalent), a Global Food Crisis Response Program grant of USD5 million, Recipient contributions of USD0.26 million and beneficiary contributions of USD 2.9 million. 11. The project included three Components covering fruit, vegetable and livestock farmers across both of Samoa’s main islands (Upolu and Savai’i), as well as institutional strengthening. Participation in the Matching Grant Program (MGP) was entirely demand-driven, with additional support offered for farmers wishing to remain operating at a subsistence level. 12. Beneficiaries were crops and livestock farmers in Samoa who were eager to move from subsistence to semi- commercial, or from semi-commercial to commercial farming. Farmers could take part in the matching grant program by providing a 50 percent contribution (of this, 30 percent could be a loan from the Development Bank of Samoa) and by providing a business plan outlining their case. Farmers were also able to take part in free training on improved farming and business practices. All activities were demand-driven. 13. Component 1: Livestock Production and Marketing (estimated cost at appraisal - USD5.51m). This component included activities to: increase farmer access to superior breeding stock; provide technical advice; foster locally grown fodder to improve livestock nutrition; and set up mobile and static slaughter units to improve meat quality and hygiene. This component also financed on-farm investments through a combination of matching grants (50 percent), optional commercial loans from the Development Bank of Samoa (DBS) (up to 30 percent) and equity (20 – 50 percent). Farmers with approved business plans were also eligible for loan guarantees through the Small Business Enterprise Sector (SBEC). At appraisal, likely investments were thought to be water facilities, fencing and stock yards and new livestock to improve breeds. 14. Component 2: Fruit and Vegetable Production and Marketing (estimated cost at appraisal - USD3.80m). Component 2 included activities to: increase farmer access to an improved variety of planting materials; provide technical advice and promote market development, including fruit and vegetable exports; and organic certification. Nursery screening trials were initiated to identify new fruit and vegetable varieties that could be successful in the 11 This indicator was measured in the ‘share’ of local produce. It had a baseline value of 5 and a target share of 20 percent. 12 This indicator was measured in the ‘share’ of local produce. It had a baseline value of 31 and a target share of 50 percent. Page 13 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Samoan climate. This component also financed on-farm investments through the MGP, under the same arrangements as Component 1. Key investments identified at appraisal included: field rock removal due to the high volume of lava rocks across the country; polytunnels to protect vegetables during the rainy season; water harvesting systems to improve production during drier months; and establishing a banana meristem partnership with the private sector. 15. Component 3: Institutional Strengthening (estimated cost at appraisal - USD6.87m). This component included activities targeted at both the government and non-government agriculture institutions which provide extension support and research services, as well as the MAF staff who monitored the project’s implementation. Activities included staff training, support to improve MAF’s work facilities, incremental staff costs and the design and implementation of a monitoring and evaluation (M&E) system which would sit in the existing Management Information System (MIS) of MAF. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets The PDO was not revised. Revised PDO Indicators 16. A Level 2 Restructure was undertaken in February 2017, which included changes to the Results Framework (revised end target figures can be seen in Annex 1). According to the ISR at the time, end target revisions were made to better reflect the findings on the ground and provide more realistic targets for project completion. Some revisions reflected the demand-driven nature of the project; for example, MGP grants were larger in scale and smaller in number than envisaged, so the number of business plans was reduced; others were a result of implementation challenges such as the cost of a static slaughter unit. The restructure also added a gender indicator. Revised Components 17. No substantive changes were made to project components/activities which impacted the PDO. As part of the February 2017 restructure the following changes were made to project components and costs, which reflected the experience on-the-ground: • Component 1: Livestock Production and Marketing: Only one of the two mobile slaughter units were procured and a smaller-scale static slaughter unit was proposed, instead of the abattoir. • Component 2: Fruit and Vegetable Production and Marketing: Renovation of Crops Division buildings was increased. • Component 3: Institutional Strengthening: Technical assistance was extended to allow selected specialists to give continued support. Other Changes 18. A second Level 2 Restructure was undertaken in December 2017 to extend the closing date and reallocate expenditure between categories. The project was extended by a cumulative total of 21 months and its final closing date was 31 December 2018. Page 14 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Rationale for Changes and Their Implication on the Original Theory of Change 19. No changes made under either restructure impacted the Theory of Change and it remained relevant for the entire project life. The major changes under the first restructure were to revise the Results Framework so it better reflected local demand for the MGP, as well as to extend the project closing date. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 20. The Project Development Objective is rated as High for relevance. 21. As highlighted in the context section, the project objectives were highly relevant at preparation stage to both the MAF Agriculture Sector Plan (2011 – 2015) and the World Bank’s Samoa Country Partnership Strategy (FY2012- 2016) (Report #67194). The PDO was also well aligned with the MAF Agriculture Sector Plan (2016 – 2020), contributing to all four of its End of Sector Plan Outcomes. 22. At project closure the PDO aligned well with both the sector needs and the World Bank Regional Partnership Framework (RPF) for the PIC-9 13 (FY17-21) (Report #120479), which is the current partnership framework between the World Bank and Samoa. Specifically, the PDO aligns with the RPF Focus Area 1 – fully exploiting the available economic opportunities – which prioritizes increasing incomes from agriculture, particularly for Samoa. To improve food security and increase small-scale production, objectives include increasing the value of sales for crop and livestock farmers and providing training to farmers on technologies, practices and resilient agriculture. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 23. The following evaluation of efficacy was undertaken in accordance with the appraisal PDO and the revised Results Framework. For the purpose of this evaluation, the PDO is broken into the following outcomes: i. Improved productivity of fruit and vegetable growers; ii. Improved productivity of livestock producers; iii. Greater advantage of market opportunities taken by fruit and vegetable growers; and iv. Greater advantage of market opportunities taken by livestock producers. 24. While there were no changes to the PDO and no changes to the key outcomes, five PDO-level indicators and five intermediate-level indicators were amended, hence a split rating was considered appropriate to ensure a comprehensive analysis (see Annex 7). The downwards revision of PDO-level indicator targets proved to be unnecessary for three of the five indicators, as the final results exceeded the original values. The two PDO level indicators which did not have their original targets met were both related to outcome number 1 (improved productivity of fruit and vegetable growers). At the time of the restructuring, US$8 million had been disbursed, representing a 62 percent weighting (this was made up of 100 percent of the Food Price Crisis Response Multidonor Trust Fund Grant and 37 percent of the IDA Grant). 13The PIC-9 includes Kiribati, Nauru, The Republic of The Marshall Islands, Federated States of Micronesia, Palau, Samoa, Tonga, Tuvalu and Vanuatu. Page 15 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Improved productivity. 25. The MGP provided one of the main vehicles for improving farmer productivity under SACEP. Additional training and improved extension support for the sector, also meant farmers outside of the MGP were able to see some benefits. A total of 740 farmers participated in the MGP and grants were provided based entirely on demand, rather than pre-determining the types on produce/stock, or the proportions for each. As a result, there were more farmers in the livestock category than fruits and vegetables (see Table 1 below). Table 1: farmer types in the MGP Farm type No. of farmers Percentage Total project cost Percentage (SAT) Cattle 405 54.7% $8,642,012 59.1% Piggery 135 18.2% $2,306,198 15.7% Poultry 43 5.8% $660,642 4.5% Sheep 21 2.8% $533,880 3.6% Bee 2 0.3% $64,000 0.4% Fruit & Vegetable 134 18.1% $2,428,023 16.6% Total 740 $14,664,755 Source: SACEP Unit 26. Three intakes were completed over the project life, with beneficiary numbers increasing exponentially, from 48 farmers initially, to 534 by intake three. Farmers who were accepted into the MGP were given a 50 percent grant (up to SAT16,000 – approx. USD6,000), the other 50 percent was provided by either their own savings, or in combination with a bank loan from DBS (max 30 percent of the total value) and in-kind contributions. Alongside guidance from MAF and SBEC, grants were intended to enable farmers to invest in new agricultural activities, to increase their on-farm productivity and improve marketing opportunities. All farmers were required to work with SBEC to prepare a business plan, outlining how money would be used, how it would provide a return on their investment, and discussing the sustainability of the investment going forward. 27. Another aspect of the project aimed at improving productivity was the strategic introduction of improved genetic livestock breeds or crop varieties. Throughout the project these were imported, tested in a controlled environment by MAF, and if successful, were shared through the relevant farmer channels. The importance of this process was highlighted as a key part of improved productivity, given the small size of Samoa and the risks involved through sharing new livestock or crop varieties without first testing their appropriateness for the environment. 28. Finally, farmer training was also provided to both MGP and non-MGP farmers and, at project closure, 1,112 farmers were using improved fruit and vegetable varieties, owned improved livestock and/or had adopted improved farming practices. Given only one family member per household was eligible to apply for a matching grant, this roughly equates to 5 percent of farming households in Samoa who benefited from training. 14 Of farmers receiving training, women made up 25 percent at closure, and 20 percent of farmers under the MGP. It should also be noted that significant spill-over impacts were found amongst farmers. MAF staff found that while new technologies, breeds and practices were trialed under SACEP, famers outside of SACEP were able to adopt them and benefit from knowledge sharing. This could be seen in the uptake of polytunnels through donor partner activities, as well as the 14Based on a finding a total of 28,880 households in Samoa (according to the Samoa Bureau of Statistics 2016 Census Brief No 1) and that 84 percent of households are involved in some level of farming. This number is an approximate only, given some non-MGP farmers may have had more than one household member take part in training and that not all farmers attending training may have used improved crops/livestock or farming practices. Page 16 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) purchase of cattle and adoption of improved practices, making a control group challenging to identify. Outcome (i) improved productivity of fruit and vegetable growers. This outcome was almost fully achieved in a ‘without restructure’ scenario and fully achieved in a ‘with restructure’ scenario. 29. PDO level indicators used to inform this are below. Indicator Original Target Revised Target Result Percentage increase in average fruit and vegetable yields of 100 50 53 growers participating in the Matching Grant Program (MGP) Percentage increase in average fruit and vegetable yields of 50 30 36 targeted subsistence farmers 30. There were 134 MGP fruit and vegetable farmers over the three intakes. These farmers saw a 53 percent increase in their average yields 15 over the project life, exceeding the final goal of 50 percent, although not meeting the original target. Productivity also increased for targeted subsistence farmers through training and extension support, as well as improved provision of resources through the Crops Division within MAF. This led to a 36 percent increase in the average yields for these fruit and vegetable growers. As with above, this exceeded the revised target of a 30 percent increase but did not meet the originally planned 50 percent. 31. A likely explanation for the increased yields is more crop cycles. Through the MGP Impact Assessment 16, the project found that MGP farmers increased their average annual cropping cycle 17 from 3.2 to 4.7, while non-MGP farmers only increased from 2.1 to 2.2 crop cycles per annum. This increase may be attributed to improved farming practices, better varieties and quality of seeds, as well as MGP investments and access to machinery through MAF, which were facilitated through the project. 32. Contributing to these results was the uptake of improved fruit and vegetable varieties appropriate for Samoa’s climate. These were selected by factoring in the anticipated impacts of climate change, and to support import substitution. Ten vegetables were trialed by the Crops Division 18, which tested population density, varieties and growth in tunnel houses versus open fields. Of these, nine were successful in their trials and seven were launched to farmers 19. Banana meristems were also imported from South Africa for production trials as there was limited genetic material available in Samoa. Using these new planting materials and trials, farmers were able to take advantage of better crops and training to increase their productivity. One hundred percent of MGP farmers surveyed had used new fruit or vegetable varieties, compared to only 20 percent of the control group. 33. Training and improved extension support was also a key driver of increased productivity. At completion, 80 percent of Crops Division staff were actively receiving training to upgrade their extension effectiveness and/or technical capacity. While there were some irregularities in the data, the indicator of Crops Division staff received training reached 100 percent during implementation. This strengthened the support they were able to offer fruit and 15 Based on average weight of all fruit and vegetable varieties, per acre of land. 16 Using a random sample of MGP and control farmers (control farmers came from communities without MGP participants). 17 Using the average of selected common products for comparison. 18 These were: Capsicums, Chinese Cabbage, Head Cabbage, Bulb Onions, Tomatoes, Lettuce, Kang Kong, Sweet Potatoes, Carrots and Irish Potatoes. 19 Carrots were unsuccessful. Sweet and Irish Potatoes undertook two of the three trials and were successful in both. Page 17 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) vegetable farmers. Several key pieces of machinery were also purchased for the Crops Division, including a tractor, bobcat (for rock removal) and a disk ripper. These were hired out to farmers at subsidized prices and have been used consistently since 2017. However, due to the relatively high distances the machines needed to drive to reach the farms hiring them, there was premature damage to the tyres and equipment. As the hire fees do not stay within the Crops Division, there was no budget for servicing and repairing the machines, which was compounded by the fact that many parts needed to be shipped in, making them more expensive, and that no trained mechanic was available; at the time of project closure, two of the machines were no longer operating. 34. Fruit and vegetable farmers were able to participate in workshops and farmer field schools, where they were taught about pesticides, crop calendarization, improved husbandry practices and post-harvest handling. Over the life of SACEP, there was a 98 percent increase in the proportion of targeted livestock and fruit and vegetable farmers who felt they were receiving satisfactory extension services. Outcome (ii) improved productivity of livestock producers. This outcome was fully achieved in both ‘with and without restructure’ scenarios. 35. While there were no PDO level indicators for this outcome, the below intermediate level indicators were used to inform this outcome. Indicator Original Target Revised Target Result Increase in reproduction rates in response to improved 60 60 68 husbandry – calving rate (cattle) Increase in reproduction rates in response to improved 7 7 8 husbandry – average number of pigs weaned per litter 36. Five types of livestock producers were involved in the MGP, totaling 606 farmers over the life of SACEP. Cattle farmers, who made up the majority of the MGP farmers (55 percent) were able to increase their calving rate from 48 to 68 percent, exceeding the target of 60 percent by project closure. For pig farmers, which made up 18 percent of the MGP, the average number of pigs weaned per litter went from 5 to 8, again surpassing the target (7). 37. For poultry, sheep and bees, the number of farmers (43, 21 and 2 respectively) were too small to meaningfully evaluate productivity through birth rates, however there were findings to suggest they saw some improvement. For chicken farmers, while there were no significant changes to the hatching rate for either group, between 2012 and 2017 MGP farmers saw a larger increase in the average number of chickens than the control group – from 65 to 77and 45 to 51 respectively 20. This suggests that although hatching rates may not have changed, mortality rates are likely to have dropped for MGP farmers, likely the result of better fencing and less access by predators. It’s also possible MGP poultry farmers may have been either more able, or more inclined to purchase new chickens for their farm. Sheep farming was only introduced in Samoa over the past decade, meaning most activities were new, and only a limited number of farmers did not participate in the MGP. Because of this, a control group comparison was not possible, however as is discussed later, MGP sheep farmers gained access to improved breeds, supplement feeds and adopted new husbandry practices including rotational pasture management for their paddocks, which all contributed to on-farm productivity and improved quality. 20 MGP Impact Assessment Page 18 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 38. As with crops, SACEP imported new livestock to increase the genetic diversity available in Samoa and improve productivity for farmers. For cattle in particular, in-breeding over the past decade, coupled with a lack of attention to nutrition and animal health had led to fewer calf births and an overall decline in the number of cattle in Samoa 21. In 2015, 46 Droughtmaster cattle were imported from Australia, a breed selected because of successful imports in the past, and their ability to adapt to the climate, terrain and feed. A breeding program was developed to inject new stock into established farms, and eight cattle multipliers were engaged to use the Droughtmaster bulls for their breeding programs. First generation (F1) offspring were then sold at a MAF-determined price to both MGP and non- MGP cattle farmers so they could improve their stock as well. A nucleus farm was also established at the APHD headquarters to conserve the high quality breed, as well to ensure the longevity and quality of the imports, and to allow further rotation of bulls in the future. The MGP Impact Assessment found that 69 percent of MGP cattle farmers purchased improved breeding stock over the life of SACEP, compared to 23 percent in the control group. 39. Pigs were imported from New Zealand at the start of 2018 and were kept at the APHD headquarters for farrow. Piglets were then sold to farmers at a reduced price, with a requirement to incorporate improved practices, such as planting cassava for improved feed. For piggery farmers in the MGP, 69 percent purchased new or obtained new breeds over the project life, while only 8 percent did from the control group. 40. Two rounds of sheep imports were made during SACEP – 110 Fiji Fantastic and 14 Dorper – in order to cross-breed the stock for increased resilience in the Samoan setting. As with the cattle, five multiplier farms were engaged across Upolu and Savai’i so that established farmers could expand the breeding program. At the start of the project there was a high mortality rate (approx. 20 percent), largely due to adapting to local climate and diseases, however by project closure this was reduced to 5 percent. The lambing percentage also increased over the life of the project, from 59 to 84 percent. Like both cattle and pig MGP farmers, sheep farmers involved in the MGP had an increased likelihood of purchasing improved breeds. 41. As the theory of change shows, improved stock is only one aspect of increasing livestock productivity – improved nutrition, husbandry practices, on-farm equipment and extension services also play a large role. Training was provided to MGP and non-MGP farmers throughout SACEP. Overall, 94 percent of MGP farmers attended this training (compared to 15 percent of non-MGP farmers in the control group) and 64 percent of MGP farmers adopted new farming practices (compared to 9 percent of non-MGP farmers in the control group). This is unsurprising and likely due to MGP farmers already being more motivated to improve their farms, as well as having the resources through the grants to apply their new-found knowledge. Some of the common changes across livestock farmers included pasture/feed improvement, stockyard development and new fencing or paddocks. 42. Training was also provided to MAF staff to strengthen the extension services offered for all livestock farmers, with the objective of improving their productivity. At project closure, 68 percent of APHD staff were actively completing training to upgrade their extension effectiveness and/or capacity, and there was a 98 percent increase in the proportion of targeted livestock and fruit and vegetable farmers who said they received satisfactory extension services. Market Opportunities. 43. For both crops and livestock farmers, several activities contributed to creating market opportunities. A lack of business awareness and training was alleviated through the MGP whereby all applicants prepared a business plan 21This was highlighted in the 2012 Ministry of Agriculture census which counted 29,000 cattle throughout the country, 6,000 fewer than in 2009. Page 19 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) and worked with SBEC to understand their business goals and objectives. Training was also provided to farmers on how to run a business, including record-keeping and financial literacy and management. A key constraint noted by supermarkets and restaurants in purchasing local produce was a lack of consistent quality and quantity. Activities related to improved productivity, husbandry and post-harvesting handling, as well as improved varieties and stock, supported farmers in overcoming this obstacle. Consumer demand was also targeted through work with the Ministry of Health to promote more healthy, local eating such as the ‘Eat a rainbow’ campaign. Outcome (iii) greater advantage of market opportunities taken by fruit and vegetable growers. This outcome was fully achieved in both ‘with and without restructure’ scenarios. 44. PDO level indicators used to inform this are below. Indicator Original Target Revised Target Result Percentage (share) of locally produced fruits and vegetables sold 30 20 51 by domestic retail and food service channels Percentage increase in the value of sales of fruit and vegetable 40 40 68 growers participating in the MGP 45. Evidence of this can be seen through an increase in the share of locally produced fruits and vegetables sold by domestic retail and food service channels (from 5 to 51 percent), as well as a 68 percent increase in the value of sales of fruit and vegetable growers participating in the MGP (compared to a target of 40 percent). The 2018 Market Survey 22 found that between 2013 and 2017, the absolute volume of local fruits and vegetables increased by 28 and 85 percent respectively. This is significant, particularly given the lower-than-expected uptake of the MGP by fruit and vegetable farmers. Several factors are likely to have contributed to this including training targeting non-MGP fruit and vegetable farmers which increased yields by 36 percent; increased demand spurred by in part by the Eat a Rainbow campaign; business plan development and support provided by SBEC and; market linkages established with the two largest supermarkets in the country. The significant increase in local produce suggests farmers were not only able to grow more produce, but also to sell it in local supermarkets, hotels, restaurants and other outlets. The difference between fruits and vegetables was likely due more vegetable varieties supported under the project. 46. Market linkages were initially delayed for fruit and vegetable farmers due to low numbers in the first two intakes of the MGP, however, the project was able to conduct a market linkage pilot during the third intake. This established partnerships between four registered fruit and vegetable farmer groups and two wholesale importers (Farmer Joe and Frankie’s supermarkets). The groups were supported through improved vegetable seeds for key import substitution varieties, such as onions and carrots, and were also able to display and sell their vegetables at the Agriculture Shows. More than 40 contracts/promises were signed by farmers, committing a list of commodities to the buyers. A Market Linkage Advisor was hired under SBEC to help the development of independent agencies and provided continued support to link them to market outlets. 47. When looking at some of the vegetables targeted under the MGP, a key finding was not only the increase in volume in the market, but also an increase in value. Prior to the project capsicums (also known as green peppers) were one of the main vegetable imports into Samoa and tended to fetch a higher price than locally grown varieties. Because of this, they were targeted through SACEP for import substitution and between 2013 and 2017, the Market Survey 22Using a sample of 22 market outlets, 8 supermarkets, 6 hotels, 5 restaurants, 2 Institutions and 1 food outlet, across Upolu and Savai’i Page 20 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) found that not only did local capsicums substantially increase in volume (local increased by 62 percent and imported increased by 39 percent), the total value increased dramatically for local compared to imported (local increased by 115 percent and imported by 15 percent). 48. The same was true for tomatoes, another crop targeted through the SACEP MGP as an import substitute option. Between 2013 and 2017, the volume of local tomatoes grew by 36 percent, compared to imported by 9 percent, and in terms of value, local tomatoes increased by 97 percent, compared to imported which increased by 14 percent. This suggests improved quality and demand for local produce supported by SACEP. 49. Support for accessing markets was also given through collaboration with SBEC as part of the MGP. All farmers accepted into the program were required to work with SBEC to develop a business plan to provide direction for the business and show the feasibility of the grant. There were 134 business plans prepared by fruit and vegetable farmers through the MGP and, while this didn’t meet the final target of the Results Framework (150), this is because of lower demand than expected by fruit and vegetable farmers; the combined number of business plans across all types of farmers (740) surpassed the combined target (600). Case Study: Banana Meristems Banana exports, particularly to New Zealand, had previously been a key export commodity for Samoa. At their peak in 1958, bananas accounted for one third of all of Samoa’s exports, however, disease, cyclones and supply chain problems led to reduced quality and supply. Exports dropped dramatically in the 1970’s, with average annual export values amounting to US$0.63 million, compared to US$6.16 million in the 1960’s.20 As the main fruit variety targeted by SACEP, MAF was able to import banana meristems from South Africa and pilot a new export value chain to New Zealand. The imported variety is more resistant to disease and also more productive, increasing the yield by up to 300 percent. The project developed banana planting materials from the meristems at the SACEP-refurbished tissue culture laboratory and distributed meristems to both established commercial farmers, as well as smaller farmers who had the potential to expand. Farmers were supported through outreach, as well as on-farm biosecurity improvements to avoid the spread of the Banana Bunchy Top Virus. A Banana Growers Association was formed under the project and drove the revival of the sub-sector. At the time of project closure, three Association members had successfully exported shipments to New Zealand. 23 Outcome (iv) greater advantage of market opportunities taken by livestock producers. This outcome was fully achieved in both ‘with and without restructure’ scenarios. 50. PDO level indicators used to inform this are below. Indicator Original Target Revised Target Result Percentage (share) of locally produced beef sold by domestic 55 50 94 retail and food service channels Percentage increase in the value of sales of livestock producers 40 30 46 participating in the MGP 23 2003by Euan Fleming and Anita Blowes. ISBN 1 86389 853 0. Working Paper Series in Agricultural and Resource Economics. University of New England. Page 21 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 51. The share of locally produced beef sold by domestic retail and food service channels increased from 31 to 94 percent over the life of the project, exceeding both targets; and the increase in the value of sales by livestock producers participating in the MGP increased by 46 percent, again surpassing the original and revised targets. These results, however, were not entirely consistent with the 2018 Market Survey which found local beef made up 84 percent of the beef market share in 2013 and 94 percent in 2017. This discrepancy in the figures is likely a result of inadequate M&E implementation, and the results still suggest an increase in locally produced livestock share of the market and price over the project life. 52. When this outcome was considered, a range of factors were looked at, which could influence the results: • Total meat volume was considered a good indication of access to markets, however market share was not, given the growing demand and consumption of meat in Samoa. • The use of hygienic slaughter services was an important factor, particularly with the introduction of the Slaughter and Meat Supply Act which required hygienic slaughter for public sale. • The number of business plans developed was also used to indicate an increased awareness and planning for market access and sustainability. • Prices contributed to the picture as well, as a possible reflection of quality and consumer preference, although this may have been influenced by supply and demand. 53. When looking at the market, the total volume of local meat increased by approximately 30 percent between 2013 and 2017, however with increased demand (and therefore supply), the overall market share for local meat only grew from 5 to 7 percent 24. For local produce, all four types of meat identified in the 2018 Market Survey (beef, chicken, mutton and pork) showed a significant increase in volume between 2013 and 2017. In particular, domestic pork increased six-fold, followed by chicken, which tripled in quantity, and mutton which doubled. Beef had the smallest increase, but was still significant at 67 percent; given its large market share at baseline, a modest increase was to be expected. The domestic growth is even more evident when compared to imported meat; with imported pork, chicken and mutton only increasing by 10-40 percent, while imported beef went down by 46 percent. 54. A total of 605 livestock were processed through hygienic field slaughter units – which met the target of 600 – reflecting the uptake of the Mobile Slaughter Unit (MSU). During field visits, farmers and local supermarkets confirmed the need for hygienic slaughter practices for meat to be sold to the public, as per the Slaughter and Meat Supply Act requirements. As with the fruit and vegetable farmers, support for accessing markets was also given through collaboration with SBEC as part of the MGP. A total of 606 business plans were prepared by livestock farmers, which exceeded the Results Framework target of 450. 55. In relation to price, small increases were seen across both imported and local produce, with the highest being in local mutton, increasing SAT $1.68/kg between 2013 and 2017. Imported beef and local pork both dropped in price, however for local pork it is possible the price dropped due to the 600 percent increase in supply which may have flooded the market. Justification of Overall Efficacy Rating 56. The split efficacy rating is Substantial/High. As outlined above, all four outcomes were partly or fully achieved with significant improvements for livestock and crops farmers in both productivity and access to markets. However, not According to the 2017 Market Survey, chicken is the largest meat import (90 percent) followed by beef (6 percent) which is 24 why significant increases in beef did not substantially impact the overall imported meat percentage. Page 22 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) all PDO-level outcomes were adequate to measure impacts, and as the theory of change illustrates, other outputs were also necessary to achieve the PDO. Because of this, Intermediate result indicators and other factors highlighted above also strengthen the justification for this rating. The project was rated Substantial for ‘without restructuring’ because, had the restructure not taken place, Outcome 1 would only have been partially achieved. 57. As this was the first agricultural investment project in Samoa in over a decade, there was not enough information to accurately assess the demand for and type of matching grants. To manage this, no beneficiary numbers were used as indicators and matching grants were flexible in their design, to suit local needs. For this reason, the reduction in intermediate-level indicators related to the number of business plans prepared by crops and livestock farmers for grant financing, was not considered a key aspect of the project and did not reduce the overall results and outcomes. C. EFFICIENCY Assessment of Efficiency and Rating 58. Economic and Financial Analysis. An Economic and Financial Analysis (EFA) was conducted at project closure to evaluate the Economic Internal Rate of Return (EIRR) and the Net Present Value (NPV). The EFA was not able to repeat the methodology used at project appraisal – which would use the actual project costs, duration and the estimated benefits based on real project data – primarily due to the demand driven nature of the project. Because of this, the investment activities envisaged at appraisal, and the investment activities supported by the project during implementation were substantially different. However, for this EFA, the various investment activities supported by the MGP were classified under similar types of investments and a cost-benefit analysis was conducted for each. The financial models included financing plans which illustrated how project activities resulted in increased incomes due to asset accumulation and working capital financed through project support and access to institutional sources of finance. In addition, a sustainability analysis was conducted for the MSU. All costs and benefits are expressed in constant prices, the analysis considers a period of 20 years. Details of the economic analysis are presented in Annex 4. 59. The ICRR EFA estimated an ERR of 13.3 percent and an ENPV of SAT 1.6 million (US$0.6 million) which is slightly lower that the appraisal estimates of 15.3 percent ERR and an ENPV of US$ 7.2 million (at 12 percent discount rate) 25. 60. The project generated short and long-term benefits for all stakeholders. Key direct benefits for MGP beneficiaries included: (i) for livestock farmers: enhanced productivity through on-farm investments, upgraded livestock, improved husbandry practices and stock management, and improved quality of meat sold in the local market; (ii) for crops farmers: enhanced productivity through on-farm investments, increased access to high yield crop varieties, improved technology and production techniques, and increased sale to retailers and foodservice operators; and (iii) increased access to extension services. Benefits also accrued to other subsistence and commercial farmers through access to improved breeds, planting material and technical advice on animal husbandry and crop production 25The ex-ante EFA for the project had assumed a social discount rate of 12 percent for calculation of the ENPV. Hence, for comparison the ex-post EFA also considered a 12 percent discount rate for calculation of the ENPV. However, as per current World Bank guidelines the discount rate should be between 1x-2x of the annual growth rate of the per capita GDP of the country. Hence, considering a social discount rate of 4% for Samoa, the ENPV will be SAT 22.2 million (US$ 8.5 million) which is higher than the ex-ante ENPV of US$ 7.2 million. Page 23 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) practices. Through SACEP, MAF increased their effectiveness and efficiency in improving the domestic production base of the country. Although not captured by the EFA, the business plan developed by the Pig Breeding Farm demonstrated it had the potential to not only be self-sustainable, but also to provide a profit for the government, if managed well. Additional benefits likely include improved nutrition and food safety, through for instance awareness raising campaigns and the introduction of the Slaughter and Meat Supply Act, as well as the increased participation of the private sector through more subsistence farmers moving towards commercial operations. 61. Aspects of the design and implementation. Implementation delays impacted the project’s efficiency. Slow procurement, particularly around the SSU and importation of livestock, meant slower results and prevented some activities from being completed by project closure. Cumulative extensions of 21 months were required to complete the majority of the MGP investments and was expected to enable the SSU to be built. A lack of adequate skills in the PCU, due to recruitment issues, likely contributed to this, particularly the procurement delays. As discussed in Section III.B. Tropical Cyclone Evan hit Samoa eight months after SACEP approval, significantly disrupting implementation for at least six months due to SACEP staff supporting the recovery project. 62. Some improvements in efficiency were seen over the life for the project, for instance the first MGP intake only had 48 farmers, compared to intake three with 534. This significant increase was only possible with improved levels of efficiency in the promotion, processing and management activities of SACEP. While staff turnover and recruitment slowed some activities, having the same Project Coordinator for the entire implementation period likely mitigated some of these impacts. 63. Efficiency Assessment. Based on the above aspects, overall project efficiency is rated as Modest. D. JUSTIFICATION OF OVERALL OUTCOME RATING 64. Based on ratings of High for Relevance, Substantial/High for Efficacy and Modest for Efficiency, the overall outcome rating is Moderately Satisfactory. The below table presents the outcome using a split rating assessment. Before Restructuring After Restructuring (03/29/12 – 02/13/17) 02/13/17 – 12/31/18) Relevance of PDO High Efficacy (PDO) Substantial High 1 Modest Substantial 2 Substantial Substantial 3 Substantial Substantial 4 Substantial Substantial Efficiency Modest Outcome rating MS MS Numerical value 4 4 Disbursement US$8 million US$5 million Share of disbursement 0.62 (or 62%) 0.38 (or 38%) Weighted value 2.48 1.52 Moderately Satisfactory Final outcome rating (2.48 + 1.52 = 4) Page 24 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) E. OTHER OUTCOMES AND IMPACTS Gender 65. At appraisal, Samoa’s achievement of gender equality targets was mixed. Samoa had ratified the UN Convention on the Elimination of All Forms of Discrimination Against Women, equal numbers of girls and boys were attending primary school, and girls were slightly over-represented in secondary education. Nevertheless, women made up less than 40 percent of the paid workforce, almost half of all women had experienced some form of domestic violence 26, and as a country Samoa was ranked 106 out of 186 countries in the 2013 Gender Inequality Index. 27 66. In 2018, 27 percent of subsistence farmers were women, and, of those, 40 percent fell below the basic needs poverty line – in the North-East of Upolu, this went up to 70 percent. 28 While specific locations or poverty levels were not targeted through SACEP, women did benefit from both the MGP and extension support and training. 67. In the MGP, 20 percent of farmers were women, with crops and poultry farmers having the highest representation (37 and 30 percent respectively). For non-MGP farmers, training such as field schools were provided, and across sheep, pigs, poultry and fruit and vegetables, women represented between 31 and 63 percent of farmers 29. Given the high level of basic needs poverty amongst female farmers, it’s likely the project included some women from this group and that they experienced the same benefits of increased productivity and access to markets as other farmers did, although more targeting could have been included to further support this group. Training provided by MAF was also disaggregated by gender, and 25 percent of farmers (both MGP and non-MGP) who participated were women. Unfortunately, this did not meet the project target of 30 percent. 68. Through collaboration with Women in Business Development Incorporated (WIBDI) the project targeted women farmers and supported their activities around organic produce opportunities and niche products. Support was also provided in the form of a double-cab pick-up truck to WIBDI’s Savai’i office, providing their only means of visiting farmers on the island and providing important extension services to them. Nutrition 69. Improving the nutrition of Samoans was a secondary, but complementary goal of SACEP. While focusing on import substitution through improved quality, production and access to markets, the project also supported activities on the demand side for consumption. Working with the Ministry of Health to develop the ‘Eat a Rainbow’ campaign (see Image 1 below), SACEP investments encouraged local fruits and vegetables be prepared in primary schools for morning tea, and public awareness activities advocated for people to eat all the colors of the rainbow. 70. While these activities were not measured, and the project is unlikely to see significant impacts on nutrition over such a short timeframe, the continued awareness raising campaign about healthy eating and local produce is likely to have had an impact. A possible proxy indication was that MGP crops farmers had average production increase by up to 122 percent, however, the ratio of quantity sold to quantity produced remained at 85 percent, indicating 26 According to the Samoa Family Health and Safety Study 27 The Gender Inequality Index is part of the Human Development Report and measures gender-based inequalities in three dimensions – reproductive health, empowerment, and economic activity. 28 Draft Country Gender Assessments of Agriculture and the Rural Sector for Five Pacific Island Countries. 29 SACEP team data. Page 25 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) increased self-consumption of fruits and vegetables. Given women in Samoa are more likely to be obese than men, this may also have a longer-term influence on their diets. Image 1. Advertisement for the Eat a Rainbow Campaign Institutional Strengthening 71. Component 3 of the project focused on institutional strengthening and included activities for both short- and long- term development. At project completion, 80 percent of Crops and 68 percent of APHD staff were completing training aimed at improving their technical capacity on extension services including in crop and husbandry practices, agricultural engineering and farm economics. There was some inconsistency in how these intermediate indicators were measured, as both achieved 100 percent during implementation, but were reduced at project closure. This was likely due to some staff turnover and different interpretations of whether the indicator measured the percentage of staff undertaking training at a point in time, or whether it measured those who had received training at some point of SACEP implementation. Training will be ongoing by MAF and supported through the follow-on project, Samoa Agriculture and Fisheries Productivity and Marketing project (SAFPROM) to ensure consistent quality and to manage staff turn-over. For the efficient operation of the MSU, and soon to be built SSU, four APHD staff undertook specialized slaughter unit training in Australia. 72. An Enhanced Extension Strategy was prepared and adopted by MAF following stakeholder discussions in 2014. It included three strategic themes: (i) priority to farming as a business, (ii) developing opportunities for private sector and industry growth; and (iii) supporting partnerships for sustainable farming livelihoods and employment skills. The project also expanded the scope of work for the APDH and Crops Divisions and supported the recruitment and onboarding of seven new roles. These included two slaughter unit staff, two meat inspectors, a Principal Abattoir Officer and two tractor operators. These new roles will support MAF to continue activities and initiatives started under SACEP. Page 26 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 73. Infrastructure was also provided through SACEP, to strengthen the Ministry’s ability to provide high quality support and services. Renovation works were carried out on APHD’s three nucleus livestock farms including farm stockyard, cattle weighing scales, pasture fencing, sheep sheds and a piggery and poultry unit. These investments will enable the APHD to continue breeding high quality livestock for distribution to Samoan farmers and allow them to advance research into improved husbandry and pasture practices. 74. For the Crops Division, a tissue culture laboratory was expanded, machines for rock removal and farm preparation were purchased and an improved quarantine laboratory was built. These activities support continued research and development in the crops sector, with new banana meristems already underway and a range of other crop varieties under multiplication. While the equipment purchases were necessary, MAF doesn’t have the budget for additional expenses such as repairs, particularly given parts often need to be shipped into the country, leaving some of the machinery already in disrepair. 75. To increase Samoa’s capacity to sell local meat, the project supported the development of the Samoa Slaughter and Meat Supply Act 2015. This was passed by Parliament in July 2015 and is aimed at improving meat quality and hygiene in the country, so local produce is better able to compete with imports. 76. An area of improvement which was highlighted by the Government of Samoa, was the use of consultants for the Project Coordination Unit (PCU) which were employed for the life of the project, rather than as permanent staff in the Ministry. This variation of contract types was believed to have three negative impacts: (i) project and institutional knowledge was lost at project closure; (ii) project activities were often seen as ‘separate’ to MAF functions and so not fully ingrained in daily operations and responsibilities; and (iii) PCU recruitment potentially competed with MAF and GoS hiring as PCU salaries were higher. 77. To mitigate future shortcomings such as those above, the GoS has established Sector Coordination Divisions in Ministries to support the implementation of donor-funded projects. For MAF and SAFPROM, the Agriculture Sector Coordination Division (ASCD) is under recruitment and includes a Project Coordinator, and four Principal Officers for Procurement, Financial Management, Safeguards and Monitoring & Evaluation. Mobilizing Private Sector Financing 78. The project included several aspects that mobilized private sector finance. Agriculture is often considered a ‘risky’ investment and farmers discussed challenges in getting credit, prior to joining the MGP. Through both the matching grant and a 100 percent guarantee by SBEC, the project was able to ‘de-risk’ investments and enable more access to finance for farmers. By requiring all grants to be matched 100 percent by recipients, all investments leveraged capital from farmers and confirmed their commitment to their activities. 80. The collaboration with SBEC to support all MGP farmers in preparing business plans encouraged households to approach their farming as a business. At project appraisal 97 percent of households engaged in some form of agriculture, but only a small percent had agriculture as their main source of income, with numbers slightly higher at project closure. 30 30In 2018, 19% of households had agriculture as their main source of income according to the Draft Country Gender Assessments of Agriculture and the Rural Sector for Five Pacific Island Countries. Page 27 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 82. By improving input supply, through new livestock breeds and genetics, as well as more climate-resistant crop varieties, the project supported increased investment by farmers into their produce. At the production and processing levels, increased productivity, diversification and quality is likely to have longer-term impacts on private sector development. More consistent and reliable produce will encourage strengthened market linkages and may lead to further import substitution. 84. New machinery purchased by the Crops Division was hired out to farmers at a reduced rate, so they could access these services. This was also true for APHD, with farmers using a pay-for-service scheme for the MSU. These activities encouraged private sector involvement and created revenue streams for the Divisions. Unfortunately, these fees do not stay within the Divisions and insufficient recurrent costs were budgeted for maintenance and repair. At project closure two Crops Division machines were broken, without funds to fix them. Poverty Reduction and Shared Prosperity 85. At project conception, extreme poverty in Samoa was low (3 percent) and basic needs poverty had been steadily declining, after a sharp rise due to the global financial crisis in 2008, from 27 percent in 2008, to 19 percent in 2013/14. 31. At appraisal around 50 percent of individuals below the basic needs poverty line and 31 percent of the extremely vulnerable lived in North-West Upolu and worked primarily in subsistence agriculture. While most activities with farmers were self-selecting, given the subsistence farming focus and that farmers came from across all parts of the country, it’s likely some vulnerable or poor farmers project beneficiaries. 86. An unintended consequence of the project was its potential to increase financial strain on some farmers. The processing requirements involved in the MGP proved difficult for some subsistence farmers who lacked basic financial skills and often were not able to write in English (so business plans were written in Samoan and translated into English, with the potential for details to be lost). Many farmers needed additional support from SBEC and the project team, who were already at capacity in terms of workload and often unable to provide the level of support needed. At project closure 20 percent of MGP grants were incomplete and 21 percent of farmers who took out loans with DBS were in arrears. This has the potential to create additional financial strain on farmers who may not have understood the loan requirements, or who did not have the skills needed to comply with the project requirements. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 87. Realistic objectives. The project’s outcomes were highly relevant and the majority of targets, both for outcomes and outputs, were appropriate. The theory of change above outlines a clear linkage between outcomes, outputs, inputs and activities and infers that suitable objectives were selected. 88. Simple design. Agriculture projects which work with small-scale and subsistence farmers are often complex in their design and SACEP was no different, particularly given the need to import live breeding stock and the construction of an abattoir. The risk rating for project design was High at appraisal, which given the challenges faced during implementation, is considered appropriate. To mitigate this, a significant amount of experienced, 31 UNDP: Samoa Hardship and Poverty Report 2016. Page 28 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) well targeted technical assistance was budgeted for to both upskill APHD and ensure sufficient capacity to implement the tasks when needed. While the design and budget allowed for adequate staff and skills, difficulty in hiring staff meant the complexity was often above the available capacity. 89. Unlike other countries where cooperatives in agriculture are promoted, the team received substantial push- back during preparation on incorporating them into the design. Anecdotal evidence was provided by multiple stakeholders on the unwillingness of farmers to enter into these arrangements. These concerns were incorporated into the design, while leaving open the potential for farmers to organically develop groups through the market linkages pilot. This approach was well received by the government and appears to have been successful given the establishment of some farming cooperatives. 90. Well-designed Results Framework. The majority of PDO-level indicators were aligned to the Project’s development objective, however the review found there were areas for improvement. The use of average fruit and vegetable yields in PDO-level outcomes one and two proved very general and challenging to measure. No PDO-level indicators aligned to outcome two: Improved productivity of livestock producers. There were, however, intermediate-level indicators which informed its evaluation, although these did not capture the entire productivity picture. The Results Framework is further discussed under the Quality of Monitoring and Evaluation section. 91. Some targets for crops may have been overly ambitious, given the first restructure included a reduction in two the PDO-level indicators which related to fruit and vegetable yields. The number of MGP participants for fruits and vegetables was also lower than originally anticipated, however the total demand was exceeded. 92. Adequacy of risk and mitigation measures identified. At the time of project preparation, SACEP’s overall risk rating was High. There hadn’t been an agriculture project between the World Bank and the Government of Samoa since 1998, meaning there was a lack of experience with implementing an IDA-financed project. A shortage of experienced, skilled, technical and managerial staff was reflected through key positions in MAF sitting vacant for long periods both in project preparation and implementation. The risk rating for the capacity of the implementing agency (MAF) was therefore also High. 93. To mitigate these risks, a Project Coordination Unit was set up and positions created for seven full-time, experienced staff, including a Project Coordinator, two Procurement Officers (one specifically for the MGP), a Financial Management Officer, an Environmental and Social Management Officer, a Matching Grants Officer and an M&E Officer. Additional short-term technical advice was budgeted for to ensure adequate knowledge and skills over the life of the project. It was suggested that MAF use Retroactive Financing to recruit the Project Coordinator and Principal Procurement Officer, so they were on board by effectiveness and could start key activities, however, recruitments were delayed and only completed in the second half of 2012, with vacancies continuing to be a challenge into implementation. 94. Readiness for implementation. There was a long delay from Concept to Approval (33 months), which was due to ongoing discussion on the Government side about the project design and stalled preparation while details were confirmed. Ahead of project approval retroactive financing was made available and was used for an important cattle census which provided much-needed information for the livestock component. However, retroactive financing wasn’t utilized for recruitment, with some PCU positions sitting vacant into the project implementation phase. The Project Implementation Manual had, though, been prepared and project design Page 29 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) had been substantially fleshed out, avoiding delays due to their preparation. B. KEY FACTORS DURING IMPLEMENTATION (a) Factors subject to the control of the government and/or implementing entity. 95. Coordination and engagement. As highlighted earlier, the PCU was seen by some staff as somewhat separate to MAF. Going forward, an Agriculture Sector Coordination Division will be absorbed into the MAF structure to encourage increased coordination and adoption of project activities. The project didn’t adopt an e-voucher system, developed through the Cyclone Ian Recovery Project, which was considered by the PCU as a missed opportunity for the MGP. While it could have facilitated online systems and tracking, increased transparency and reduced ineligible expenditures, it should also be acknowledged that it would have needed significant tailoring for SACEP. 96. While the project design recognized that more vulnerable farmers (such as women and those falling into basic needs poverty) were unlikely to make up the majority of matching grant recipients, more attention could have been paid in encouraging them to apply and/or ensuring they were part of the wider training provided. 97. Human resources and organizational capacity. Having the right skills and adequate staff was a challenge throughout the life of the project, although this was less a MAF-specific issue and largely reflective of limited local supply and Samoa’s extreme remoteness. While project preparation ensured suitable roles and skills were budgeted for, it was not until a year after approval that the unit was fully staffed. Eighteen months after project approval the matching Grant Program Officer passed away and was never replaced and the Procurement Officer role remained vacant for much of the project’s life. Having said this, the Project Coordinator role was held by the same person from approval to closure, providing much-needed continuity and institutional knowledge. 98. While there were some delays in the importation of live cattle, sheep and pigs, these activities were innovative for Samoa and extremely complex in terms of procurement and transport. For example, for the importation of Droughtmaster Cattle and Dorper Sheep a flight had to be chartered from Australia to Samoa as transport by sea would require stops in other countries and lead to extensive waiting times at quarantine stations. On the other hand, the construction of the Static Slaughter Unit was also the first of its kind in Samoa and the delays encountered meant it could not be completed by project closure. This was due not only to its complexity, but also MAF’s capacity and having access to skills locally to manage the procurement process. 99. Appropriate selection of stakeholders. The Matching Grant component was demand driven, meaning beneficiaries were self-selecting. This approach encouraged farmers who were already eager to move to more commercial operations to take part, which was important given the effort required and potentially played a role in the high success rate (80 percent) of matching grant investments. However, no means testing was done which also risked an element of elite capture, where farmers with more income and financial literacy were able to better understand the systems and benefit from the MGP. 100. Fiduciary. Throughout the project there were delays in both procurement and financial management (FM) activities. This was largely due to a lack of skills or people available to carry out tasks and manage their progress. Country systems were followed for FM activities; however, staff were often unaware of legislative requirements or processes. World Bank Procurement Guidelines were used for the project, however for significant periods of project implementation there no procurement staff in the team, leading to significant Page 30 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) delays in many of the activities. 101. MGP Monitoring and Compliance. Overall MGP monitoring and compliance was adequate, however, the scale of monitoring participants did prove challenging during project implementation and led to some compliance issues and delays. Over the three MGP intakes, 80 percent of investments were completed; 10 percent were incomplete due to slow project implementation and so couldn’t be finished before SACEP closure; approximately 5 percent were cancelled due to factors such as death, migration or land disputes; and just less than 5 percent were incomplete due to non-compliance issues such as mis-use of funds or ineligible households. 102. Of these non-compliance grants, MGP procedures were able to catch a significant number of these, with no ineligible investments receiving all three tranches of grant payments, and the majority were stopped after the first payment was made. However, the team was inadequately staffed for the amount of supervision required to manage this and were not able to provide enough support to get slow-moving projects back on track. The PCU also didn’t have the right skills for grant fund recovery, when investments were deemed ineligible. 103. Attempts were made for better screening and more streamlined processes for farmers, and incomplete investments reduced over the three intakes, from 35 percent in intake one, to 15 percent in intake three, even though the actual number of MGP investments grew by more than 10 times the original number. (b) Factors subject to the control of the World Bank. 104. Adequacy of supervision. The first restructure in February 2017 included reduction of multiple targets in the Results Framework. This was done four years into project implementation and after the third and final intake of matching grant applications had been received. Given the numbers in intakes one and two, it was foreseeable that the project would not meet the anticipated number of MGP farmers or achieve some of the PDO level outcomes. For this reason, the team could have been more proactive and restructured the project earlier. 105. Adequacy of reporting. The team visited the project on average twice per year and provided thorough and candid updates through the ISRs, aide memoires and management letters. (c) Factors outside the control of government or implementing entities. 106. Natural disasters. Eight months after project approval, in December 2012, Tropical Cyclone Evan hit Samoa. One of the worst cyclones to hit Samoa, Cyclone Evan killed 14 people, destroyed more than 600 homes and was estimated to cause damage and losses equivalent to 30 percent of Samoa’s GDP. 32 Due to the significant impacts on agriculture and fisheries, the Samoa Agriculture and Fisheries Cyclone Response project was initiated, and the SACEP PCU supported a significant amount of its preparation. This impacted SACEP implementation, particularly in the first 18 months of effectiveness and led to some delays. 32 WBG Resilient Recovery in Samoa after Cyclone Evan (2014) Page 31 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 107. The project’s theory of change showed a well-organized design with clear linkages and outputs to reflect progress. The Results Framework for the project was generally thorough, with a range of outputs and outcomes used to measure impact; each indicator also had a defined data source, methodology and responsible party for collection, however given some of the weaknesses in implementation, is it likely they were not commensurate with PCU capacity. Data sources were kept to a minimum, with standard MAF data used for several indicators and two annual surveys planned for indicators it couldn’t cover, surveying both project participants and markets (e.g. supermarkets, hotels and restaurants). 108. A range of activities and outputs in the theory of change are captured in the Results Framework, however, as highlighted above, there was no PDO-level indicator to monitor “Improved productivity for livestock producers”. This was, instead, monitored through intermediate-results indicators. A potential gap between the theory of change and the Results Framework was a lack of monitoring farmers’ access to finance. 109. A key challenge was selecting what to measure and how, this was due to the broad range of both livestock and crops supported by the project, as well as various ways to measure produce (for example, by weight, number, frequency of birth/planting cycle). For crops, an average across all varieties was used, this proved too general and led to variations in measurement over the life of the project as farmers used different measurements which were hard to reconcile into one ‘average’. For livestock the calving rate and litter size was used for cattle and pigs respectively. This missed the opportunity to capture other forms of reproduction rates, such as litters per year. No specific indicator on poultry, sheep or bees, which were the three lowest, in terms of number of MGP farmers. 110. It was found that some indicators were phrased similarly, but with different meanings, which caused some confusion for the implementing team. This includes the use of ‘Percentage increase in…’ and ‘Increase in share/proportion of…’. This was present throughout PDO and intermediate level indicators. M&E Implementation 111. As highlighted above, a key challenge for M&E is the lack of standardization across the crops and livestock being supported. This was highlighted as an issue, particularly when measuring crop yields, as farmers would use different measurements, such as a bunch of cabbage or a bag of tomatoes, but these might change in number or weight throughout the year, depending on season and supply. This created inconsistencies in data which made it hard to compare results over the life of the project. 112. Another key issue which emerged was the lack of record-keeping by farmers. Although MGP farmers were provided training and encouragement to keep up-to-date records, only 57 percent of crops and 34 percent of livestock farmers did. This, though, was a substantial increase from the control groups where only 4 percent of crops and 0.5 percent of livestock farmers kept records 33. Not having accurate or up-to-date records impacted M&E accuracy and made data collection more time consuming as SACEP staff would visit farms individually to work through numbers. 33 MGP Impact Assessment Report (2018) Page 32 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 113. Serval target values included in the original Results Framework proved too ambitious for the project, and so were revised down in the first restructure of 2017. Given the MGP was already in its third intake and a strong indication of numbers was available, this could have been done earlier in the project life. The revised targets were generally more appropriate, and the majority were achieved by project completion. Some changes were proven unnecessary as the final results met the original targets. 114. At project completion data on PDO-level indicator #3 (Percentage of locally produced beef sold by domestic retail and food service channels) was questioned due to the substantial increase; from 30 to 94 percent over the life of the project. It was found that the methodology for the baseline data was different to the results, where the baseline was determined using DBS food import numbers, and then averaging this across all livestock animals, the results data used the market survey which had beef-specific numers and so was more accurate. Had the Market Survey data been used for the baseline, it would have been 84 percent of locally produced beef sold on the domestic retail market. The use of two different methodologies for one result reflects a significant inconsistency in the M&E implementation, however both still show an increase in domestic beef sold at market. M&E Utilization 115. The M&E surveys undertaken through the project, including the MGP Impact Assessments and Market Surveys, were used to inform project activities and planning, as well as to communicate results internally. Additional resources were directed to areas showing weakness, such as increased outreach for safe and hygienic slaughter when monitoring data showed slow uptake. Targets were amended in the first restructure to reflect current M&E findings, however it could be argued that the first restructure could have been done earlier, given the data available. A Management Information Sydney (MIS) for MAF was originally envisaged as part of the project, and its development would have encouraged more utilization of SACEP data. The MIS was not completed, however the data sourced through the project is expected to provide much of the foundations for an MIS to be developed under SAFPROM, as well as to inform its Results Framework. Justification of Overall Rating of Quality of M&E 116. The overall quality of M&E is rated Modest. The design was clear and appropriate for the project and was generally sufficient to assess achievements, test the links with the results chain and provide a good foundation for attribution. However, some shortcomings were present regarding their clarity and implementation. The surveys undertaken through the project provide a solid foundation for future activities and World Bank-funded projects and were also valuable to provide additional data relevant to the PDO. Not establishing a fully functional MIS was a missed opportunity to expand the value and impact of the data collected. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 117. The project triggered two safeguards policies - OP4.01 Environmental Assessment and OP 4.09 Pest Management – implementation of which was governed by an Environmental and Social Management Framework (ESMF). The key environmental safeguards risks were associated with the establishment of the static slaughter unit (SSU), however as this sub-project was not progressed under the project (and will be completed under SAFPROM) the risk oversight was not required. Overall safeguard compliance was satisfactory; however there is an important opportunity to continue working on integrated pest management capacity building under SAFPROM. This will require alignment with Government of Samoa initiatives on ecosystem services and cooperation with the Ministry of Natural Resources and Environment (MNRE). Page 33 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 118. The Fiduciary risk rating varied between Substantial and Moderate throughout the Project. At appraisal, the assessed Financial Management risk was Substantial, however it was noted that it could be reduced to Moderate, should mitigation measures be implemented. The overall Procurement risk rating was High, however it was also proposed to lower this to Moderate, should the agreed actions be implemented. 119. Implementation support missions were conducted on average twice a year during the Project. Financial Management and Procurement performance was generally rated as Moderately Satisfactory. The Bank’s Financial Management and Procurement Specialists provided implementation support to help the client improve implementation. C. BANK PERFORMANCE Quality at Entry 120. Strategic relevance and approach. The project was highly aligned with the World Bank’s Country Partnership Strategy for Samoa (FY2012-2016) (Report #67194), as well as the Government of Samoa’s 2012-2016 Strategy for the Development of Samoa, and MAF’s Agriculture Sector Plan (2011 – 2015). It was the World Bank’s first foray into the agriculture sector since 1998 and came at a time when agriculture was seen as a key investment for Samoa’s development. Given the implementing agency’s lack of experience in implementing World Bank-funded projects, the project approach was relatively simple, with only core activities included and direct links to achieving the PDO. Importantly, the design also reflects local requirements, for example not using collective grants for farmer groups, as these received a lot of push-back from stakeholders. 121. Technical, financial and economic aspects. The project’s Economic and Financial Analysis at approval was strong and thorough. The anticipated NPV for farm enterprise investments, computed at a 12 percent discount rate, was approximately US$7.2 million. The financial and economic IRR for the entire project was estimated at 13.5 and 15.3 percent respectively. 122. Poverty, gender and social development aspects. At the time of entry, basic needs poverty, particularly for subsistence farming households, was high and many smallholder farmers were vulnerable to increased food and fuel prices, as well as the growing impacts of climate change. Because of this, it’s likely the project supported many of these farmers, however no active measures were taken to ensure it. The project lacked targets for poor or marginalized groups such as women and as the MGP was demand-driven, it was exposed to elite capture, with large farms and successful businessmen using some of the grants. 123. Environmental aspects. The project actively supported climate resilient investments, through poly-tunnels and rainwater harvesting systems to support farmers during periods of heavy rain or drought, climate-resilient crops were also brought in and encouraged, and livestock which suited the local climate and environment were used, or interbred for better characteristics, as was the case for sheep. 124. Implementation arrangements. As the implementing agency, MAF had limited experience in implementing a World Bank-funded project. To mitigate this, additional staff positions were included in the PCU to support capacity development and a significant amount of experienced, and targeted technical assistance was budgeted to both upskill APHD and ensure sufficient capacity to implement the tasks when needed. Given the delays experienced, and that the SSU was not able to be completed, additional efforts may have been beneficial to support MAF’s implementation capacity. An additional three positions were also added in SBEC, to support MGP participation. Page 34 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 125. M&E arrangements. As discussed in the M&E section, quality at entry was moderately satisfactory. It included a sound framework and data collection strategy, however some indicators only showed some of the attribution story. An M&E officer position was created to manage these tasks for the life of the project. 126. Risk Assessment. At entry, SACEP’s risk rating was High which, given the challenges faced during implementation, is considered appropriate. Quality of Supervision 127. Focus on development impact. The Bank was well-focused on achieving its PDO and undertook regular missions to monitor progress and resolve issues. The team undertook two restructurings to better align project results and to extend the project to allow time for activities to be completed. In hindsight, first restructure could have been undertaken earlier in project implementation, given there would have been a good indication of MGP numbers and results after intake two. 128. Supervision of fiduciary and safeguard aspects. Safeguards and fiduciary supervision were adequate and commensurate with the project risks. For Safeguards there were moderate risks associated with the small-scale interventions financed under the project and supervision advice enabled screening mechanisms included in the original ESMF to be revised and simplified to suit local circumstances. Both PRIMA and PRAMS assessments were carried out throughout project implementation, with FM and Procurement staff joining implementation missions on a regular basis and/or meeting with relevant staff at other times. 129. Adequacy of supervision inputs and processes. Overall, the task team undertook 13 full implementation support missions, with an average of two per year. To assist in addressing challenges during implementation, missions undertook site visits and held regular discussions with relevant stakeholders such as SBEC, DBS and farmers. Critical issues that emerged during implementation and for which the WB was able to provide support include the importation of livestock, the purchase and amendments to both the MSU and SSU, and management of incomplete MGP activities. 130. However, the project was first restructured only one month prior to its original closing date, which was not as pro-active as it could have been. All three MGP intakes had been started by that stage and it could be argued there was a good indication of the lower demand and higher cost of investments before this. The project was also rated throughout its life as Moderately Satisfactory or above for both Implementation Projects and progress towards the Development Objective. The need for a restructure, coupled with having only disbursed 62 percent one month before closing date should have merited a Moderately Unsatisfactory rating at that time. The project also extended twice in 12 months which suggests the first extension was overly optimistic. Justification of Overall Rating of Bank Performance Based on the above, the overall outcome rating for World Bank performance is Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 131. At the time of project completion, several initiatives had been taken to reduce the risk to its development outcome, however some hazards remained, these are discussed below. 132. MAF’s Agriculture Sector Plan 2016-2020 had been prepared as part of SACEP implementation and Page 35 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) provided a framework to guide MAF’s work in this space. Key activities in this plan which will continue to contribute to SACEP’s development outcome include: o Strengthening the institutional capacity of MAF, particularly in carrying out a sector-wide approach. This will include amendments to the policies, laws and regulations in an effort to foster competitive agricultural markets and inclusive value chains. Activities will be supported by the establishment of an ASCD, which will manage donor-funded projects across MAF and will initially be funded by the World Bank as part of a follow-on agriculture project. This will support sector growth and quality, encouraging farmers to become more commercially-oriented. o Enhancing smallholder productivity, improving the supply chain quality and reducing post-harvest losses. This was supported by SACEP activities and will continue to promote new technology adoption, strengthen farm business management skills and market linkages. o Value chain promotion, particularly in areas of competitive advantage for Samoa, will be prioritized. The promotion of country origin, branding and organic quality for exports will support many of the crop farmers, particularly those growing bananas. 133. The indicative cost of implementing the plan (approx. SAT $117 million) was largely equivalent to the development funding available, increasing the likelihood of the activities being implemented. 134. The Government of Samoa has approached the World Bank to begin a follow-on operation; the Samoa Agriculture and Fisheries Productivity and Marketing (SAFPROM) project. This is under preparation and will build on the work of SACEP, encouraging more farmer groups as well as extending the matching grants to fishers. The project will mean much of the support to farmers will continue, allowing time for the sector to develop further. The Static Slaughter Unit will be completed and work will be done to encourage its use, further supporting market opportunities in the livestock sector. The project will also support MAF to prepare the next Agriculture Sector Plan, which will put in place a strategy for objectives to be sustained. 135. There are, though, risks that remain. Ownership is discussed below as a key lesson learnt and poses a potential threat to SACEP’s development outcome. While SACEP staff were expected to remain on interim contracts after project completion – both to provide continuity and to support the preparation of SAFPROM – for a range of reasons this did not eventuate and by April 2019 no SACEP staff remained in MAF. This meant a lot of institutional knowledge was lost and without full integration of SACEP activities into MAF core functions, it also has the potential to slow down much of the work started under SACEP, as well as to delay activities planned under SAFPROM. 136. Lack of adequate maintenance budget is also a risk to its development outcome. As previously discussed, revenue from MAF Division activities, such as hiring out machinery in the Crops Division or pay- for-service use of the slaughter unit(s) in the APHD, doesn’t stay within the division and recurrent budget for maintenance and repairs was highlighted as a potential risk. To mitigate this, all large investments had business plans developed to determine their sustainability and were only undertaken if appropriate, however if budget is not allocated, it could halt these activities and reduce the extension services available for farmers. 137. While SACEP specific knowledge and skills are important, MAF capacity overall was consistently highlighted as a challenge, both in terms of being able to recruit the right candidates, and for having the human resources available for the work required. This is particularly apparent when considering the Page 36 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) veterinary capacity of MAF. Currently MAF has only one vet, needing to service the entire country. With staff turnover, this could create significant delays and challenges for supporting livestock farmers. 138. As with many Pacific Island countries, climate change, natural disasters and economic shocks remain as significant risks to development objectives. For Samoa and SACEP, this is especially true, given the high vulnerability agriculture has to climate change and natural disasters, and that Samoa is predicted to experience more frequent and extreme rainfall events, longer drought events, extreme winds, and high air and water temperatures.34 While climate resilient investments were encouraged through SACEP, imported crops and livestock were selected for their resilience, and the Government of Samoa is actively investing in resilience activities 35, depending on the type and severity of disasters there is still the risk of undoing much of the progress made. V. LESSONS AND RECOMMENDATIONS 139. Several lessons and recommendations can be drawn from SACEP. These have been broken into those relevant to Samoa and the Pacific, and those which are applicable to the agriculture sector in general. Lessons and recommendations related to the Samoan/Pacific context: 140. Inadequate staff and skills can delay project implementation. Throughout project implementation, the PCU struggled to hire and retain people of the required caliber and with the necessary skills to support efficient and effective implementation. This was particularly apparent for procurement and MGP management. While positions were advertised extensively, thin capacity in the country meant the right skills weren’t available when needed. While there are limited options to address this, some possibilities include: o Ensure budget is available for supplemental consultants to provide surge support, or just-in-time support as-needed during implementation. Particularly at key milestones and busy periods of project implementation, hiring international consultants to support the project may be necessary, however they will have to be managed sufficiently to ensure value. o Increase training for MAF staff. As the required skills proved difficult to find locally, in the future it may be necessary to hire a more junior staff member and up-skill them for the role. A skill gap analysis would initially help; however, its successful implementation would also require sufficient budget, careful monitoring and a consistent approach to mitigate staff turnover. o More closely monitor procurement and recruitment activities. Staff and skill shortages are not unique to this project or Samoa, but are a common challenge across the Pacific portfolio. Being more realistic about timelines required for recruitment and increasing implementation support may benefit project implementation. 141. Distinctions between the project team and Implementing Agency can reduce ownership. Ownership was raised as an area for improvement with many tasks being the responsibility of SACEP staff, and not integrated into MAF day-to-day functions and reporting. This was exacerbated by some SACEP project staff being hired outside of MAF human resource standards and on project-length contracts. Related aspects such as ongoing work and maintenance are also impacted by this. Going forward, this will be addressed through the introduction of a Centralized Technical Services and Support Unit (CTSSU), housed within MoF, as well as 34 COP23 country forecast: https://cop23.com.fj/samoa/ 35 For example, the Pacific Resilience Program in Samoa (P154839) Page 37 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) an Agriculture Sector Coordination Division (ASCD) which will form part of MAF’s Organizational Chart. The ASCD will be comprised of Principal Level Officers in Procurement, Financial Management, M&E and Safeguards and will managed by an ACEO as Project Coordinator. This has the intention of absorbing roles and activities into MAF, to improve not only coordination and ownership, but also reduce knowledge loss when projects close. While originally the intent was for SACEP PCU positions to be absorbed into MAF at the closure of the project, it did not eventuate, and all project staff needed to reapply for roles within the follow-on project. Sector Coordination Divisions will also be created in the lead agency of other sectors which are implementing donor-funded projects. As seen under SACEP, finding and maintaining the right skills will be an important factor in the Sector Coordination Divisions’ success. 142. Monitoring and Evaluation (M&E) should be as simple as possible. M&E is often a challenge given the limited data available during preparation and throughout project implementation. In agriculture, where many famers keep little-to-no records and surveys are time consuming and expensive, this is exacerbated. While initially envisaged, a Management Information System (MIS) was not developed through SACEP. Lessons and recommendations related to the agriculture sector: 143. Communicating with external stakeholders is important for project implementation and outcomes. Communication was consistently highlighted both by MAF and SACEP team members, and by external stakeholders, as an area of weakness during project implementation. Misunderstandings about the process for applying for matching grants, how the 100 percent guarantee from SBEC worked, how to receive payments and why payments were not received were all common concerns. For example, the ICR team found that while delays in payments were often due to a lack of record-keeping or presentation of receipts, farmers were unaware of these requirements, which caused them financial stress and frustration. Uptake of the MSU could also have been increased if more farmers were aware of it and understood its value in terms of the Slaughter and Meat Supply Act. While some communications and outreach was undertaken by the PCU and reasonable budget was allocated for this during project preparation, the outputs either lacked the impact anticipated, or were not completed as planned 36. 144. Adapting to local contexts is critical. While most matching grant activities target agribusinesses or farmer groups 37 this project was designed for individual farmer grants, largely due to strong feedback from stakeholders on what would work in the Samoan context. This was an important aspect of the project, and likely influenced the project’s success. At the same time, a pilot market linkage activity was included which supported the inception of farmer groups. Having a project design that reflected local needs while setting the foundations for global good practices means future projects can build on its successes. A significant contributor to this contextual approach was a complementary ASA38 done prior to project approval which informed the Government of Samoa’s Agriculture Sector Strategy. Through the ASA, World Bank staff were able to gain a thorough understanding of the sector and its constraints. 145. It’s important to consider the benefits and impacts of beneficiary selection or targeting. Matching grants provided under the project were demand-driven and provided on a first-come, first-served basis, if they met the eligibility criteria. Outreach about the grants was done through the MAF farmer database and 36 A strong communications plan for matching grants, that starts at project approval, was highlighted as good practice by the World Bank’s Agriculture Finance Note #1 - Lessons Learned from World Bank Projects Using Matching Grants (June 2017) 37 Designing and Implementing Agriculture Innovation Funds, World Bank (May 2010) 38 The Samoa Agriculture Sector Strategy (P120701) Page 38 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) other public mediums such as radio and newspapers, with no beneficiary types targeted. This had positive impacts as it meant MGP farmers were already commercially-oriented, which proved important given the processing steps involved, and not being supply-driven meant all farmers wanted to be there which is likely to support the sustainability of outcomes. However, some negative consequences were also noted, such as a lack of female farmers 39 and elite capture of grants. Some considerations to mitigate these risks include: o Having more targeted outreach to encourage vulnerable farmers to apply for grants and providing the budget/staff to enable this. o Applying targets for priority beneficiaries such as poorer or female-led farms. o Making grants ineligible to farmers who earn over a certain amount. While this won’t entirely eliminate elite capture, it will reduce the number of farmers who already have a taxable income and are less in need of the grant. o Specific investments which are likely to have longer-term benefits, such as climate-smart agricultural activities (i.e. rainwater harvesting or improved crops/livestock breeds) and private sector development (hygienic slaughter or organic certification) should be prioritized over general farm upkeep and investments. 40 146. Matching grant percentage contributions were appropriate, however ways to reduce loan defaults should be considered. Matching grant contributions vary across the World Bank, with the majority (60 percent) offering a matching of 50 percent or above. Good practice has found beneficiary contribution must be set high enough to ensure ownership and to crowd in commercial credit 41, while also reflecting the capacity of the beneficiary 42. SACEP provided a 50 percent grant with the option of farmers taking a 30 percent loan and offering some contributions in-kind and appears to have been appropriate given the uptake and increased access to finance. However, a significant number of accounts ended in arrears (21 percent of farmers who took out loans with DBS). Some possible mitigation measures include: o Scaling the grant amount itself to reflect farmer capacity to repay it. o Improving MGP business plan assessments could reduce investments unlikely to succeed. DBS acknowledged overstretched capacity when reviewing MGP business plans, thus increasing resources around these assessments and ongoing support could improve loan repayments. 147. Stronger linkages between activities may enhance results. While training was given to farmers on how to improve their practices and the matching grants increased access to new and innovative technologies, more connections between the project activities and matching grants might have amplified the successes. For instance, more requirements for matching grant recipients in the livestock sector to use the hygienic slaughter units may have accelerated uptake and general awareness; more incentives for farmers to use new or innovative technologies may have increased the climate resilience. . 39 While SACEP achieved 20 percent female farmer participation, as a sector 30 percent of subsistence farmers are female. 40 This recommendation is from the Agriculture Finance Note #1 - Lessons Learned from World Bank Projects Using Matching Grants, World Bank (June 2017) 41 Agriculture Finance Note #1 - Lessons Learned from World Bank Projects Using Matching Grants, World Bank (June 2017) 42 Designing and Implementing Agriculture Innovation Funds, World Bank (May 2010) Page 39 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: 1: Improved productivity of fruit & vegetable growers and livestock producers Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage increase in Percentage 0.00 100.00 50.00 53.00 average fruit and vegetable yields of growers 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 participating in the MGP Comments (achievements against targets): Target achieved. Data as of December 2018 from the MGP Impacts Survey 2017. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage increase in Percentage 0.00 50.00 30.00 36.00 average fruit and vegetable yields of targeted subsistence 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 farmers Page 40 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Comments (achievements against targets): Target achieved. Data as of December 2018 from survey results of subsistence farmers participating in the Farmer Field School program. Objective/Outcome: 2: Greater advantage of market opportunities by fruit & vegetable growers and livestock producers Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of locally Percentage 5.00 30.00 20.00 51.00 produced fruits and vegetables sold by domestic 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 retail and food service channels Comments (achievements against targets): Target achieved. Data as of December 2018 from Market Survey 2017. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage increase in the Percentage 0.00 40.00 40.00 68.00 value of sales of fruit and vegetable growers 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 participating in the MGP. Comments (achievements against targets): Page 41 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Target achieved. Data as of December 2018 from Government of Samoa's Project Completion Report. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of locally Percentage 40.00 55.00 50.00 94.00 produced beef sold by domestic retail and food 28-Feb-2011 28-Mar-2017 13-Feb-2017 31-Dec-2018 service channels Comments (achievements against targets): Target achieved. Data as of December 2018 from Government of Samoa's Project Completion Report. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage increase in the Percentage 0.00 40.00 30.00 46.00 value of sales of livestock producers participating in the 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 MGP Comments (achievements against targets): Target achieved. Data as of December 2018 from MGP Impacts Survey 2017. Page 42 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) A.2 Intermediate Results Indicators Component: Component 1: Livestock Production and Marketing Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increase in response to Percentage 0.00 60.00 60.00 68.00 improved husbandry - calving rate 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 Comments (achievements against targets): Target achieved. Data as of December 2018 from calving rate reported by MGP cattle farmers. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increase in reproduction Number 5.00 7.00 7.00 8.00 rates in response to improved husbandry - 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 average number of pigs weaned per litter Comments (achievements against targets): Target achieved. Data as of December 2018 from average weaning rate reported by MGP pig farmers. Page 43 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of MGP and Number 0.00 400.00 400.00 882.00 subsistence farmers owning improved livestock and/or 28-Feb-2012 13-Feb-2017 13-Feb-2017 31-Dec-2018 adopting improved animal husbandry practices (improved pastures, new fencing, etc.) Comments (achievements against targets): Target achieved. Data as of December 2018 from Project Coordination unit's records for MGP and subsistence farmers (APHD). This target was added to results framework under the project's Restructuring Paper in 2017, MGP and subsistence livestock farmers were separated from MGP and subsistence farmers for crops. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Cumulative number of Number 0.00 1100.00 450.00 606.00 business plans prepared by livestock 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 producers/entrepreneurs and approved for grant financing Comments (achievements against targets): Page 44 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Target achieved. Data as of December 2018 from final MGP figures. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of livestock Number 0.00 1400.00 600.00 605.00 processed through hygienic slaughter services 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 Comments (achievements against targets): Target achieved. Data as of December 2018 from MSU slaughtering services log book. Component: Component 2: Fruit and Vegetable Production and Marketing Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of MGP and Number 0.00 600.00 200.00 230.00 subsistence farmers using improved fruit and vegetable 28-Feb-2012 28-Mar-2017 17-Feb-2017 31-Dec-2018 varieties and adopting improved husbandry practices (IPM, prudent fertilizer use) Comments (achievements against targets): Page 45 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Target achieved. Data as of December 2018 from Project Coordination unit records for MGP and subsistence farmers (APHD). A revised indicator under the project's restructuring paper as subsistence and MGP livestock farmers were separated from subsistence and MGP farmers for fruit and vegetable production. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Cumulative number of Number 0.00 500.00 150.00 134.00 business plans prepared by fruit and vegetable 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 producers/entrepreneurs and approved for grant financing Comments (achievements against targets): 89 percent of target achieved. Actuals correspond to the number of fruit and vegetable beneficiaries who singed Grant Agreements. Data as of December 2018 from final MGP figures. Component: Component 3: Institutional Strengthening Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increase in the proportion of Percentage 0.00 50.00 80.00 98.00 targeted livestock and fruit and vegetable producers 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 Page 46 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) receiving satisfactory extension services Comments (achievements against targets): Target achieved. Figures include both MGP and subsistence farmers. Data as of December 2018 from Government of Samoa's Project Completion Report. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of Crops Division Percentage 10.00 100.00 100.00 80.00 staff completing training programs to upgrade their 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 extension effectiveness and/or technical capacity Comments (achievements against targets): 80 percent of target achieved. Data as of December 2018 from Government of Samoa's Project Completion Report. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of APHD staff Percentage 0.00 100.00 100.00 68.00 completing training programs to upgrade their 28-Feb-2012 28-Mar-2017 13-Feb-2017 31-Dec-2018 extension effectiveness Page 47 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) and/or technical capacity Comments (achievements against targets): Target not achieved. Data as of December 2018 from Government of Samoa's Project Completion Report. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of female farmers Percentage 0.00 30.00 30.00 25.00 participating in training provided by MAF extension 28-Feb-2012 13-Feb-2017 13-Feb-2017 31-Dec-2018 officers Comments (achievements against targets): 83 percent of target achieved. Indicator was added under the project's restructuring paper in 2017. Data as of December 2018 from Government of Samoa's Project Completion Report. Page 48 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Improved productivity of fruit and vegetable growers 1. Percentage increase in average fruit and vegetable yields of growers participating in the MGP. Outcome Indicators 2. Percentage increase in average fruit and vegetable yields of targeted subsistence farmers. 1. Component 2: Number of MGP and subsistence farmers using improved fruit and vegetable varieties and adopting improved husbandry practices (IPM, prudent fertilizer use). 2. Component 3: Proportion of CD staff completing training programs Intermediate Results Indicators to upgrade their extension effectiveness and/or technical capacity. 3. Component 3: Increase in the proportion of targeted livestock and fruit and vegetable producers receiving satisfactory extension. 4. Component 3: Proportion of female farmers participating in training provided by MAF extension officers 1. Average fruit and vegetable yields for MGP participants increased by 53%. 2. Average fruit and vegetable yields for targeted subsistence farmers increased by 36%. 3. 230 MGP and subsistence farmers accessed and used new, higher Key Outputs by Component yielding fruit and vegetable varieties and adopted improved (linked to the achievement of the Objective/Outcome 1) technology and production techniques. 4. 80 Crops Division staff completed training programs to improve their technical capacity on extension services. 5. 98% of targeted MGP and subsistence farmers received satisfactory extension services. 6. 25% of farmers who received extension services were female. Page 49 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Objective/Outcome 2: Improved productivity of livestock producers. Outcome Indicators 1. Component 1: Increase response to improved husbandry – calving rate (cattle). 2. Component 1: Increase in reproduction rates in response to improved husbandry – average number of pigs weaned per litter. 3. Component 1: Number of MGP and subsistence farmers owning improved livestock and/or adopting improved animal husbandry practices (improved pastures, new fencing, etc.). Intermediate Results Indicators 4. Component 3: Proportion of APHD staff completing training programs to upgrade their extension effectiveness and/or technical capacity. 5. Component 3: Increase in the proportion of targeted livestock and fruit and vegetable producers receiving satisfactory extension 6. Component 3: Proportion of female farmers participating in training provided by MAF extension officers 1. Average calving rate increased by 41% as reported by MGP cattle farmers. 2. Average number of pigs weaned per litter increased by 60% to 8 from a baseline of 5. 3. 882 MGP and subsistence farmers were recorded owning improved Key Outputs by Component livestock and/or adopting improved husbandry practices. (linked to the achievement of the Objective/Outcome 2) 4. 68% of APHD staff completed training to strengthen the capacity of staff for delivering extension services to farmers. 5. 98% of targeted MGP and subsistence farmers received satisfactory extension services. 6. 25% of farmers who received extension services were female. Page 50 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Objective/Outcome 3: Greater advantage of market opportunities taken by fruit and vegetable growers. 1. Percentage of locally produced fruits and vegetables sold by domestic retail and food service channels Outcome Indicators 2. Percentage increase in the value of sales of fruit and vegetable growers participating in the MGP. 1. Component 2: Cumulative number of business plans prepared by Intermediate Results Indicators fruit and vegetable producers/entrepreneurs and approved for grant financing. 1. The share of locally produced fruit and vegetables sold by domestic retail and food service channels increased by 51%. 2. The value of sales of MGP fruit and vegetable growers increased by Key Outputs by Component 68%. (linked to the achievement of the Objective/Outcome 1) 3. 134 business plans were prepared and approved for grant financing by fruit and vegetable producers/entrepreneurs. This number corresponds to the number of fruit and vegetable beneficiaries who signed Grant Agreements. Objective/Outcome 4: Greater advantage of market opportunities taken by livestock producers. 1. Percentage of locally produced beef sold by domestic retail and food service channels Outcome Indicators 2. Percentage increase in the value of sales of livestock producers participating in the MGP. 1. Component 1: Cumulative number of business plans prepared by livestock producers/entrepreneurs and approved for grant Intermediate Results Indicators Financing. 2. Component 1: Number of livestock processed through hygienic slaughter services. Page 51 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 1. The share of locally produced beef sold by domestic retail and food service channels increased by 94%. 2. The value of sales for MGP livestock producers increased by 46%. Key Outputs by Component 3. 606 business plans were prepared and approved for grant financing (linked to the achievement of the Objective/Outcome 2) by livestock producers. This number corresponds to the number of livestock producers who signed Grant Agreements. 4. 605 livestock have been processed through the MSU Slaughtering Service since commencement of operations. Data as of December 2018 from the Government of Samoa’s Project Completion Report; January 2019. Page 52 of 63 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Mona Sur Task Team Leader Gitanjali Ponnambalam Team Assistant Miriam Witana Procurement Specialist Stephen Paul Hartung Financial Management Specialist Maeva Natacha Betham Vaai Liaison Officer Isabella Micali Drossos Senior Counsel Victoria Florian S. Lazaro Operations Officer Gerardo F. Parco Safeguards Antonia Wong Senior Program Assistant Supervision Mona Sur Task Team Leader Kofi Nouve Task Team Leader Stephane Forman Task Team Leader Marianne Grosclaude Task Team Leader Sudesh Ponnappa Team Member Isabella Micali Drossos Team Member – Legal Gitanjali Ponnambalam Team Assistant Brenna Moore Agricultural Specialist Eric Leonard Blackburn Procurement Specialist Joyce Miriam Denise Witana Procurement Specialist Mark David Ansell Procurement Specialist David Bruce Whitehead Financial Management Specialist Page 53 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Stephen Paul Hartung Financial Management Specialist Thomas John Callander Social Specialist Victoria Florian S. Lazaro Social Specialist Josefo Tuyor Social Specialist Ross Butler Social Specialist Nicholas John Valentine Environmental Specialist Gerardo Pio Francisco Parco Environmental Specialist Josefo Tuyor Environmental Specialist Wolfhart Pohl Safeguards Advisor/ESSA Maeva Natacha Betham Vaai Team Member Solange A. Alliali Team Member ICR Wendao Cao Task Team Leader Kara Mouyis Lead Author Kundan Singh Agricultural Economist (FAO) Eric Leonard Blackburn Procurement Specialist(s) David Bruce Whitehead Financial Management Specialist Thomas John Callander Social Specialist Nicholas John Valentine Environmental Specialist Jim Waldersee Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY10 19.585 153,838.43 Page 54 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) FY11 30.327 222,593.45 FY12 6.859 113,010.69 FY13 8.248 65,206.55 Total 65.02 554,649.12 Supervision/ICR FY11 0 2,830.56 FY12 3.157 15,866.44 FY13 1.625 110,535.07 FY14 4.092 120,591.54 FY15 12.237 219,678.05 FY16 13.125 125,375.29 FY17 8.275 72,497.30 FY18 6.050 84,893.72 FY19 13.015 114,617.72 Total 61.58 866,885.69 Page 55 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Livestock Production and 3.60 3.48 97% Marketing Fruit and Vegetable 1.80 2.14 119% Production and Marketing Institutional Strengthening 5.70 6.60 116% Total 11.10 11.88 107% NB: Actual costs per Component are as of September 2018 IFR. Amounts shown are IDA/TF financing only and do not include contingencies costed at appraisal, or Recipient and farmer/beneficiary contributions. Page 56 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 4. EFFICIENCY ANALYSIS Economic and Financial Analysis (EFA) at Appraisal 1. The main objective of SACEP as anticipated at project appraisal was to support Samoa’s production for the domestic market, including related import substitution needs and to take advantage of the export opportunities offered by niche markets. It was anticipated that the project would provide support for investments in selected farm enterprises with strategic relevance for the above goals, adoption of improved technologies, through investments in extension services and access to matching grants. At appraisal, the Economic and Financial Analysis (EFA) for the project was based on ex-ante financial analysis of the following: (i) on-farm investments to be undertaken by the direct participants and (ii) processing investments to be undertaken by the project. The Net Present Value (NPV) for farm enterprise investments computed at a 12 percent discount rate was approximately US$7.2 million. The financial and economic IRR for the entire project was estimated at 13.5 and 15.3 percent respectively. Table 1: Activities to be supported by the project (at appraisal) Sl. Activities Unit of production Total No 1 Pasture improvement 5 acres 93 Cattle (10 acres of improved pasture) - with hygienic 2. a 10 heads slaughter facility Cattle (10 acres of improved pasture) - without 2. b 10 heads hygienic slaughter facility 2 Cattle (combination of 2a and 2b) 105 10 local sows, 1 improved 3 Pig rearing - level 1 65 boar 5 improved sows, 1 4 Pig rearing- level 2 28 improved boar 5 Sheep rearing 1 ram, 5 ewes 45 20 local hens and 2 6 Poultry - level 1 490 improved roosters 5 cycles of 40 broiler 7 Poultry - level 2 36 chickens per year 8 Beekeeping 10 hives 50 9 Cassava production 3 acres 160 10 Cassava processing units 600 MT, group of 20 farmers 8 11 Mixed vegetables (improved) 0.1 acre - 1 cycle per year 543 Page 57 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 12 Mixed vegetables (category 1 - rock removal) 1 acre - 1 cycle per year 217 Mixed vegetables (category 2 - rock removal, 13 1 acre - 1 cycle per year 136 irrigation) Mixed vegetables (category 3 - rock removal, 14 1 acre - 2 cycles per year 94 polytunnel) Mixed vegetables (category 4 - rock removal, 15 1 acre - 2 cycles per year 30 irrigation, poly tunnel) 16 Fruits (improved) 1 acre 40 17 a Fruits (Oranges - category 5 - drip without intercrop) 1 acre Fruits (Oranges - category 5a - rock removal, drip 17 b 1 acre with intercrop) 17 c Fruits (Papaya - category 5 - drip without intercrop) 1 acre Fruits (Papaya - category 5a - rock removal, drip with 17 d 1 acre intercrop) 17 Fruits (combination of 17a, 17b, 17c and 17d) 36 18 Mobile slaughter unit 2 2 19 Static slaughter unit 1 1 2. The EFA at appraisal does not take into account the project impact on the ‘improved subsistence’ farmers who would also benefit from the broader project activities concerning extension, applied research and provision of planting material. Similarly, the anticipated benefits in terms of nutrition and food safety would be substantial. However, they are difficult to quantify, and therefore were not taken into consideration in the analysis. Data Collection 3. The data for building the financial models has been obtained by tracking the business plans prepared by the MGP beneficiaries, data on beneficiaries from the SACEP MIS system, MGP Impact Evaluation 2017 report, Market Survey 2017 report and the MGP Incomplete Projects Report. The project supported 740 on-farm investment projects through the MGP which comprised of 405 cattle; 135 piggery; 134 crops (fruits and vegetables); 43 poultry; 21 sheep, and 2 bees. Of the total MGP projects, 594 (80 percent) were successfully completed while 146 projects (20 percent) were in-complete. The SACEP team tracked the fund utilization and performance of each of the business plans prepared under the MGP. Further, the MGP Impact Assessment report undertook a comprehensive study to capture the benefits accruing to the MGP beneficiaries through a survey that covered 592 MGP farmers and 240 farmers in a control group. The Task Team and the ICR team also undertook a number of field visits to various MGP beneficiaries and other MAF sites to ascertain the validity of data being reported and understand other implications of the project activities on the beneficiaries. Page 58 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 4. The financial prices for inputs have been collected from a sample of business plans prepared under the MGP while the market prices for outputs were collected from the Market Survey report. Methodology 5. For the purpose of this ICRR, it was not possible to follow the standard approach to the EFA at project completion, that is, to repeat such an analysis with the actual project costs and duration and the estimated benefits based on real project data. This was primarily because of the demand driven nature of the project. Hence, the investment activities envisaged at appraisal (Table 1) and the investment activities supported by the project during implementation were substantially different (Table 2). For the EFA at ICRR, the various investment activities supported under the MGP have been classified under similar groups and a cost-benefit analysis has been conducted for each of the activity groups. The financial models for each activity group includes financing plans that illustrate how project activities have resulted in increased incomes as a result of asset accumulation and working capital financing through project support and access to institutional sources of finance. In addition, a sustainability analysis was conducted for the MSU. Benefits and Beneficiaries 6. The project has generated short and long-term benefits for all its stakeholders. Key direct benefits accruing to MGP beneficiaries include: (i) productivity enhancement through enhanced on-farm investments, upgraded livestock, improved husbandry practices and stock management, and improved quality of meat sold in the local market for livestock producers; (ii) productivity enhancement through enhanced on-farm investments, increased access to high yield crop varieties, improved technology and production techniques, and increased sale to retailers and foodservice operators for fruit and vegetable producers; and (iii) increased access to extension services. In addition, benefits have also accrued to other subsistence and commercial farmers through access to improved livestock breed and planting material and improved technical advice on animal husbandry and crop production practices. Through SACEP the MAF has increased their effectiveness and efficiency in improving the domestic production base of the country. 7. The project was originally designed to reach 1,072 livestock farmers, 477 vegetable farmers and 36 fruits farmers through the MGP and 543 subsistence vegetable farmers and 40 subsistence fruits farmers through improved technical advisory services. In addition 2 mobile slaughter units and 1 static slaughter unit were planned to cater to the livestock farmers. 8. At project closure, the project covered 740 farmers under the MGP (Table 2) of whom 594 completed the projects. The project undertook the construction of 1 MSU which supported the hygienic culling of 605 animals. Further, 230 farmers (including MGP beneficiaries) received training on improved agriculture production practices and 882 farmers (including MGP beneficiaries) received training on improved animal husbandry practices. In addition, 200 MAF staff completed various training programs aimed at upgrading their technical capacity. Table 2: MGP Beneficiaries covered under SACEP % Category 2012 2013 2014 2015 2016 2017 2018 Total Completed Completion Cattle rearing# - - 52 69 214 70 - 405 339 83.7% Page 59 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Piggery - - 10 17 73 35 - 135 108 80.0% Poultry - - 4 8 18 13 - 43 26 60.5% Sheep rearing - - 8 6 5 2 - 21 19 90.5% Beekeeping - - - 1 1 - - 2 2 100.0% Fruits and Vegetables - - 24 26 57 27 - 134 100 74.6% Total MGP Beneficiaries 740 594 80.3% # - Includes multiplier farmers and rearing farmers 9. In order to support productivity enhancements for the farmers, improved breed of livestock was imported to Samoa. Imported cattle, sheep and pigs were distributed to selected MGP beneficiaries that acted as multiplier farmers for multiplication and rapid distribution of offspring (by sale) to other commercial and subsistence producers. For cattle, 13 bulls were imported from Australia and distributed to nine MGP (multiplier) farmers while three were retained at MAF’s parent nucleus farms. The average calving rate of these multiplier farms is 64 percent, a significant improvement of the 2016 rate of 26 percent. (The calving rate calculated from the 2012 National Cattle Census was only 35 percent.) Table 3: Key SACEP Outputs Category UoM Total Fruits and Vegetables – subsistence farmers farmers 78 Cattle slaughtered at MSU heads 605 Pasture improvement acre 10,000 Stockyard improvement support unit 37 Staff trained number 200 10. Similarly, for the sheep farmers, the project tried to improve the existing Fiji Fantastic (FF) sheep stock with newly imported FF sheep (93 ewes, 12 rams, 12 ewe hogget and 9 ram hogget) and crossbreeding these with imported Dorper sheep to improve carcass size and quality. The project had identified nine MGP beneficiaries as multiplier farmers. The lambing and mortality rate results were mixed, with the nucleus farmers achieving higher rates (lambing rate of 148 - 192 percent) while the multiplier farmers achieved moderate success (average lambing rate of 89 percent). 11. The project supported production of fruits and vegetables through the MGP for a wide variety of fruits and vegetables. The project productivity and production improvements through support to increase in area under cultivation, improving the quality of land through operations such as land leveling, stone removal, etc. Further, he project supported productivity gains by promoting improved production practices, increased access to better quality inputs, etc. The MGP beneficiaries reported an increase of 21 percent in net area under cultivation and an increase in cropping intensity from 227 to 271 percent. The MGP beneficiaries also reported an increase of 53 percent in average yields. While the average production went by up 122 percent, the ratio of quantity sold to quantity produced remained at 85 percent, thus indicating increased self-consumption of fruits and vegetables. 12. In addition to the MGP beneficiaries the project also worked with subsistence farmers engaged in the Page 60 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) production of fruits and vegetables. These farmers underwent technical trainings and were provided other support to sustain production. At project closure, 78 subsistence farmers have continued production activities. Table 4: MGP fruits and vegetable beneficiaries: Key figures Increase UoM 2013 2017 (%) Net area under production acre 0.36 0.43 21.3 Gross area under production acre 0.81 1.18 44.9 Cropping intensity % 227.4 271.6 19.4 Average production per kg 1,554 3,457 122.5 farmer Average sales per farmer kg 1,339 2,939 119.5 Average yield kg/acre 1,909.5 2,932.9 53.6 13. The project also supported the upgrading of facilities of MAF including (i) procurement of 13 pick-ups trucks for MAF; (ii) renovation works for the APHD’s three nucleus livestock farms; (iii) establishment of a nucleus piggery unit at Vaea; (iv) establishment of a tissue culture laboratory at Nu’u; (v) civil works for the SSU facilities at Nu’u; and (vi) establishment of a new quarantine post-entry laboratory at Vailima. Table 5: Summary of MGP producer models – Key parameters, interventions and benefits MGP producer Key parameters and benefits model Average herd size of 90 heads Calving rate and calf mortality at 26% and 15% Control respectively; 65% of sales continue to go to traditional fa'alavelave market; 100% of remaining sales go to supermarket Marginal improvement in herd size to 100 heads, calving rate improves to 64% Cattle - because of improved breed and pasture management. The improved breed bull multiplier is leased to each farmer for two years and is subsequently rotated among the Treatment MFs. Average steer offtake for meat is 12% and bull calves (13% of herd size) sold to other cattle rearers. Sale to fa’alavelave strongly discouraged but still prevalent. Average herd size increases from 10 to 15. Calving rate is 55% and herd Control mortality is at 8%. Sales – 36% as live weight and to retailers while 27% sold to fa'alavelave, remaining either donated or for self-consumption Cattle rearing Average herd size increases from 20 to 30. Calving rate improves to 68% and herd mortality improves to 5%. Sales – 50% as live weight and to retailers while Treatment 27% sold to fa'alavelave, remaining either donated or for self-consumption Average herd size of 38 heads; average weaning rate of 7 piglets per year and a Control mortality rate of 8%. Sales – 3% as live animals and 3% to retailers; 16% sales to Piggery fa'alavelave and donations; 70% for self-consumption Page 61 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) Average herd size of 47 heads; average weaning rate of 8 piglets per year and a mortality rate of 8%. Sales – 3% as live animals and 20% to retailers; 28% sales to Treatment fa'alavelave and donations; 40% for self-consumption Average herd size of 56 birds; mortality rate of 27%. Sales – 1.5% to retailers; Control 62% to fa'alavelave and self-consumption; Poultry Average herd size of 135 birds; mortality rate of 13%. Sales – 2% as live animals and 32% to retailers; 25% consumption during fa'alavelave and within own Treatment family; 25% sales to fa'alavelave Control No suitable control exists as sheep farming is a new activity to Samoa Average herd size of 44 heads; average lambing rate of 80% and a mortality rate Sheep of 6%. Sales – 30% as live animals and 10% to retailers; 33% sales to fa'alavelave Treatment and donations; 17% for self-consumption Average area under cultivation 0.7 acre. Key crops include: Chinese Cabbage, Control Pumpkin, Head Cabbage, Banana and Eggplant Fruits and Average area under cultivation 1 acre; Key crops include - Chinese cabbage, Vegetables head cabbage, cucumber, tomato, pumpkin, beans, lettuce, capsicum, spring Treatment onions and pineapple. Productivity increase of 54%, annual production increase by 122% and an increase in total value of sales by 72%. Bee keeping has not been included in the financial analysis due to their low numbers (Only 2 farmers were engaged in bee keeping). Results of the EFA 14. For the purpose of this EFA, the emphasis is given to understanding the underlying economics and cash flow of on-farm investments carried out by the MGP beneficiaries. The financial analysis looks at the actual costs incurred (tracked by the SACEP M&E team) and returns to farmers (MGP Impact Assessment Survey and Market Survey), and compares these results with the baseline situation to understand the impact of SACEP on the farmer incomes and the financial viability of the on-farm investments. Table 6: Summary of financial indicators of key SACEP activities Cattle Cattle Pig Sheep Poultry F&V – F&V – UoM multiplier rearing rearing rearing rearing MGP# Subsistence# Unit of production heads 100 35 47 45 140 0.43 0.15 Grant + Loan + farmer SAT 32,000 32,000 29,000 28,000 15,000 21,500 - contribution Revenue SAT 115,444 22,024 18,498 19,800 6,619 13,006 3,188 Gross Margin (not % 90.3 48.2 51.3 54.5 63.7 91.8 88.8 including labor) Net incremental income SAT 59,144 5,113 6,074 4,800 4,219 11,945 2,830 # - The unit of production is acres Page 62 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) 15. The financial results show that among the animal husbandry activities the returns are highest for cattle multiplier while the returns are approximately similar for rearing of cattle, pig and sheep. The option of taking up cattle multiplication as an activity is limited due to the higher demands of pasture and land availability. Table 7: Summary of Investments by MGP Beneficiaries (Completed Projects, SAT) Total Loan/ Total Grant Beneficiary Cash Category Disbursed Contribution* Contribution# (50%) (20%) (30%) Cattle 3,621,657 2,172,994 1,448,663 Fruits & 845,100 507,060 338,040 Vegetables Honeybee 32,000 19,200 12,800 Piggery 905,926 543,555 362,370 Poultry 220,803 132,482 88,321 Sheep 246,151 147,691 98,460 Total 5,871,636 3,522,981 2,348,654 # - Only those beneficiaries that could not contribute the required 30% in cash availed loans from banks * - The beneficiary contribution of 20% is a mix of in-kind and cash contribution 16. For the overall Internal Rate of Return (IRR) calculation for the project, the overall project costs includes the loans and beneficiary contribution for investments in the MGP business. The details of the investments for the completed MGP projects is provided in table 7. The beneficiary contribution for the MGP investments is a mix of cash and in-kind contribution. However, it was difficult to estimate the beneficiary contribution under the cash and non-cash components, hence it was assumed that all beneficiary contribution was in cash. 17. The total project costs exceed the estimated costs at appraisal in SAT terms. Due to appreciation of XDR against US$ by over 10 percent between appraisal and closure, the IDA credit reduced to US$ 7.2 million against the appraisal estimate of US$ 8 million (1.55 at appraisal, 1.39 at closure). Of this, the IDA credit disbursed was only US$ 6.8 million (95.4 percent). However, due to depreciation of the SAT against the US$ between appraisal and closure by over 16 percent (from 2.24 to 2.61), the total project expenses in SAT terms was at 107 percent of appraisal estimates. The total beneficiary contribution was SAT 5.86 million against the appraisal estimate of SAT 6.5 million (90 percent) primarily due to the lower farmer participation in the MGPs. After including the government and beneficiary share, the total project expenses was 103 percent of estimate in SAT terms (SAT 37.5 million against the appraisal estimate of SAT 36.2 million). Table 8: Summary of project costs At appraisal At closure % Utilization US$ SAT US$ SAT (in SAT) (million) (million) (million) (million) Page 63 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) IDA credit+ Trust 13.0 29.1 11.8 31.0 107% fund Government 0.26 0.58 0.22 0.58 100% Beneficiary 2.9 6.5 2.2 5.8 90% Contribution Total 16.16 36.20 14.34 37.5 103% 18. The ICRR EFA estimated an ERR of 13.3 percent and an ENPV of SAT 1.6 million (US$ 0.6 million) which is slightly lower that the appraisal estimates of 15.3 percent ERR and an ENPV of US$ 7.2 million (at 12 percent discount rate) 43. This was in spite of the substantially lower number of farmers mobilized in the MGP program that that was estimated at appraisal. These returns are however conservative, as benefits generating from increased capacity of beneficiaries, improved extension services through MAF, nutritional and health benefits resulting from increased consumption of fruits, vegetables and meat have not been quantified in monetary terms and hence have not been included in the overall IRR calculations. Sustainability of the Mobile Slaughter Unit (MSU) 19. The MSU was established to cater to the large and lucrative market for the fa’alavelave cattle, and to have a hygienic slaughter facility that could be run at low costs. Being mobile, the MSU could serve a larger number of farmers and be easier to manage. The MSU was also a pilot project to test the capabilities of the APHD before mobilizing further investments in a static slaughter unit. 20. At project closure, the MSU had been used to slaughter 605 animals. A charge of SAT 50 is charged per animal to the farmers. A fixed cost of SAT 5,886 is incurred on the MSU for the registration and maintenance and a weekly cost of SAT 200 is incurred on diesel. The breakeven volume for the MSU is 326 cattle per annum. The MSU has been in operation since 2017 thus operating at near breakeven volumes. 43The ex-ante EFA for the project had assumed a social discount rate of 12 percent for calculation of the ENPV. Hence, for comparison the ex-post EFA also considered a 12 percent discount rate for calculation of the ENPV. However, as per current World Bank guidelines the discount rate should be between 1x-2x of the annual growth rate of the per capita GDP of the country. Hence, considering a social discount rate of 4% for Samoa, the ENPV will be SAT 22.2 million (US$ 8.5 million) which is higher than the ex-ante ENPV of US$ 7.2 million. Page 64 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS The World Bank shared the draft ICRR with the Government of Samoa on May 23, 2019 and was advised on June 10, 2019 that they had reviewed the report and had no comments for inclusion. Page 65 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) By the Borrower: 1. ICR Report by MAF 2. MGP Impact Assessment Report 3. The 2018 Market Survey Report 4. PIM 5. FMM 6. MOU of MGP 7. Impact of Fa’alavelave Slaughter on Retail Meat Marketing in Samoa 8. Agricultural Sector Strategies 9. Agriculture Sector Plans 10. The Strategy for the Development of Samoa By the Bank: 1. Project Concept Note 2. PCN Review Meeting Minutes 3. Mission Aide Memoires 4. Project Appraisal Document December 28, 2012 5. Restructuring Papers 6. Review paper of the Matching Grant Program of SACEP 7. Country Partnership Strategy for Samoa (2012–2016) Page 66 of 67 The World Bank Samoa Agriculture Competitiveness Enhancement Project (P115351) ANNEX 7. SPLIT RATING 148. The below table presents the outcome using a split rating system. Before Restructuring After Restructuring (03/29/12 – 02/13/17) 02/13/17 – 12/31/18) Relevance of PDO High Efficacy (PDO) Substantial High 1 Modest Substantial 2 Substantial Substantial 3 Substantial Substantial 4 Substantial Substantial Efficiency Modest Outcome rating MS MS Numerical value 4 4 Disbursement US$8 million US$5 million Share of disbursement 0.62 (or 62%) 0.38 (or 38%) Weighted value 2.48 1.52 Final outcome rating Moderately Satisfactory (2.48 + 1.52 = 4) Page 67 of 67