Document of The World Bank Report No.: 76197-SL PROJECT PERFORMANCE ASSESSMENT REPORT THE REPUBLIC OF SIERRA LEONE NATIONAL SOCIAL ACTION PROJECT (IDA-37480, 3748A, TF-92671, AND TF-95661) FOOD PRICE CRISIS RESPONSE DEVELOPMENT POLICY GRANT (TF-92619) March 27, 2013 IEG Public Sector Evaluation Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Sierra Leonean Leones (Le) 2003 US$1.00 Le2370.01 2004 US$1.00 Le2455.01 2005 US$1.00 Le2882.59 2006 US$1.00 Le2882.59 2007 US$1.00 Le2976.37 2008 US$1.00 Le2979.92 2009 US$1.00 Le2950.03 2010 US$1.00 Le3217.61 2011 US$1.00 Le3925.95 Abbreviations and Acronyms ADB African Development Bank BIA Beneficiary Impact Assessment CAS Country Assistance Strategy FAO Food and Agriculture Organization IDA International Development Association IEG Independent Evaluation Group ICR Implementation Completion Report IMF International Monetary Fund Le Leone (Sierra Leone Currency) MT Metric Ton NaCSA National Commission for Social Action PAD Project Appraisal Document PPAR Project Performance Assessment Report Fiscal Year Government: January 1 to December 31 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Public Sector Evaluation : Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation : Ms. Martha Ainsworth (Acting) Task Manager : Mr. Ismail Arslan iii Contents Principal Ratings ................................................................................................................. v Key Staff Responsible......................................................................................................... v Preface............................................................................................................................... vii Summary ............................................................................................................................ ix 1. Background and Context................................................................................................. 1 2. National Social Action Project........................................................................................ 5 Objectives, Design, and Relevance ................................................................................. 5 Objectives ................................................................................................................... 5 Relevance of Objectives ............................................................................................. 7 Design ......................................................................................................................... 8 Relevance of Design ................................................................................................... 9 Design of Monitoring and Evaluation....................................................................... 10 Implementation Arrangements.................................................................................. 10 Implementation ............................................................................................................. 10 Achievement of the Objectives ..................................................................................... 12 Restoring Community Infrastructure and Services ................................................... 13 Building the capacities of Local Communities for Collective Action ...................... 19 Assisting Vulnerable Households to Access Temporary employment ..................... 22 Efficiency ...................................................................................................................... 26 Ratings .......................................................................................................................... 27 Outcome .................................................................................................................... 27 Risk to Development Outcome ................................................................................. 28 Quality of M&E ........................................................................................................ 29 Bank Performance ..................................................................................................... 29 Borrower Performance .............................................................................................. 30 3. Food Price Crisis Response Development Policy Grant .............................................. 31 Objectives, Design, and Relevance ............................................................................... 31 Objectives ................................................................................................................. 31 Relevance of the Objectives...................................................................................... 32 Design ....................................................................................................................... 32 Relevance of Design ................................................................................................. 33 Implementation ............................................................................................................. 35 Achievement of the Objectives ..................................................................................... 37 Mitigating the Impact of the Food Prices Increases on Consumer Welfare ............. 37 This report was prepared by Keith Oblitas (Consultant), who assessed the operations in February 2012. The Task Manager is Ismail Arslan. The report was peer reviewed by Gershon Feder and panel reviewed by John Heath. Yezena Yimer provided administrative support. iv Continued Delivery of Basic Services for Vulnerable Groups ................................. 40 Ratings .......................................................................................................................... 41 Outcome .................................................................................................................... 41 Risk to Development Outcome ................................................................................. 41 Bank Performance ..................................................................................................... 42 Borrower Performance .............................................................................................. 43 Implementing Agency Performance ......................................................................... 43 Monitoring and Evaluation ....................................................................................... 44 4. Lessons .......................................................................................................................... 44 References ......................................................................................................................... 47 Annex A. Basic Data Sheet ............................................................................................... 51 Annex B. Additional Tables and Figures .......................................................................... 56 Annex C. Design and limitations of survey data used in the assessment of the National Social Action Project ........................................................................................................ 59 Annex D. List of Persons Met........................................................................................... 62 Boxes Box 1. The Social Action Project‘s Components and Costs ............................................... 8 Box 2: Observations at a Primary School Sub-Project ..................................................... 15 Box 3. The Cash for Work Program in Bombali District ................................................. 24 Box 4. Lessons from the Cash-for-Work Program ........................................................... 26 Box 5. Indicative Impact of the Rice Tariff Reduction on Consumer Prices ................... 38 Tables Table 1. World Bank Projects Relevant to the Social Action Project and Food Crisis Grant ................................................................................................................................... 5 Table 2. Distribution of Community Driven Projects by District and Sub-Project Type . 13 Table 3. Household Participation at Initiation of Sub-projects......................................... 20 Table 4.Indicators of Sustainability of sub-projects financed under the Social Action Project ............................................................................................................................... 21 Table 5. Distribution of Cash-for Work - Sub-projects by District .................................. 23 Table 6. The Food Crisis Grant‘s Results Framework ..................................................... 33 Table 7. Sierra Leone Fiscal and Macro-economic Indicators ......................................... 36 Figures Figure 1. International Prices of Rice, Thailand, 5% (2000-2010, Current Prices) ............ 3 v Principal Ratings NATIONAL SOCIAL ACTION PROJECT (IDA – 37480, 3748A, TF92671 AND TF95661) ICR* ICR Review* PPAR Outcome Satisfactory n.a. Satisfactory Risk to Moderate n.a. Moderate Development Outcome Bank Performance Satisfactory n.a. Satisfactory Borrower Moderately Satisfactory n.a. Moderately Satisfactory Performance FOOD PRICE CRISIS RESPONSE DEVELOPMENT POLICY GRANT (TF-92619) ICR* ICR Review* PPAR Outcome Satisfactory Moderately Satisfactory Moderately Satisfactory Risk to Moderate Negligible to Low Significant Development Outcome Bank Performance Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Satisfactory Moderately Satisfactory Satisfactory Performance * The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. “n.a.� = not available. Key Staff Responsible NATIONAL SOCIAL ACTION PROJECT Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Eileen Murray Alexandre V. Abrantes Mats Karlsson Completion John Van Dyck Lynne D. Sherburne-Benz Ishac Diwan FOOD PRICE CRISIS RESPONSE DEVELOPMENT POLCIY GRANT Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Douglas M. Addison Antonella Bassani Ishac Diwan Completion Douglas M. Addison Antonella Bassani Ishac Diwan vi IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to ad apt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. vii Preface This is the Project Performance Assessment Report (PPAR) for the National Social Action Project and the Food Price Crisis Response Development Policy Grant in the Republic of Sierra Leone. A Credit (IDA 37480) for the National Social Action Project was approved on April 24, 2003 for an amount of $35.0 million equivalent. The project closed on December 31, 2010, two years after the scheduled closure date, in order to implement a cash-for-work program responding to Sierra Leone‘s food price crisis. The project was restructured on August, 18, 2008, with an amendment of the project‘s objectives to accommodate the cash-for-work program and was accompanied by a grant of $4.0 million from the Food Crisis Response Trust Fund (TF92671). On November 12, 2009, a further $4 million was provided comprising $3.0 million from the same Trust Fund and $1.0 million from the Food Price Crisis Multi-Donor Trust Fund (TF95661) to scale-up the cash-for-work program. The Food Price Crisis Response Development Policy Grant (TF-92619) was a development policy grant of $3.0 million. It was approved on August 6, 2008 and closed as scheduled on June 30, 2009. The grant was disbursed as a one-time budget transfer on August 27, 2008. The report presents findings from the two operations based on review of the project and program documents, the Implementation Completion and Results Reports, aides memoire, supervision reports, and other relevant material. A mission to Sierra Leone was undertaken by Keith Oblitas (consultant) from end January to early February, 2012, during which interviews were conducted with government officials, Bank staff in Washington and Sierra Leone, development agencies, beneficiaries and other involved persons. The report aims, first, to serve an accountability purpose by evaluating whether the operations achieved their intended outcomes. Second, the report draws lessons to inform future social funds and emergency budget transfers, in particular the lessons from the food price crisis. Third, the assessment also contributed to the forthcoming IEG evaluation of the World Bank Group‘s response to the global food crisis. Following standard IEG procedures, the government of Sierra Leone was invited to comment on the draft PPAR, however no comments were received. xi Summary This Project Performance Assessment Report reviews the experience and lessons from two operations in Sierra Leone - the National Social Action Project (2003-1 0), and the Food Crisis Response Development Policy Grant (2008-09). The Social Action Project supported recovery from a civil war, and both operations supported mitigation of the impacts of the 2007-2008 food price crisis on welfare. In 2002-2003, when the Social Action Project was being prepared, Sierra Leone was the poorest country in the world. It had just emerged from a devastating ten-year civil war; seventy-six percent of the population had incomes below $2.00 per capita per day; much of its social and economic infrastructure had been destroyed, especially in rural areas; and basic services such as primary education and health care were dysfunctional or non- existent. The human costs of the war were major. Some 20,000 of Sierra Leone's five million people had been killed and many more had been maimed or sexually abused. Among the most vulnerable were the poor, women, orphans, and unemployed youths. Since the war, economic activity in Sierra Leone has significantly recovered, improving welfare, but development has been uneven. As recently as 2009, the mortality rate for children under five years old was 192 per 1,000 births. This situation has presented many challenges. First, there is a chronic need to restore and develop services providing for basic needs such as health and education. Second, economic growth needs to continue apace and be socially inclusive, to enable families to emerge from the pervasive poverty of the past. Third, tensions between the former rival factions of the civil war need to give way to a socially cohesive society. More recently, as if these challenges were not enough, a fourth challenge arose stemming from the steep rise in the prices of internationally traded food commodities in 2007-2008. Due to this "food price crisis", the price of rice - Sierra Leone's dominant food staple - increased, especially in urban areas, by 50-100 percent. The hike in prices significantly increased the cost of living of an already impoverished population. National Social Action Project The National Social Action Project, financed through an International Development Association (IDA) Grant of$35 million equivalent, was a social fund assisting communities to construct social infrastructure following a community-driven development approach. The project's original objectives were to "assist war affected communities to restore infrastructure and services and build local capacity for collective action," with the priority to areas "not previously serviced by the government, newly accessible areas (those which were under rebel control until January 2002), and the most vulnerable population groups within those areas." In 2008, in response to the food price crisis, the objectives were expanded to include assistance to "vulnerable households to access temporary employment opportunities." To achieve this objective, an additional component was added, a "cash-for-work" program, accompanied by two provisions of additional financing of $4 million each. Thus, in essence, the restructured project had three objectives: the two original objectives (restoration of infrastructure and services, and building the capacities of local communities for collective action) and the additional objective to assist vulnerable households to access temporary employment. xii Under the community-driven development program, communities chose the social infrastructure that best corresponded to their needs. They usually chose primary schools or health clinics, which represented about one-third each ofthe sub-projects. Various other investments, such as a village community hall or market, made up the balance. Each community worked collectively to build and manage the facilities. Decision making and implementation processes were formulated to promote participation of the community as a whole, and these processes appear to have been influential. A survey found that 96 percent of men and 87 percent of women had contributed to construction, mostly in the form of labor. Of the 600 community sub-projects commenced under the project, 589 were completed and over 90 percent of these were functional two years later. Assessment of the benefits from the schools and clinics is complicated by possible attribution problems due to the influence of similar facilities constructed under the concurrent Bank-financed health and education projects. The number of children enrolled in the project's primary schools increased by 27 percent overall (33 percent for girls, 23 percent for boys). About two-thirds of patients walked to a medical center before the project, whereas after the clinic was operational. nearly all users walked. And at maternal health clinics, 29 percent of pregnant women made two or more antenatal visits after the clinic was established compared with 17 percent beforehand. Staffmg also improved; nearly doubling for health workers and increasing by one-third for teachers. The cash-for-work program primarily reached youths aged 15-35 (75 percent of beneficiaries), many of whom were former combatants with little or no education and no job or income, and women (30 percent).The design and implementation both had shortcomings, such as deficient selection of target groups, limited involvement of communities, and limited flexibility in choice of activities that could be suitable for women. It only partly reached the poorest amongst targeted groups, as selection of individual participants did not follow a standardized procedure and was sometimes influenced by vested interests. Contractors were provided too much power relative to communities, were often late in paying beneficiaries, and could produce poor quality works. The outcome ofthe project under its original objectives was satisfactory. Its objectives and design were substantially relevant, it substantially achieved both its objectives and, with relatively low overhead costs for a community-driven development project (about 10 percent), it was efficient in use of resources. Due to the modest achievement of the additional objective to provide vulnerable households access to temporary work opportunities and modest efficiency in achieving it, the outcome under the revised objectives (the two original objectives and this additional objective) was moderately satisfactory. However, as the majority of the project's resources were disbursed before the restructuring, the overall project outcome is rated satisfactory. The main risk to development outcome is the possibility that Government will fall short in financing the supplies and staffmg to enable continued operation of the clinics and schools - a situation that arose during the early part of the project period, which slowed implementation. Counteracting this is, first, the political pressures to finance the operating costs of schools and clinics that the increasingly empowered communities are likely to exert on government and, second, larger government revenues are expected from 2013 as a major Xlll iron ore mine begins exploitation. This would ease government's fiscal situation and make public expenditures more affordable. Risk to development outcome is moderate. The Bank's performance was satisfactory. It provided technical assistance on the highly participatory processes brought in by the project. The cash-for-work program was · prepared and implemented remarkably quickly as befitting an emergency project, and while it had a number of problems, these were identified during its two-year implementation period and adjustments made for a follow-on youth employment project. Government championed the project, and effectively backstopped the National Commission for Social Action and its coordination with other government agencies, but the shortfall in counterpart funding, albeit at a time when all sectors were affected by critical fiscal constraints, reduces Government performance to moderately satisfactory. The Commission was an effective implementer and coordinator of the community driven development approach. This was new to Sierra Leone, was national in scale, and required major training for staff in the relevant technical ministries and local councils. The Commission's performance was satisfactory, but taking account of the moderately satisfactory performance of the Government, overall Borrower performance was moderate! y satisfactory. Food Price Crisis Response Development Policy Grant The Food Price Crisis Response Development Policy Grant was a development policy operation of $3 million prepared on emergency basis to assist the Government in responding to the soaring international prices of food commodities during 2007-2008. The grant was provided shortly after international prices peaked in mid-2008. The operation's objective was to "support the Government's poverty reduction strategy in 2008 by providing the authorities with needed fiscal space to partially compensate for the lost revenues resulting from the recently reduced tariffs on food and fuel imports. This support would help mitigate the impact of such price increases and contribute to continued basic service delivery for vulnerable groups." For evaluation purposes the two intended outcomes were: (i) mitigating the impact of food price increases on consumer welfare; and (ii) contributing to continued basic service delivery for vulnerable groups. The objective of mitigating the impact ofthe food price increase on welfare was modestly achieved. Government reduced import tariffs on basic foodstuffs in an attempt to lower domestic prices. The most significant of these reductions was for rice, Sierra Leone's dominant food staple. The five percentage point reduction in rice import tariffs (from 15 to 10 percent) is estimated to have reduced the import price for rice by about 6 percent. This is dwarfed by the 50 to 100 percent increases in domestic rice prices influenced by international markets. But the impact ofthe tariff reduction was much smaller. Based on a number of assumptions, without the tariff reduction, a doubling in the price of rice would have increased a "modest income" urban family's cost of living by about 7.5 percent. With the tariff reduction, the increase would have been reduced by only one percent. This is not altogether surprising, as the initial tariff was not high and the reduction was even smaller. Yet, for these small reductions in the cost of living, the tariff reductions reduced revenues, straining the fiscal balance and the ability to finance social services. xiv The objective of contributing to continued basic service delivery for vulnerable groups, such as children and patients at health facilities, was reached in terms of the feeding program's numerical targets. These ongoing institutional feeding programs· financed by the government were at risk because ofthe sharp increase in the cost of food. Nearly 3,900 children in boarding schools and remand homes continued to be fed three times a day as against a target of3,500 children; and 8,200 lactating mothers with children under five and 12,700 patients at community hospitals or health centers received food three times a day, as against targets of 5,300 and 12,800 respectively. The additional costs of these programs as a result of the hike in food prices were approximately covered by the fiscal space created by the Grant's $3 million budget support. There is no empirical evidence on the quality of the feeding program or on its educational or health benefits. For schools, donor agencies consider the main benefits to be greater ability of children to concentrate; for health programs for young children, the benefits are in terms of improved nutritional status. None of these were measured for the program, but the counterfactual for this intervention would have been a cessation or sharp reduction in the institutional feeding program. It is not known to what extent other sources of support might have materialized had the program been cut. Efficacy of continuing the feeding program is likely to have been substantial. The Food Crisis Grant's outcome was moderately satisfactory. Its objectives met an emergency need; the Grant was timely and supported the Government's budget at a time of fiscal crisis; and feeding services were substantially maintained. However, it only modestly achieved its objective to mitigate the impact of food price increases on consumer welfare, in part due to deficiencies in the relevance of design. With the low level of the rice tariff to begin with, its small reduction, and both of these in relation to the very high increases in international prices, the likelihood of a significant impact on welfare was small ex ante, but certain to lower revenue and available resources for vulnerable groups. The risk to development outcome is significant, as Sierra Leone remains highly vulnerable to future food price shocks. The Bank's performance was moderately satisfactory. It responded rapidly to the food price crisis, and the understanding with Government on maintaining social services enhanced the utility of the grant. But the operation could have been designed in the broader context of the · effectiveness and costs of tariff reductions compared with the other actions government was taking to reduce the impact of the food price increases. Government's performance was satisfactory; notably, it was strongly committed to the operation, and its steering committee ensured that the feeding program was fully funded. Lessons Two broader considerations have emerged from this review. First is the question of what to do beyond a short-term emergency response to improve resilience to future shocks. The Food Crisis Grant provided some measures for short-term relief to the food price crisis and the components added to the National Social Action Project provided temporary employment for the poor. A government campaign has been launched to increase rice production and two strategy papers on food security have been prepared by Government. These papers, however, have gaps and in most respects lack detail. In the absence of a more comprehensive food security action plan and its implementation, there XV would be little change in Sierra Leone's vulnerability to future surges in international food prices. Second, the government undertook other short-term mitigating actions in response to the food crisis, ranging from export restrictions, agreements on price ceilings with rice traders, monitoring of trader and retail prices, to a special deal with a shipper for a consignment of rice imported at below market prices. The Food Crisis Grant and the Social Action Project's cash-for-work program filled niches in the country's response, but if the overall picture had been considered, this might have influenced the Food Crisis Grant's design. Tariff reductions, with their resultant lower budgetary revenues, are an expensive way to achieve reduced prices and, given the small impact on the cost of living of the tariff reductions, Government's other actions are likely to have been just as effective, or more effective, at lower cost. The experiences of the two operations, particularly as concerns their role in the food price crisis, yield the following main lessons: • Attempts to lower food prices through reducing tariffs on imported foodstuffs have direct consequences on government revenues yet may have only marginal impact on consumer prices relative to the much larger impacts of international price changes. The impact of the tariff reductions on rice prices was small, and insignificant relative to the doubling of food prices influenced by international market forces. Sierra Leone took about 10 different actions to mitigate the impact of the price increases, which may have had greater impact on consumer prices than the tariff reductions. • In a food emergency operation there may be a trade-off between a rapid, larger-scale response, at the cost of some reduction in quality, compared with a better prepared operation which is slower to mitigate the impacts of price increases. The cash-for-work program had an outcome less than its potential, but in responding to the emergency, speed was critical. • A cash-for-work program needs clear criteria and processes that ensure objective selection of beneficiaries. Lack of such processes, and of community involvement in selection, reduced the program' s effectiveness at reaching the most vulnerable groups. • High community participation helped the Social Action Project's communities to "own" their sub-projects and foster interest in maintaining their investment and operating the service. The project's community sub- projects were mostly effective in establishing community institutions, infrastructure and services; there are some indications that they might be sustained beyond the short term. ..r-·/ ~ UCJt2~dr- caroline Heider Director-General Evaluation 1 1. Background and Context 1.1 In 2002-2003, the time when the first of the two operations considered in this review (the National Social Action Project) was being prepared, Sierra Leone was the poorest country in the World. In 2003, 76 percent of the population had incomes below $2.00 per capita per day, and 53 percent had per capita incomes of less than $1.25 per day. It was also at the bottom of the Human Development Index. The country had just emerged from a devastating ten year civil war affecting all of the nation‘s districts. Much of the country‘s social and economic infrastructure (schools, health clinics, roads, markets and other facilities) had been destroyed or seriously damaged, especially in rural areas, and the corresponding services were dysfunctional or non-existent. The human costs of the war were major. Some 20,000 of Sierra Leone‘s 5 million people had been killed and many more had been maimed. Post-traumatic stress, women and children who had been sexually abused, and children who had lost their parents, were ubiquitous. Other groups that were vulnerable were the poor, women, and unemployed ―youths‖ (defined as those between 15 and 35 years old), including the former combatants, a potential source of unrest. 1.2 By 2009, gross national income per capita was $340, relatively better than in 2003 but among the 10 countries in the world with the lowest per capita incomes. Sierra Leone‘s social statistics told the same story. In 2008, 28 percent of children under five years old were malnourished, life expectancy at birth was 48 years and child mortality was 192 per 1000 children.1 1.3 Sierra Leone‘s economy has significantly revived since the civil war. The turnaround in economic activity since the end of conflict reflects a catch-up effect, mainly driven by reconstruction, a substantial surge of aid flows (including debt relief), and improved macroeconomic policies undertaken by the authorities starting from 2002. GDP growth averaged 10.2 percent per annum between 2002 and 2007. It then slowed to about 5 to 6 percent per annum from 2008 to 2012, influenced by the decline in growth rates in the global economy and a sharp reduction in the prices of the country‘s mineral exports. The rural sector has contributed to GDP growth but does not appear to have been a major driver, either as a significant source of additional labor for the urban sector, or as a lead source of growth through its own development. The percentage of Sierra Leone‘s population that is rural fell from 67 percent in 1990 to 62 percent in 2010–not a major reduction for a period of two decades. Agriculture‘s share of GDP fell from 58 percent in 2000 to 49 percent in 2010. Still, with population increasing by about 3.5 percent per annum from 2000 to 2010, coupled with some movement from rural to urban areas, the urban population doubled from 1 million in 1990 (33 percent of the country‘s total population) to 2 million in 2010 (38 percent). 1.4 Looking ahead, there are prospects for substantial revenues from new iron ore mines that are beginning to be exploited. The country will have the opportunity to use its increasing wealth to progressively improve incomes and social services, especially for the poor, and to invest in agricultural services and related infrastructure. There is a long way to go, however. 1 World Development Indicators, 2010 and 2011. Sources for other data in this section include FAO/FAOSTAT and Government of Sierra Leone statistics. 2 The per capita income is only one-third of the average for Sub-Saharan Africa, social welfare statistics illustrate widespread deprivation, and the 2007-2008 ―Global Food Price Crisis‖ found the country highly vulnerable to fluctuations in international prices of food commodities. 1.5 Rice is Sierra Leone‘s primary food staple. Other staples such as cassava, sweet potatoes, pulses, and palm oil are much smaller presences in diets and are mainly produced domestically. By contrast, one third of rice consumption is imported making the country vulnerable to changes in international prices. 1.6 In 2007/2008, international rice prices doubled, increasing by about 140 percent from January to April 2008. This had already been preceded by a significant climb in rice prices in earlier years (Figure 1). The prices of other basic commodities, such as other cereals, petroleum, and fertilizer also rose, though not as much as rice (Annex B, Figure B1). The hike in international prices affected domestic rice prices for urban consumers more than consumers in rural areas.2 In urban and peri-urban areas rice prices typically increased by between 50 to 100 percent, the larger increases tending to be where the local market was particularly exposed to import prices, while those inland agricultural areas that were largely self-sufficient in rice were only marginally affected. 1.7 The Food Price Crisis generated broad ranging short-term efforts by Government to mitigate its impact. The government reduced tariffs on rice, wheat, wheat flour, sugar, and petroleum. Other measures taken included: requests to rice importers and wholesalers to moderate their price increases; monitoring against hoarding and monopolies; banning rice exports and closing border posts with neighboring countries; distributing improved rice seed; and efforts to expand social safety net programs. Assistance was also requested from development agencies working in Sierra Leone which to the extent feasible in a short-term context, stepped up their programs. Aid was primarily for social safety net programs in health, education, feeding, youth employment, and intensified agricultural extension. Several donors also provided budget support. Agencies supporting welfare activities included the World Bank, African Development Bank, United Nations Development Program, European Community, Department for International Development (UK), World Food Program, GTZ/KFW (German aid), United Nations Children‘s Fund, Islamic Development Bank, Irish Aid, JICA (Japanese development aid) and the nongovernmental organization (NGO) community. Budget support was provided by the World Bank, African Development Bank, Department for International Development, and the European Community. 2 Trends in the consumer price indices suggest a segmentation of the rice market between urban and rural areas in their susceptibility to fluctuation in international prices. The trend in the consumer price index for food in Freetown, which was broadly similar to the general price index until 2006, increased faster than the general index from 2007, while national data on the food and general price indices showed no significant differentiation. Such apparent trends may, however, understate or misrepresent the situation at local levels. First, the consumer price index for food only partly relates to the specific prices for rice. Second, there are substantial differences between market prices depending on location, even between adjacent localities. 3 Figure 1. International Prices of Rice, Thailand, 5% (2000-2010, Current Prices) Source: DECPG –GEM Commodities Data (as of March 22, 2012) (www.worldbank.org/ddp) 1.8 The World Bank‘s involvement in the food crisis response in Sierra Leone was primarily through the two operations under review in this PPAR, which between them contributed both to fiscal support and the social safety net. The National Social Action Project, hereafter referred to as the ―Social Action Project‖ (FY03, $35 million plus $8 million additional financing) was a social fund to assist village communities to implement and manage small investments for the common good. It was launched as one of several projects following the civil war–among them the Health Sector Reconstruction and Development Project ($20 million) and the Rehabilitation of Basic Education Project ($20 million), also approved in FY03 (Table 1). Most investments of the Social Action Project were for social infrastructure, the most popular being primary schools and small health clinics, but a component for cash-for-work targeted to youths was added in 2008 to mitigate the impact of the food crisis. The Food Crisis Response Development Policy Grant (FY08, $3 million), termed in this report ―Food Crisis Grant, was a budget transfer to Government disbursed in one tranche. This was to provide emergency financial assistance to partially compensate in the short-term for the reduced revenues resulting from lowering tariffs on food imports, thereby: (i) helping to mitigate the impact of food price increases on consumer welfare; and (ii) facilitating continued provision of food as part of education and health services. 1.9 Thus, the experiences under the Social Action Project and the Food Crisis Response Grant potentially provide between them a number of lessons for future community driven development programs, for food security in a vulnerable state, for how to enhance the development impact of an emergency budget support operation, and for reducing poverty. The projects also provide contrasts between the relative advantages and disadvantages of two commonly applied approaches and lending instruments for a food price crisis. On the one hand, food import tariff reductions and the budget transfer instrument of a Development Policy Grant as used under the Food Crisis Grant operation, which can be applied with little 4 preparation. And on the other hand, the addition of an emergency component to an investment operation, as done for the cash-for-work program added to the Social Action Project. This provided more directly targeted relief measures, but with speed and quality of implementation limited by the implementation capacity of the institutions involved. It also posed a trade-off between the speed of implementation and possible loss in quality of the program. 1.10 The Bank was the sole financial partner with Government for both the Social Action Project and Food Crisis Grant, and there were no formally established links between the Bank and other donors. However, the two projects were prepared and implemented within a web of other projects and involved donor agencies in the same sectors, and sometimes with closely similar activities. Specifically, the Health Sector Reconstruction and Development Project and the Rehabilitation of Basic Education Project ran concurrently with the Social Action Project and were in the same sectors as most of its community sub-projects. The health and education projects were not directly linked with the Social Action Project, although the general strengthening of these two sectors would likely have created a stronger base for the Government‘s responsibilities under the project–provision of teachers, nurses, drugs and materials. 1.11 There are five other projects with some relevance to the National Social Action Project (Table 1). The Community Reintegration and Rehabilitation Project (FY99, $38 million), preceded the project and was closed two months after the Social Action Project‘s approval. It launched community level projects, but involved little community participation, the essence of the Social Action Project. It also strengthened the implementing agency – the National Commission for Reconstruction, Resettlement and Rehabilitation, which was the predecessor of the National Committee for Social Action (NaCSA), the implementing agency for the Social Action Project. The Youth Employment Support Project (FY10, $20 million) continues the Social Action Project‘s Cash-for-Work Program. The Decentralized Services Delivery Project (FY10, $20 million) is strengthening the capacity of Local Councils–a process to which the Social Action Project had contributed to through the councils‘ involvement in project implementation. And there are two projects supporting rural development: the Rural and Private Sector Development Project (FY07, $30 million) focuses on development of agricultural marketing chains and includes support to improve agricultural technologies, and the Infrastructure Development Project (FY06, $55 million) includes development of rural roads. 5 Table 1. World Bank Projects Relevant to the Social Action Project and Food Crisis Grant Project Project Number FY Approved Credit/Grant Community Reintegration and Rehabilitation Project P040649 FY99 $38 million Health Sector Reconstruction and Development Project P074128 FY03 $20 million Rehabilitation Basic Education Project P074320 FY03 $20 million National Social Action Project P079335 FY03 $35 million Infrastructure Development Project (Transport) P078389 FY06 $55 million Rural Development and Private Sector Development P096105 FY07 $30 million Food Crisis Grant P113219 FY09 $3 million Youth Employment Support Project P121052 FY10 $20 million Decentralized Services Delivery Project P113757 FY10 $20 million Note: a. Plus $8 million additional financing. Source: Operations Portal, World Bank, Sierra Leone Lending Portfolio 1.12 There have been no follow-on operations to the Food Crisis Grant, although development of agricultural marketing and rural roads will support agricultural growth. If the need for rice imports were to fall, Sierra Leone‘s vulnerability to future food price shocks would be reduced. Government-financed development programs are also relevant. Most notable is the extensive group of actions that Government took to mitigate the food price crisis, which puts the role of the Food Crisis Grant in a contributory rather than primary perspective. The Bank‘s lending program since the close of that operation in 2009 contains little in the way of comprehensive measures to reduce Sierra Leone‘s vulnerability to future food crises, although a number of projects contribute toward this.3 1.13 A caveat is in order regarding data sources. Statistical capacity, while improving, is limited, and development related data are both scarce and of varying quality. Official statistics from the Food and Agriculture Organization, the World Bank and other sources also vary in quality due to their substantial reliance on Sierra Leone source data. The analysis in this report has, thus, supplemented available empirical information with use of proxy information, interviews with stakeholders from government, civil society and beneficiaries, and reference to the literature. These sources are typically qualitative rather than quantitative, and often limited to only a few observations that might not be representative of the situation as a whole. One technique has been to triangulate between such sources for common ground with which to assess general directions and magnitudes. Situations where data are particularly limiting are highlighted in the section concerned; the design and limitations of the surveys used are discussed in Annex C. 2. National Social Action Project Objectives, Design, and Relevance OBJECTIVES 2.1 The original objectives of the Social Action Project, as articulated in the Project Appraisal Document (PAD), were to: ―Assist war affected communities to restore infrastructure and services and build local capacity for collective action. Priority would be 3 In marketing, rural roads, health, and youth employment (Table 1). 6 given to areas not previously serviced by the government, newly accessible areas (those which were under rebel control until January 2002), and the most vulnerable population groups within those areas‖ (World Bank 2003). 2.2 The objectives expressed in the Credit Agreement are consistent with the PAD version, but focused on the contribution of community-driven development on longer-term poverty-reduction objectives: ―to assist the borrower in its efforts to generate sustained poverty reduction by financing and supporting community driven development initiatives, focusing on newly accessible areas and the vulnerable people within these areas.‖ In January 2008, the project underwent a second-level ―adaptive‘ restructuring to align the two statements of objectives, modifying the Credit Agreement to align with the objectives in the PAD. The objectives expressed in the PAD will be used in this evaluation as its original objectives, but with some clarification regarding the target groups in the second sentence (see below). 2.3 A third objective was added in August 2008 (the original closing year) so as to play a role in mitigating the adverse effects of the 2007/08 food price crisis. The revised objectives, accompanied by a two year extension of the project period, were: "To assist war affected or otherwise vulnerable communities to restore infrastructure and services, build local capacity for collective action and assist vulnerable households to access temporary employment opportunities, with priority given to areas not previously serviced by the government, newly accessible areas (those which were under rebel control until January 2002), food insecure areas, and the most vulnerable population groups within those areas"4 (World Bank 2008). There were no further modifications during approval of the second Additional Financing in 2010. 2.4 Thus, the two original project objectives and the third one added in response to the food crisis can be broken down to its three primary intended outcomes:  Restoration of infrastructure and services.  Building the capacities of local communities for collective action.  Assisting vulnerable households to access temporary employment. 2.5 A clarification is needed regarding the interpretation of the target population. First, as concerns classification of affected areas: other than in the wording of the objectives, the PAD makes no references at all to any specific geographic area. Rather, throughout the PAD, targeting is oriented to types of communities and households, the common denominators being the degree of poverty and deleterious impacts of the war. Second is the ubiquitous nature of the ―war affected‖. Persons interviewed by the IEG mission emphasized that all districts in Sierra Leone were war affected. Hence, war-devastated communities could be found throughout the country.5 Further, specifically for the project‘s communities, qualitative references suggest that the project did indeed primarily benefit the war-affected. The Social 4 The added parts of the project objectives are highlighted in italics. 5 Quantitative assessment of the extent of war damage to communities and infrastructure by district is generally not included in Sierra Leone data sources. 7 Action Project‘s 2010 Beneficiary Assessment commented ―The overwhelming majority of these projects were implemented in communities that had lost such facilities and services as a result of the war.‖6 In fact, in the project‘s selection process, only communities that had damaged or destroyed infrastructure were considered, thereby practically assuring that the project, while geographically dispersed, went to the war-affected. Hence, vulnerability characteristics such as poverty and war damage rather than geographic situations will be used in this evaluation. RELEVANCE OF OBJECTIVES 2.6 The project‘s original objectives were responsive to Sierra Leone‘s major challenge of addressing the destitution and divisions left by the civil war. The Bank‘s ―Transitional Support Strategy‖ of March, 2002, the policy current at the time of project preparation and which was prepared just after the formal cessation of hostilities, emphasized improving access of the poor to basic services such as health and education, and for improving the welfare of the most vulnerable victims of the war–the very poor, landless, amputees, war widows, displaced persons, orphans and sexually abused women. The policy document at project completion - the March, 2010 Country Assistance Strategy – articulated Sierra Leone‘s security issues in greater detail, also commenting that improved incomes and social welfare would be a stabilizing influence for civil society. There was also the Second Poverty Reduction Strategy Paper (June 2009), more commonly known as the ―Agenda for Change‖. This is largely consistent with the earlier strategy documents, but emphasizes that communities should be more involved in management of village infrastructure. Progressive decentralization of responsibilities from central government to the District Local Councils was a common theme in all documents. The project‘s objectives were consistent with these policies - in the aftermath of the war, restoring destroyed and dilapidated social infrastructure and services was a basic need; and social cohesion within formerly divided communities could be facilitated if all factions worked together to a common purpose. The Relevance of the original Objectives was Substantial. 2.7 The objective added to the project at the time of the food price crisis - to provide temporary employment for the particularly vulnerable (the poor, women and unemployed youths) - was a means of addressing poverty on an emergency footing to mitigate, on a short term (within 10 months) basis, the additional hardships imposed by the steep rise in food prices and the cost of living. Targeting unemployed youths (often former combatants) had a strategic as well as poverty alleviation dimension - lack of income, idle time, no or minimal education, and general disaffection with the status quo could pose difficulties for civil stability. Employment for youths was a strategic need recognized in Bank and Government policy documents– as expressed in the 2010 Joint Country Assistance Strategy — ―Youth unemployment ….. remains a social, economic and security challenge.‖ And ―Youth unemployment may increasingly become an issue of peace and stability that needs to be managed.‖ An urgent response to the food crisis had become an additional priority, making 6 Dalan Development Consultants 2010, page 10. The report also comments (page 10) ―There is clear evidence to show that CDD created assets have been consistent with the original NSAP (Social Action Project) design of reestablishing basic social infrastructure and services in war affected communities …‖ 8 the additional objective substantially relevant. The Relevance of the revised objectives (comprising all three objectives) was Substantial. DESIGN 2.8 The project as approved had three components (Box 1). An existing institution – the National Commission for Social Action (NaCSA) - was chosen for coordination of the project. With respect to the first component, local communities were to decide on the type of sub-project, and be involved in oversight of contractors. The menu for the sub-projects included social infrastructure such as schools and clinics, agriculture-related investments, and investment in other productive assets. A community could apply for only one sub-project. No maximum cost for a sub-project is specified in the PAD, although an average cost (all project types) of $28,000 was assumed at appraisal. Box 1. The Social Action Project’s Components and Costs 1. Community-Driven Program (Appraisal cost, $22.5.0 million; actual costs $27.6 million): restoring social and economic community infrastructure and services using community mobilization and collective action to select, implement and maintain small-scale sub-projects. 2. Pilot and Special Programs in Newly Accessible Areas (Appraisal cost $4.0 million; actual costs $1.5 million): (i) rehabilitating basic chiefdom and district feeder roads, drainage, and other infrastructure through a public works program employing demobilized soldiers, unemployed youths and other vulnerable groups; and (ii) construction of shelters for poor and vulnerable families. 3. Project Management and Innovative Activities (Appraisal costs $9.0 million; actual costs $10.7 million) : capacity building for communities, chiefdoms and district authorities, information, education and communication, M&E, and technical assistance for social, beneficiary, poverty and vulnerability and environmental assessments, and technical and financial audits. (Primarily, this component was the technical assistance and training part of the community-based approach.) 4. Cash for Work Program (Additional component at restructuring, estimated costs $8.0 million; actual costs $8.1 million.): Short-term employment, in response to the food price crisis, of vulnerable persons - primarily youths, women and the poor – for labor-intensive construction or rehabilitation of small-scale infrastructure such as feeder roads, thereby providing income to improve the welfare of these vulnerable groups. 2.9 A community‘s proposal for a sub-project was to be reviewed by NaCSA and required the endorsement of the line ministry concerned (e.g. Ministry of Health for clinics) which required that design standards matched the norms established by each ministry. Technical assistance during construction was to be provided by the Commission and the concerned line ministry. A monitoring and technical assistance process was established, involving NaCSA‘s specialist units (engineering, procurement, financing, project management). The Local Councils were to be substantially involved in the project, eventually as the conduits of funding for new sub-projects and the administrators of the program in their respective districts. The Ministry of Health was responsible for providing and funding the nurses, materials and medicines for the health clinics, and the Ministry of Education was responsible for providing and funding the teachers and pedagogical materials for the schools. The communities were responsible for maintaining the infrastructure, using their own funds, 9 materials and labor, and jointly responsible (with the nurses or teachers) for management, with the main say in how the services would be provided. A Project Management Committee, which after construction became the maintenance committee, was established by each community for these purposes. 2.10 The pilot program to provide shelters for particularly war-affected persons was to be implemented by NGOs. Communities were intended to identify beneficiaries, but participatory processes were not clearly laid out. The sub-component was also quite large for a ―pilot‖ – some 1,700 shelters at an estimated cost of $2 million. This was well beyond the needs for a pilot. The PAD comments that shelter programs had mixed performances in other countries, and in these circumstances a smaller program would have been better. RELEVANCE OF DESIGN 2.11 The original design was well framed to achieve both of its objectives. The social fund approach was suited to the multiple needs of communities for restoration of key social services in Sierra Leone‘s post-conflict situation where both institutions and community cohesion were weak. The design benefitted from the lessons learned under an earlier project – the Community Reintegration and Rehabilitation Project - specifically that community participation had to be much more than a consultative role. Transparency and clearly defined implementation processes were also important. Thus, the Social Action Project adopted a new and highly participatory community-driven development approach, along with clear processes designed to maximize community ―ownership‖ and transparent implementation. 2.12 The project‘s design was simple, comprising only two main activities: (i) the rehabilitation of infrastructure; and (ii) doing so by a community driven development approach. All sub-projects had the same operational processes for mobilizing communities, procurement, and provision of technical assistance and funds. This straightforward design directly responded to both of the original project objectives. However, the construction of shelters for poor and vulnerable families (part of the second component), while valuable to the potential beneficiaries, wasn't logically linked to the objectives of restoring infrastructure and services or increasing the capacity of communities for local action. The Results Framework in the PAD provided quantitative targets which would contribute to achieving the project objectives. However, while the causal chain between inputs, outputs and outcomes could be inferred from the performance indicators, this could have been clearer, and the monitorable indicators more sharply framed to include more quantitative data for assessing the project‘s outcome.7 Overall, the Relevance of the Social Action Project‘s Design to its Original Objectives was Substantial. 7 Thus, for the objective to increase the capacity of communities for collective action, the indicators provided (PAD Annex 1) – communities to have conducted a participatory needs assessment and have management structures in place, and 90 percent of communities to have assessed their projects as ―successful‖ (a qualitative and subjective term) – cover some inputs and outputs but not outcomes, or proxies for outcomes. The percentage of the community that contributed to the work and the average time or materials provided, the regularity and attendance of committee meetings, the percentage of the community that contributed to a maintenance fund, and the measured functionality of the facility (for example,. numbers of vaccinations given and prenatal visits) are quantifiable indicators that would get nearer to assessing achievement of this outcome. 10 2.13 The cash-for-work component was a response to the food price crisis, and the component‘s design was directly linked to the temporary employment objective. This points to a Substantial Relevance of Design for the project‘s Revised Objectives (the two original objectives and the additional temporary employment objective). DESIGN OF MONITORING AND EVALUATION 2.14 The monitoring and evaluation (M&E) system‘s design had the capacity to measure the project‘s monitorable indicators. Several reporting systems were to be used - operational reports from NaCSA‘s division leaders, quarterly and yearly reports of data relevant to management decisions, and more evaluative reporting through social surveys. The weak area was that the monitorable indicators and the data to be collected were mainly of an output nature (for instance, the number of schools constructed), with less on outcomes such as primary school enrollment rates. This was partly compensated by including social assessments (later termed Beneficiary Impact Assessments or ―BIAs‖) in the M&E‘s design which included some more outcome related data, albeit with significant gaps. IMPLEMENTATION ARRANGEMENTS 2.15 The National Commission for Social Action was a natural choice for the implementing agency, as it had gained a partial understanding of community-driven approaches implementing the Community Reintegration and Rehabilitation Project (under NaCSA‘s predecessor, the National Commission for Reconstruction, Resettlement and Rehabilitation). The roles of NaCSA and the line ministries, the latter providing technical expertise and the Commission as a coordinator, and trainer and promoter for the community driven development program, made best use of the existing institutions. The most fundamental alternative to the project‘s design would have been to continue with the top- down approach of the Community Reintegration and Rehabilitation Project. But this project‘s experience had indicated that a more participatory program would have greater chance of success. Implementation 2.16 The project was prepared in 2002/2003, shortly after the end of the civil war. (The concept paper review was in September 2002, and the project was approved on April 24, 2003, about 9 months later. It closed on December 31, 2010, two years after the original closing date of December 31, 2008. This prolonged the project duration from 5 ½ to 7 ½ years. The original IDA Grant was SDR 25.4 million or $35 million equivalent. The extension of the project period was made in 2008, the original final year of the project in order to contribute to Sierra Leone‘s overall response to the food price crisis. Thus, in August 2008, shortly after the peak of rice prices in April-May, 2008, the project was restructured, its objectives revised, and provided with additional financing of $4 million from the Food Price Crisis Response Trust Fund, for addition of the cash-for-work program. In November 2009, in order to continue and scale-up the cash-for-work, a further additional financing amounting to $4 million was provided through a further grant of $3 million from the Food Price Crisis Response Trust Fund and a grant of $1 million from the Food Price Crisis Multi- donor Trust Fund. 11 2.17 The mandate of the implementing agency (NaCSA) already included leading social infrastructure projects. But it was still an inexperienced organization with limited practice in the community oriented activities and support services that the Social Action Project was to introduce. NaCSA progressively improved its capacity throughout the project period. A pilot program in three districts for provision of shelter for vulnerable groups (part of the second component) was terminated at the Mid-Term Review due to poor performance. The intentions at appraisal were for the construction of 1,700 shelters8 at a total estimated cost of $2.0 million. The two NGOs implementing the program were ineffective, construction costs had escalated due to major increases in the costs of cement and other building materials, and the majority of shelters were not reaching the most vulnerable people. Moreover, progress was slow. By the Mid-Term-Review 908 shelters had been constructed (some were still under construction but were subsequently completed). Savings from the closed shelter program (the amount is not specified in project documents) helped finance the community- driven program, which had also been affected by the increase in construction costs but was considered a much higher priority for improving welfare and capacity for collective action. 2.18 For the Social Action Project‘s primary activity, the community driven development program, the target number of sub-projects was significantly reduced at the Mid-Term Review, from the appraisal target of 1000 sub-projects to a target of 600 sub-projects. The primary reason was the major increase in the costs of materials; cement prices increased by 130 percent in the first two years of the project. As a result, contractors slowed down construction and new contracts reflected the increase in construction costs. In these circumstances, it made sense to reduce the target to stay within the financial limitations. This may also have been sensible considering the pace appropriate to effectively respond to the objectives. 2.19 The project was categorized as ―at risk‖ in 2004, its first full year, based on four risk flags raised–two were country based (general country record and governance environment) and two were project-based, related to weaknesses in NaCSA‘s financial management and delays in provision of counterpart funds. In response to the first project flag, NaCSA tightened monitoring and coordination of participating agencies and local governments. Capacity building efforts were strengthened and project processes and fiduciary arrangements were further developed and monitored. These actions substantially resolved the problems, and thereafter, the Commission‘s institutional capacity progressively improved. Concerning the second risk flag, delays in provision of counterpart funds caused delays in implementation in the first years of the project and an overall shortfall of counterpart funds during the life of the project of $1.5 million. The appraisal estimate of the Government‘s counterpart funding was $3.5 million, whereas $2.0 million was provided. Exchange rate changes provided an extra $2.9 million in IDA funding, thus covering the shortfall in Government funds. Beneficiary contributions, mainly of labor and local materials, were estimated at appraisal to value about $3.5 million, whereas at project completion, the value was estimated at about $2.1 million, mainly reflecting a higher share of costs borne by contractors. 8 Prior to the Mid-Term-Review the target was reduced to 1,200 shelters. 12 2.20 Disbursements: All of the original SDR 25.4 million (US$ 35.0 million) grant was disbursed which, due to exchange rate depreciation, amounted in dollar terms to $37.9 million by project completion. Virtually all this amount was disbursed by the project‘s original closing date. The extension of the project by two years was for the purpose of implementing the cash for-work program, which used all of the $8 million in additional grants. 2.21 M&E Implementation: M&E staffing (a lean 2 person cell in the Strategic Planning and Information Division) provided a base for data coordination and collection although, without full-time field enumerators, the cell relied on data collection through NaCSA‘s District Coordinators and other operational staff. Data for output indicators were regularly collected, but the lack of regularly reported outcome indicators in the M&E system‘s design persisted. Exceptions were two beneficiary impact assessments - in 2006 (about mid-term of the project) and at project completion in 2010 – which between them provide a broader understanding of project impacts, although the range of information is limited and mostly qualitative. Management Information System reporting was regular, and the data became a useful management tool. 2.22 Fiduciary Issues: According to the ICR, NaCSA provided accounting and auditing reports on schedule and to required standards. Initial weaknesses in financial reporting were resolved with the Bank‘s technical advice, and guidelines for financial management and procurement were issued. Procurement followed Bank guidelines with the exception of one case of misprocurement which led to dismissal of the contractor. No other fiduciary issues are reported. 2.23 Safeguards Compliance: The project was Environmental Category B. An Environmental and Social Management Framework and a Resettlement Policy Framework were issued during project preparation with checklists to identify issues and mitigation measures and to prepare an action plan for each type of sub-project. Actual implementation of these systems was weak. An assessment of environmental safeguards practices was made in February 2010, the last year of the extended project period. The main problems were assessed to be: use of general rather than project-type checklists, inadequate numbers of environmental officers and weak training, and inclusion of NaCSA officials in environmental committees. Efforts were made by the Commission to address these issues. Although all sub- projects and cash-for-work schemes had been completed, it introduced a Safeguards Manual clarifying criteria and standards for screening sub-projects and developing mitigation plans, increased staff training, and tightened monitoring. There is no information on whether these measures improved environmental screening and mitigation, but the actions, though too late for the project, are clearly in the right direction. Achievement of the Objectives 2.24 Assessing the project‘s efficacy must contend with limited direct evidence on project achievements and the fact that there were parallel health and education projects. Nevertheless, from the BIAs, other commentaries or proxy data, and findings from IEG field visits and discussions, an indicative picture emerges. There are a number of concerns and limitations with respect to the BIA survey design and sampling, the occasional use of leading 13 questions, and the degree to which benefits can be ascribed to the project as opposed to other influences, as discussed in Annex C. RESTORING COMMUNITY INFRASTRUCTURE AND SERVICES 2.25 A total of 589 sub-projects were completed. The bulk of the investments chosen by the communities were for primary schools (205 schools, 34 percent of the sub-projects), general or maternal health clinics (185 clinics, 31 percent), and community infrastructure such as a meeting hall or village market (111 projects, 19 percent, Table 2). The remaining 99 subprojects (16 percent) financed investments in irrigation, skills training, and other choices. The sub-projects were a mix of new constructions and rehabilitations of destroyed infrastructure. No break-down between the two has been noted in the beneficiary surveys and ICR, although comments made in these documents infer that the great majority were for new construction or rebuilding of virtually destroyed infrastructure.9 The quality of the sub- project investments (infrastructure and adequacy of initial equipment) is not empirically assessed in the BIAs, although the Borrower comments in the ICR indicate that, for 95 percent of the sub-projects, construction met the prescribed standards of the line ministries.10 Table 2. Distribution of Community Driven Projects by District and Sub-Project Type Type of Sub-Project DISTRICT COMMUNITY EDUCATION HEALTH OTHER INFRASTRUCTURE TOTAL Bo 12 8 10 10 40 Bombali 20 23 6 5 54 Bonthe 27 7 5 7 46 Kailahun 16 13 5 14 48 Kambia 11 19 13 5 48 Kenema 17 17 5 14 53 Koinadugu 16 15 9 3 43 Kono 6 15 17 8 46 Moyamba 12 10 5 12 39 Port Loko 19 15 8 4 46 Pujehun 16 10 5 14 45 Tonkolili 23 16 3 2 44 Western Area 10 17 20 1 48 Total 205 185 111 99 600a Percent 34 31 19 16 100 Note: a. Of these, 589 were completed by project closing. Source: Beneficiary Impact Assessment, 2010. 9 ―The overwhelming majority of these projects were implemented i n communities that had lost such facilities and services as a result of the war.‖ ( 2010 BIA, page 33). 10 The line ministries also had to approve construction standards before a facility could be authorized for construction (Borrower‘s ICR, page 62 of the ICR.) 14 2.26 Restoring Education (Primary School) Services. The 2010 BIA found that of the 25 schools in the survey sample, 23 had been completed by the end of the project (92 percent) and two (8 percent)) had been abandoned during construction. This was about the same rate of abandonment as the average for all (including all types of investment) of the sub-projects sampled, for which 10 of the 90 sub-projects in the survey (11 percent of the projects) were abandoned. A precise breakdown of the reasons why construction was stopped is not available, but according to NaCSA, contractual disputes were the primary reason, usually related to the steep rise in prices of cement and other materials, which had not been foreseen at contract signature. Some contractors could not afford to continue the works under these circumstances. Focus groups in the 2010 BIA made similar comments (page 53). 2.27 The more significant question is whether the investments made are being sustained in good operational order. Available data is qualified by the short time periods observed in the surveys, but there are some indications. In the 2010 BIA, 20 of the 23 (87 percent) completed schools were found to be in ―good or acceptable‖ condition, and for all types of sub-project a similar situation (90 percent) was found. The average age of the sub-projects would have been over two years as about one-third of the sub-projects were completed in FY06, four years before the survey.11 An alternative source of information is directly from NaCSA‘s district offices. For six districts, 94 percent of completed sub-projects (all types) were still operating two years after construction. Observations made to the IEG mission by NaCSA and a District Chairman were consistent with these data – general views tended to be that all or virtually all sub-projects were functioning well, and the Bank team assessed that 96 to 98 percent of sub-projects were operational at the time of the ICR. 2.28 From NaCSA data, the number of children enrolled in primary school in communities near (not defined) project schools increased by 27 percent; for girls the increase was by 33 percent and for boys by 23 percent. The number of teachers increased by 37 percent.12 The share of families with a school-aged girl who sent a girl to a NaCSA school rose from 19 to 37 percent. The BIA also comments from focus group discussions with beneficiary communities (without quantitative data) that before the constructed/rehabilitated school, the average time spent by children getting to school was two hours. IEG‘s field visit to a village school supported by the project in the Western Area District confirmed the increased access to schooling and continued participation by parents, while also pointing to continuing shortages of teachers and materials (Box 2). 2.29 To some degree, the primary schools upgraded under the Rehabilitation of Basic Education Project (implemented from 2003 to 2010, about the same project period as the Social Action Project) can be used as a counterfactual – what was happening elsewhere? The education project was a broad-based program providing improvements to schools in 11 World Bank 2005, 2006 and 2007: Status of Projects in Execution, FYs 2005 and 2007. 212 projects were ―under completion in FY05, and the same number were reported as ―completed in FY07. (The construction program began in 2004 and ended in 2008, two years prior to project closure at the end of 2010, and only a few months after the 2010 BIA. 12 These figures may be underestimates of the eventual increases, as data were reported as of March 2007. The bulk of the schools would have been constructed less than two years earlier, and the full impact may not have been fully achieved in such a short time period. 15 furnishings, equipment, teacher training, and materials through service providers funded by ―partial‖ government grants for 776 schools and ―full‖ grants for 192 schools that also had to be constructed or rehabilitated. Between them the 968 schools represented 41 percent of the total number of primary schools in the seven districts covered under the project. There are few performance indicators enabling direct comparison of the two projects, as the parameters measured differ.13 Nevertheless, both projects showed improvement – facilities and pedagogical equipment were provided, the number of teachers increased,14 school enrollment, especially of girls, went up, pass rates at the end of primary school increased and communities appear to have greatly appreciated the schools.15 Box 2: Observations at a Primary School Sub-Project IEG‘s visit to the Our Lady of Kwama Primary School, Western Area District, illustrates some of the issues encountered in the community driven development program.. Kwama school had been financed under the project, and its operations were being financed and supplied (teachers and teaching materials) by the district Local Council, with grants from the Ministry of Education. Maintenance of the school was done by parent volunteers. A group of parents explained that the community had hired and paid for additional teachers to reduce class sizes, with contributions from each parent of Le 5000 (about $ 1.70) per child per annum. While not officially certified, the parents felt that these additional teachers were skilled enough for primary school teaching. The parent-teachers association reportedly met every term. The main subjects covered in the most recent meeting were the shortage of teaching materials, overly large class size and the problem of access by pupils from the more distant villages (some pupils had to walk up to five miles to reach the school). The main general problems mentioned by the parents were too few teachers, underpaid teachers, and insufficient training materials. Notwithstanding, enrollment of girls had increased from being a distinct minority to parity with the boys by 2011. IEG‘s site inspection found the school clean and children‘s notebooks neat and informative. There was an evident rapport between the teaching staff and parents. In a discussion with parents they advised that they were committed to maintain the school, and examples of actions to date were mentioned (weeding, cleaning). The parents felt that the school had helped the healing process after the war as families had to work together through the parent-teachers association, the maintenance committee, and other interactions. Source: IEG mission school visit 13 For instance, primary school completion rates increased in the education project‘s seven districts from 50 percent to 65 percent, but there is no comparable data for the social project other than views expressed in the beneficiary survey‘s focus groups that the completion rates had improved. 14 For both projects, teachers and pedagogical equipment and materials were provided through the Ministry of Education. Financing of specialist equipment was by government in the case of the social project, and through project funds under the education project. 15 Based on data from the 2005 and 2010 BIA‘s for the Social Action Project and the ICR for the education project. ―Satisfaction‖ (the term used in the ICRs of both projects and in the social project‘s beneficiary survey) has limited utility as the term is subjective and may not be representative. Satisfaction was assessed for the education project based on a questionnaire circulated to participants in the end-of-project workshop (ICR page 46) primarily comprising government staff, civil society, teachers and politicians (Workshop proceedings in ICR page 65 and list of participants at page 103) with limited representation of beneficiaries. The very high satisfaction rate for each project input (furniture, textbooks, teacher training, etc.) under the education project does not necessarily reflect the views of beneficiaries given that they were largely excluded from the survey. The social project‘s beneficiary survey (1656 respondents) rated whether their sub -project had been ―successful or very successful‖ at 79 percent, but this was for all sub -projects, thus not education alone. 16 2.30 The fundamental difference in the orientations of the projects was the degree of community participation. The Social Action Project put community participation at the fore, with project management committees and engagement with the committees established before consideration of any investment. Decisions, first-line construction supervision, and maintenance were their responsibility, whereas the education project put emphasis on equipping schools, in which function it was successful. However, in the education project, only 291 school management committees received training, about one-third of the schools that were refurbished. In a survey of one of the project districts (sample of 31 schools), four years after construction, one-third of the schools financed by the education project were found to need repairs (World Bank 2010).16 Direct comparison with the Social Action Project is not possible as the measurement time-frame for the project is shorter. 2.31 Restoring Health Services (Clinics). The other main choice by communities was small general or maternal health care clinics. Access to clinics improved due to the substantial reduction in travel time and costs (before construction/rehabilitation the only recourse to medical attention was to district hospitals). The clinics enabled nearly all users (98 percent) to walk to the clinic whereas before construction only two thirds could access by foot.17 The rest had to go by public transport as functioning clinics were few and far between. Another indication of utility is the number of antenatal visits to a clinic – from proxy information, 29 percent of pregnant women made two to three visits after a clinic was established, compared with 17 percent before the clinic. 18 An encouraging sign, both of appreciation and likely long-term commitment, is the growth in clinic staff (nurses or assistants) by project closure.). Staff numbers increased by nearly 90 percent (based on NaCSA data from 43 clinics). The focus group discussions in the BIA found that communities highly appreciated the clinics (and schools).19 There are also cases, both among the schools and the health facilities, where the school parents or the women visiting clinics had paid, with their own funds, for additional assistant teachers and nurses (IEG mission field visits and discussions). 2.32 There was a concurrent health sector Bank operation20 at the time of the social project, but the health project‘s health outpost sub-component was relatively small: 50 clinics in four districts were constructed and provided with nurses, specialist equipment and medicines supplied, as with the Social Action Project, through the Ministry of Health. It is 16 ―While two-thirds of school buildings are still in good condition after four years, one-third of schools need repairs, including cracked walls, potholed/coarse floors, leaking ceilings and coarse blackboards that consume too much chalk‖ (World Bank 2010). 17 BIA, 2010. Sample of 408 householders from 25 Social Project clinics. 18 BIA, 2010. Sample of 47 mothers from 18 clinics, each with two infant children, one borne before the clinic and one borne afterwards. Visits for the first child were before establishment of the clinic, and for the second child, after the clinic was established. 19 For instance, a comment from one women‘s focus group discussion in Laiya, Bombali: ―We are utilizing the health center effectively, because, when people are sick, we take them to the health facility; likewise pregnant women, we don‘t take them to Kamakwie.‖ 20 The Health Sector Reconstruction and Development Project, approved in February 2003 and closed in December 2009 (IDA $20 million). 17 not clear from the project documents whether the equipment and facilities were similar to the Social Action Project clinics. There are also limited performance data for inter-project comparisons, although evidence showing improved access to health care is consistent with the Social Action Project clinics. As with the education project, the key difference between the health and social action projects was in the degree of community participation. There is virtually no mention of this in the health project‘s PAD, and this limitation is critiqued in the ICR as a shortfall and a lesson for more community involvement in the future.21 2.33 Building Village Markets, Village Courts and Meeting Halls. The third most popular choice for a sub-project was community infrastructure such as a meeting hall, a village court or a small market. Sixteen of the 90 communities in the BIA had chosen such community infrastructure and a (very small) sample of 75 individual respondents provided feedback. Over 90 percent stated they were ―satisfied‖ with their sub-project.22 For those in communities that had chosen courts, 85 percent considered that their court had become since construction the main forum for settling disputes; over 90 percent of respondents visited their markets ―often;‖ and about 90 percent of those using the market were women. 2.34 Piloting Shelters and Rural Public Works. The shelter program was a pilot activity separate from the community driven development program and with a different, and far less participatory design. It was closed down at the Mid-Term Review due to performance problems. Of the originally targeted 1,700 shelters, 908 were constructed. The cost of the program amounted to $1.5 million or three percent of total project costs at the Social Action Project‘s closure. There were a variety of implementation difficulties, but the principal weakness lay in weak processes for involving communities and selecting beneficiaries. Communities were meant to select the poorest and most vulnerable people, but in practice, for the majority of shelters this was not done, or only partially done (World Bank 2011). The rural public works program was a demand-driven program to build community access roads, drainage systems and other infrastructure implemented in the country‘s three poorest districts. It was targeted primarily to demobilized soldiers and unemployed youths, and each sub-project required a contribution by the community or local authorities of 10 percent of the value of the works. Some 3,200 beneficiaries were involved in the works. The degree to which the program reached the intended beneficiaries is not provided in project documents or the 2010 BIA, although the task team considers that the program provided useful lessons in community involvement for the cash for work program. 2.35 Central Services. The relevant line ministries and Local Councils were and remain a central part of community services. For the schools, the Ministry of Education pays the teachers‘ salaries, and provides a basic supply of teaching materials. Likewise, the Ministry of Health provides for the nurses‘ salaries, medicines and vaccines. The Councils progressively took on the administrative and financing role of the community development 21 There was … ―very little involvement of the local communities that were beneficiaries of the facilities.‖ …… And, ―The lesson learned there was that local beneficiaries should be involved in every aspect of the process. This will increase ownership, and introduce another useful control mechanism. (World Bank 2010, page 44) 22 As for the schools and clinics, ―satisfaction‖ is open to different interpretations; hence this can only be broadly indicative. 18 program.23 The adequacy or otherwise of these human and material resources, frequently an issue in Africa, is not assessed in the ICR, or in the BIA, but proxies suggest that this was not perceived as a major issue. Thus, 79 percent of households stated they were satisfied with the services provided by their sub-projects (sample of 1683 respondents across all of the 90 communities in the BIA survey). 2.36 Overview documents such as the 2010 BIA and the ICR are silent on the adequacy and timeliness of medical and school supplies, even when specifically referring to the role of the line ministries, sustainability, and borrower performance. None of these proxies can be used to conclude that supplies and staffing were adequate, but together they suggest at least a basic level of supplies, teachers, and nurses.24 The IEG mission site visits and field discussion are consistent with this. 25 These various sources suggest that, while there were situations in which materials and regularity of delivery were inadequate, this does not appear to be a predominant issue.26 Coordination between the National Committee for Social Action and the line ministries, and at district levels, the local councils, helped in meshing the activities of the institutions concerned. At the central level there was a monthly meeting between NaCSA and the line ministries, and similar coordination existed at district level which also had monthly meetings between district level staff of NaCSA, the line ministries, and the Local Councils. 2.37 The observations above provide indications that: the majority of completed subprojects provided a useful service: they made a difference to welfare by providing services that were needed; access to these services has improved; and the communities appreciate the services. There are also some preliminary signs that the services will be maintained. The Social Action Project‘s Efficacy at Restoring Infrastructure and Services was Substantial.27 23 This is in line with the Government‘s decentralization program, through block grants to each district and transfer of responsibilities from the center to the Local Councils. For the Social Action Project, this includes responsibilities for general monitoring of communities, coordination of the line agencies, and channeling of funds. 24 Consistent with these pointers, there is no discussion in the health project‘s ICR of supplies being an issue for its clinics. 25 For one of the schools and one of the clinics visited by the IEG mission, when parents or patients were questioned by the mission as to whether the teaching materials and medicines supplied by these two ministries were adequate, they commented that the materials were insufficient, and sometimes came late. But problems did not appear serious. Medicines, vaccines and teaching materials did arrive, albeit sometimes irregularly, usually responding to the clinic‘s or school‘s critical needs; and the quantity and timelines of supplies was not one of the key issues raised by the staff or communities. However, insufficient staffing was evidently an issue for parents and users of the clinics. In both facilities, the communities, with their own money, had hired extra assistant teachers or nursing assistants. 26 More specific data would have been desirable on the adequacy of materials, staff, and medicines, as international experience has found that shortfalls in these areas are quite common. 27 This evaluation excludes the pilot shelter program. This was only 3 percent of actual project costs and followed a largely non-participatory approach, different from the rest of the Social Action Project. 19 BUILDING THE CAPACITIES OF LOCAL COMMUNITIES FOR COLLECTIVE ACTION 2.38 The preparation and implementation processes and institutional arrangements required under the Social Action Project were a foundation for empowering communities to take collective actions. The project brought in structured institutional arrangements and processes that necessitated interactions within a community. The first step was the establishment by a community of a Project Management Committee. At this stage NaCSA was closely involved: it required that members should be chosen with as much participation of the community as possible, and the Commission‘s staff were facilitators rather than leaders in the meetings. Focus group discussions undertaken in the 2006 and 2010 BIAs indicated that this was generally a democratic process, setting the stage for further cooperation in later steps. There were some requirements for the committees: a membership of 10 persons; women to be represented on the committee; illiterate as well as literate persons to be eligible for membership; and no government officials or chiefs to be members. These stipulations were generally complied with although there were exceptions (for instance, situations with a low representation of women, whereas a minimum of four female members was NaCSA‘s standard advice). Community meetings were held for major decisions such as location of the facility, and the committee organized supervision (on site monitoring) of the contractor. 2.39 Available information, most from the sample survey in the 2006 BIA (a sample of 630 community members from only 17 sub-projects across nine districts), indicates that the communities were fairly active participants in most of these processes and in subsequent implementation of their sub-projects, though this varied by stage of the process, and in some aspects by gender (see Annex C for the limitations of the data). 2.40 Initiating the process was the weakest stage (Table 3). While the community meeting was the primary avenue for informing the people (78 percent of male and 61 percent of female respondents), nearly all survey respondents felt that there should have been a longer consultation process and/or community outreach when NaCSA and other involved agencies introduced the project. Nevertheless, about three-quarters of the respondents said that the Project Management Committees were selected by vote at a community meeting and another 10 percent stated that it was by consensus during the meeting. There were very strong views (90 percent of respondents) that voting was important in selecting the Project Management Committee. 2.41 At the sub-project construction stage the great majority of respondents - 96 percent of the men and 87 percent of the women - stated that they contributed to the work (the NaCSA requirement is at least a 10 percent contribution from the community to sub-project costs), although the actual participation may have been lower if some respondents in the survey were embarrassed to say no. Nearly all respondents stated they made contributions in kind, either in labor (62 percent of respondents) or land (31 percent). Participation in project management was also found to be high. About 82 percent of respondents affirmed that there were monthly project implementation meetings. Procurement and financial management aspects of the committees‘ work had lower participation. Some 74 percent of respondents felt that procurement processes were transparent, and about 45 percent considered financing and financial management to also be so. Women respondents had somewhat lower assessments of 20 participation when it related to meetings. They were particularly marginalized at the beginning of the process - only 36 percent stated that they had participated in identifying the type of sub-project to be chosen, whereas the corresponding figure for the men was 68 percent. Table 3. Household Participation at Initiation of Sub-projects Male Female Question Answer respondents respondents (percent) (percent) What was the main way that your Through community meeting 78 61 community was informed about 18 the CDD program? Word of mouth 9 Radio 11 17 Other means 2 4 If you could change the way such Longer consultation 60 62 sub-projects are announced to the 30 communities what is the main Better outreach in the beginning 37 thing you would do? Scrap the Project Management 8 2 Committee More transparency 0.3 - Radio and other - 0.4 Is there a need for spending time Voting necessary 91 90 on democratic processes when it 5 might be better to just move on Voting unnecessary 3 with delivery (implementation of Don‘t know 6 the project)? 5 How were Project Management By vote at community meeting 74 76 Committees elected? By consensus at community meeting 9 11 Were not elected or by Local 17 13 Councils Source: 2006 Beneficiary Impact Assessment for the Social Action Project; Nordic Consulting Group (Sample of 630 beneficiary householders from 17 completed sub-projects) 2.42 As concerns construction, while there is no quantitative evidence, the 2010 BIA comments (page 39) that, during the sub-project construction phase ―providing general oversight to contractors in the course of implementation‖ was one of the two main activities of a project management committee (the other was to ―work closely with local authorities in mobilizing community-level resources, including unskilled labor and local materials‖). There are also qualitative comments elsewhere in the report that infer that contractor oversight was commonly carried out by committees and that good oversight was an important element to achieve quality construction.28 Similar comments were made to IEG by NaCSA and by 28 ―In most of the communities visited, they expressed this feeling that the quality of facilities constructed was to large extent linked to the fact that they had community representative who acted as deterrent against theft and 21 communities during site visits. Thus, triangulation from these sources suggests that oversight was a common practice and may have been a significant contributor to construction quality. 2.43 An important indicator of the capacity for acting collectively is whether the community can play its role in operating and maintaining the infrastructure and the service provided. A direct empirical assessment of the sustainability of the infrastructure and services is limited by the still short period of operations and, especially, the period covered by existing data. But the information that is available is consistent with a start in the right direction. The 2010 BIA found that 95 percent of sub-projects (all types) were still operational after 24 months (Table 4). At an aggregate level most communities were found in the BIA to be satisfied with the services and showing strong ―ownership‖ of their sub- projects. Overall, 79 percent of sampled households considered their sub-projects successful, with higher percentages (though on relatively small samples) satisfied with their court barays or community centers (92 percent) and 96 percent satisfied with their market sub-projects. Some caution is necessary however. The terms ―satisfied‖ and ―successful‖ are subjective terms, and there is little beyond these indicators and the functionality of the infrastructure over a fairly short time period with which to predict sustainability. More empirical data such as on the percentage of households that contributed to a community maintenance fund, or contributed labor or materials for operations and maintenance would have provided greater assurance on sustainability. Table 4.Indicators of Sustainability of sub-projects financed under the Social Action Project Percent of Sub-Projects in ―Good or Acceptable Condition‖ (n=90)a 80% (as assessed to be ―functional and kept in good or reasonable state‖) Percent of sub-Projects from 5 districts that were funded and operational 24 months after constructionb 95% Percent of sub-projects funded in 2004 (n=55) and still operative in 2007 96% Percent of sub-projects funded in 2005 (n=48) and still operative in 2008 90% Percent of sub-projects funded in 2006 (n=64) and still operative in 2009 95% Percent of surveyed householders who considered their sub-project as successful (n=1,656)c 79% Source: a. Beneficiary Impact Assessment 2010, sample of 90 out of 600 sub-projects. b. NaCSA district offices records of Kambia, Kenema, Koinadugu, Port Loko, Tonkolili districts. c. Beneficiary Impact Assessment 2010 2.44 There are also indications that the project has contributed to building social cohesion, evidently desirable, if not essential, for collective action, and also key to Sierra Leone‘s peace building process. In the 2006 BIA, 70 percent of respondents considered that the level of trust had improved over the previous two years (page 23). In a group discussion conducted by IEG with parents of children at a primary school, the general view was that the participatory processes, and in particular the way the parents were intimately involved in school management and hiring new teachers, naturally ―brought people together.‖ Similar views were expressed at two health centers. Regardless of former divisions, people visited poor quality of work on the part of the contractor. At the same time, the committee acted as motivators for influencing contractors towards building an asset that was rich in quality.‖ (2010 BIA, page 40) 22 the health centers when necessary. Conversations while waiting for treatment between people from all social groups were helpful. The regular village visits of a nurse for open discussions of preventative and basic health also helped. ―Bridges were built between us‖ was one comment. And, reflecting on all of the project‘s activities in his District, a Deputy Councilor commented that (community driven development) had ―really transformed our communities.‖ 2.45 The various sources of information above, while substantially qualitative in nature and often based on small samples, suggest that, taken overall, the project, through its participatory processes has Substantially helped Build the Capacities of Local Communities for Collective Action. As to be expected, however, each community has its own unique social character, and the individual experiences of the communities were notably diverse, both in how the communities built and managed their facilities, and in the quality of implementation and utilization. ASSISTING VULNERABLE HOUSEHOLDS TO ACCESS TEMPORARY EMPLOYMENT 2.46 This objective was added to the project in August 2008 to provide employment and incomes to those particularly affected by the food price crisis. Rice prices had peaked only three months earlier. A ―cash for work‖ program was established to benefit, on a temporary basis, the most food-vulnerable (the poor, unemployed youths, and women). The program covered all of the country‘s 14 districts (Table 5) and involved a total of 727 sub-projects in both urban and rural areas.29 There are two main sources of information about the program: the 2010 Beneficiary Impact Assessment and the ―Qualitative Assessment of Cash for Work Program for Sierra Leone‖ (Dalan Consultants 2010).30 2.47 The great majority (80 percent) of the cash-for-work sub-projects were for rehabilitation of small rural and urban unpaved roads. Restoration of small earthen irrigation channels for valley swamp rice made up another 5 percent, land clearing and preparation for other crops (predominantly cassava and vegetables), a further 11 percent, and the balance (four percent) was for environmental projects such as urban drainage. Sub-projects were required to have a minimum of 60 percent of costs in unskilled labor. Daily wage rates varied by district but were typically about Leones 7,000 (US$ 1.60). Rates were set at levels about 70 percent of average wages for unskilled labor in the private sector.31 In an attempt to provide a self-selection incentives structure, it was presumed that those who were employed and getting remuneration above these rates would not seek to join the program. 2.48 The program provided temporary employment for about 39,400 persons, higher than the 31,000 beneficiaries anticipated, due mainly to a reduction in days per beneficiary practiced at some sites to accommodate excess demand for work (World Bank 2009d). This 29 A breakdown between rural and urban sites is not available. 30 The methodologies and limitations of these surveys are discussed in Annex C. 31 The rate was generally within the range of 6,000 to 7,500 Leones per day. The difference between private sector and cash-for-work wage rates could be significantly greater in urban areas – in the western urban areas (primarily Freetown) cash-for-work rates were less than half of private sector rates (Dalan Consultants, 2010). 23 resulted in about two million person days of employment. The average number of workdays per beneficiary was 50, and typical total earnings of a beneficiary were about Leone 350,000 ($80). The 2010 BIA, states that 341 sub-projects had been completed and were in use out of 347 sub-projects that had commenced under the first phase of the program. (This may overstate the completion rate as the Qualitative Assessment found that beneficiaries raised the issue of contractors that didn‘t complete their job.) There are few, even qualitative, statements about the quality of the work, but concerns about future maintenance of the rehabilitated roads are raised. In principle, Local Councils and line agencies such as the Sierra Leone Roads Authority are expected to take charge of maintaining the rehabilitated roads. But experience in Sierra Leone is that development programs are seldom sustainable without continual external financial support. Table 5. Distribution of Cash-for Work - Sub-projects by District District Number of Sub-projects Western rural 68 Freetown 96 Kono 58 Bonthe 43 Kambia 27 Kailahun 33 Port Loco 68 Tonkolili 37 Bombali 84 Kenema 38 Moyamba 57 Pujehun 34 Bo 64 Konadugu 20 TOTAL 727 Source: NaCSA (covers all of Sierra Leone’s 14 districts including Freetown) 2.49 The program was effective in reaching its primary demographic target groups. About 75 percent of beneficiaries were youths, nearly 30 percent were women, and most of these groups were also war affected. Unemployed youths were particularly targeted to help mitigate the risks of conflict. Many were uneducated because they had been combatants during the war, and were typically poor and needing employment to occupy their time and to generate much needed income.32 The predominant views of Government officials, and as also reported in the 2010 BIA, are that the program had reached youths as well as other vulnerable groups, and had been a stabilizing influence. In a similar vein, the BIA reports that focus groups felt that the program reduced the risk of conflict by occupying the youths‘ 32 As reported by Platt (2009), 70 percent of the Nation‘s youths were unemployed in 2009. 24 time and providing subsistence. Several government officials and a Local Council expressed the same views to the IEG mission. 2.50 The targeting of potential beneficiaries was through a two-stage process, first with the choice of areas for involvement and communities within these areas, and secondly, during the choice of individual beneficiaries within the communities. The selection of the areas was based on a national ―Comprehensive Food Security and Vulnerability‖ analysis carried out in 2008 by the World Food Program and Food and Agriculture Organization which was based on prevalence of poverty and food insecurity, unemployment and nutrition rates. Selection of communities within these areas was done by a Project Approval Committee comprising the Local Council, NaCSA, and representatives of line ministries. The Qualitative Assessment report commented that community selection was based on the official cash-for-work guidelines, and considered that the program had largely reached the poorest communities. A NaCSA official and a Local Council official met by IEG advised that the committee, by having multiple entities involved and a Local Council that was answerable to its citizenry, was protected from bias. This was not always the case, however. The Qualitative Assessment also comments that some council representatives attempted to lure the choice of areas to their own wards. 2.51 The primary shortcoming in the beneficiary selection process was the method of choosing individuals. The Qualitative Survey, based on focus group discussions with beneficiaries from the first phase of the cash-for-work program, found that selection of individuals in three districts had involved the community, apparently without complaints. But the majority of the sample communities in two other districts complained that participants came from outside the community (page 75). In one site visited by the IEG mission (Bombali), selection of participants was primarily by one individual (Box 3). To counter such problems, NaCSA brought in a more institutionalized approach through a multi-party committee ensuring more rigorous selection. From accounts to IEG, the presence of representatives from the Local Council, town council, and village elders influenced a more objective selection process. Box 3. The Cash for Work Program in Bombali District A cash-for-work road reconstruction site in Bombali district rehabilitated a 3 meter-wide dirt road in an urban community and had been completed about three years ago. The community had done light maintenance since then. The scheme provided work for ―Youths‖ (men or women 15 -35 years old) to work on a temporary basis over an 8-week period. This site had provided such temporary jobs for about 50 youths at any one time. On average about 30 percent of the youths were women. The Bombali District Youth Chairman recounted the method used to select the participants. The objective was to employ the most vulnerable ―street youths.‖ A combination of identification based on apparent poverty and of self-selection was used. First, he made house- by-house visits and identified the poorest youths who were not employed. This would inevitably contain some subjective judgment, but he considered the next step would tend to only attract the very poor. The daily wage rate of 6,000 Leone was well below rates for laborers hired by contractors who paid about 15,000 to 18,000 per day for the same kind of work. Thus, from his experience, only those who genuinely needed the income tended to come forward. He also felt that food for work schemes would be better, earnings would be entirely for family nutrition, and in kind payment would provide greater buffering against food price increases. Source: IEG mission 25 2.52 The substantial degree to which cash-for-work reached youths and women was not matched by the program‘s achievement reaching the most impoverished of the population. A 2012 World Bank study33 found that only 5.5 percent of the poorest of beneficiaries were in the poorest quintile, and that even when the poorest and the second poorest quintiles are taken together, only 20 percent of beneficiaries were in these categories.34 2.53 In implementing the cash-for-work program, the Qualitative Survey cites a number of issues based on its focus groups discussions: selection of individuals could be by a contractor without much regard for social concerns; might target relatives; could involve politicians; and might even select individuals from outside the locality. Whether these are isolated cases or occurred frequently is not known. Nevertheless, two issues are given particular prominence–performance of contractors, and lack of community involvement–and appear to be at the heart of the program‘s problems. Only 27 percent of communities were satisfied with the performance of their contractor, the most commonly cited issues being that the contractor might marginalize community involvement, might provide wages to communities late or below the determined wage, sometimes did not complete a job, lacked transparency, and might be dishonest. Lessons emerging from the project experience with cash-for-work are summarized in Box 4.35 2.54 In summary, these various sources indicate that the cash-for-work program had some success at reaching youths and women and in improving the welfare of its beneficiaries, and was partially successful in reaching the target groups. The lax processes for selecting beneficiaries enabled cases where vested interests might determine who participated, contractors had too much power, which enabled malpractices such as late payments, selection of participants from outside of a community, and poor monitoring of quality. And communities were often marginalized in decision making and beneficiary selection. There is no way to quantitatively determine the extent of these problems, but in both the 2010 BIA and the Qualitative Assessment, such cases were often referred to during focus group discussion, especially as concerns the contractors. In turn, these issues pulled down the effectiveness of targeting the poor, and the ―ownership‖ of the investments by local communities. Hence, for the project objective to Provide Temporary Employment to Vulnerable Households, Efficacy was Modest. 33 Backiny, Yetna and Wodon as reported in Andrews, Ovadiya, and Wodon (2012). 34 By contrast, the cash-for-work program in Liberia reached 60 percent of beneficiaries in the two poorest quintiles. According to the draft Social Protection Assessment (World Bank 2012) in 2007 some 57 percent of Sierra Leone‘s population lived below the poverty line. 35 The design of the follow-on Youth Employment Support Project (FY10) appears to have taken note of a number of these lessons (Annex 9 of the PAD). 26 Box 4. Lessons from the Cash-for-Work Program Beneficiary communities should be much more involved in the cash-for-work program: in selection of project location, type of project, and participants, and in monitoring of the works. Targeting of communities and participants should be significantly improved. Minimize the use of contractors or the range of their responsibilities, with communities taking over selection of sites and participants, and monitoring the contractors‘ activities (the contractor‘s role being mainly restricted to construction). Women‘s participation should be improved through inclusion of more projects other than roads (where strength is a primary need), or tasks should be differentiated within the sub-project by activities more suitable for women (fetching water, gravel, cooking, etc.), and arrangements for looking after children. Communities should be trained in management and maintaining the works. They should also be left with the tools to undertake maintenance. Sources: (i) Dalan Development Consultants, August, 2010: Qualitative Assessment of Cash for Work Program for Sierra Leone; (ii) World Bank, June 16, 2010: Youth Employment Support Project. PAD; and (iii) IEG discussions with NaCSA and Manager of cash-for-work program for Bombali District, February, 2012. Efficiency 2.55 Economic rates of return for the project were not calculated at appraisal or in the ICR, on the grounds that benefits would be difficult to quantify, especially as concerns welfare impacts - the essence of a social fund. There are, however, a number of observations that can contribute to an overall assessment of efficiency. For the original objectives of the project these are:  Cost-effectiveness: NaCSA administrative and operating costs were about 12 percent of project costs, which is reasonable for a community development project where extensive capacity building is needed. While sub-project investment costs were higher than envisaged at appraisal, this was primarily due to the unanticipated hike in costs of materials, especially cement, which affected construction costs of infrastructure generally, across all projects. The ICR reports that investment costs per classroom and per clinic under the Social Action Project were lower than the costs of similar facilities constructed by government and under an Economic Union project (although these comparisons are qualified by lack of clarity on whether the facilities were the same and whether they were implemented under the same conditions).36  Operational efficiency: The average time taken from application to approval of a sub-project decreased during implementation from about 21 months to 8 months - the latter is not excessive given the extensive meetings and sequential steps needed for effective community participation. 36 Such as, whether the facilities are similar in size and furnishing, new construction or rehabilitation, inclusive or not of community contributions, and other possible dissimilarities. Also, the costs compared are the direct investment costs and exclude management and training. The community driven approach could be expected to have higher overheads than programs with less participation. 27  Allocative Efficiency: Communities had free choices of the kind of subproject they wanted, and most made their choices through democratic processes. Most communities chose education or health because they felt that these would maximize their welfare. Thus, the high level of participation in community discussions would generally cause project funds to be allocated to investments where socioeconomic impact would be greatest. 2.56 Based on such observations, the Efficiency of the Social Action Project‘s Original Objectives is assessed Substantial. 2.57 Under the added objective to provide temporary employment to vulnerable households, all of the $8.0 million allocated to the cash-for-work program was disbursed. This was within the 2 additional project years provided for the program, and the total number of labor days reached 2.0 million compared with the target of 1.7 million. There was also widespread interest in participating in the program, because it was seen as significantly improving family incomes and welfare. Nevertheless, the weaknesses in selection of beneficiaries resulted in only partly reaching the poorest and most vulnerable households, hence under-achieving the program‘s potential benefits. The Efficiency of the temporary employment objective is assessed Modest. This also reduces the efficiency of the project under the Revised Project Objectives (combining all three objectives) to Modest. Ratings OUTCOME 2.58 The civil war had resulted in widespread destruction of Sierra Leone‘s infrastructure, and there was a crucial need for war-torn communities to recover from the divisions of the past. The Social Action Project‘s design was well framed to achieve these objectives. The project successfully contributed to restoring community infrastructure and services and in building the capacity of local communities for collective action. It was cost-effective, and allocation of funds followed community decisions on what they perceived as the most beneficial for their needs. Under the Original Objectives, project Outcome was Satisfactory. 2.59 The additional objective to help the vulnerable access temporary employment, implemented through the cash-for-work program, was a relevant emergency response to help mitigate the hardships caused by the food price crisis. The program created jobs and improved incomes. It benefitted participants and provided learning for future scaling up of the program. However, it was less effective at reaching the most vulnerable, had limited involvement of communities, and little flexibility in choice of activities suitable for women. Both Efficacy and Efficiency were modest for this additional objective. Taking into account the results for all three of the objectives, the Outcome under the project‘s Revised Objectives (the original objectives plus the additional objective) is Moderately Satisfactory. 2.60 Under standard IEG/OPCS procedures for assessing a project with formally revised objectives, the outcome of the project under the original objectives and under the revised objectives is estimated separately, and the overall Outcome is a weighted average of the two ratings, with the weights based on the share of final disbursements disbursed as of the time of 28 restructuring. On this basis, as disbursements under the original objectives are significantly larger than under the revised objectives, the Social Action Project‘s Outcome was Satisfactory.37 RISK TO DEVELOPMENT OUTCOME 2.61 Political and institutional commitment to support the Social Action Project‘s investments appears favorable. Government is strongly backing the community investment program, which it considers central to poverty alleviation and development of grass-roots society. The necessary support to the implementing agencies can be expected to continue, although this needs to be reflected in sufficient budget allocations, a problem during the project period (refer below). For the agencies themselves, the key needs will be that: NaCSA continues to provide good coordination, and technical assistance; the Local Councils further develop their capacity to manage the programs in their districts; and the line ministries continue their support through technical assistance and provision of teachers, nurses, and regular supplies of medicines and materials. As concerns the cash-for-work program, the provision of temporary employment which was the program‘s objective, has already been implemented, although, for maintenance of the roads that were constructed, continued funding will be needed. 2.62 The most fundamental need is for continued commitment by the communities. Actual or proxy findings related to the project‘s achievements indicate broad-based participation within most communities, appreciation of the services provided by their sub-projects, and motivation to maintain their investment. Consistent with this, most sub-projects, although still recently constructed, were operating and in good condition at the time of the 2010 BIA. 2.63 Insufficient government financing of the recurrent costs of providing the services associated with the infrastructure is the project‘s main risk. A shortfall in counterpart funds was an implementation problem during the earlier years of the project. The gaps and delays in Government funding might be expected to continue. As government finances teachers, medical personnel and materials such as medicines and pedagogical supplies, this would disrupt services. Two factors mitigate this risk. First is Government‘s high regard for the community and cash-for-work programs, making chronic financing deficiencies less likely. Second, the country‘s fiscal situation will likely improve from 2013, once the new iron ore revenues come on-stream. Risk to Development Outcome is Moderate. 37 Disbursements at the time of restructuring were $37.94 million. Disbursements for the additional objective were $ 8.1 million, resulting in total disbursements at project closure of $ 45.94 million. The weights are, thus, 37.94/45.94 = 0.826 for the original objectives, and 8.0/45.94 = 0.174 for the revised objectives. Outcome ratings are given scores of one to six, six being for highly satisfactory, five for satisfactory, four for moderately satisfactory, and so forth, descending to one for a highly unsatisfactory project. Under the original objectives the satisfactory outcome gets a score of five, and under the revised objectives the moderately satisfactory rating gets a score of four. The share of the original objectives in the overall outcome is 5 X 0.826 = 4.130 and the share for the revised objectives is 3 X 0.174 = 0.696. The overall outcome is 4.130 + 0.696 = 4.826. Rounding to the nearest whole number, the outcome is five, which translates to a ―Satisfactory‖ rating for the project‘s overall outcome. 29 QUALITY OF M&E 2.64 M&E Design had a systematic reporting system for operational monitoring, but few monitorable indicators to evaluate the quality and outcomes of the project, as discussed earlier. 2.65 M&E Implementation provided regular output type data for operational purposes, but the gap in regular monitoring of outcome data persisted. The two Beneficiary Impact Assessments in 2006, at about mid-point of the project, and in 2010, the last year of the project, only partially made up for this. 2.66 M&E Utilization: The 2006 and 2010 BIAs have helped inform NaCSA management and Bank staff on the aspects of the project that have been beneficial and what needs revising. Both have recommendation sections to this effect. However, as noted above, outcome-oriented data have been limited and, where they exist, tend to be based on focus group discussions, which largely provide qualitative rather than quantitative assessments. In addition, the BIA surveys and the Qualitative Assessment survey have some methodological and sampling limitations (Annex C). 2.67 Use of monitoring systems for project outputs has been more successful. NaCSA‘s Management Information System has become a well-integrated and important part of its management. The M&E cell‘s quarterly reports (based mainly on data provided by the various headquarter divisions and district coordinators) are the main documents used in management meetings. An action plan is prepared after each meeting, which is then itself monitored for actions taken, and as needed, followed up with instructions to lagging units. 2.68 Although operational monitoring was good, given the weaknesses in monitoring outcomes and the limited evaluative evidence collected, the overall quality of the project‘s M&E system was Modest. BANK PERFORMANCE Quality at Entry 2.69 In the difficult circumstances of the immediate post-war period, the Bank designed a project to address two critical needs: reestablishment of social infrastructure and services; and, building the capacity of local communities for collective action. The Bank launched a particularly participatory community-driven development approach, devising effective institutional, implementation, and fiduciary arrangements. M&E was well designed for management decision making, though wanting in assessing project outcomes. Nevertheless, given that the project‘s design was in other respects good, Quality at Entry is rated Satisfactory. Quality of Supervision 2.70 The Bank provided experienced staff specialized in areas relevant to the project. Supervision was regular and intensive, the team working closely with Government and NaCSA, which helped resolve most project implementation problems. Fiduciary 30 management was assiduous. The lack of outcome measurements in the M&E system was partially made up for through the beneficiary impact assessments, but there were still gaps. The Bank reacted to the food price crisis expeditiously. Under emergency procedures the first Additional Financing grant was approved on July 29, 2008 – shortly after the peak in rice prices in April-May. All NaCSA and government officials met by the IEG mission had strongly favorable views of the Bank and its supervision (and preparation) team. While the cash-for-work program had a number of difficulties, the task team made adjustments, and the lessons learned were reflected in improved design features for future programs. 2.71 The follow-on activities for the community-driven development program could have received greater attention. While the cash-for-work program was continued as a component of the Youth Employment Support Project, there was no such provision for the more complex community-driven program. Neither continued Bank support nor a planned exit strategy was implemented. In the event, the task team informally persuaded the Islamic Development Bank to finance continuation of the community-driven program, but a planned and specifically designed exit strategy or a follow-on community development project would have provided more assurance that continued funding would be provided, as well as the necessary technical guidance. However, as concerns the project period, in the difficult task of pioneering and steering a new community driven development approach responsive to Sierra Leone‘s needs, supervision was strong and a key factor in the Social Action Project‘s success. Overall, the Bank‘s Supervision Performance was Satisfactory. Taking account of both Quality at Entry and Supervision performance, the Bank‘s Overall Performance was also Satisfactory. BORROWER PERFORMANCE Government Performance 2.72 Government strongly owned the project and the community-driven development approach. It provided an enabling policy and institutional environment, including dependable back-up to NaCSA in the difficult area of decentralizing activities and creating institutional cooperation between NaCSA, the line ministries and the Local Councils. Government was also proactive in helping resolve implementation problems encountered as the project proceeded. Cooperation with the Bank was excellent. 2.73 The main difficulty was in provision of counterpart funds. The country‘s critical fiscal constraints contributed to financing shortfalls from Government in the early years of the project, and slowed implementation. This reduces an otherwise strong Government Performance to a Moderately Satisfactory performance overall. Implementing Agency Performance 2.74 At project commencement, NaCSA had limited overall capacity, and minimal experience with community-driven development approaches. NaCSA took on the new approach and was a proactive learner and implementer throughout the project period, progressively increasing its effectiveness. NaCSA vigorously embraced the challenge of bringing together the line ministries involved, including monthly meetings between NaCSA 31 and the line ministries, both at central and district levels, the latter also including the Local Councils. Particularly challenging was helping the councils to build capacity for their future lead role in field coordination of the program. And most challenging of all for NaCSA was the motivation, technical assistance, teaching and monitoring of the 600 community-driven and over 700 cash-for-work sub-projects. The cash-for-work program had mixed performance, but this was in part due to the need for scale-up on an emergency basis based on only limited experience. With a less established and field-based institution than NaCSA was becoming by the time the cash-for-work program commenced, the implementation problems would likely have been worse. As concerns M&E, monitoring of project outcome was weak, However, NaCSA had a good management information system which was effectively used to track project progress and outputs, and the beneficiary assessments partly made up for the limited attention to outcomes in regular monitoring. Implementing Agency Performance was Satisfactory. 2.75 Taking into account the Satisfactory Implementing Agency Performance and Government‘s Moderately Satisfactory performance, Borrower Performance overall is rated Moderately Satisfactory. 3. Food Price Crisis Response Development Policy Grant Objectives, Design, and Relevance 3.1 The Food Crisis Grant was an emergency operation responding to the 2007-2008 Food Price Crisis that affected numerous countries world-wide, particularly in 2007 and 2008. The Grant provided budget support to Sierra Leone at the height of the surge in food prices and was one of the two main actions – the other being the addition of the cash-for- work program under the Social Action Project - taken by the World Bank to assist the country‘s short-term efforts to mitigate the hardships brought on by the price hikes. OBJECTIVES 3.2 According to the Program Document for this development policy operation, the objectives of the Food Crisis Grant were to: "Support the Government's poverty reduction strategy in 2008 by providing the authorities with needed fiscal space to partially compensate for the lost revenues resulting from the recently reduced tariffs on food and fuel imports. This support would help mitigate the impact of such price increases and contribute to continued basic service delivery for vulnerable groups" (World Bank 2008).‖ 3.3 These objectives can be broken down into two anticipated outcomes in the short run, against which the operation will be assessed:38 38 The two objectives and results chain are presented in Annex 2 (―Program Matrix‖) of the Program Document. Providing fiscal space (budget support) to partially compensate for the lost revenues resulting from the reduced 32  Mitigating the impact of food price increases on consumer welfare; and  Contributing to continued basic service delivery for vulnerable groups. RELEVANCE OF THE OBJECTIVES 3.4 The two objectives were both critical needs in Sierra Leone as it faced the hardships brought on by the food price crisis. Reducing poverty and improving social services were well articulated priorities in the Country Assistance Strategy for FY 2006-09, the relevant policy document at both appraisal and closure of the grant operation.39 Also, the Bank‘s March 2010 Country Assistance Strategy and Government‘s 2009 Second Poverty Reduction Strategy Paper, the ―Agenda for Change,‖ included human development and inclusive agricultural growth as two of four development targets (the other two were development of energy and transport infrastructure). The operation‘s objectives were, therefore, broadly enveloped within the overall strategy, although, interestingly, food security, and specific actions for reducing the country‘s vulnerability to future price shocks received only minor attention in the Country Assistance Strategy. They were framed recognizing the limits of what a modest grant of $3 million could achieve–―supporting‖ the poverty reduction strategy, and ―partially compensating‖ and ―helping mitigate‖ the impacts of the food price increase, and it was recognized as a short-term intervention (the 5 months until the end of 2008). The objectives fit the strategic needs of Sierra Leone at a critical (emergency) juncture, and were Substantially Relevant in this capacity. DESIGN 3.5 The operation was funded by a Development Policy Grant of $3 million, disbursed in a single tranche as emergency budget support to help compensate for the reduced revenues that the Government was receiving as a result of its response to the food price crisis. A Letter of Development Policy from the Minister of Finance and Economic Development described the actions that would be assisted with the extra fiscal space provided by the grant. Prior actions (implemented before grant approval) were reductions in import tariffs for several basic food commodities and fuel. These reductions were also specified in the operation‘s results framework in the Policy Matrix of the Bank‘s Program Document (as summarized in Table 6). The two policy areas–food security and human development– were based on two pillars of the Poverty Reduction Strategy:  Food Security: This policy area supported measures to mitigate the impact of food price increases on consumer welfare. This was to be done by maintaining the lower import tariffs on main food commodities and oil.40 tariffs on food and fuel imports is a means to achieve the objectives – rather than an outcome. It will thus be discussed as an action intended to contribute to the two objectives. 39 There were a number of other policy documents at that time. The June 25, 2008 (first) Poverty Reduction Strategy Paper was issued six weeks before the Grant‘s approval, and a Progress Report for the 2006 -2009 Country Assistance Strategy was released on November 5, 2008, three months after approval. 40 As discussed in the Framework Document for the Global Food Crisis Response Program (World Bank, June 2008), ―food security‘ has multiple dimensions; amongst them: policy development, short -term response strategy, agricultural production, post-harvest losses, marketing and storage, commodity trading, social safety 33  Human Development: The second policy area contributed to continued basic service delivery to vulnerable groups, despite the increased costs resulting from the food price hike. The services specified were: (i) feeding for patients in hospitals and community health centers; (ii) provision of food 3 times a day for lactating mothers and children under five in Ministry of Health and Sanitation Hospitals; and (iii) provision of food three times a day for school children, handicapped children, and children in remand homes and approved schools. Table 6. The Food Crisis Grant’s Results Framework Policy Objectives Actions Outputs Outcomes Areas Mitigate the Food Reduce import Reduction in: Increased impact of security tariffs on rice, rice tariff from 15% to 10% consumer food price flour, wheat, flour tariff from 20% to 10% welfare increases on sugar and wheat tariff from 5% to 2.5% consumer petroleum sugar tariff from 20% to 10% welfare products.a petrol tariff from 5% to $20/mt. Contribute to Human Maintain 2007 Maintained at the 2007 level or better: Poverty basic service Develop levels of basic - Number of hospital patients in district hospitals reduction delivery for ment services by and community health centers that receive food vulnerable allocating maintained at 12,800 groups additional funds - Number of lactating mothers and children under to cover rising five in hospitals of the Ministry of Health and costs of food and Sanitation that receive feedings three times a day fuel maintained at 5,300/year - Number of pupils in government boarding schools and handicapped children who are fed three times/day maintained at 3,000 - Number of children in remand homes and approved schools who receive food maintained at 500/year Sources: Government Letter of Development Policy and Program Matrix at Annex 2 of Program Document Notes: a. This was a prior action, which was fully met before the project was approved. RELEVANCE OF DESIGN 3.6 The Results Chain. The causal chain between the Food Crisis Grant‘s objectives and the impact of the operation on these objectives could be expected to be through the following sequence: The steep rise in international commodity prices would likely be transmitted to domestic prices. The reduction in import tariffs on cereals and oil could be expected to reduce the increase in domestic prices of cereals. This in turn would reduce the increase in the costs of foodstuffs for consumers, thereby somewhat mitigating the negative impact of nets, advance warning systems, communications and coordination, institutional strengthening, fiscal support, tariff reductions, investment in infrastructure for production and marketing (e.g., irrigation, feeder roads), targeted programs for vulnerable groups, and other act ions. The primary focus of the PPAR‘s analysis for the Food Crisis Grant‘s consumer welfare objective is narrowed down to the action‘s and impacts of the food import tariff changes that the Grant enabled, although reference is made to other options where relevant. 34 the international price increases on welfare. The poor would benefit more, as a larger share of their family incomes is spent on food. The tariff reduction would also reduce revenues for the treasury, reduce fiscal space, and increase difficulty in funding of public sector activities, including for social services. The Grant‘s fiscal injection would contribute to maintaining basic service delivery for vulnerable groups, the operation‘s second objective. However, achievement of the first objective assumes that tariff reductions would be transmitted to consumer prices, which rely upon an efficient and fully responsive domestic rice market. Inadequate data preclude quantitative assessment of the degree of carry-through, but experience elsewhere, such as in Djibouti (IEG 2012),41 suggests that market distortions can significantly reduce pass-through to the consumer. 3.7 The relation between the program‘s outputs and outcomes is logical, even if not quantifiable. If a tariff reduction prevents prices rising as high as they otherwise would, the cost of living and consumer welfare would be improved. However, given the low level of tariffs to begin with (from 5 to 20 percent depending on commodity) and the relatively small absolute reductions, it would be unlikely that this alone would have a major effect on mitigating price increases42 and improving welfare, while the reduction in revenue could be predicted. Thus, for the first objective, the design relevance was not strong. The second objective had a more tangible outcome. Continued school feeding, or food programs for hospitalized lactating mothers and children under five years old would be expected to mitigate the impact of the food price crisis on vulnerable groups, who otherwise might lose those benefits because of budget constraints imposed by higher food prices. 3.8 Choice of Instrument. There was an evident advantage in the Development Policy Grant instrument, as it could be prepared and disbursed quickly. It also came in at a time when the crisis was most evident – shortly after the peak in international rice prices. Neither the Government nor the Bank could presume what would happen next, and a continuation of the surge in prices would add further hardship for the country‘s already destitute people. A fear of yet more price increases was developing in Sierra Leone. Protests in urban areas were beginning to mount, and the prospects of resumed civil unrest carried considerable risk to the country‘s still fragile social fabric. 3.9 An investment loan would have taken longer to appraise than the Grant and disbursements would have been linked to specific activities over a much longer time period. A short cut would have been to embed these funds in an ongoing project such as with the Social Action Project‘s additional financing for the cash-for-work program which was prepared quickly and approved in August, 2008, the same month as the Food Crisis Grant. However, disbursements against actual expenditures, as in any investment loan, could not create the fiscal space that budget support could provide. The $3 million budget transfer is not large relative to annual government revenues and expenditures, but has greater 41 The PPAR for the Food Crisis Response Development Policy Grant in the Republic of Djibouti comments that ―There is statistically strong evidence of some pass through from world prices to domestic prices, although the correlation coefficient (71 percent) is sufficiently less than one to suggest that there are some other more dominant factors at play. It is possible that removal of taxes increased traders‘ margins rather than being passed through to consumers.‖ 42 This was a point made in the Global Food Crisis Response Program framework document. 35 significance when compared to the projected budget deficit at that time of $48 million, the estimated loss in revenues due to the tariff reductions ($5.4 million), and the expected increase in the costs of delivering basic services of $3.5 million. 3.10 The relevance of the funding of social programs to vulnerable groups was substantial, but in Sierra Leone‘s circumstances, the tariff reductions would have had little impact on food prices and consumer welfare. The Relevance of the Grant‘s Design to its objectives was Modest. Implementation 3.11 The Food Crisis Grant was appraised in July 2008 and approved 12 days later, on August 6th. The government‘s Letter of Development Policy had been sent to the Bank on July 25, 2008, and the tariffs had been reduced in March 2008 (prior actions). The grant was disbursed in a single tranche on August 27, 2008. The operation was formally closed on June 30, 2009. The Ministry of Finance and Economic Development was the implementing agency, and it was intended that two committees would be established for monitoring the program‘s action plan: at the policy level, an Inter-ministerial Steering Committee chaired by the Minister of Finance and Economic Development and, at more operational level, a Technical Committee chaired by the Secretary of Finance and comprising relevant ministry and agency heads. There is no record that the Technical Committee was established. 3.12 Maintaining Macroeconomic Stability. The Grant‘s results framework does not include the measures that Government would take for fiscal stability, but anticipated that food and fuel prices would rise faster than export prices, resulting in deterioration in the terms of trade, which would start to be compensated from 2012 by very large earnings from new iron ore mines. The medium-term objective would be to dampen the inflationary impacts of the commodity price increases, and to maintain economic growth within a stable macroeconomic framework, while continuing to finance poverty reduction programs. Prudent fiscal policy and tightened monetary policy would limit price increases and debt build-up to sustainable levels. 3.13 Sierra Leone‘s economic situation during and immediately after the food price crisis showed mixed performance (Table 7). The fiscal balance (overall balance excluding grants) deteriorated in 2008 to a budget deficit of 9.2 percent of (non-iron ore) GDP. The budget deficit deteriorated further in 2009 to 11.1 percent of GDP. In contrast, the budget deficit was lower in 2007 due to a major Multilateral Debt Relief Initiative by the World Bank and the African Development Bank. The fiscal deficit started to decline from 2010 onwards, but it is still too high to ensure macroeconomic stability. Inflation in 2008 and 2009 was kept at 10- 11 percent per annum. GDP growth slowed from its average of about 10.2 percent per annum in the 2002-2007 period to about 5.5 and 3.2 percent per annum in 2008 and 2009 respectively, and then picked up starting from 2010. Macroeconomic management was less disciplined in 2010 and 2011. Revenues rose, but expenditures increased by over 40 percent between 2009 and 2010, and the deficit more than doubled. Inflation rose from 11 percent in 2009 to 18 percent in 2010. The IMF 2011 country report expressed concern with the increased inflation rate but noted a number of actions by Government to tighten fiscal and 36 monetary policy. The prognosis, providing macro-economic management is sound, was considered good, led by the expected high revenues from the new iron mines in 2012-13. 3.14 As a proportion of the country‘s overall budget, education expenditures remained remarkably uniform at about 18 percent from 2005 to 2009. Health expenditures fell somewhat – from about 10 percent of budget before the price shock to about 8 percent during and after the crisis (2008 and 2009) - but in 2010, health expenditures more than doubled as a new national initiative was launched.43 Table 7. Sierra Leone Fiscal and Macro-economic Indicators 2007 2008 2009 2010 2011 2012 est. (IN BILLIONS OF LEONES, CURRENT TERMS, EXCEPT WHERE STATED) Total revenues and grants 2130 935 1250 1552 2171 2527 revenues 537 670 750 1008 1462 1826 a grants 1593 265 500 544 709 701 Expendituresb 877 1207 1452 2073 2752 2907 current 661 848 1000 1286 1603 1926 capital 216 359 452 787 1149 981 Overall fiscal balance (deficits) Overall balance including grants (in Leone billions) 1253 -272 -202 -521 -581 -380 As percentage of non-iron ore GDP 25 -4.7 -3.2 -5.1 -4.6 -1.9 Overall balance excluding grants (in Leone billions) -340 -537 -702 -1065 -1290 -1081 As percentage of non-iron ore GDP -6.8 -9.2 -11.1 -10.5 -10.2 -6.6 Annual inflation rate (end of period ) 11.7 12.2 10.8 18.4 16.9 11.1 c Real GDP growth 6.4 5.5 3.2 5.3 6.0 6.3 Notes: a. Includes Multilateral Debt Relief Initiative in 2007. b. 2007 expenditures include contingency spending. c. Excluding iron ore sources starting from 2011. Source: IMF Country Reports for Sierra Leone - Years 2008 - 2011, and World Bank Country Assistance Strategy 2010. 43 The Health Care Initiative, which provided subsidized medicines and increased the number of health care professionals, targeted in particular to vulnerable groups (IMF Country Report, 2010). 37 Achievement of the Objectives MITIGATING THE IMPACT OF THE FOOD PRICES INCREASES ON CONSUMER WELFARE 3.15 Reduction in tariffs. In March 2008, import tariffs on key foodstuffs and petroleum were reduced as follows: the rice import tariff was reduced from15 percent to 10 percent; the wheat tariff from 5 percent to 2.5 percent; wheat flour from 20 percent to 10 percent; sugar from 20 percent to 10 percent; and petroleum 5 percent to $20/mt. 3.16 Changes in Domestic Food Prices. Reliable data on domestic prices of foodstuffs are very limited, including for rice. Nevertheless, the trend in the domestic prices of rice is discernible and appears to have followed the profile of international prices. From remarks made to IEG by government and consumers, and documents from the time by the media, NGOs, and other development agencies, domestic rice prices typically increased by between 50 to 100 percent. The change varied depending on: (i) location (urban or rural); (ii) for a rural area, the degree of self-sufficiency in rice; and (iii) the time frame. As a generalization, the more that a local market was exposed to international prices (for instance, the urban and peri-urban areas, especially Freetown), the larger the increase in the consumer price of rice. A 50 to 100 percent range would have been typical in urban, peri-urban, and more accessible rural areas where rice consumption exceeds production. 3.17 Rice was the only main food staple in Sierra Leone significantly influenced by international prices. The prices of cassava and sweet potatoes, largely domestic products, did not follow the price profile of rice (Annex B, Table B.5). The domestic prices of higher value commodities such as bonga (dried fish), milk, and eggs also bear no particular relation to the spike in rice prices. 3.18 Impact of the tariff reductions on food prices. Due to the many variables influencing prices, and some gaps in data, only an indicative assessment of the impact of the tariff changes is possible. The tariff reduction for rice of 5 percent (together with the adjustment in the reference price) would have caused consumer prices to be about six percent lower than without the tariff reduction (Box 5). Even if import tariffs had been removed in their entirety, with all the budget implications that this would have had, this would still have had a relatively small impact on consumer prices compared with the influence of the international rice market. Further, the reduction might have been less if the tariff changes were not fully carried through to the consumer.44 Several other simplifications in the simulation of impacts made above would also tend to reduce the effects on welfare of both the price surge and the reductions in tariffs.45 3.19 The impact of the rice price increases on consumer welfare and the poor. In the absence of survey data on consumption, a rough simulation of a household food consumption budget provides an indicative assessment of the effect of a doubling of rice prices on the cost 44 As in Djibouti where very little of a tariff reduction was reflected in lower consumer food prices (IEG 2012). 45 This is not the full impact of the tariff reductions. While changes in the price of rice would have had greatest impact on the cost of a family‘s food budget, the aggregate impact of the tariff reductions would be marginally larger if wheat is included, as well as the effect on transport costs of the reduced petroleum tariff. 38 Box 5. Indicative Impact of the Rice Tariff Reduction on Consumer Prices An indicative calculation of the impact on the domestic price of rice of the reduction in the rice import tariff from 15 percent to 10 percent, coupled with a change in the reference price for rice is as below:  The average international price of rice (current terms) before the food crisis was about $300/metric ton (MT) (average of prices of Thailand A1 special, and 5% and 25% broken).  Before the food price crisis a tariff of 15 percent and a reference price of $260/MT was used. (The ―reference price‖ is the price set by Government for the purpose of levying the tariff.) .  These values increase the import price of rice by $39/MT leading to a price of $339/MT as follows: $300 plus 0.15 X 260 = $339/MT.  In 2008, international rice prices doubled, resulting in the international price of rice increasing from $300/MT to $600/MT.  Assume first that there are no changes in the way that Government estimates the overall levy on rice imports. Hence, the tariff of 15% remains, and the ratio of the reference price to the import price (260/300 = 0.87) also remains the same. This ratio results in a new reference price of $522/MT (0.87 X 600). Hence the overall levy is 0.15 X $522 = $ 78/MT, and the price of rice imports after the levy becomes $600 + 78 = $678/MT.  Taking now the actual changes, the tariff is reduced from 15% to 10% and the reference price is adjusted to only $375 (well short of the 0.87 ratio applied before the crisis). The import price of rice becomes $600 + 0.1 X 375 = $638/MT.  Thus, the import price of rice after levies has decreased from $678/MT to $638/MT; a reduction of about $40/MT or 6%.  By comparison, the import price will have gone from $339/MT in 2007 to $638/MT, an increase of 88%. Source: IEG estimation of food consumption.46 This is based on an IEG focus group discussion with a group of urban women, the simulation being for a ―modest income‖ urban household (with sufficient income to afford what was considered a reasonably balanced diet).47 3.20 A typical food basket for a family of 2 adults and 2 children would be about Leone 20,000 per day (about $4.6 for the family or a little under $1.00 per person) at early 2011 prices (Annex B, Table B.6). In this diet, family rice consumption would represent Leone 3,000 ($0.70) per day. This is about 15 percent of the family‘s total food expenditures. With a doubling of the market price for rice, the daily consumption cost of rice would rise to Leone 6,000, an increase in the family‘s food budget of 15 percent (by about 7.5 percent in the case of a 50 percent rice price increase). The average share of food costs in overall household expenditures in Sierra Leone is just over 50 percent. Hence, if rice prices doubled, the overall cost of living for this household can be expected to have gone up by about 7.5 percent. The impact from the tariff reduction would have been much smaller – about a one percentage point reduction in the increase in the cost of living. Thus, the reduction in the 46 The field work for a national household survey with one section specifically on consumption, expenditures and incomes was completed in 2011, but analyzed results are still to come. 47 From a focus group comprising one woman (professional, mother) as main interlocutor in an interactive discussion with other participating women. The group was asked to provide a possible food budget for a typical modest to low income urban family. 39 tariff would have had little discernible impact for such a family. In actuality the impact is likely to be smaller because the simulation assumes that the tariff reduction is fully passed through to consumer prices48 and that consumers do not alter their diets by substituting other food staples in place of rice.49 3.21 The effect of other government actions on food prices. In addition to the reduction in tariffs, Government took a number of other measures to temper the rise in rice prices. An understanding was reached between the Government and Sierra Leone‘s three major importers of rice to limit the extent and pace of their rice price increases; a consortium of local wholesalers was established for dialogue between Government and rice traders; traders in Freetown and the districts were monitored to counter any hoarding and local monopolies; selective food distributions at subsidized prices were made to vulnerable groups; rice exports were banned and border posts with Guinea and Liberia were charged with enforcing these restrictions. 3.22 Most of Government‘s actions were short-term in nature and would be unlikely to stand-up to market forces in the longer term. Nevertheless, for the approximately 12 month period of high international rice prices these actions are likely to have had some impact on rising rice prices. Further, most did not require substantial Government expenditure. This begs the question as to whether actions such as these might be more influential on prices, and more cost effective to implement, than tariff reductions. This question is reinforced by the generally lower increases in the domestic prices of rice compared with international prices. International prices more than doubled, whereas, according to the sporadic data available, the increases in domestic prices were typically below 100 percent for urban consumers, nearer to half that in some estimations, and lower than 50 percent outside the urban and peri-urban areas. If the gap between domestic and international prices is indeed substantial, it would be hard to explain the lower domestic prices without ascribing some influence from the non- tariff actions above. 3.23 To summarize, the budget transfer helped enable Government to maintain its tariff reductions at a time when the fiscal balance was particularly tight. The direct impact of the tariff reductions was to mitigate the increase in rice prices by about 6 percent (that is, consumer prices would have been about 6 percent higher without the tariff reductions). This was a small impact relative to the estimated 50 to 100 percent increases in consumer rice prices in the urban and peri-urban areas. The grant‘s significance is even less when considered within the context of Government‘s overall action program to address the food 48 International experience suggests that full pass-through is not necessarily the case, such as in the PPAR for the Djibouti Food Crisis Response Development Policy Grant (IEG 2012). 49 The simulation contains a number of simplifications for estimation purposes, in particular: (i) Complete pass- through to consumers of international price changes, whereas this could be more or less than the percentage change in border prices depending on trader behavior, collusion, other Government actions and other factors; and (ii) the price change is assumed not to alter the amount of rice that a household eats. For the ―modest income‖ household simulated, this may be substantial ly correct given the cultural significance of rice in Sierra Leone diets. But destitute households would likely reduce rice consumption and substitute cheaper food staples, such as cassava; and (iii) the price impacts are calculated only for rice whereas tariffs were reduced for several other food staples (though these commodities are minor cost elements in typical Sierra Leone diets. 40 price crisis, and the decision to reduce already modest food tariffs increased the fiscal deficit, making it necessary to intervene to protect ongoing feeding programs for the vulnerable. The Grant‘s Efficacy mitigating the impact of the food price increases on consumer welfare is therefore assessed as Modest. CONTINUED DELIVERY OF BASIC SERVICES FOR VULNERABLE GROUPS 3.24 An Action Plan acceptable to the World Bank to protect the provision of basic services to vulnerable groups was a prior action for the operation and was contained in the Letter of Development Policy sent to the Bank on July 25, 2008. 3.25 The services to be supported were for continuing institutional feeding programs, either as part of health services or for schools. Funds were needed to absorb the substantial rise in the cost of meals due to the food price crisis. Thus, the need was to provide short-run (defined as the five month period from August to December 2008) financial support to the government for continuing these existing services. The Letter of Development Policy had laid out the numbers of beneficiaries who would continue to benefit from its feeding program, and according to government monitoring reports, these targets were achieved. About 3,500 children in boarding schools or children who were handicapped continued to be fed as against a target of 3,000 children; and 380 children in approved schools or remand homes were fed compared with a targeted 500 (reduced as the number of children in homes declined). On the health facilities side, 8,200 lactating mothers and children under five were fed compared with the targeted 5,300; and 12,734 patients in district hospitals or community health centers were fed, nearly at the targeted 12,800 patients. All of these beneficiaries were from vulnerable groups 3.26 Monitoring of the program was overseen by the Ministry of Finance and Economic Development, with responsibility for data collection under the ministries of education and health for their respective sectors. Data were provided directly to these ministries by the institutions benefitting from the program. A draft implementation report was prepared by the Ministry of Finance and Economic Development in June 2009, and a final report was issued in December 2009. Data from the district hospitals and remand homes were provided late, but the ICR comments that by the ministry‘s final report the information had been received (with the exception of the information from one hospital). 3.27 While monitoring of the number of beneficiaries appears to have been thorough, no information is available on the quality of the government feeding program or the number of calories provided per beneficiary. There is some empirical information on feeding programs financed by development agencies, but these are not necessarily equivalent. Donor funded programs may provide the concerned institutions with more technical support, and the nutritional quality of the meals may be different. Further, the Government program is all for residential feeding whereas the programs financed by the development community tend to be for non-residential beneficiaries. This makes some yardsticks of quality only marginally relevant. Nevertheless, accepting these caveats, the benefits stemming from the donor funded feeding programs may at least partially apply for the government program. Also indicative is the sustained interest and involvement in feeding programs of a number of development 41 agencies and NGOs. Such commitment would not be there if the feeding programs were not providing benefits. 3.28 The feeding program at health facilities was maintained, facilitated by the increased budget enabled by the Food Crisis Grant. Breastfeeding mothers and children under 5 years old are particularly in need of adequate nutrition, and in the absence of the program, patients in district hospitals and community centers typically received minimal or no food, relying on family or the good will of others; even from the doctors‘ and nurses‘ own salaries in the case of one hospital.50 3.29 In summary, while empirical data are limited, there is evidence that the services were continued, in most cases at 2007 levels or higher, though it is not known how much was actually provided and whether it was sufficient for adequate nutrition. However, assuming that the counterfactual would have been no food or reduced food provided by the institutions (because of inadequate government budget), the continuation of the feeding programs likely prevented a reduction in welfare among the beneficiaries. The Food Crisis Grant‘s Efficacy contributing to continued basic service delivery for vulnerable groups during the last five months of 2008 was Substantial. Ratings OUTCOME 3.30 The Food Crisis Grant‘s two objectives – mitigation of the impact of the increases in food prices on consumer welfare and continued delivery of social services to the vulnerable – were substantially relevant to Sierra Leone‘s efforts to mitigate a rapidly developing food crisis. The extreme poverty of a large proportion of the population left them particularly vulnerable to the domestic price increases‘ effects on their costs of living and family welfare. Government tried to mitigate the price increases by lowering import tariffs. However, given the low initial level of the food tariffs, their small reduction, and the very high increase in food prices, lowering the tariffs was not likely to have an appreciable impact on the food price increase or consumer welfare. The operation‘s design relevance was therefore modest. Maintaining social services was relevant, but threatened by a critical shortage of government funds due to reduced revenues. In helping to maintain a part of the social safety net, the grant was valuable. The Grant‘s efficacy in mitigating the short-term impact of food price increases on consumer welfare was modest, and its impact on consumer prices is likely to have been significantly smaller than the impact of the other short-term actions taken by the government to counter the spike in rice prices. Overall, the Food Crisis Grant‘s Outcome was Moderately Satisfactory. RISK TO DEVELOPMENT OUTCOME 3.31 The Food Crisis Grant‘s development outcome is, from a project perspective, already over. The Grant was intended as a short-term response to the food crisis, not as an investment. In the broader perspective, however, food security risks remain. The chances of 50 As reported for the Sierra Leone Psychiatric Hospital by the Concord Times newspaper on April 23, 2008. 42 price shocks in the international rice market do not appear any different now than before the food crisis and could become more so if large countries such as China need periodic substantial importations of rice to compensate for a poor harvest. A major effort has been made by Government to increase food production (cereals, cassava, and other staples), especially of rice,51 and several Bank projects are also contributing to agricultural development.52 This has had some success. The three year average of milled rice production increased by 70 percent over the six years from 2005-2007 to 2010-2012. But consumption and imports of rice also increased over the same period.53 As a result, while Sierra Leone‘s reliance on imports has fallen slightly (from 23 percent of consumption in 2005-2007 to 17 percent of consumption in 2010-2012), the gap still leaves the country vulnerable to international price fluctuations, especially as concerns the population in urban areas. Thus, both in terms of the possibility of a surge in international rice prices, and the likely impact from such an event, risks from food price shocks remain. Repeat operations of similar design would not be a sustainable solution. The Grant‘s Risk to Development Outcome is, therefore, rated Significant. Nevertheless, a more strategic approach to food security is developing, which could lower the risk environment. A food security ―framework‖ was issued by Government in 2009 with assistance from USAID, and a ―policy implementation plan‖ was issued in 2012.54 These lay out a number of the elements for an overall strategy, and may form the basis for more comprehensive food security actions over time.55 BANK PERFORMANCE Quality at Entry 3.32 The Bank responded rapidly (13 days between Government‘s Letter of Development Policy and Board Presentation) to the food price crisis. Bank staff worked well with Government, supporting its short-term response to the crisis. The understanding reached with 51 Among the measures taken have been campaigns to improve agronomic practices for main food staples such as rice and cassava, introduction of a higher yielding, shorter duration rice variety (Nerica), promotion of larger private sector farms, using higher technologies tha n smallholders, and investment in irrigation. The ―National Sustainable Agriculture Development Plan, 2010-2030‖ (September 2009) provides a schematic overview of some of these actions. 52 Notably, the Rural Development and Private Sector Development Project (FY07), financing agricultural value chains, and the Infrastructure Development Project (FY06) which includes rural roads. 53 All data are from the US Department of Agriculture as of February 2013, expressed as milled rice. The three year average for rice production 1n 2005-2007 was 410,000 metric tons, and in 2010 -2012 was 696,000 MT. Imports averaged 122,000 MT in 2005-2007 and 141,000 MT in 2010-2012. 54 ‗Sierra Leone Food Security Country Framework FY 2010 -2014‖, USAID, 2009; and ―Sierra Leone Food and Nutrition Security Policy Implementation Plan, 2012-2016. 55 The framework document provides a comprehensive overview of food security issues and possible actions, but its purpose was not to detail an actual plan. The food and nutrition security policy implementation plan is in considerably more detail. It is particularly detailed in reviewing social issues such as nutrition, maternal and infant health, hygiene and common diseases, and it also overviews areas such as agriculture and disaster preparedness. These areas, especially agriculture and short-term response, would have merited further detailing; and other matters such as an ordering of priorities, capacity development, and a roll-out plan for the main actions, might also have been included or given greater prominence. 43 Government regarding maintenance of social services was practical, and fitted within Government‘s crisis response actions. M&E design covered tracking of outputs such as numbers of beneficiaries served but lacked plans to assess the quality of the feeding programs. But the principal shortcoming of the operation‘s quality at entry was the lack of assessment of the low probable impact of the tariff reductions on consumer welfare. The speed of processing did not allow deliberate assessment of policy options provided in the GFRP Framework Document. This qualifies an otherwise strong performance of the Bank, leading to a Moderately Satisfactory Quality at Entry overall. Quality of Supervision 3.33 Supervision was part of the broader policy dialogue with Government which was sufficient for a short-term grant related to ongoing social services that had no particular need for technical assistance. As concerns the feeding program‘s progress, this was monitored closely and delays in M&E reporting appropriately noted and discussed with Government, resulting in complete data on project outputs by the end of the project. There were two Implementation Status Reports, with issues (primarily the M&E delays) proactively followed up with Government. Supervision Performance was Satisfactory. 3.34 Taking account of both the quality at entry and of supervision, the Bank‘s Overall Performance was Moderately Satisfactory. BORROWER PERFORMANCE Government Performance 3.35 Government was strongly committed to the operation and was clear on what it wanted and the actions that it would take as part of its overall policy. This was well conveyed in its Letter of Development Policy. The High-level Inter-ministerial Task Force ensured that social actions were implemented and that government funds went in timely manner to the appropriate ministries. Despite the numerous development agencies involved with mitigating the food crisis, including several other budget support operations, Government effectively ensured that there was no duplication of activities between the agencies. Preparation for the Grant was fast, and relations with the Bank were seamless. Government Performance was Satisfactory. IMPLEMENTING AGENCY PERFORMANCE 3.36 The Ministry of Finance and Economic Development effectively headed the Inter- ministerial Task Force, coordinated the ministries involved, and was the primary government actor preparing the project and partnering the Bank during implementation. It assiduously followed project progress and was instrumental in getting project monitoring up to date. The intention at appraisal was to have an inter-ministerial technical committee headed by the Secretary of Finance and Economic Development, to monitor the Food Crisis Grant, but there is no indication that it was ever established. The two main agencies implementing the feeding program were the Ministry of Health and Sanitation and the Ministry of Education, Science and Technology. The feeding programs reached the numbers of beneficiaries 44 intended. Both Ministries were slow in reporting on progress, though this was primarily due to their need to rely on the promptness with which the various hospitals and schools provided data. But taken as a whole, the performance of the Implementing Agencies was Satisfactory. Taking account of both the performance of Government and the performance of the implementing agencies, the Borrower‘s overall Performance was Satisfactory. MONITORING AND EVALUATION 3.37 M&E Design: Monitoring was to be under a technical committee formed from the ministries involved which would overview collection of data by each line agency. Data would be provided to the committee by each agency on a monthly basis. The line agencies were to collect the data. Data to be collected were primarily outputs such as the number of persons benefitting from each project activity, rather than outcome related indicators such as the quality of the feeding programs. 3.38 M&E Implementation: Monitoring was enough to measure output achievements such as the number of schools assisted. More outcome oriented aspects – including information on the amount of food and nutritional status - would have been desirable. The frequency of data reporting varied between agencies, some being very slow. Reporting during the project was sporadic. 3.39 M&E Utilization: Despite the weaknesses above, data for most social activities appears to have been produced (though irregularly), and complete and up-to-date data were available at project closure. However, reflecting the original design, data were restricted to outputs, limiting the degree to which the quality and impact of the feeding program on health and education could be empirically assessed. 3.40 The quality of M&E overall was Modest. 4. Lessons 4.1 Attempts to lower prices through reducing tariffs on imported foodstuffs have direct consequences on government revenues yet may have only marginal impact on consumer prices relative to the much larger impacts off international price changes. The impact of the tariff reductions on rice prices (about six percent) was small, and insignificant relative to the doubling of food prices influenced by international market forces. The impact on households‘ cost of living was even smaller. Meanwhile, Government had taken an array of separate actions to buffer the increase in rice prices, which are likely to have had greater short-term impact and at lower cost than the tariff reductions. This raises the question as to whether the operation was an efficient way of mitigating the price increases. 4.2 The effectiveness of a cash-for-work program can be compromised in its effort to target benefits to particular social groups in the absence of: (i) clear criteria and processes that ensure objective selection of communities and individual beneficiaries; (ii) communities as the primary deciders on the type of works to be done and selection of the participating individuals; and (iii) greater women’s participation and differentiation of activities better suited for women. The Social Action Project‘s cash-for- 45 work program primarily reached its target groups, especially for youths. But selection processes for the individual participants were not uniformly applied, allowing situations where vested interests influenced selection of the individuals. Communities felt that their participation could have assured more objectively based selection of the beneficiaries, and choice of investments responding better to the communities‘ priorities. Women‘s participation would have been higher if work activities had been differentiated between more strength-based activities for men and less physical tasks for women. 4.3 High community participation helped the Social Action Project’s communities to “own� their sub-projects and foster interest in maintaining their investment and operating the service. The project‘s community sub-projects were mostly effective in establishing community institutions, infrastructure and services, and there are some indications that they might be sustained beyond the short-term. An IEG review of the Bank‘s experience with social funds found that a common experience with such funds was that community infrastructure was not adequately maintained and that the facilities and services were relatively short-lived. Whereas a common approach for social funds is to place the infrastructure as first consideration, with participatory processes often in a less dominant role, the Social Action Project placed communities and community processes at the fore. Decisions, implementation and operations were the community‘s responsibility, and necessarily required a high degree of interaction and cooperation which helped create collective ―ownership‖ of the asset and service. The luxury of choosing a highly participatory approach may not be there for social funds responding to urgent disaster recovery, but where possible, participatory approaches may have better chances of sustainability. 4.4 Add-on emergency components to existing projects, which experience has shown often encounter implementation difficulties, have greater chances of success with an established institutional base. The addition of the cash-for-work program to the Social Action Project benefitted from an experienced field-oriented agency (NaCSA) operating in all the country‘s districts. It had also done a small pilot for cash-for-work giving it at least a minimal base for scale-up. 4.5 In a food emergency operation there may be a trade-off between a rapid, larger- scale response at the cost of some reduction in quality, compared with a better prepared operation that is slower in mitigating the emergency. The cash-for-work program was prepared quickly, when rice prices were still high, and was implemented quickly. However, had the program been prepared in a more deliberated way, and implemented with less haste, it is likely that there would been fewer design and implementation shortcomings. The way forward 4.6 While these operations addressed the short-run impact of food price increases, and Government‘s 2009 food security framework and 2012 policy implementation plan provide many of the actions that would enhance food security further, there is no comprehensive longer-term food security plan showing the inter-linkages between the various actions and institutions involved. The design and value of the food crisis grant and the cash-for-work program helped Sierra Leone respond to the immediate needs of the food emergency. The Bank portfolio as a whole also potentially contributes to improved food security (such as the 46 projects related to health, education, youth employment, roads and marketing chains). But without a comprehensive plan to integrate such operations within an overall food security strategy, to consider gaps in emergency response, to reduce future vulnerability, and to think through institutional responsibilities and coordination, relevant needs and potential synergies may be missed. Such a plan would also benefit from a more integrated agricultural development program to draw the Government‘s ongoing actions to promote agricultural development into a more interlinked strategy. 47 References Andrews, Colin, Mirey Ovadiya, Christophe Ribes Ros, and Quentin Wodon. 2012. ―Cash for Work in Sierra Leone: A Case Study on the Design and Implementation of a Safety Net in Response to a Crisis.‖ Social Protection and Labor Discussion Paper, no. 1216. World Bank, Washington, DC. Backiny-Yetna, Prospere, and Quentin Wodon. 2012. ―Targeting Performance and Cost- Effectiveness of Labor-Intensive Public Works in Sierra Leone: Results from a Light Evaluation Survey.‖ World Bank, Washington, DC. Dalan Development Consultants. 2010a. ―Qualitative Assessment of Cash for Work Program for Sierra Leone.‖ Dalan Development Consultants. 2010b. ―National Social Action Project. Beneficiary Impact Assessment.‖ Conducted for the National Commission for Social Action. Government of Sierra Leone. 2007. ―Prospects for the Poor in the Face of National Agriculture Reforms: A Poverty and Social Impact Analysis of Changes to Agricultural Policy in Sierra Leone.‖ Economic Policy and Research Unit, Ministry of Finance and Economic Development, Freetown. May. _____. 2009a. ―An Agenda for Change – Second Poverty Reduction Strategy 2008- 2012.‖ Freetown. _____. 2009b. ―National Sustainable Agriculture Development Plan - 2010-2030.‖ Freetown. _____. 2009c. ―Economic Bulletin Financial Year 2008.‖ Ministry of Finance and Economic Development, Freetown. June. _____. 2010. ―Sierra Leone Food and Nutrition Security Policy Implementation Plan 2012-2016.‖ Freetown. Independent Evaluation Group. 2012. ―Project Performance Assessment Report for the Food Crisis Response Development Policy Grant, Republic of Djibouti.‖ World Bank, Washington, DC. International Monetary Fund. 2009 to 2011. Annual Country Reports, Sierra Leone. Mantymaa. A. 2009. ―Sierra Leone: School Meals Make a Difference,‖ World Food Program Web Update. July 3, 2009. Nordic Consulting Group. 2006. ―Beneficiary Impact Assessment and Technical Audit for the National Social Action Project.‖ Prepared for the National Commision for Social Action. Platt, S.A. 2009. ―The Republic of Sierra Leone. National Draft Policy for Social Protection.‖ Stanley, R., E. Brigges, S. Patel, and L. Wonneh. 2008. ―The Severe Impact of Rising Food Prices – A Situational Assessment of the Food Crisis in Sierra Leone.‖ Food and Agriculture Organization/World Food Program. June. 48 Unauthored. 2009. ―The Global Food Price Crisis and Household Hunger: a Review of Recent Food Security Assessments.‖ Humanitarian Exchange Magazine,.Issue 42, March. UNICEF. 2007. ―Therapeutic Feeding Improves Child Nutrition in Sierra Leone.‖ Bulletin. April. United Nations Integrated Peace-Building Office. 2009. ―Joint Vision for Sierra Leone.‖ May. USAID. 2009. ―Sierra Leone Food Security Country Framework FY2010-2014.‖ October. Wiggins. S,, J. Compton J., S. Keats, and M. Davies. 2010. ―Country Responses to the food price crisis 2007/08 – Case Studies from Bangladesh, Nicaragua and Sierra Leone.‖ Overseas Development Institute. World Bank. 1984. ―Sierra Leone Agricultural Sector Review.‖ World Bank, Washington, DC. March. _____. 2003a. ―Project Appraisal Document. Health Sector Reconstruction and Development Project.‖ World Bank, Washington, DC. January. _____. 2003b. ―Sierra Leone. National Social Action Project – Project Appraisal Document. ― Report No. 25105-SL. World Bank, Washington, DC. March. _____. 2003c. ―Project Agreement – National Social Action Project.‖ World Bank, Washington, DC. June. _____. 2003d. ―Community Reintegration and Rehabilitation Project. Implementation Completion Report.‖ World Bank, Washington, DC. December. _____. 2005a. ―Country Assistance Strategy for the Republic of Sierra Leone, FY2006- 2009. ― World Bank, Washington, DC. May. _____.2005b. ―Sierra Leone. Country Assistance Strategy FY2006-2009.‖ Report No. 31793-SL. World Bank, Washington, DC. May. _____. 2007. ―Project Appraisal Document. Sierra Leone Rural and Private Sector Development Project.‖ Report No. 39571-SL. World Bank, Washington, DC. _____. 2008a. ―Framework Document for Loans, Credits, and Grants in the Amount of US$ 1.2 Billion Equivalent for a Global Food Crisis Response Program.‖ Washington, DC. _____. 2008b. ―Sierra Leone. Program Document for the Food Crisis Response Development Policy Grant.‖ Report No. 44830-SL. World Bank, Washington, DC. July. _____. 2008c. ―Food Price Crisis Response Trust Fund. Project Paper‖ Co-financing of the National Social Action Project. World Bank, Washington, DC. July. _____. 2009a. ―Proposed Second Additional Grant – Co-financing of the National Social Action Project.‖ World Bank, Washington, DC (November 2). 49 _____. 2009b. ―Sierra Leone. Implementation Completion and Results Report for the Food Crisis Response Development Policy Grant.‖ Report No. ICR-1189. World Bank, Washington, DC. December. _____. 2009c. ―Emergency Project Paper for the Social Action Project‘s Second Additional Financing.‖ World Bank, Washington, DC. _____. 2010a. ―Pilot IDA Crisis Response Window - Emergency Project Paper.‖ Youth Employment Support Project. June 16, 2010 _____. 2010b. ―Emergency Project Paper. Sierra Leone. Youth Employment Support Project.‖ Report No. 54419-SL. World Bank, Washington, DC. June 16. _____. 2010c. ―Beneficiary Impact Assessment – Final Report.‖ World Bank, Washington, DC. _____. 2010d. ―Health Sector Reconstruction and Development Project. Implementation Completion and Results Report.‖ _____. 2010e. Sierra Leone Public Expenditure Review. World Bank, Washington, DC. _____. 2010. ―Implementation Completion and Results Report. Health Sector Reconstruction and Development Project.‖ World Bank, Washington, DC. May. _____. 2010g. ―Implementation Completion and Results Report. Rehabilitation of Basic Education Project.‖ World Bank, Washington, DC. June. _____. 2010h. World Development Report 2010. Washington, DC. _____.2010i. ―Youth Employment Support Project, Project Appraisal Document.‖ World Bank, Washington, DC. _____. 2011a. ―Sierra Leone. Implementation Completion and Results Report for the National Social Action Project.‖ World Bank, Washington, DC. Report No. ICR- 1560. June 28 _____. 2011b. World Development Report 2011. World Bank, Washington, DC. _____. 2012. ―Social Protection Assessment (draft).‖ World Bank, Washington, DC. World Bank and African Development Bank. 2010. ―Sierra Leone. Joint Country Assistance Strategy for FY10-FY13.‖ Report No. 52297-SL. Washington, DC, World Bank. March. 51 Annex A. Basic Data Sheet NATIONAL SOCIAL ACTION PROJECT (IDA – 37480, 3748A, TF-92671, 95661) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 42.0a 47.9b 124.4 b Loan amount 35.0 44.9 128.4 Cofinancing - 1.0 - Cancellation - - - a.Includes $3.5 million contribution each from the government and local communities. b.Includes $37.9 million from the original IDA credit and $7 million of additional financing from the Food Crisis Response Trust Fund (TF92671). Cumulative Estimated and Actual Disbursements FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 Appraisal 0.70 8.36 18.26 30.34 35.00 35.00 35.00 35.00 35.00 estimate (US$M) Actual (US$M) 0.0 2.69 11.83 22.18 31.05 37.59 41.33 44.09 44.94 Actual as % of 0 32.20 64.80 73.10 88.71 107.39 118.09 125.97 128.40 appraisal Date of final disbursement: June 2011 Project Dates Original Actual Initiating memorandum 09/10/2002 09/17/2002 Negotiations 02/10/2003 02/21/2003 Board approval 03/31/2003 02/24/2003 Signing 06/02/2003 Effectiveness 08/11/2003 08/11/2003 Closing date 12/31/2008 12/31/2010 52 Staff Inputs (staff weeks) Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ Thousands (including No. of staff weeks travel and consultant costs) Lending FY03 41 175.37 Supervision/ICR FY04 21 68.96 FY05 17 73.13 FY06 12 69.33 FY07 4 29.56 FY08 12 66.99 FY09 9 27.58 FY10 26 124.02 Total: 101 459.54 Task Team Members Responsibility/ Names Title Unit Specialty Lending Eileen Murray Senior Operations Officer TTL Julie Van Domelen Lead Social Protection Specialist Daniel Owen Senior Social Development Specialist Evelyn Awittor Operations Officer Jacob Saffa HD Specialist Fred Yankey Financial Management Specialist Mbuba Mbungu Senior Procurement Specialist Samantha de Silva Social Protection Specialist Gregoria Dawson-Amoah Team Assistant Franco Russo Language Program Assistant Edward Dwumfour Environment Specialist Charles Annor-Frimpong Agriculture Specialist Elizabeth Adu Chief Counsel, Africa Practice Group Lisa Lui Senior Counsel David Webber Lead Financial Management Specialist David Rudge Senior Engineer Bharat Devokta Consultant, Shelter Jim Edgerton Consultant, Institutional and M & E issues Johanne Angers Operations Officer Aissatou Chipakou Language Program Assistant Supervision/ICR 53 ANNEX A Responsibility/ Names Title Unit Specialty Eileen Murray Task Team Leader/Country Manager MNCTN TTL Giuseppe Zampaglione Country Manager LCCHN TTL Mirey Ovadiya Senior Social Protection Specialist HDNSP TTL John Van Dyck Senior Operations Officer AFTSP TTL Osman S. Ahmed Lead Economist MNC02 Colin Andrews Social Protection Specialist HDNSP Ferdinand Tsri Apronti Procurement Specialist AFTPC Noora Arfaa Consultant MNSED Kathryn Suzanne Bach Junior Professional Associate AFTHE Gregoria Dawson-Amoah Program Assistant AFCW1 Robert Wallace DeGraft- Financial Management Specialist AFTFM Hanson Edward Felix Dwumfour Senior Environmental Specialist AFTEN Paul Geli Consultant MNSHD Engilbert Gudmundsson Country Manager AFMSL Manush A. Hristov Senior Counsel LEGAF Kristine M. Ivarsdotter Senior Social Development Specialist LCSSO Eva Jarawan Sector Manager AFTHE Alie Barker Kargbo Human Development Specialist AFTH2 - HIS Mbuba Mbungu Consultant AFTFM Ruth Afandi Mulahi Senior Program Assistant AFREX Jonathan Nyamukapa Senior Financial Management Specialist AFTFM Pia Peeters Senior Social Development Specialist AFTCS Oluwole Pratt Financial Management Analyst AFTFM Jacob Jusu Saffa Human Development Specialist AFTH2 - HIS Frederick Yankey Senior Financial Management Specialist AFTFM Samantha de Silva Senior Social Protection Specialist SASED Josiane Luchmun Program Assistant AFTSP Other Project Data Borrower/Executing Agency: Follow-on Operations Operation Credit no. Amount Board date (US$ million) Youth Employment Support Project 4790-SL 20 2010 54 FOOD CRISIS RESPONSE DEVELOPMENT POLICY GRANT (TF-92619) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 3.0 3.0 100 Loan amount 3.0 3.0 100 Cofinancing - - - Cancellation - - - Cumulative Estimated and Actual Disbursements FY09 Appraisal 3.00 estimate (US$M) Actual (US$M) 3.00 Actual as % of 100.0 appraisal Date of last 08/27/2008 disbursement Project Dates Original Actual Negotiations - 07/24/2008 Board approval 08/04/2008 08/06/2008 Signing - 08/20/2008 Effectiveness 08/22/2008 08/20/2008 Closing date 06/30/2009 06/30/2009 55 ANNEX A Staff Inputs (staff weeks) Staff Time and Cost (Bank Budget Only) Stage USD Thousands (including No. of staff weeks travel and consultant costs) Lending and Supervision (FY09) 3.08 2.99 Task Team Members Responsibility/ Names Title Unit Specialty Lending Douglas Addison Senior Economist AFTP4 Team Leader Kathryn Suzanne Bach Junior Professional Associate AFTH2 Social Dev. Yusuf Bob Foday Economist AFTP4 Economics Anna Victoria Gyllerup Operations Officer AFTRL M&E Mirey Ovadiya Senior Operations Officer AFTH2 Social dev. Manush Hristov Senior Council LEGAF Legal John Nyaga Sr Financial Management Specialist AFTFM Fiduciary Lead Financial Management Trichur Balakrishnan AFTFM Fiduciary Specialist F. Tsri Apronti Procurement Specialist AFTPC Procurement Tidiane Toure Lead Procurement Specialist AFTPC Procurement Rajiv Sondhi Senior Finance Officer CTRFC Loan Accounting Suzanne Morris Senior Finance Officer CTRFC Loan Accounting Supervision Douglas Addison Senior Economist AFTP4 Team Leader Yusuf Bob Foday Economist AFTP4 Economics Kathryn Suzanne Bach Junior Professional Associate AFTH2 Social Development Mirey Ovadiya Senior Operations Officer AFTH2 Social Development 56 Annex B. Additional Tables and Figures Table B.1: Proportions of Budget Allocations by Main Expenditures (2005-2010) (percent of Total Budget) 2010 Sector 2005 2006 2007 2008 2009 (projected) Education 18.8 19.6 18.4 18.9 18.3 17.3 Health 10.6 9.8 10.4 7.9 6.7 7.2 Agriculture 4.2 5.8 7.0 5.2 5.8 5.5 Infrastructure, (roads, energy, 21.8 18.1 21.6 26.8 26.8 24.8 water) General and admin. 18.2 19.9 17.0 18.6 18.1 16.9 Other expenditures 26.2 26.6 25.4 22.4 24.1 27.8 Total Expenditures 100 100 100 100 100 100 Source: World Bank, October 2010; Sierra Leone Public Expenditure Review Table B.2: Changes in modes of access and staffing for Social Action Project rehabilitated clinics Survey Measure Before clinic After Clinic Percent of persons visiting clinics that travelled by public 33 2 transport (n=381 before, n=408 after) Percent of persons visiting clinic who walked to the clinic 67 98 by public transport (n=381 before, n=408 after) Number of staff provided by the Ministry of Health (n= 43 48 92 clinics) Source: BIA 2010 (Dalan consultants) and NacSA. Table B.3: Indications of Community Satisfaction with Court Barrays and Markets Percent of Number of Survey Measure households households Households ―satisfied‖ with Court Barray or Community Center 92 320a Households considering that their court barray had become since 260a 85 construction the main forum for settling disputes 96 101 Households expressing satisfaction with their market center Households who visited the market ―often‖ 94 94 Percent of those using the market who were women 92 - Source: NSAP Beneficiary Impact Assessment, 2010 (Dalan consultants) Note: a. 101 households were interviewed for the market centers. The overall sample size for both court barays and market centers is not recorded in the survey, but would, from the survey design for the number of persons interviewed per community, be a maximum of 320. 57 ANNEX B Table B.4: How Communities Selected the Type of Sub-project they wanted Percentage of Respondents HOW DECISIONS WERE MADE SELECTION OF SELECTION OF PRIMARY SCHOOL HEALTH CENTER Attended Community Meeting Called by NaCSA 60% 57% Attended Meeting Called by Elders 24% 25% NaCSA meets with chiefs and elders 6% 8% Not involved/other 10% 10% Source: NSAP Beneficiary Assessment, 2010 page 35. Table B.5: Local Prices of Selected Food Commodities (In Leones) Price increase from Commodity Unit 2007 2008 2009 2010 2007 to 2009 (%) Cassava Kg 943 798 884 813 -6 Dried Bonga Kg 6298 6894 8348 9494 33 Sweet Potatoes Kg 3182 3566 3885 5835 18 Palm Oil Pint (0.33 ltrs.) 6928 6894 8348 9494 20 Milk Tin (0.33Kg) 1917 2363 2665 2850 39 Eggs 1 dozen 6125 6104 6151 7708 26 Source: EPRU, Ministry of Finance Table B.6: Calculation of a Daily Household Food Budget for a Moderate Income Family Measurement Price per Unit Number of Total Expenditure per Food Item Unit (Leone) Units per day day (Leone) Rice Cup (0.2 Kg) 1000 3 3000 Vegetables (leafy & (pile) 1000 3 3000 miscellaneous) Oil pint 1800 1 1800 Salt unit 500 500 500 Dried Bonga (fish) pile 5000 1 5000 Beans unit 1500 1 1500 Peanuts cup 2000 1 2000 Charcoal pile 1000 2 2000 Magi lump 500 2 1000 Total Daily Expenditure 19,800 Source: Based on an IEG mission interview with a woman (professional, mother) with several other persons participating. The woman was asked to provide a representative daily food budget for a moderate income family of 5 (2 adults, 3 children). 58 Figure B.1: International Prices of Wheat, Maize, and Rice (2005-2010, Current Prices) Source: DECPG –GEM Commodities Data (as of March 16, 2012) (www.worldbank.org/ddp) 59 Annex C. Design and limitations of survey data used in the assessment of the National Social Action Project 2010 Beneficiary Impact Assessment56 Design. The 2010 assessment was primarily of the community-driven sub-projects under the Social Action Project‘s original design. All 13 of Sierra Leone‘s districts other than Freetown were sampled (the sampling was not linked to the 2006 impact assessment). A sample was taken of 90 sub-projects, out of the Social Action Project‘s total of 600 sub-projects, using a two-stage cluster sampling method. First, the 600 sub-projects were stratified by district, and a sample of sub-projects (about 7 per district) was then selected randomly using a probability proportional to size method to provide a self-weighted mix of sub-project types. For the selected sub-projects, a random sample of 10 households was taken from the population of households and two adults ( husband and wife as first preference) were chosen from each household. Through this method, 1683 respondents were selected, a 6% shortfall against the theoretical number of respondents (90 X 10 X 2 or 1800 respondents) due to attrition and other factors. The survey was supplemented by a focus group discussion in each community with either an all-female group or a male group. A third source of information was through in-depth interviews with other stakeholders such as NGOs, and a larger number of local councils (25), sub-project management committees (27) and NaCSA field and national level officials. Quantitative data analysis used Statistical Packages for Social Scientists. Secondary sources of information (reports) were also used to inform the report. For each survey process (individual interviews, focus groups, interviews of other stakeholders) observation checklists were provided to provide background guidance of methodology and question areas, and the household interviews were based on structured questionnaire forms. The broad assessment areas that the survey attempted to cover (Appendix 2 of the survey) included: effectiveness of maintenance of the sub-projects, degree of social inclusion in community decisions and actions, social capital, capacity building and organizational development, degree of community participation in sub-project selection, design and implementation, measures of the degree to which community sub-projects achieved intended impacts (e.g. school enrollment, utilization of health facilities, views on whether the sub- project had been beneficial/successful), community ―ownership‖ of sub-projects, NaCSA interrelations with the communities and the contributions of Local Councils and other secondary stakeholders. Limitations. The survey is a useful source of information, but had a number of limitations and potential biases which need circumspect interpretation of varying degree, depending on the specific question area, how questions were framed, and the sample sizes where data are disaggregated. Attribution is the most difficult issue. There was no non-project sample; analysis was longitudinal, with no comparison group outside the project. This then raises the 56 Sources: National Social Action Project Beneficiary Impact Assessment. Final Report, August 2010, Dalan Development Consultants; and Beneficiary Impact Assessment, 2006. Nordic Consulting Group, Denmark and Business-Engineering-Science and Technology consultants, Freetown. (nb: the cover of the 2006 impact assessment report erroneously dates the report as 2005.) ANNEX C 60 question as to how much of a change can be attributed to the project itself and how much due to the influence of other factors such as general development or related programs. For instance, enrollment and test scores increased for project primary schools, but were there also other influences such as government programs to increase the quality of teachers and to provide books? Schools outside the project may also have had improved results. The health and education sector projects that ran concurrently with the project are thus discussed in this PPAR for these reasons. There is also the difficulty that the survey sample had no precursor survey (the 2006 Beneficiary Impact Assessment was a different sample). Thus, for some questions, requiring retrospective responses from interviewees of situations ―before the project‖ compared with current situations. Secondly, the survey had a number of leading questions (for example, ―How have you benefitted‖) that might bias the responses. As concerns sample size, this can vary by type of infrastructure and other matters, and may be too small for some measures. Sample size has thus been reported in the PPAR‘s tables and text, and has also influenced the selection of data to be considered. Finally, the survey relies to a substantial extent on qualitative rather than quantitative information making comparisons and evaluation of impact more difficult. 2006 Beneficiary Impact Assessment Design. The 2006 BIA was commissioned by NaCSA with Social Action Project funding and carried out by a team of consultants from the Nordic Consulting Group, Denmark, and Business-Engineering Science and Technology consultants, Freetown. Information was obtained through (i) a sample survey of individual beneficiaries; (ii) beneficiary focus groups; and (iii) interviews with about 50 involved stakeholders (nurses, teachers, NGOs, District Councils, NaCSA, and staff from the ministries of health, education and other involved agencies). The main activity was the household sample survey. A sample of 17 sub- projects from nine Districts was taken by the consultants from a list provided by NaCSA of the first 100 sub-projects completed as of February 2006. The projects were stratified by district and main types of subprojects (schools, clinics, community centers, etc.), and by gender a category for youths, and thereafter was random from these strata. The overall sample size was 630 individuals. The focus group exercise was based on five focus groups for each sub-project community of which four would be ordinary community members and one would be of the project management committee. The total number of focus groups was 96. Additionally, 3 communities without a Social Action Project were selected. Field interviews were pre-tested in February 2006, the bulk of fieldwork undertaken in March 2006, and the report issued in May 2006. Limitations. The 2006 Beneficiary Impact Assessment shares most of the survey limitations of the 2010 survey, including the attribution difficulty and the need to partly rely on qualitative information. Nevertheless, the sample selection process is more completely described, adding some credibility in thus respect. As with the 2010 survey, where possible the PPAR has triangulated from the different sources of data (household survey, focus groups and interviews with stakeholders), plus IEG mission field observations and interviews, to better assess findings. Due to its smaller sample size and use of substantial stratification, the 2006 BIA may be less reliable than the 2010 survey and findings, especially where there is little possibility of triangulating from other sources, and the results risk misleading. 61 Qualitative Assessment of the Cash-for-Work Program57 Design. Primary data were from a stratified sampling process. In each district two cash-for- work communities were selected; one from the District town and one from a locality outside the town (16 rather than 14 communities were in the event selected as in one district four sites were chosen; no reason is provided). Selection of the communities was done randomly. From the 16 sample communities, two focus groups were formed, one of primary beneficiaries (actual participants in the program) and one of secondary beneficiaries from the locality but not directly involved. Thus, there were 32 focus groups in the sample, representing nine percent of the total number of completed sites. Focus group discussions followed a structured guide to enable comparison of findings. Field work was done in February 2010. In addition, the consultants interviewed other stakeholders such as NaCSA, the Local Council, chiefs, women‘s leaders and youth leaders. Written material as available was also used. Limitations. The main limitation is the qualitative nature of the survey. Assessment findings are presented in a discussion form organized by topic (such as wages given, implementation issues, performance of contractors, and other matters). The bulk of the discussion is a verbal, qualitative discussion based on the consultants‘ interpretation of these various information sources. The discussion is considered and appears balanced, providing valuable insight and perspectives, but there is little in terms of quantitative data to back this up. The Cash-for-Work part of the Beneficiary Impact Assessment Design. The survey was also of the first phase of the cash-for-work program, covering the seven districts involved. Sample size was 46 cash-for-work communities (out of an initial target of 52). Sampling methodology is not entirely clear but appears to have been that the sample was first stratified by District with communities then chosen randomly. The 46 sites represent 13 percent of the total number of sites (341) completed at the time of the survey. For each community a focus group was formed comprising ―mixed groups of direct and secondary beneficiaries.‖ (BIA, page 73). Limitations. Somewhat similar to the Qualitative Assessment in lacking empirical content. Nevertheless, a broad discussion based on the consultants‘ findings is provided, and the variability of experiences of the communities is illustrated with cited examples and candid and more detailed commentary where particular issues, such as the performance of contractors, were encountered 57 Sources for the discussion of the Cash-for-Work program: Beneficiary Impact Assessment, 2010, and Qualitative Assessment of Cash for Work Program for Sierra Leone, August 2010. Dalan Development Consultants. 62 Annex D. List of Persons Met Government Alimamy Bangura, Director, Economic Policy and Research Unit, Ministry of Finance and Economic Development Peter S. Bangura, Statistics Sierra Leone Mathew Dingie, Director of Budget, Ministry of Finance and Economic Development Lansana Fofanah, Economist, Economic Policy and Research Unit, Ministry of Finance and Economic Development Sallaymatu Fofana, District Coordinator West, Mohamed K. Koroma, Agricultural Statistician, Statistic Sierra Leone John Ngebeh, Director, National Commission for Social Action Saiduh Conton Sesay, Commissionner, National Commission for Social Action Imran Shahryar, Economist, EPRU, Ministry of Finance and Economic Development Francis B. Tommy, Acting Statistician General, National Strategy for the Development of Statistics, Statistics Sierra Leone Soriba Yansaneh, Planning, Monitoring and Evaluation Officer, National Commission for Social Action Civil Society Ibrahim Jalloh, Chairman, Bombali Youth Program Parents Association and school maintenance volunteers, Our Lady of Kwame Primary School, Western Area District. Headmistress and teachers, Our Lady of Kwame Primary School, Western Area District. Chief, Fogbo Area, Western Region. Women‘s group (informal, on household food expenditures), Freetown Nurses and volunteer assistants, Maternal and Child Health Clinic, Western Region Mothers and pre-natal women, Fogbo maternal and Child Health Post. Medical staff and volunteers, Robot Maternal Child health Post, Bombali District, Northern Region. Executive Committee, Robot area villages, Robot Maternal Child Health Post, Northern Region. Donors Yero H. Balde, Resident Representative, African Development Bank David Chalila, Head, Logistics Unit, United Nations World Food Program M. Fatmata, World Food Program Francis Y. Kumah, Resident Representative, International Monetary Fund Peter Mostert, Consultant, Natural Resources Management, German Aid David Mwesigwa, Program Implementation Officer, FAO Mathew Sandy, Economist, International Monetary Fund Resident Mission Eva Taylor, Sierra Leone Representative, African Development Bank Miyuki Yamashita, Program Officer, United Nations World Food Program World Bank Douglas Addison, Task Team Leader, Food Crisis Grant 63 Bidemi Carrol, Human Development Specialist, Freetown Resident Mission Sergiy Kulyk, Program Coordinator, Sierra Leone Mirey Ovadiya, Task Team Leader Daniel Owen, Senior Social Development Specialist Vijay Pillai, Country Manager, Sierra Leone Cyrus Talati, Senior Economist John Van Dyck, Task Team Leader, National Social Action Program