ICRR 14419 Report Number : ICRR14419 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/27/2014 Country : Togo Project ID : P110943 Appraisal Actual Project Name : Community US$M ): Project Costs (US$M): 17.8 33.0 Development Project L/C Number : CH409 Loan/ US$M ): Loan /Credit (US$M): 17.2 32.9 Sector Board : Social Protection US$M): Cofinancing (US$M ): 0.6 0.6 Cofinanciers : Communities Board Approval Date : 06/26/2008 Closing Date : 06/30/2012 09/30/2013 Sector (s): Primary education (30%); General agriculture fishing and forestry sector (25%); Health (18%); Other social services (17%); General public administration sector (10%) Theme (s): Participation and civic engagement (33% - P); Health system performance (17% - S); Education for all (17% - S); Other social development (17% - S); Rural services and infrastructure (16% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Hjalte S. A. Sederlof Denise A. Vaillancourt Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: The Project Development Objective (PDO) as set out in the July 16, 2008 Financing Agreement (p. 5) and in the June 3, 2008 Emergency Project Paper (p. 4) was to provide poor communities with improved basic socio-economic infrastructures and income generating activities, by financing at least 350 Sub-projects to be identified and implemented directly by communities . The PDO contains both outcomes and outputs . The assessment will use the outcome element : to provide poor communities with improved basic socio -economic infrastructures and income generating activities, The Project Appraisal Report included the following key indicators : 1. Allocation of financing of subprojects consistent with regional targeting criteria as estimated by a core welfare indicator survey undertaken in 2006; 2. Number of subprojects functioning one year after completion . Revised during implementation The project was revised in September 2008, when additional financing was introduced to mitigate the impact of the international food price crisis . This revision does not affect the version of the PDO that is being assessed . No new outcome indicators were added to the project . The project was revised a second time in June 2010, when additional financing was again introduced to mitigate the impact of the international financial crisis . This time the PDO being assessed was also changed to read : to provide selected poor communities with improved basic socio -economic infrastructures, income -generating and labor intensive activities and access to food, At the same time, new project outcome indicators were added to the original ones, measuring the number of direct project beneficiaries, including the share of female beneficiaries, in the following categories : (i) beneficiaries of public works program; (ii) students receiving at least a meal per day; (iii)farmers with access to fertilizer and seeds; (iv) people directly involved in new or improved income -generating activities; (v) students enrolled in constructed/rehabilitated schools; (vi)people with access to an improved water source; and (vii)area under improved land management. In addition, one original outcome indicator was amended, as “number of new sub-projects…� was changed to “percentage of new sub-projects…�. Because the PDO was revised once (at second additional financing ) and key indicators revised at that time, ratings will be assessed against the original project and the second additional financing . At the time of this restructuring, US$ 19.2 million (59.8 percent of total actual project costs ) had been disbursed. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 06/10/2010 c. Components: The project originally included three components : 1. Community subprojects (estimated cost at appraisal US$ 13.1 million; actual cost US$ 12.2 million). The component was to facilitate access of the population in poor communities to basic socio -economic infrastructures, and to income generating opportunities . Subprojects were to be implemented by communities with the support of existing regional development associations, Agences d’appui aux initiatives de base (AGAIB) that combined government and civil society organizations that finance projects in poor communities . The AGAIB was to make sub-grants to communities to finance approximately 200 basic infrastructure subprojects, and approximately 150 income generating activities. Communities were to contribute an additional 5 percent of the cost of the sub-project. Component resources were to be distributed to the five regions in the country according to criteria based on population and poverty levels . 2. Training (estimated cost at appraisal US$ 1.7 million; actual cost US$ 2.1 million). Training was to be conducted at the community and regional levels, and with AGAIB, on community -driven development methodologies and operational procedures, and at the national level to support liaison functions (financial management, procurement and reporting ). 3. Management and Operating costs (estimated cost at appraisal US $ 3.0 million; actual cost US$ 2.7 million). This component was to finance the salaries and operating costs of the five AGAIB staff and the Technical Secretariat as well as all Monitoring and Evaluation activities, information and communication campaigns, the financial and technical audits and other consulting services . This component was also to finance limited provision of goods. Revised components (i) First revision: During the first revision, the amount allocated for community development was increased from the appraisal estimate of US$ 12.5 million to US$ 14.5 million. The additional 2 million was to finance a school feeding operation that consisted of the preparation and distribution of daily lunches to 16,800 primary school students in 84 schools, over two school years, beginning with the 2008-2009 school year. In addition, a new, fourth, component was introduced into the project : 4. Enhanced Cereal Production (estimated cost US $5 million; actual cost US$ 3.5 million). The component was to support the purchase and delivery of inputs to increase the domestic production of cereals by about 14,000 additional tons of maize by the end of 2009. There were two sub-components: • Procurement and distribution of fertilizers, foundation seeds and improved seeds to 14,000 selected food-crop producers in 2009; • Coordination and technical support for capacity building activities within the Ministry of Agriculture, Fisheries and Livestock (AFL) to facilitate project implementation and supervision . (i) Second revision During the second revision, the school feeding program was expanded from 84 schools to approximately 180 schools and run the program for two additional school years (2010-2011 and 2011-2012) In addition, a sub-component was added to the community sub -projects component: • Labor Intensive Public Works (estimated cost US$ 4.4 million: actual cost US$ 4.7 million). The sub-component was to provide revenue support to more than 25,000 people through labor intensive public works. Public works will contribute to the rehabilitation of degraded lands and reforestation, thus reducing vulnerability of populations to the consequences of soil erosion and the impact of extreme climate phenomena . During the second revision, the Closing Date was extended from June 30, 2012 to September 30, 2013. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Financing: Of the original IDA grant of SDR 10.6 million, 10.55 million (or 99.5%) was disbursed. The follow-on IDA Grant of SDR 5.8 million under the Pilot Crisis Response Window was 100% disbursed. US$ 6.94 million was used of the total $7.0 million grant from the Food Price Crisis Response Fund (or 99 percent). Community contributions amounted to US$0.6 million or about 2.5% of the total cost of Component 1, less than the target of 5%. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High for the original and the revised objective . The project follows an extended period of political and economic crisis, with over 60 percent of the population living in poverty, mainly in rural and remote communities . A new political administration came into power in 2006, determined to change the economic environment for the better . A poverty reduction strategy was introduced in 2009, and donors, including the Bank, started re -engaging. The project directly supported the community development focus of the poverty strategy, which remains a central feature of the strategy . The project also is in line with the Bank ’s strategy in Togo Interim Strategy Note (FY12-FY13) and is an integral element of its second objective that includes “improved access of communities to basic social and local development services�. Conformity with Government and Bank strategy is also reflected in a second community development project currently underway . The elements that were brought into the project through the two additional financings were aimed at addressing the effects of particular shocks – first the food price crisis (through school feeding and agricultural inputs ) and then the international economic crisis (through public works). They were in line with government and Bank policy to support urgent social needs as they arose . b. Relevance of Design: Substantial for the original and the revised objective . The original design – a community development project - was relevant to the project objective in addressing both infrastructure and income generating objectives . Its focus on building community -level social and economic infrastructure, and creating resilience at grassroots levels, allowed it to proceed as an emergency operation under OP/BP 8.0. The introduction of the AGAIBs provided a strong grassroots engagement beyond participating communities, and introduced an element of continuous support to community initiatives beyond the project period. Geographic targeting ensured that the resources would be primarily allocated to poor regions . The results framework was straightforward, linking outputs to outcomes . The design of the revised project included the original design, which maintained its relevance, and introduced public works as well as agricultural inputs, which both directly supported the revised objective . Likewise, the results framework, while adjusted, maintained a clear statement of outputs linked to objectives . 4. Achievement of Objectives (Efficacy): Original PDO : to provide poor communities with improved basic socio -economic infrastructures and income generating activities Sub -objective 1: to provide poor communities with improved basic socio -economic infrastructures Sub- Efficacy for this sub-objective is rated substantial for the revised project . Outputs : • 290 basic socio-economic infrastructure sub-projects completed, exceeding the projected target of 200 (including 621 classrooms built against 360 projected; 23 health facilities constructed /renovated/equipped against 40 projected; and 76 community water points constructed or rehabilitated against 30 projected); • 4,275 tons of fertilizer procured and distributed to farmers, equal to the target; 858 tons of seeds procured and distributed to farmers, against a target of 370 tons; as a result, 13,774 farmers had access to fertilizer and seeds, against a target of 14,000. Outcomes : • 96 percent of basic socio-economic infrastructure assessed as having satisfactory technical quality against 90 percent target; • 97,367 students enrolled in constructed /rehabilitated schools, against a target of 72,000; • 63,440 people with access to an improved water source, against a target of 30,000; • The area under improved land management totaled 2,032 hectares, against a target of 2,200 hectares, and a baseline of zero hectares; • As a result of cereals inputs, rice and corn yields were 2.90 and 2.03 tons per hectare, respectively, exceeding long term trends in yields of 1.57 to 2.83 and 1.15 to 1.29 tons per hectare for rice and corn, respectively. The project met or exceeded all the targets under the sub -objective, and the impact of the cereals inputs raised production per ton above long term trends . With regard to socio-economic infrastructure, legitimate outcomes in terms of behavior change at the community level, beyond improved access to infrastructures, is difficult to determine. But choices are made by communities, and presumably reflect welfare maximization on their part, within the constraints set by the project operating manuals . A beneficiary survey showed a high degree of satisfaction with the sub -projects. Some 98 percent were satisfied; of those 55 percent were very satisfied . 94 percent felt that the sub-projects belonged to their community; 95 percent were satisfied with the quality of the works . Sub -objective 2: to provide poor communities with income generating activities Sub- Efficacy for this sub-objective is rated substantial for the revised project . Outputs : • 294 income generating sub-projects were undertaken, engaging 5,981 individuals; against a target of 150 sub-projects engaging 1,500 individuals. The projects related mainly to trading in grain, grain processing, and investing in livestock. Outcomes : • The end-of-project evaluation indicates that for the most profitable enterprises reached financial rates of return of 30 percent. A total of 75 percent of activities supported improved livelihoods of beneficiaries at the end of the project, with 58 percent “working well’, and 17 percent ‘moderately well’. 25 percent were structurally unprofitable or abandoned. 78 percent of beneficiaries interviewed thought their livelihoods had improved . Outcomes common to both sub -objectives : • Village development associations established, trained, and representative, with 68 percent of members women (target 25 percent), and at least one woman in a leadership position in 97 percent of cases (target 90 percent); • 268,119 people, of which 44 percent women, were direct beneficiaries of the community development component, against a target of 199,500, including 50 percent women; • 96 percent of sub-projects were functioning after one year, against a target of 90 percent; • The distribution of financing of sub -projects was consistent with regional targeting criteria based on a core survey of welfare indicators, showing a 3.5 percent variation from the survey; against a target of a less than 10 percent variation Revised PDO : to provide selected poor communities with improved basic socio -economic infrastructures, income-generating and labor intensive activities and access to food . The assessment of the Revised PDO will include (i) the assessment contained in the Original PDO, which includes basic socio-economic infrastructure and income generating activities; to which will be added (ii) an assessment of labor-intensive activities and access to food . Sub -objective 3: to provide selected poor communities with labor intensive activities Sub- Efficacy for this sub-objective is rated substantial for the revised project . Outputs : • 80 percent of the public works schemes were completed with satisfactory technical quality, against a target of 75 percent • The total cost allocated to wages was 70 percent, equal to the target Outcomes : • 1.1 million person-days of work was provided, against a target of 1 million; and 26,600 people were employed under the program • Female participation totaled 40 percent, against a target of 50 percent; and participation by youth 58 percent against a target of 75 percent • The annual net increase in cash income of participants exceeded that of a control group by 30 percent Sub -objective 4: to provide selected poor communities with access to food Sub- Efficacy for this sub-component is rated high for the revised project. Outputs : • 17 million meals provided to primary school students in 256 schools (against a target of 180 schools); as a result, 60,934 students were receiving at least a meal per day, against a target of 57,000 Outcomes : An evaluation of the school feeding program indicates that the program provided two thirds of the daily needs of the recipients. In addition it had indirect effects : all food was produced locally, and local women were engaged in the school feeding process . 5. Efficiency: Efficiency is rated substantial . No quantitative estimates of efficiency were undertaken, either in the PAD or the ICR . Instead, the ICR draws on three indicators of efficiency – operating costs as a share of investment costs, which are found reasonable and in line with other CDD operations at 15 percent; unit rate costs for construction, in this case of classrooms, which were found to be below comparable works in public agencies; and labor costs which met the target of 70 percent of total costs in public works . Similar analysis of the school feeding program and the enhanced cereal sub-component indicates that they compare favorably to international comparators . The school feeding program was estimated at a standardized cost of US$ 28 per student per year compared to international costs that range from US$ 28 to US$ 63. The yields generated by the inputs into cereals production and indicated above in the Efficacy section, are above long -term trends. Income generating projects improved the livelihoods of 78 percent of participants, and 75 percent of activities were working well or moderately well . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The ratings for relevance of objectives are high for the original project and for the revised one . Both are directly in line with Government policies and Bank strategy . The rating for design is substantial in both cases. On the positive side, this reflects the choice of original project design (financing of community sub-projects), as well as the subsequent introduction of new elements (school feeding and cereals production ) that immediately addressed the effects of international crises . A rating of high was tempered by the cereals production element, introduced under the original PDO; one of the lessons drawn under the project is that introducing that component under an agricultural project might have yielded better results . Achievement of objectives is rated substantial for both sub-objectives under the original PDO; reflecting the fact that where targets were set, they were either met or exceeded; where targets had not been set (for income generating projects ), results appear to be substantial . Regarding the efficacy of the revised PDO, three of the four sub -objectives were rated substantial, the two first ones as described for the original PDO, and the third one reflecting that two out of three outcome targets were met; the rating of high for the fourth sub-objective, reflecting its impact on food intake and local activity. Efficiency is rated substantial . Based on these ratings, the outcome rating for the original objective is satisfactory ; likewise, the outcome rating for the revised objective is satisfactory . Since both have a satisfactory outcome rating, no weighing will be undertaken. The combined outcome rating is therefore satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome is rated moderate . The risk to the development outcome pertains mainly to the sustainability of the community development sub-projects that were implemented during the project . With the withdrawal of project support, operations and maintenance of infrastructures will depend on the beneficiary communities and the resilience of the capacity building that was provided to them during the sub -project cycle. Initial indications are positive: 96 percent of infrastructure sub-projects were functioning one year after completion; and retention of concepts of community development was strong among 95 percent of community members. The risk is higher in other areas of the project. The sustainability of income-generating projects may be more unpredictable, and a function of the business acumen of the participants . School feeding programs and public works schemes were temporary initiatives, not expected to be sustained . Still, the Government has committed to the school feeding program, and it is continuing under a second Bank -funded community development project . The sustainability of the works completed under the public works scheme largely is a government responsibility and dependent on the capacity of the relevant ministry to maintain a large number of scattered works . More generally, the principle of a community development approach remains viable, and is reflected in a second community development project having been launched with the continued assistance of the Bank . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: In preparing the project, the Bank team built on the experience of a prior emergency program that had applied a community development approach to maintaining social services . And it drew on inputs from a broad set of stakeholders – national and sub-national public authorities, as well as grassroots organizations . For implementation, the team drew on regional associations (AGAIB) to act as channels for financing sub-projects, and for working with the communities to design and implement them . They would prove to be strong intermediaries, as is indicated by the sub -component outcomes. Additional support was introduced to ensure that fiduciary requirements would be met . Recognizing the recovery aspect of the intervention in the wake of crisis, the Bank team opted for processing the project as an emergency response project . The chosen design was a straightforward community development project that was able to deal with limited local capacities. The results framework was straightforward, and monitoring and evaluation design was thorough (see Section 10a). Still, some of the indicators would prove to be difficult to monitor, and as a result, would be only sporadically collected . at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: Facing an inexperienced client, the Bank team undertook regular field supervision as well as communicating frequently with the client. Supervision missions were used as occasions for training and technical assistance, as well as more formal oversight . Particular focus was given to safeguards and fiduciary aspects. Supervision dealt less effectively with monitoring, where persistent staffing problems were addressed slowly and as a result, the intended monitoring system was fully operational only in 2012. Moreover, indicators that would have allowed regular monitoring of community empowerment were dropped during implementation. These could have provided signals on potential risks to sustainability . Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government ministry (Ministry of Development, Cooperation and Regional Planning ) that successively was in charge of overseeing the project was supportive of most necessary adjustments to the project during implementation (essentially staffing changes and changes to the training program ). A decision in 2012 to cut staff from the implementing agencies may have been justified in terms of reduced workload as the project was coming to closure, but did nevertheless adversely affected supervision and capacity building, and slowed procurement, and consequently somewhat reduced project effectiveness and efficiency . Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: A project coordinating unit (PCU) at the central level and regional AGAIP agencies, supported by local NGOs and consultants formed the implementation team . The PCU focused mainly on fiduciary functions, and the AGAIPs field level supervision and support . Both operated in a competent manner, even as the project took on a more demanding set of activities . Their performance was adversely affected by the reductions in staff, but they are nonetheless seen as having been key in facilitating a relatively smooth implementation and satisfactory outcomes. Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: M&E design aimed at capturing outcomes and outputs of project activities : the quantity and quality of physical infrastructure, geographic targeting of project resources, gender inclusion, community empowerment, and project processes and procedures . Relevant indicators were introduced . A computerized management system at the central level was to consolidate results monitoring data produced by the AGAIBs; and a series of impact evaluations, including two technical audits, two evaluations of income generating activities, an evaluation of school feeding, an environmental and social audit, and annual financial audits, were included . b. M&E Implementation: Implementation was slow to get underway due to staff problems (the M&E specialist fell ill and was only belatedly replaced), and consequently slow consolidation of data at the central level . The problems were addressed only in 2012, when qualified specialists were brought in and the monitoring system became fully operational. In addition to the initial indicators, new ones were introduced with project revisions, as were Bank-wide mandatory core indicators. To contain the number of outcome indicators, two key indicators on community empowerment were dropped, weakening the sustainability analysis .. Moreover, data collection for certain indicators proved difficult, and as a consequence such indicators were not regularly updated To compensate for some of the monitoring and evaluation problems during implementation, an end -of-project stocktaking survey had to be commissioned . c. M&E Utilization: While information was collected, its timely utilization for project implementation was constrained as a result of the implementation problems signaled above . However, once these were straightened out – albeit belatedly – the system proved functional and, with the end -of-project stock-taking survey, formed a solid means of M&E . . Of some concernwas the absence of community empowerment indicators to serve as measures of sustainability of the works (sub-projects) being introduced. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The project triggered environmental, social and resettlement safeguard policies . Potential impacts were assessed at appraisal to be limited, and the environmental category was B . An Environmental and Social Management Framework (ESMF) and a Resettlement Policy Framework (RPF) were prepared, focal points were assigned, and training was conducted . All works contracts contained provisions for environmental and social protection. Instances of non-compliance with requirements occurred but these were corrected through further training and did not affect project results . An end-of-project audit found only minor anomalies and the Togolese Ministry of Environment issued a certificate in October 2013 to confirm that the project had complied with environmental and social protection requirements, according to the ICR . b. Fiduciary Compliance: Financial management . According to the ICR, financial management complied with Bank procedures and no major problems were identified during either statutory audit or verification by Bank financial management specialists. A fiduciary issue did arise in 2011 when the AGAIB salaries were raised without prior approval of the Bank. This led to temporary downgrading of the overall project rating (to Moderately Satisfactory), but was corrected when staff repaid the unauthorized increases . The ICR provides no information on whether audits were unqualified or not. Procurement . According to the ICR, procurement complied with Bank procedures and no major problems were identified. c. Unintended Impacts (positive or negative): n/a d. Other: n/a 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Significant Moderate Risks raised in the ICR appear to have Outcome : been significantly mitigated by the introduction of a second project . Bank Performance : Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lesson drawn from the ICRR process : • Sustainability of investments may be an inherent risk in community infrastructure projects . While it may not be possible to insure against it, it can be reduced through strong capacity building at community and local authority levels and careful monitoring of the quality of that capacity building . A longer-term programmatic engagement including through a community development fund may be a way to build sustainability . Lessons drawn from the ICR (abridged ): • Although the rationale for including a cereals production activity was persuasive, its development impact was limited. Benefits might have been greater had this activity been integrated into an ongoing support program to agriculture. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provided a thorough description of the project cycle and the necessary evidence of results . A more thorough analysis of the Borrower ’s performance during implementation would have been useful . The document was internally consistent and consistent with the ICR guidelines . a.Quality of ICR Rating : Satisfactory