Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004663 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF017708 ON A SMALL GRANT IN THE AMOUNT OF US$2.85 MILLION TO THE CATHOLIC RELIEF SERVICES FOR COMMUNITY DRIVEN NUTRITION IMPROVEMENT PROJECT (P145491) (March 31, 2019 ) Health, Nutrition & Population Global Practice East Asia And Pacific Region Currency Unit = US Dollars FISCAL YEAR January 1 – December 31 Regional Vice President: Victoria Kwakwa Country Director: Rodrigo A. Chaves Senior Global Practice Director: Timothy Grant Evans Practice Manager: Enis Barış Task Team Leader(s): Elvina Karjadi ICR Main Contributor: Elvina Karjadi ABBREVIATIONS AND ACRONYMS CDNIP Community Driven Nutrition MAF Ministry of Agriculture and Improvement Project Fisheries CNC Community Nutrition Coordinators MEAL Monitoring Evaluation Accountability and Learning CNE Community Nutrition Educators M&E Monitoring and Evaluation CNF Community Nutrition Facilitators MOH Ministry of Health CDNIP Community Driven Nutrition MOU Memorandum of Understanding Improvement Project CRS Catholic Relief Services MTR Mid-Term Review CVTL Cruz Vermelha de Timor NFNSP National Food and Nutrition Leste/Timor Leste Red Cross Security Policy DHS Demographic and Health Survey NGO Non-Governmental Organization EBF Exclusive Breastfeeding NNS National Nutrition Strategy ECOP Environmental Code of Practice ORS Oral Rehydration Salts FAO Food and Agricultural Organization PDO Project Development Objective FGD Focus Group Discussion POM Project Operational Manual FM Financial Management PRA Participatory Rural Appraisal GOTL Government of Timor Leste IA Implementing Agency SISCa Integrated Community Health Services (Servisu Integrado Saude Communitaire) GOTL Government of Timor Leste TF Trust Fund IFR Interim Financial Reports TLSLS Timor-Leste Survey of Living Standards IYCF Infant Young-Child Feeding TLSDP Timor Leste Strategic Development Plan JSDF Japanese Social Development Fund KDP Kolega de Paz TOT Training of Trainers IFR Interim Financial Reports UNICEF United Nations Children Fund IYCF Infant Young-Child Feeding WB World Bank JSDF Japanese Social Development Fund WHO World Health Organization KONSSANTIL Konselho Nasional Seguransa Aihan no Nutrisaun Timor Leste/National Council for Food Security, Sovereignty and Nutrition in Timor Leste TABLE OF CONTENTS DATA SHEET ........................................................................................................................... I I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 1 II. OUTCOME ...................................................................................................................... 6 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME .................................. 8 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .... 9 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 11 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 13 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 22 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 23 ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 29 The World Bank Community Driven Nutrition Improvement (P145491) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P145491 Community Driven Nutrition Improvement Country Financing Instrument Timor-Leste Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Catholic Relief Services Catholic Relief Services - Timor-Leste Project Development Objective (PDO) Original PDO The Project Development Objective (PDO) is to improve nutrition practices targeted to children under the age of two and pregnant and lactating women in targeted least developed communities. FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-17708 2,850,000 2,839,426 2,839,426 Total 2,850,000 2,839,426 2,839,426 Total Project Cost 2,850,000 2,839,426 2,839,426 Page i of 32 The World Bank Community Driven Nutrition Improvement (P145491) KEY DATES Approval Effectiveness Original Closing Actual Closing 16-May-2014 12-Aug-2014 31-Aug-2017 31-Aug-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 01-Sep-2016 0.99 Change in Results Framework Change in Loan Closing Date(s) Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 18-Dec-2014 Satisfactory Satisfactory 0.00 02 07-Jun-2015 Satisfactory Satisfactory 0.30 03 12-Nov-2015 Moderately Satisfactory Moderately Satisfactory 0.62 04 27-May-2016 Moderately Satisfactory Moderately Satisfactory 0.90 05 08-Oct-2016 Moderately Satisfactory Moderately Satisfactory 1.13 06 01-Mar-2017 Moderately Satisfactory Moderately Satisfactory 1.31 07 23-Sep-2017 Moderately Satisfactory Moderately Satisfactory 1.83 08 23-Mar-2018 Satisfactory Satisfactory 2.36 09 12-Oct-2018 Moderately Satisfactory Satisfactory 2.65 Page iii of 32 The World Bank Community Driven Nutrition Improvement (P145491) ADM STAFF Role At Approval At ICR Regional Vice President: Axel van Trotsenburg Victoria Kwakwa Country Director: Franz R. Drees-Gross Rodrigo A. Chaves Senior Global Practice Director: Xiaoqing Yu Timothy Grant Evans Practice Manager: Toomas Palu Enis Baris Elvina Karjadi, Eileen Brainne Task Team Leader(s): Eileen Brainne Sullivan Sullivan, Puti Yasmin Chrysanti ICR Contributing Author: Elvina Karjadi Page iv of 32 The World Bank Community Driven Nutrition Improvement (P145491) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. Context 1. Timor Leste is a country in transition, and remains fragile with the challenge of stagnant human development outcomes. After emerging from a long struggle for independence between 1999 and 2006, the country had its first peaceful democratic election for President in 2012. Although the country achieved lower middle-income status in 2011, poverty remains persistently high, particularly in the rural areas where 70% of the population reside. Timor Leste ranks 147 out of 187 countries on the United Nations Development Program Human Development Index for 2011, and performance on key socioeconomic indicators lags behind that of neighboring countries. 2. Timor Leste continues to experience high rates of fertility and infant mortality, while maternal mortality is among the highest in the world. This is a reflection of the lack of access to health care, poor infrastructure, and insufficient care for mothers during pregnancy and delivery. High maternal and child undernutrition contribute to poor health outcomes, low school achievement and productivity, and a cycle of impoverishment. An epidemiological transition, though at the early stages, is underway. Infectious, communicable, maternal and nutritional challenges remain the key causes of the disease burden, but new challenges are rapidly emerging; in particular, the growing burden of non-communicable diseases and associated risk factors has gone largely unaddressed. Malnutrition has serious—but preventable—adverse consequences for health, human capital, and economic development. Recent estimates indicate that malnutrition leads to US$41.0 million annually in lost economic activity, equivalent to nearly the entire Ministry of Health (MOH) budget and roughly 2% of the economic activity in the non-oil sector. 3. Based on the 2016 Demographic Health Survey (DHS), Timor Leste has one of the highest burden of stunting globally, affecting about 46% of children nationally, and reaching levels as high as 72% in one district; wasting affects 23% of infants in the first 5 years. Anemia and other micronutrient deficiencies, particularly vitamin A and iodine, are still a significant problem. Nearly one in three (63.2%) children (6-59 months) and 2 in 5 women aged 14-60 (39.5%) were anemic based on the 2013 Timor Leste Food and Nutrition Survey. While 2009-2010 and 2016 DHS does not have data on vitamin A deficiency, the coverage with vitamin A supplements among women and children is estimated at about 50%. In addition, 7 to 12% of children 1–5 years of age, and 2% of pregnant women, were reported to have signs of night blindness (e.g., bumping or tripping over objects in the evening). 4. Child malnutrition is due to immediate causes related to inadequate dietary intake and disease, as well as underlying causes such as food insecurity, improper feeding and caring behaviors, insufficient health care services, poor sanitation, hygiene and environment. An assessment carried out using Timor Leste Survey of Living Standards (TLSLS) 2007/08 confirmed that short-term food insecurity as measured by the quality and quantity of food consumed is significantly correlated with measures of child malnutrition. Even though agriculture dominates the economy and employs 70% of the population, most producers are subsistence farmers with an average landholding of 0.08 hectare1 and yields are among the lowest in Asia, resulting in a national shortfall of 39,000 tons of maize 1 FAO-MAF, Strategic Program for Promoting Agricultural Growth and Sustainable Food Security in Timor Leste, Bangkok, Thailand. March 2010 Page 1 of 32 The World Bank Community Driven Nutrition Improvement (P145491) and 50,000 tons of rice per annum. The deficit is made up through imports of rice (approximately 78,000 tons per annum), and substitution with cassava and other root crops.2 In addition, there are seasonal shortages of food between December to March, when households cope by reducing meal size, reducing the number of meals per day (often to one), and substituting less nutritious foods.3 On average, rural households experience 3.8 months without enough rice or maize to eat.4 However, increasing the availability of food preferred by Timorese, such as rice alone, may not be sufficient to make major reductions to malnutrition. Inadequate quality of consumed foods (e.g., low iron consumption, exposure of grain to aflatoxins, not initiating proper complementary feeding with protein and micronutrients, etc.), delayed planting season, and high food expenditures at the household level (including on nutritionally poor foods such as instant noodles) are also significant factors leading to child stunting.5 5. Evidence demonstrates that cultivation of diversified crops in home gardens, along with nutrition education, can lead to diversified diets and better nutrition.6 This might be due to the reduced risk of micro/macro-nutrient deficiencies, which can make children less susceptible to childhood illnesses. Child feeding and caring practices also have a profound effect on overall child health and nutrition. Evidence from the Lancet Series on Maternal and Child Malnutrition reinforces the importance of promoting complementary feeding of appropriate foods through strategies such as nutrition counseling as a way of reducing stunting incidence and the related burden of disease.7 6. In Timor Leste, child illnesses, gender disparities (including imbalances in decision making power about food choices), poor access to basic services, inadequate feeding and caring behaviors (well known to be one of the most important factors affecting nutritional status of children throughout the world) are also significantly correlated with increased risk of malnutrition. For example, living far from a clinic (over 2 hours) is associated with an increase in the risk of stunting and being underweight. However, even though demand for health services varies considerably across the country, the overall demand remains weak. Demand is particularly low among families living in rural areas that are unaware of available services and their benefits, or often have to walk long distances to access critical preventive and curative health care. Although direct delivery of health and nutrition services, especially through the country’s network of integrated community health service delivery system, known as Servisu Integrado Saude Communitaire/Integrated Community Health Services (SISCa)8, at each Suco, has improved utilization of key services, the current system of a monthly visit is inadequate to provide the intense support needed during the critical period of pregnancy to up to two years of a child’s life. 7. According to the March 2012 WHO/UNICEF Joint Monitoring Program on the use of improved sanitation facilities, less than half of the population of Timor Leste (47%) has access to any kind of improved sanitation facility. Improvements in rural sanitation are even scarcer, with just over 37% of the population accessing any kind of improved latrine. In fact, 43% of rural populations still actively practice open defecation. Preventable, fecal- borne illnesses such as diarrheal diseases and typhoid are a key contributor to mortality in children under five 2 Ibid. 3 Ibid. 4 Ibid. 5 Ibid. 6 Tung, Alexandra et.al. (2013) Nutrition Sensitive Agriculture for Timor-Leste: A Compendium of Resource 7 Bhutta et al., What works? Interventions for maternal and child under nutrition and survival. Lancet. 2008.371:417-40 8 Through a “six-table assistance system� SISCa provides: (i) population registration assistance; (ii) nutrition education; (iii) maternal and child health services; (iv) personal hygiene and sanitation counseling; (v) curative health care services; and (vi) broad-based health education. Page 2 of 32 The World Bank Community Driven Nutrition Improvement (P145491) years of age and are directly linked to inadequate water supply, sanitation and hygiene issues. Diarrheal diseases are also one of the most significant causes of malnutrition and nutrient malabsorption, which can lead to impaired physical growth (stunting), reduced resistance to infection, and long-term gastrointestinal disorders.9 The 2008 Lancet series recommends using zinc supplementation to manage diarrhea.10 8. The Government of Timor Leste (GOTL) has a cross sector commitment to address malnutrition. This includes Timor Leste’s Strategic Development Plan (TLSDP) 2011-2030 for the importance of nutrition for health and development of children, the National Nutrition Strategy (2014-2019-NNS), and the GOTL has also endorsed the Zero Hunger Challenge and Sustainable Development Goals. In 2012, the National Council for Food Security, Sovereignty and Nutrition in Timor Leste (KONSSANTIL) was formed to strengthen relevant sectoral ministries in nutrition planning and coordination. However, institutional and implementation arrangements to support the NNS are yet to be functional partly due to: (a) inadequate human resources and capacity; (b) sub-optimal coordination between sectors including unclear roles and responsibilities of line ministries; and (c) inadequate financial resources for activities and personnel. 9. The Community Driven Nutrition Improvement Project (CDNIP) was a four-year Project led by Catholic Relief Service (CRS), the implementing agency (IA) financed by the World Bank (WB) with funding from the Japanese Social Development Fund (JSDF), and implemented by three local partners (Caritas Baucau, Timor-Leste Red Cross (CVTL) and Kolega de Paz (KDP), and an additional partner, Fraterna, which implemented the aquaculture pilot. The CDNIP was implemented in 49 Sucos of Baucau and Viqueque municipalities, impacting close to 4,500 children and more than 5,000 pregnant and lactating women. The Grant Agreement was signed, and became effective in August 2014; however, the actual implementation only began when the Project Operational Manual (POM) and Procurement Plan were finalized after six months. Given the delayed start, among other reasons, an extension of the Closing Date by one year, from August 31, 2017 to August 31, 2018, was part of the restructuring in August 2016. The JSDF provided US$2,850,000 through the WB, and the total CRS contribution to the Project was US$1,298,880, resulting in a total cost of CDNIP of estimated US$4,148,880. 9 Murray, Lopez (1997); Guerrant et al. (1999); Baqui et al. (1993); Schneider et al. (1978); Humphrey (2009). 10 Raew, Kathryn et al. (2012). A Life Free From Hunger: Tackling Child Malnutrition. STC. London. Page 3 of 32 The World Bank Community Driven Nutrition Improvement (P145491) Project Development Objectives (PDOs) 10. The PDO of the Project was to improve nutrition practices targeted to children under the age of two and pregnant and lactating women in targeted least developed communities. Key Expected Outcomes and Outcome Indicators 11. The following four PDO indicators were used to measure the success of the Project: (a) Proportion of children < 6 months of age who are exclusively breastfed; (b) Proportion of children 6-23 months provided with all 3 minimum infant and young child feeding practices; (c) Proportion of children under two years with diarrhea who are provided continued feeding along with ORS11 and/or increased fluids; and (d) Proportion of mothers of children under two years of age who consumed iron-rich foods. Components 12. The Project had four Components: 13. Component 1: Community Mobilization, Awareness Raising and Participatory Planning (US$0.602 million original, US$0.434 million restructured). The objective of this Component was to increase community’s understanding and participation in identifying and addressing malnutrition. The objective was to be achieved through the following strategies: (a) district and sub-district level orientation of Project objective and identification of target communities; (b) orientation of Project objective at Suco level in consultation with Suco councils; (c) participatory mapping of key issues in child malnutrition at community level (e.g. types, causes and impact as well as options to address malnutrition based on gender analysis to understand the roles of women and men in decision making); and (d) community-led participatory planning including nutrition-specific and sensitive interventions and selection of community volunteers to be Community Nutrition Educators (CNE). Under this Component, the IA-CRS and its local partners (e.g., civil society organizations12, etc.) was responsible for facilitating Project orientation, coordination and participatory planning processes at the national, district, Suco, and Aldeia levels. 14. Component 2: Community Led Delivery of Nutrition Specific Interventions (US$0.786 million original, US$1.135 million restructured). The objective of this Component was to improve accessibility of community driven nutrition specific interventions by the targeted households. While the SISCa and clinic-based initiatives have shown some progress to address malnutrition in Timor- Leste, they still did not able to reach the outlying households; most of the households did not have 11 Oral Rehydration Salts (ORS). 12 The civil society organizations included local NGOs, faith-based organizations, etc., currently operating in the target areas. Local partners would be selected per the simplified World Bank Procurement/Consultant Guidelines for Fragile and Small States. Page 4 of 32 The World Bank Community Driven Nutrition Improvement (P145491) frequent contact with providers of health services.13 This Project was to incorporate aspects of the Care Group model that was able to multiply efforts through the use of CNEs that have shown promising results to improve the level of global under-nutrition scale at a low cost in Africa.14 The CRS had experience in Timor Leste in reaching large numbers of communities with high levels of engagement from community facilitators/volunteers. This was a mixed approach that has combined a volunteer-based strategy with the provision of incentives for CNE. The Component’s objective was achieved by: (a) provided Community Nutrition Coordinators (CNC) with training of trainers (TOT) on nutrition and hygiene counseling and facilitating community led learning sessions; (b) CNC trained CNEs on nutrition and hygiene counseling and conducted home visits and small group discussions; and (c) CNEs built relationships with households and provided ongoing, individualized, dynamic counseling. 15. Component 3: Community Led Delivery of Nutrition-Sensitive Interventions (US$1.018 million original, US$0.794 million restructured). The objective of this Component was to improve accessibility of selected nutrition-sensitive activities that address the underlying causes of malnutrition and create an enabling environment for households and communities to engage in the behavior change for improved nutritional outcomes. Under this Component, communities were to implement their community action plans: the communities selected one of three options that best addressed their needs, interests and natural resources. These three nutrition-sensitive interventions had been identified as having the most promising effects on reducing maternal and child malnutrition.15 The experience of CRS in Timor-Leste with community-based Projects had shown that there was increased participation and better outcomes when the community is provided with a limited number of interventions to select from that address the community’s prioritized problems. A review of nutrition-sensitive agriculture interventions of Timor-Leste16 identified the following as having positive impacts on nutritional outcomes at the household level: (a) increasing the availability and production of bio-fortified foods such as orange flesh sweet potato; (b) increasing vegetable production and homestead gardening combined with nutrition education; and (c) increasing aquaculture production, post-harvest handling and storage. This information was used to create the 3 nutrition-sensitive intervention options for this Component, which were increased availability of (a) nutritious staple and minor crops; (b) vitamins and diversified food through homestead gardens; and (c) protein and Omega 3 through creation of household level fish ponds. 16. Component 4: Monitoring, Evaluation and Project Management. Sub-Component A: Monitoring and Evaluation (US$0.148 million original); and Sub-Component B: Project Management (US$0.296 million original) for a total of US$0.444 million original, and US$0.487 million restructured). The objectives of this Component were twofold: under Sub-Component A, the Project was to (a) provide technical advisory services and other material support to facilitate implementation of nutrition- specific and sensitive interventions by target communities; (b) improve coordination among various actors through creation of a communication flow map (e.g., ministerial counterparts, district teams, Suco, Aldeias, etc.); and (c) strengthen monitoring and evaluation (M&E) activities associated with 13 TLSLS 2007. 14 Davis, Thomas P. et al. Reducing child global undernutrition at scale in Sofala Province, Mozambique, using Care Group Volunteers to communicate health messages to mothers. Global Health: Science and Practice 2013. 15 Tung, Curran and Fanzo (2013) “Nutrition Sensitive Agriculture for Timor -Leste: A Compendium of Resources� 16 Ibid. Page 5 of 32 The World Bank Community Driven Nutrition Improvement (P145491) the verification and measurement of Project results at the community level. The M&E system incorporated community monitoring to ensure beneficiary accountability and community ownership of information which was gathered and analyzed. Sub-Component B included (a) financial supportive supervision of Project implementation and monitoring system to facilitate systematic data collection, Project implementation, and coordination of multiple local implementing agencies; (b) preparation of semester progress reports including unaudited financial reports; and (c) implementation completion and results report incorporating results of the final evaluation. 17. At Approval, the total Project cost was US$2.85 million, financed by the JSDF. See Annex 2 for the details of the original and restructured Project costs by Component. 18. Significant changes during Implementation. The Project was restructured on August 31, 2016. The PDO and components were not revised. When the project was designed, the baseline data was taken from DHS 2009/2010. Therefore, when the baseline survey from the Project was completed in November 2015, there were changes made to the Results Framework with some adjustments to the baseline and targets indicators (see Annex 1). Annex 4 presents a snapshot of the original baseline and targets, as well as the revised targets and achievements. 19. The rationale for the restructuring. The restructuring was to provide sufficient time for implementation, revise the target outcomes based on more up-to-date information available, and reallocate funding within the components based on the needs of the activities to be implemented. The changes included in the restructuring were: (a) extension of the Closing Date from August 31, 2017 to August 31, 2018; (b) the results framework was revised, with the revision of baseline and target values; (c) reallocation of funds under the Grant Agreement categories by reallocation of funds from Part 1 of the Project (Component 1) to Parts 2 and 3 (Components 2 and 3), because certain goods were not needed and procured reducing the funds under Component 1. II. OUTCOME Assessment of Achievement of Each Objective/Outcome 20. PDO 1. Proportion of children < 6 months of age who are exclusively breastfed. This indicator was not met; achieving 70% against a target of 75% from a baseline of 71.3%. Although the indicator did not achieve the target, based on the results of endline survey of the Project, there was a significant increase among children 4-5 months from 42% to 76%, when the sample was stratified the sample by age. In addition, the proportion of children put on the breast within the first hour of birth had significantly improved from a baseline of 65% to 77% at the end of the Project. 21. PDO 2. Proportion of children 6-23 months provided with all 3 minimum infant and young child feeding practices. The target was exceeded by more than 100%. It increased from a baseline of 3.10% to 18%, against a target of 3.25%. For those children who were not breastfed, the increase was 1.7% from a baseline of 0% against a target of 0.5%. The increment was due to the fact that the Community Nutrition Facilitators (CNFs) were sufficiently trained on the topic of maternal and infant nutrition (including child feeding practices) and had given good advice on feeding practices utilizing local food. Page 6 of 32 The World Bank Community Driven Nutrition Improvement (P145491) 22. PDO 3. Proportion of children under two years with diarrhea who are provided continued feeding along with ORS and/or increased fluids. The indicator did not achieve the target of 59% from a baseline of 53.8%, and only attaining 31%. This result was due to a reduction in the proportion who provided continuous feeding during diarrhea. Informal interviews indicated that feeding anything outside of fluids when children experience diarrhea is a more difficult behavior change to address in communities and requires more time and messaging to be fully adapted by families, as many families believe that feeding more food will worsen the diarrhea. Although PDO3 target was not achieved, children aged 0-24 months with or without diarrhea who were provided with either ORS or increased fluid has increased from 82% to 87% and the provision of zinc supplements increased from 0 to 11% (p<0.001) based on the endline survey of the Project. 23. PDO 4. Proportion of mothers of children under two years of age who consumed iron-rich foods. The target for this indicator was surpassed (>100%), reaching 37% against a target of 20%. The proportion of mothers of children under two years old who consumed iron-rich foods was more than doubled during the Project. The findings showed that the benefits of improved dietary knowledge resulted in improved maternal nutrition as well, since 96% of mothers consumed at least three meals in the previous 24 hours. 24. Overall, the results of the Project show that two of four PDO indicators have surpassed their targets and two PDO indicators did not achieve their targets although progress was made against both. Most of the intermediate results indicators (eight out of nine) were achieved (surpassed) and only one indicator was partially achieved. The original and revised baseline and target values for the PDOs, along with a summary of achievements of PDO indicators at the end of the Project is presented in Annex 4, Table 1. Based on the above assessment, and considering the weighted average of surpassed PDO indicators, the rating of efficacy is Substantial. Overall Outcome Rating 25. The PDO was highly relevant to the commitment of GOTL which had put in place a number of national policies such as The National Food and Nutrition Security Policy (NFNSP) and the NNS. These policies were to guide the Government’s effort to eradicate hunger and malnutrition in all their forms, and to contribute to poverty reduction by 2020 which is one of the goals of the TLSDP (2011-2030). The PDO was also highly relevant to the current WB Group’s Country Partnership Framework (CPF) (2019-2023) which are fully aligned with Timor Leste’s long- and medium-term national development priorities articulated in the Strategic Development Plan 2011-2030 and Government’s current five-year program (2018-2023). 26. Based on the perspective of Project design, actual achievements and disbursements, the Project was efficient because all planned activities were completed by the time of the final implementation support mission, with 99.6% funding disbursements, although the overall timeframe of the project was extended. Considering this delay, the overall implementation efficiency was modest. 27. Based on high relevance, substantial efficacy and modest efficiency, the overall rating is Moderately Satisfactory Page 7 of 32 The World Bank Community Driven Nutrition Improvement (P145491) Other Outcomes and Impacts (a) Gender. The positive impact of this Project was the empowerment of women (mothers) in their decision in breastfeeding, proper feeding practices and treatment of their sick children (diarrhea). However, the involvement of men (husbands) needs to be improved, and this Project triggered the awareness of mothers and family members about nutrition and agricultural investments. (b) Household income. Based on the endline survey results, the positive feedback of the nutrition sensitive intervention (agricultural) for this Project reported that 44% of households surveyed grew better crops, 22% earned more income, and 17% increased production yields. (c) Replicable best practices. After the completion of this Project, another development partner has been providing support to 30 of 49 Sucos covered by this Project through their TOMAK project (Farming for Prosperity Program in Timor Leste to support agriculture and food security sector). III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME 28. Readiness for implementation. The Grant Agreement was signed in July 2014, however implementation was delayed by six months, with WB concurrence, to allow for the completion of the Project Operational Manual. In addition to requiring the Manual to provide guidance on processes, there was also delays associated with the preparation of a Procurement Plan, contracting of consultants to support the implementation, the provision of intensive socialization prior to the commencement of activities. 29. Good collaboration and coordination. During implementation, there was good collaboration and coordination between the MOH and Ministry of Agriculture and Fisheries (MAF) at National, District, Sub- district and Suco level which increased the involvement of the relevant national, District and sub-district staff. Also, the participation from CRS staff in the national and District level nutrition meetings, as well as the cross- sectoral working group (KONSSANTIL) and harmonization meetings also had benefits and the Project was well received by all levels, including community level authorities, where full cooperation and collaboration was established. However, there was still a challenge to increase the Government’s ownership of this project considering their involvement during implementation was still limited. 30. The extremely dry climate conditions. Extremely dry conditions were triggered by El-Niño during Project implementation and resulted in a late planting season, which reduced families’ abilities to feed their children. Also, since vegetable production does not continue in the dry season, many communities complained that they could not grow the vegetables due to a lack of water; Baucau was particularly hard hit in this respect. Therefore, while the agricultural training under the Project helped farmers to cope to a certain degree, the conditions were so severe that it still affected the Project impact. This is evidenced by the intermediate results, where the dry weather affected the achievement of harvesting beans. The CRS, in anticipation of the El-Niño year, implemented the nutrition-specific and sensitive activities in parallel, and quickly moved on the nutrition- sensitive activities to counter the upcoming dry weather. 31. Political tensions in country. Community Actions Plans were finalized by mid-November 2015; however, the registration of beneficiaries was only completed in March 2016. This delay was due to political tensions Page 8 of 32 The World Bank Community Driven Nutrition Improvement (P145491) that occurred over a period of three to four months during which the Project staff could not travel to some of the sub-Districts (Laga, Baguia, and to a lesser extent Quielikai); the difficulties in reaching District/sub-Districts areas was because of the potential volatile situations which could arise due to the political conflicts. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. BANK PERFORMANCE 32. The Project design and strategy remains relevant. The Project design and strategy were simple and clearly stated in the Project documents. However, the estimated time required for the Project to achieve the targets need to be considered for more than 4-years project duration. 33. Risk identification and mitigation measures at appraisal were taken into account. The overall risk rating was Moderate which took into consideration that while CRS Timor Leste had experience implementing similar programs in the target areas, it had not implemented any WB financed projects, and the Project required multi- sectoral interventions in a relatively weak coordination environment. Based on this, the WB staff worked with the CRS to mitigate the risk factors, including some which arose during implementation, which fed into the adjustments/restructuring. 34. Despite several changes in task team leaders during the Project, close supervision was essential to ensure the quality of Project implementation. During the Project implementation, the change of task team leader three times did not impact on the progress of Project implementation because of their availability for just-in-time support/responses. The task team leaders continuously provided guidance throughout the process from start up to Project closing, including on the technical, monitoring, and fiduciary aspects. After the MTR, the WB’s staff carried out supervision missions every 6 months and gave constructive feedback after each visit. The participation of WB’s staff in one of the visits to the Project sites (District) was very useful to oversee the field conditions at the local level. 35. M&E Implementation. In the first half of the Project, in addition to the indicators from the Results Framework, there were a large number of indicators required by CRS in the M&E plan which were being captured using their Monitoring Evaluation Accountability and Learning (MEAL) system, which made it difficult for implementing partners to provide what was required for reporting and program planning. The forms did not have formats that readily captured the required information which necessitated conducting follow-up visits to collect missing data. Complaints about the MEAL system were raised at the MTR and CRS responded to these complaints by reducing the complexity of the MEAL system and increasing resources to help the partners manage the MEAL requirements. However, after the MTR, there was a more consistent approach to M&E and additional resources were provided to implementing partners (e.g., hiring of M&E officers). The partners still found the M&E system challenging, with too many indicators to report on, and M&E system was often lacking and delayed. However, the level of skills required to maintain the M&E system was still beyond available skills in the local labor market. Therefore, CRS tried to support staff from the three local partners through monitoring and capacity building. Furthermore, it was found that M&E officers in place at the end of the Project were completing the required monitoring documentation and providing it on-time and of satisfactory quality. Page 9 of 32 The World Bank Community Driven Nutrition Improvement (P145491) 36. M&E Utilization. During data collection, the paper-based system was used and required the staff to manually enter all the data. This took a long time, and occasionally resulted in data being lost, and data was not being provided in a timely manner to assist planning for the next quarter. The M&E was not optimally utilized and communicated to stakeholders. Justification of Overall Rating of Bank Performance 37. Overall, the ICR rates WB Performance over the entire Project cycle as Satisfactory. This rating reflects the WB’s ability to remain flexible and to find alternative implementation arrangements based on the ground realities. The WB’s agility to adjust the implementation process with closely supervision were essential to the Project’s achievements. B. ENVIRONMENTAL, SOCIAL AND FIDUCIARY COMPLIANCE 38. Environmental Safeguards. The Project triggered OP/BP 4.01, environment assessment. Physical investments were very small and included household type of agriculture activities to improve the nutrition content available to the communities. An Environmental Code of Practices was utilized as the instrument to prevent and minimize impact to the environment. The instrument was reviewed and simplified after the MTR to make it more user friendly. Training was provided to beneficiaries before implementation of activities such as preparation of homestead gardening, simple farming techniques, and storing seeds. No significant environment impact was found during Project implementation. 39. Social. The social safeguard compliance was satisfactory. The Project triggered the WB’s Indigenous People Policy (OP 4.10) and the elements under the Policy were integrated into the Project design and included in the Project Operational Manual. The Project ensured the communities participated in, and benefitted from, the Project and that the Project interventions would not adversely affect communities’ social and cultural condition. The safeguards aspects were well implemented at field level through involving local partners, including local facilitators who spoke the local languages and understood local cultural conditions. The Project established a Grievance Mechanism based on consultation with communities, which was put in place and implemented effectively. 40. Procurement. Procurement performance was consistently satisfactory. Preparation of the Project Operational Manual, which included procurement implementation guidance for the Project, was a significant piece of work carried out by the Project. All procurement of goods, and consulting services of the Project were deemed to have been carried out in accordance with provisions of the Manual, WB Procurement and Consultant Guidelines, and CRS’ approval procedures. The Procurement Plan was updated frequently throughout the Project implementation. Despite the diligent manual/guidance preparations, there were some minor deviations relating to the application of the recommended procurement process and templates. 41. Financial Management (FM). The FM performance under the Project was rated as moderately satisfactory during the life of the Project. The internal controls were adequate overall and were guided by CRS policies and adhered to the procedures outlined in the Project Operational Manual. The FM reviews did not raise any internal control issues. The Project internal Interim Financial Reports and bank reconciliations were prepared monthly. The advances monitoring system was effective, and the list of fixed assets was maintained and up to date. The filing system was well organized. Annual audited financial statements were submitted on time except Page 10 of 32 The World Bank Community Driven Nutrition Improvement (P145491) for fiscal year 2016 (October 1, 2015 to September 30, 2016); the report was due by March 31, 2017 but was only submitted on May 21, 2017. All the audit reports were unqualified, and no issues were noted; as such no management letters were submitted. The final audit covered the period of October 1, 2017 to December 31, 2018 (including the 4 month grace period) was acceptable. Based on the final IFR submitted by the Project and the client connection, there is an undisbursed balance of US$10,573, meaning that the Project was 99.6% disbursed. C. RISK TO DEVELOPMENT OUTCOME 42. The risk to development outcome is Moderate. Although there was generally a lack of ownership by Government and limited information sharing during this Project, the Australian Government will be supporting this approach in 30 Sucos, and there was a tentative plan that the Project will enter into a memorandum of understanding with the MOH. This should increase the sustainability of project investments. V. LESSONS LEARNED AND RECOMMENDATIONS 43. Using an IA with in-country experience at the grass roots level, and their support to capacity building of local partners had a positive impact on the Project’s achievements. These results were shown by CRS’s successful collaboration with both implementing (local) partners, as well as fulfilled the requirements from the WB. The implementing partners were mostly satisfied with CRS’s investment in their organizational strengthening, finance training, policy development, and TOT in agriculture investment. 44. Improving access to antenatal care and the quality of nutrition advice provided at these sessions provided an additional avenue to introduce or reinforce the Project’s messages to pregnant and lactating women and their spouses. There were a number of areas where people’s understanding of nutrition has shifted due to the project. The three most significant areas were: (1) The importance of including diverse and nutritious food in infant and young children’s porridge (sasoro); (2) the importance of exclusive breastfeeding; and (3) the importance of continuing breastfeeding after 6 months along with nutritious complementary food. 45. Building new practices to improve nutrition based on existing family practices and including community members as teachers increased the adoption of the strategies. Good results were shown by (a) using change agents drawn from the communities themselves, such as CNF, to serve as community mobilizers and focal points, rather than as the main training resource communities, was likely to be easier to manage and allows great control of training quality and consistency; (b) building new practices based on existing family practices was a highly successful strategy to generate adoption, e.g., the preparation of sasoro (rice porridge). 46. Including water, sanitation and hygiene along with nutrition specific interventions could help to introduce and reinforce the linkage between these aspects. The Project had a component of hygiene awareness due to correlation between enteric diseases and malnutrition – a child with diarrhea or helminthiasis will not gain optimal growth. However, until the end of the Project, the endline survey found that the link was not well understood by the communities. Rain and soil moisture were reduced through at least half of the Project due to El Niño driven drought conditions. Many communities complained that they could Page 11 of 32 The World Bank Community Driven Nutrition Improvement (P145491) not grow the vegetables due to a lack of water. Although this Project delivered smart agricultural training that helped farmers cope somewhat, the dry conditions affected the vegetable production nonetheless. Consideration should be given to expanding future support of this type by including water, sanitation and hygiene projects to complement the nutrition-specific interventions. 47. Building awareness on specific targets for men and female elders. In this Project, the involvement of men was limited. The men were generally of the view that the Project was for mothers only, and they had not participated. The lack of involvement of men had made it difficult for the implementing partner staff who sometimes had to do the heavy physical work preparing demonstration agriculture plots that was too demanding for pregnant and lactating women. In addition to the men (husbands), the grandmothers also influenced the dietary decisions in a family. For future projects of this type, male and female elders should be engaged for increasing knowledge and interest in nutrition, as well as encouraging men/husbands/fathers to participate in agricultural productions. . Page 12 of 32 The World Bank Community Driven Nutrition Improvement (P145491) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve nutrition practices targeted to children under the age of two and pregnant and lactating women in targeted least developed communities. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of children < 6 Percentage 71.30 60.00 75.00 70.00 months of age who are exclusively breastfed 31-Aug-2016 12-Aug-2014 31-Aug-2016 15-Aug-2018 Comments (achievements against targets): Although the Project target of 75% was not achieved, there is only a slight decline from baseline or 71.30% (baseline) and 70% (endline). Nationally, the decline of exclusive breastfeeding was from 62% in 2013 to 50% in 2016 (Timor Leste Demographic Health Survey). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of children 6-23 Percentage 3.10 40.00 3.25 18.00 Page 13 of 32 The World Bank Community Driven Nutrition Improvement (P145491) months provided with all 3 31-Aug-2016 12-Aug-2014 31-Aug-2016 15-Aug-2018 minimum infant and young child feeding practices Comments (achievements against targets): It increased from a baseline of 3.10% to 18%, against a target of 3.25%. The increment was due to the fact that the Community Nutrition Facilitators (CNFs) were sufficiently trained on the topic of maternal and infant nutrition (including child feeding practices) and had given good advice on feeding practices utilizing local food. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of children under Percentage 53.80 68.00 59.00 31.00 two years with diarrhea who are provided continued feeding 31-Aug-2016 12-Aug-2014 31-Aug-2016 15-Aug-2018 along with ORS and/or increased fluids. Comments (achievements against targets): The indicator did not achieve the target of 59% from a baseline of 53.8%, and only attaining 31%. This result was due to a reduction in the proportion who provided continuous feeding during diarrhea. Informal interviews indicated that feeding anything outside of fluids when children experience diarrhea is a more difficult behavior change to address in communities and requires more time and messaging to be fully adapted by families, as many families believe that feeding more food will worsen the diarrhea. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of mothers of Percentage 15.40 58.00 20.00 37.00 children under two years of age who consumed iron-rich foods 31-Aug-2016 12-Aug-2014 31-Aug-2016 15-Aug-2018 Page 14 of 32 The World Bank Community Driven Nutrition Improvement (P145491) Comments (achievements against targets): The target for this indicator was >100%, reaching 37% against a target of 20%. The proportion of mothers of children under two years old who consumed iron-rich foods was more than doubled during the Project. The findings showed that the benefits of improved dietary knowledge resulted in improved maternal nutrition as well, since 96% of mothers consumed at least three meals in the previous 24 hours. A.2 Intermediate Results Indicators Component: Component 1: Community Mobilization, Awareness Raising and Participatory Planning Component 2: Community Led Delivery of Nutrition Specific Interventions Component 3: Community Led Delivery of Nutriti Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of targeted Number 0.00 200.00 200.00 262.00 communities where a Participatory Rural Appraisal 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 (PRA) is conducted by second year. Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of community-led Number 0.00 200.00 200.00 238.00 action plans for nutrition interventions (Component 2 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 and 3) available by project year Page 15 of 32 The World Bank Community Driven Nutrition Improvement (P145491) 2 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of Community Number 0.00 17.00 18.00 25.00 Nutrition Coordinators trained as ToTs (Training of Trainers) 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 by end of project Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percent of mothers/caretakers Percentage 0.00 70.00 70.00 73.00 attending small group sessions or received home visit by end 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 of project. Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of trained Percentage 0.00 80.00 80.00 94.00 Page 16 of 32 The World Bank Community Driven Nutrition Improvement (P145491) houseshold practicing one or 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 more of the recommended planting to post-harvest practices for the production of nutritious staple and minor crops by end of project Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percent of target households Percentage 0.00 80.00 70.00 77.00 producing vegetables through homestead gardens by the end 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 of project. Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of target Percentage 0.00 70.00 70.00 53.00 households with established homestead protein production 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 by end of project Comments (achievements against targets): Page 17 of 32 The World Bank Community Driven Nutrition Improvement (P145491) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of Community Number 0.00 500.00 350.00 368.00 Nutrition Facilitators trained by Community Nutrition 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 Coordinators per annum Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of Community Number 0.00 577.00 833.00 3555.00 Nutrition Facilitator Learning Sessions conducted by end of 12-Aug-2014 12-Aug-2014 31-Aug-2016 15-Aug-2018 project Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of mothers of Percentage 15.40 20.00 20.00 62.00 children 0-23 months of age who report consuming iron- 31-Aug-2016 12-Aug-2014 31-Aug-2016 15-Aug-2018 rich food in the previous 24 hours Page 18 of 32 The World Bank Community Driven Nutrition Improvement (P145491) Comments (achievements against targets): Page 19 of 32 The World Bank Community Driven Nutrition Improvement (P145491) A. ORGANIZATION OF THE ASSESSMENT OF THE PDO PDO: To improve nutrition practices targeted to children under the age of two and pregnant and lactating women in targeted least developed communities. 1. Proportion of children < 6 months of age who are exclusively breastfed. 2. Proportion of children 6-23 months provided with all 3 minimum infant and young child feeding practices Outcome Indicators 3. Proportion of children under two years with diarrhea who are provided continued feeding along with ORS and/or increased fluids 4. Proportion of mothers of children under two years of age who consumed iron-rich foods 1. Number of targeted communities where a PRA is conducted by second year. 2. Number of community-led action plans for nutrition interventions (Component 2 and 3) available by project year 2 3. Number of CNC trained as ToTs by end of project 4. % of mothers/caretakers attending small group sessions or received home visit by end of project 5. % of trained household practicing one or more of the Intermediate Results Indicators recommended planting to post-harvest practices for the production of nutritious staple and minor crops by end of project 6. % of target households producing vegetables through homestead gardens by the end of project 7. % of target households with established homestead protein production by end of project 8. Number of CNF trained by CNC per annum Page 20 of 32 The World Bank Community Driven Nutrition Improvement (P145491) 9. Number of CNF Learning Sessions conducted by end of project 10. % of mothers of children 0-23 months of age who report consuming iron-rich food in the previous 24 hours 1. 262 targeted communities were reached where the PRA was conducted 2. 238 of community-lead actions plans for nutrition interventions available 3. 25 CNC trained by end of project 4. 73% mothers/caretakers attending small group sessions or received home visits by end of project 5. 94% of trained household practicing one or more of the recommended planting to post-harvest practices for the production of nutritious staples and minor crops by end of Key Outputs by Component project (linked to the achievement of the Objective/Outcome ) 6. 77% of target households produced vegetables through homestead gardens by the end of project 7. 53% of target households established homestead protein production by the end of project 8. 368 of CNF trained by CNC per annum 9. 3,555 CNF learning sessions conducted by the end of project 10. 62% of mothers of children 0-23 months of age reported consuming iron-rich food in the previous 24-hours Page 21 of 32 The World Bank Community Driven Nutrition Improvement (P145491) . ANNEX 2. PROJECT COST BY COMPONENT Percentage of Amount at Amount at Actual at Project Closing Components Approval Approval Restructuring* (US$M) (US$M) (US$M) (US$M) Community Mobilization, Awareness Raising and 602,000 434,000 433,752.33 100% Participatory Planning Community Led Delivery of Nutrition Specific 786,000 1,135,000 1,170,042.19 103% Interventions Community Led Delivery of Nutrition-Sensitive 1,018,000 794,000 733,183.52 92% Interventions Sub-Component A: Monitoring and Evaluation - 444,000 487,000 502,447.98 103% Sub-Component B: Project Management Total 2,850,000 2,850,000 2,839,426.02 99.6% *Amount at approval based on revised version at Project restructuring (August 31, 2016) Page 22 of 32 The World Bank Community Driven Nutrition Improvement (P145491) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Summary of Borrower's ICR • Assessment of the operation’s objective, design, implementation, and operational experience; Funded through the Japanese Social Development Fund, the Community-Driven Nutrition Improvement Program (CDNIP) is a four-year program led by Catholic Relief Services (CRS), in partnership with the World Bank, and implemented by three local partners Caritas Baucau, Timor-Leste Red Cross (CVTL) and Kolega de Paz (KDP). One partner, Fraterna, implemented the aquaculture pilot. CDNIP is implemented in 49 sucos of Baucau and Viqueque municipalities, impacting close to 4,500 children and more than 5,000 pregnant and lactating women. The project agreement was signed in August 2014. Implementation was delayed by six months with the World Bank’s permission, to finalise the Operational Manual and procurement plan. Consequently, the end date of the project was also shifted from 31 August 2017 to 31 August 2018. CDNIP’s project development objective (PDO) is to improve nutrition practices targeted to children under the age of two years old and pregnant and lactating women in the Asian Development Bank identified least developed communities (sucos). Project Design The problem of stunting in Timor-Leste is among the worst in the world. One of the significant drivers of stunting is an inadequate regular intake of nutritious food amongst infants and small children17. Government stakeholders, CRS and partner staff, and now community members acknowledge that Timorese food preparation usually consists of only carbohydrates, which is insufficient to meet full nutritional needs. This is understood to be central to poor health of children and the wider population. Before the project, nutrition was not appreciated as a priority by residents of Baucau and Viqueque municipalities. Therefore, the project design appropriately targeted increasing awareness, knowledge, and self-reliance to produce nutritious food for children under two as well as pregnant and lactating women. Project Implementation and Effectiveness Project implementation was a success. The project accomplished all of its’ community awareness and mobilization objectives. In many instances it significantly over-achieved against its targets. However, the plans to facilitate bespoke Community Action Plans at the beginning of the project were adapted to be a select range of agricultural initiatives. The project’s nutrition-related objectives were attained, while all agriculture-related objectives were attained except for one related to bean harvesting. The endline evaluation conducted in May 2018 found that CNF were sufficiently trained on topics of maternal and infant nutrition. It also found that they had good coverage of the women in their aldeias with 94% of mothers being familiar with their work. 80% of mothers attended at least two mothers’ group sessions in the last two years, mobilized by the CNF, and 95% of those found the meetings useful. Regular attendance was much lower with only 11% attending a meeting at least once every two months, although there was follow up home visits by the CNF to convey session messages. The project, and the use of CNF, addressed a hitherto unmet need for nutrition education in these municipalities, given that three quarters of mothers cited the project’s CNF as their main source of information about 17 World Health Organisation. At URL: http://www.who.int/nutrition/topics/globaltarges_stunting_policybrief.pdf Page 23 of 32 The World Bank Community Driven Nutrition Improvement (P145491) nutrition. The project targeting did not strongly include other people who influence the decisions of pregnant and lactating women. Such people, like mothers, mothers-in-law and husbands are less likely to give good advice to these women. What’s more, the lack of access by pregnant women to antenatal counselling in the government health system also meant the project lacked natural allies to reinforce the project’s messages. • Assessment of the outcome of the operation against the agreed objectives; Achieving higher-level outcomes is much more difficult than meeting implementation targets because changing behaviors in communities is affected by complex internal, societal and environmental dynamics that are beyond the project’s control. Nevertheless, the project met or exceeded two of its four Primary PDO indicators and sustained baseline levels in one (exclusive breastfeeding). PDO 1: What is the proportion of children 0 - 5 months of age who are exclusively breastfed? Exclusive breastfeeding (EBF) is vital for protecting infants from disease, and a major objective of the project. The proportion of mothers practicing EBF was unchanged since baseline. However, focus groups revealed that women in project communities have a much greater understanding and appreciation of the importance of EBF now, compared to before the project. Given that rates of EBF have been declining across Timor-Leste in recent years (from 62% 201318 to 50% in 201619) maintaining these rates in the project areas is, in some ways, a success but still an area requiring significant work nationally and in the project sites in the future. The project recorded a significant increase in the proportion of children put on the breast within the first hour of birth, from 65% at baseline, to 77% at endline (p=0.000). Almost all of this improvement was observed in Baucau. This is also a shift that should contribute to reducing infant morbidity and mortality. PDO 2: What is the proportion of children 6-23 months provided with all 3-minimum infant and young child feeding practices? The minimum acceptable diet requires children to receive: (i) at least four of the seven food groups plus (ii) the age appropriate number of meals, plus (iii) the appropriate milk feeds. Overall, the project achieved significant improvements in the diets of breastfed and non-breastfed children. The proportion of breastfed and non-breastfed children 6-23 months receiving all three components of the minimum infant and young child acceptable diet increased. For breastfed infants this rose from 3.1% at baseline to 18% at endline. For non-breastfed infants, this rose from 0% at baseline to 1.7%. For non-breastfed children, the cause of disadvantage compared to breastfed children is lack of provision of milk feeds. While, at endline, almost a quarter of non-breastfed young children (23%) received at least the minimum dietary diversity and number of meals, the third requirement for this group to meet the minimum feeding practices, or to have two or more milk feeds, was only 1.7%. The proportion of children who received the minimum dietary diversity (food from four or more food groups in the 24 hours prior to the survey) increased six-fold from 3.9% of all children (aged 6-23 months) at baseline to 23% at endline. The most commonly consumed food groups in the previous 24 hours were grain-based foods (primarily sasoro – rice porridge), vitamin A-rich fruit and vegetables, 18 Timor-Leste Food and Nutrition Survey 2013 19 Timor-Leste Demographic and Health Survey 2016 Page 24 of 32 The World Bank Community Driven Nutrition Improvement (P145491) other fruit and vegetables, and eggs. PDO 3: What is the proportion of children under two years with diarrhoea who are provided continued feeding along with ORS or increased fluids? By endline, the proportion of children with diarrhoea whose parents gave them ORS and/or increased fluid intake increased from 82 to 87% (p=0.35 – not significant) and giving zinc supplements increased from 0 to 11% (p<0.001 – extremely significant). Yet, the evaluation recorded a deterioration in the overall target indicator of the proportion of households who correctly treat children under 2 years old experiencing an episode of diarrhoea by providing continued feeding along with ORS or increased fluids. This reduced from 54% at baseline, to 31% at endline. The cause of the decline was due to a reduction in the proportion who provide continuous feeding during diarrhoea. In relation to diarrhoea prevention, those who understood that handwashing is an effective way to reduce exposure to enteric illnesses almost doubled since baseline from 47% to 81%. Still, many other factors that cause diarrhoea were outside the scope of the project and not addressed in communities and rates of diarrhoea in targeted communities has increased since baseline. PDO 4: What is the proportion of mothers of children under two years of age who consumed iron- rich foods? As we saw with young children’s diets, the benefits of improved dietary knowledge translated to improved maternal nutrition as well. The proportion of mothers of children under two years old who consumed iron- rich foods more than doubled over the lifetime of the project from 15% to 37% of mothers. In addition to the improved intake of iron rich foods, overall 96% of mothers consumed at least three meals in the previous 24 hours. Though an encouraging result, no comparative data were presented at baseline. Attribution: Endline survey respondents were asked who else supported them with assistance to “feed your children better, in the last one or two years�. Half (53%) of all mothers reported that CDNIP was the only source of such advice and support. The remaining 47% reported there was support from one or more other source. 25% observed that one or more MOH function gave nutrition advice as well. 8% received some form of support from the MAF, and 23% received some support from another NGO, though the latter may refer to the partner implementing NGOs. Thus, overall, we conservatively calculate that CDNIP is responsible for at least three-quarters of the positive influence evidenced in this evaluation. When asked what factors reduced families’ ability to feed their children, 51% of mothers cited very dry weather, affecting production. This observation is supported by climatic measurements of an El Niño drying effect over Timor-Leste in recently years. • Evaluation of the borrower’s own performance during the preparation and implementation of the operation, with special emphasis on lessons learned; CRS has been successful in carrying out its implementation plan. All intermediate results were achieved by the end of the project, with the exception of one: harvesting beans. Two key factors constrained impact beyond the control of CRS and implementing partners: 1) the limited timeframe of the project implementation period; and 2) the extremely dry conditions triggered by the El Niño event that Page 25 of 32 The World Bank Community Driven Nutrition Improvement (P145491) occurred during program implementation. Nevertheless, the project model, management and implementation partners have been very successful in embedding positive changes to knowledge and dietary norms for infants, young children and pregnant and lactating women, in line with its intended results. This is evidenced by the achievement of major improvements in the quantity and diversity of nutritious foods consumed in those segments of the community. Even where the indicator results could not show progress, the project has accomplished major shifts in some of the dynamics that will enable those indicators ultimately to change. Specifically, in regard to exclusive breastfeeding, the levels of understanding and respect for the need for EBF was very high; and in relation to correct treatment of children with diarrhoea, the project increased the proportion of families who increase fluid intake, administer ORS, and zinc supplements. Despite these achievements, the MoH reminded the evaluators that rates of stunting are still high in these communities, indicating that food security and nutrition are still priority sectors requiring long term investment. With the foundations laid by this project in these 49 sucos, and the lessons learned along the way, much more improvement in IYCF and maternal nutrition can be accomplished in the near future with sustained or increased program interventions by CRS and/or other community development partners. Challenges: 1. There were significant challenges in the M&E system in the first half of the project with confusion around data collection procedures. After the mid-term review the M&E system was revised and increased human resources and accompaniment provided to implementing partners. This improved the quality of M&E products and also improved the capacity of partners and improved the relationship between CRS and implementing partners. 2. The government did not feel that the project adequately engaged them, likely due to engagement being focused on specific areas rather than continuous engagement throughout the project in all project activities. The main area of collaboration was on chicken vaccinations with MAF, where CRS worked with MAF to train CNFs to provide chicken vaccinations to over 2,700 beneficiaries. CRS also worked with WFP and MoH to increase reach of messaging around supplements available at clinics. Some CNF’s were also Community Health Workers, and CRS helped inform and coordinate the rollout of the Viqueque Primary Health Care Curriculum with HAI and MoH, as well as printing 65,000 Lisios (Mother Child Book) and installing 51 water filters in clinics in Baucau and Viqueque at the request of MOH.CRS would like to have engaged more with governments officials, particularly with MoH, but it was unclear how the government wanted to be involved. Such expectations could be clarified in the future through a higher-level MOU with MoH at the national level that clearly spells out roles and responsibilities. 3. Involvement of men in the project was limited. Men were generally of the view that the project was for mothers, not for men. Men thought the project had been for mothers only and they had not participated. In other locations there was some involvement of men and some showed interest in the agriculture activities. The lack of involvement of men had made it difficult for Page 26 of 32 The World Bank Community Driven Nutrition Improvement (P145491) the implementing partner staff who sometimes had to do the heavy physical work preparing demonstration agriculture plots that was too demanding for pregnant and lactating women. CRS and the partners recognised this shortcoming and see this as one of the major lessons. 4. Another operational lesson is the relatively short timeframe of the project which made it challenging to implement the full spectrum of training modules across so many remote villages. A longer timeframe would also have allowed for some review and follow up and identification of refresher trainings. For the 30 sucos that CRS will continue to work with under TOMAK, this opportunity will be realised. Lessons: 1. Use of change agents drawn from the communities themselves, such as CNFs should be preserved, although using them more as community mobilisers and focal points, rather than as the main training resource for communities, is likely to be easier to manage and allows great control of training quality and consistency. 2. Interactive learning such as cooking demonstrations should be the norm, rather than the exception to promote attendance, participation and learning. 3. Building new practices on existing, familiar and simple practices was a highly successful strategy to generate adoption. In this project, CNFs promoted modifications to the preparation of sasoro (porridge). This was universally accepted because it required little change, compared to the low likelihood of change if they were taught completely new recipes or food styles. 4. Some FGDs referred to selling some of their vegetable production. In almost every rural community, vegetables attract a higher profit at market than staples and grains. The potential for income as well as added nutrition can be used as an incentive to increase interest and increase men’s motivation and involvement in vegetable production. • Evaluation of the performance of the Bank, any co-financiers, or of other partners during the preparation and implementation of the operation, including the effectiveness of their relationships, with special emphasis on lessons learned; CRS appreciated the support of Team Leader Puti Marzoeki and the World Bank team in Dili, who provided guidance throughout the process from project start up to project close, especially in helping CRS understand the JSDF requirements. Puti was flexible with revising monitoring indicators to reflect realities on the ground, and willing to adapt to the different processes associated with working with a non-governmental organization. The CRS team felt that the World Bank team was very constructive and helpful. CRS also appreciated the frequency of six monthly reports, as well as biannual visits from the Bank. The participation of the World Bank’s Country Representative in one of the visits to the program sites was also greatly appreciated as a sign of support at the local level. One challenge was that the Bank required CRS to segregate their funds in a separate bank account. CRS understands this is a World Bank requirement, but it was time consuming and takes extra effort to manage. Another challenge was the amount of time needed to set up the POM, which unfortunately delayed implementation by six months. CRS, however, found the Manual very useful once in place to Page 27 of 32 The World Bank Community Driven Nutrition Improvement (P145491) provide guidance on processes. Finally, CRS would recommend more team coordination in general because of some process such as procurement, finance was needed, and communication involved both the CRS and the Bank. Overall, CRS felt that the relationship was very constructive and positive and appreciates the World Bank for their collaboration and support. • Description of the proposed arrangements for future operation of the project Knowledge and practices that do not require external inputs are promising. Such practices are likely to become embedded in the collective wisdom of most communities, with potential for expansion within and between communities. In particular, the new knowledge and practices around giving a nutritionally diversified diet to children over 6 months every day, giving colostrum to newborns, continuing to breastfeed beyond 6 months and awareness about the reasons for EBF. Project accomplishments that have a more fragile likelihood for sustainability relate to food production, exacerbated by consecutive years of El Niño and the adverse effects of climate change. Families are harvesting some crops too early, not saving the seeds to replant, are not coping with reduced rains and invasive pests, and have no access to agricultural extension beyond the project lifecycle (although 30 of the sucos will receive assistance through CRS’ new project TOMAK). • Monitoring and Evaluation (M&E) Design, Implementation, and Utilization In the first half of the project the large amount of indicators required in the M&E plan made it difficult for implementing partners to provide what was required for reporting and program planning. Forms did not have formats that readily captured the required information resulted in follow up visits to collect missing data. After the MTR, there was a more consistent approach to M&E and additional resources were provided to implementing partners (e.g. hiring of M&E officers). Partners still found the M&E system challenging with too many indicators to report against, but a M&E system was in place and able to provide consistent data for reporting. It was found, however, that the level of skill required to maintain the M&E system is still beyond the skills available in the local labour-market. Therefore, while CRS tried to support the three local partner NGOs’ staff through monitoring and capacity building, the incumbents in the M&E officer roles continued to struggle to complete the required monitoring documentation and provide it on-time and in a satisfactory quality. The paper-based nature of the data collection required someone in CRS to manually enter all the data. This took a long time, and occasionally resulted in data being lost. Even by the end of the project, data was not being provided in a timely manner to assist planning in the next quarter. Suggestions for future M&E systems include limiting the number of indicators in the M&E plan and using tablets for data collection instead of paper. Page 28 of 32 The World Bank Community Driven Nutrition Improvement (P145491) ANNEX 4. SNAP SHOT OF ORIGINAL AND REVISED BASELINES AND TARGETS Table 1: Achievements of the PDO Indicators 2014-2018 Original Baseline Revised Original Revised Results Level of PDO indicators (DHS Baseline Target Target achieved Achievement 2009/2010) (2016)* (2014) (2016)* (2018) 1. Proportion of children < 6 months of age who are 52 71.3 60 75 70 Not achieved exclusively breastfed (%) 2. Proportion of children 6-23 months provided with all 3 30 3.1 40 3.25 18 Surpassed minimum infant and young child feeding practices (%) 3. Proportion of children under 2 years with diarrhea 63 53.8 68 59 31 Not achieved who are provided continued feeding along with ORS and/or increased fluids (%) 4. Proportion of mothers of children under two years of 53 15.4 58 20 37 Surpassed age who consumed iron-rich foods (%) Intermediate Result Indicators Original Original Revised Results Level of Baseline Target Target* achieved Achievement (2014) (2014) (2016) (2018) 1.1 Number of targeted communities where a PRA 0 200 200 262 Surpassed is conducted by the end of project year 2 1.2 Number of community-led action plans for nutrition 0 200 200 238 Surpassed interventions (Component 2&3) available by project year 2 2.1 Number of CNC trained as trainers (Training of 0 17 18 25 Surpassed Trainers) by end of project. 2.2 Number of CNF by CNC per annum. 0 500 350 368 Surpassed 2.3 Number of CNF Learning Sessions conducted by the 0 577 833 3555 Surpassed end of the project. 2.4 % of mothers / caretakers attending small group 0 70 70 73 Surpassed sessions or received home visit by end of project 3.1 % of trained household representatives practicing one 0 80 80 94 Surpassed or more of the recommended planting to post-harvest practices for the production of nutritious staple and minor crops by end of project. 3.2 % of target households producing vegetables through 0 80 70 77 Surpassed homestead gardens per annum by the end of the project. 3.3 % of target households with established homestead 0 70 70 53 Partially protein production per annum by the end of the project. achieved *Revised baseline and target after the baseline survey completed and MTR Page 29 of 32