Report No: AUS0001036 . India Impact Evaluation of JEEViKA Multisectoral Convergence Initiative in Bihar: Engaging women’s groups to improve nutrition Summary Report October 2019 . HNP . . © 2019 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2019. India: Impact Evaluation of JEEViKA Multisectoral Convergence Initiative in Bihar: Engaging women’s groups to improve nutrition. Summary Report. © World Bank.” All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. JEEViKA Multisectoral Convergence Initiative in Bihar: Engaging Women’s Groups to Improve Nutrition Ashi Kohli Kathuria1 The report is a summary of the impact evaluation of the JEEViKA Multisectoral Convergence Initiative in Bihar, India and its findings and recommendations. The full technical impact evaluation report is available at https://www.ifpri.org/publication/engaging-womens-groups-improve-nutrition-findings-evaluation. 1 Senior Nutrition Specialist, World Bank Table of Contents List of Tables .................................................................................................................................. i List of Figures ................................................................................................................................. i Acknowledgments ......................................................................................................................... ii Acronyms and Abbreviations ..................................................................................................... iii I. Introduction ............................................................................................................................. 1 II. The Multisectoral Convergence Initiative Pilot....................................................................... 2 III. Evaluation of the Multisectoral Convergence Initiative Pilot.............................................. 6 IV. Results .................................................................................................................................. 8 V. Costing Study ........................................................................................................................ 21 VI. Discussion: Impact, Implementation, and Financial Cost ................................................. 23 A. Impact: Evidence from Health and Nutrition Outcomes ................................................... 23 B. Implementation: What Worked, What Didn’t, and Why ................................................... 24 C. Costs: How Much did it Cost to Implement the JEEViKA-MC Pilot? ............................. 27 D. Recommendations ................................................................................................................ 27 VII. Conclusions ........................................................................................................................ 30 List of Tables Table No. Page No Table 1 : Estimates of per-unit costs under different scenarios 22 List of Figures Figure No. Page No Figure 1 : Proportion of index children consuming each food group at baseline, by 9 treatment arm Figure 2 : Proportion of index children consuming each food group at baseline 10 and endline, by treatment arm Figure 3 : Proportion of women attaining MDD at baseline and endline, by 11 treatment arm Figure 4 : Proportion of respondent women consuming each food group at 12 baseline and endline, by treatment arm Figure 5 : Proportion of households that engage in correct sanitation and 14 hygiene practices at endline, by treatment arm Figure 6 : Proportion of youngest children meeting core and optional IYCF 15 recommendations at endline, by treatment arm Figure 7 : Proportion of households experiencing the HFIAS conditions at 16 baseline and endline, by treatment arm Figure 8 : Exposure to, trial of, and adoption of message at endline, by 20 treatment arm Figure 9 : Allocation of total costs by activities 22 i Acknowledgments The team is grateful to the Jeevika leadership and team for their sustained partnership in conceptualization, designing and implementing this multisectoral convergence pilot, and their commitment to research to understand the possible value of leveraging the women’s self-help group platform to improve nutrition and health outcomes. We thank the Jeevika team in Saharsa for their constant support during the implementation of the pilot and for facilitating the evaluation. We appreciate the guidance we received from Vinay Kumar Vutukuru and Mohini Kak from the World Bank Jeevika task team and their assistance in resolving implementation issues. On-the-ground assistance from Rajni Khanna, Vinay Singh, Alok Kashyap, and Anand Kothari during the pilot and its evaluation was critical to the pilot implementation, quality assurance and to the research. We are grateful to Vinay Singh for his contributions to the costing analysis in this report. The Impact evaluation was conducted by the International Food Policy Research Institute (IFPRI) and led by Neha Kumar. We gratefully acknowledge the excellent work of the Impact evaluation team from IFPRI - Neha Kumar, Purnima Menon, Kalyani Raghunathan, Shivani Gupta and Shinjini Pandey, and Rasmi Avula, who led the process evaluation that complemented this Impact evaluation. greatly improved the understanding of the pilot. We gratefully acknowledge the support of Shruti Viswanathan, Vinaya Padmanabhan, Vipul Singhal, and the entire team at Oxford Policy Management (OPM) who led the data collection for this impact evaluation. This work has received core financial support from the World Bank’s Strategic Impact Evaluation Fund (SIEF), and valuable technical inputs from the SIEF team from time to time. The Japan Trust Fund for Scaling Up Nutrition provided resources for the Process Evaluation to complement the Impact Evaluation. We are thankful to the peer reviewers, Alaka Holla, Sitaramachandra Machiraju and Anne Marie Provo, of the World Bank for reviewing the research outputs and for their valuable suggestions to strengthen the report. The team gratefully acknowledges the editorial support provided by Patricia Rogers. ii Acronyms and Abbreviations BCC Behavior Change Communication BMI Body Mass Index CC Coordination and Convergence CLF Cluster-Level Federations CM Community Mobilizers CNRP Community Nutrition Resource Persons CRP Community Resource Persons FLW Frontline Workers FSF Food Security Fund HFIAS Household Food Insecurity Access Scale HRF Health Risk Fund HSC Health Subcommittee ICDS Integrated Child Development Scheme IFA Iron Folic Acid IFPRI International Food Policy Research Institute IYCF Infant and Young Child Feeding MDD Minimum Dietary Diversity ORS Oral Rehydration Salts SHG Self-Help Groups VHSND Village Health Sanitation and Nutrition Day VO Village Organizations WASH Water, Sanitation and Hygiene iii I. Introduction 1. JEEViKA is a program that was launched in 2006 by the Bihar Rural Livelihoods Promotion Society (an autonomous body under the Department of Rural Development), with the support of the World Bank. It targets women in poor rural households, helping them improve their livelihoods and enhance household incomes through a core set of interventions: organizing the women into self-help groups (SHGs), training and strengthening the SHGs, federating the SHGs into village organizations (VOs) and cluster-level federations (CLFs), and establishing bank linkages for the SHGs and their federations. Over time, these groups can become membership-based social service providers, business entities, and valued clients of the formal banking system. Over its 13 years of operation, JEEViKA has reached some 1.8 million women in thousands of villages and has proven to be very successful at empowering women through their increased economic contributions. 2. This report presents a summary impact evaluation of a pilot program—the JEEViKA Multisectoral Convergence Initiative—that tested the use of the JEEViKA structure to address undernutrition in women and children. Under the pilot, SHG members received messages about mother and child nutrition and about various safe water, sanitation, and hygiene practices. To assess whether the approach is effective and cost-effective, the evaluation compares changes in practices among these women with those of a similar group that did not receive the intervention. 3. Following this introduction, Section II describes the pilot program, Section III describes the evaluation process, Section IV summarizes the evaluation findings, and Section V analyzes the cost of the intervention. Section VI discusses the impact, implementation, and costs of the pilot and presents recommendations for increasing the effectiveness of similar future programs. Finally, Section VII presents conclusions. 1 II. The Multisectoral Convergence Initiative Pilot 4. The JEEViKA Multisectoral Convergence Model (JEEViKA-MC), developed by the Bihar Rural Livelihoods Promotion Society with technical support from the World Bank, was designed to use the JEEViKA model and build on its structure to address undernutrition among women and children. The program was targeted to women who were members of the SHGs formed by JEEViKA in 12 Gram Panchayats of Saharsa, Bihar, with a special focus on households with young children, mothers of young children, and pregnant women. It added two sets of complementary interventions to the core package of JEEViKA activities. 5. Component 1. Promoting household behavior change. The JEEViKA Community Mobilizers (CMs) were trained to deliver intensive behavior change communication (BCC)— that is, messages on maternal and child nutrition and health, water, sanitation, and hygiene behaviors (see Box 1) at bimonthly SHG meetings. The CMs also received pico-projectors and a series of six videos, developed by Digital Green, to show to community members. These videos covered topics such as the importance of kitchen gardens, the preparation of oral rehydration salts (ORS), different food groups and the importance of dietary diversity, and the need for good hygiene. The BCC delivered by the CMs was complemented by targeted home visits, peer group meetings, and community events organized by the Health Subcommittee (HSC) members—three women per VO, serving on a volunteer basis, who were trained in health and nutrition by health Community Resource Persons (CRPs) and by Master Trainers under the supervision of the World Bank team in Saharsa. Box 1. Content of the BCC messages The BCC messages focused on such topics as maternal, infant, and young child feeding practices; diets during pregnancy; early initiation of and exclusive breastfeeding; timely and appropriate complementary feeding; antenatal and postnatal care; awareness of the benefits of iron-folate supplementation and vitamin A supplementation (for children); institutional delivery; routine immunization; Integrated Child Development Scheme (ICDS) entitlements such as supplemental food; ways of improving household food security through improved use of JEEViKA’s Food Security Fund (FSF); the cultivation of kitchen gardens; and the importance of safe water, sanitation, and hygiene practices, including the use of latrines. Recipients of the BCC were also made aware of the Health Risk Fund (HRF) that individual SHG members can access to finance medical treatment. 6. Component 2. Improving access to and use of key nutrition-specific and nutrition- sensitive 2 public services. This component aimed to (a) create awareness among VO members about key public services and the entitlements of the community, and (b) systematically facilitate members’ access to and use of the services. The component involved creating convergence and coordination committees at the Gram Panchayat, block, and district levels, comprising representatives from several departments: (a) Integrated Child Development 2 Nutrition-specific interventions directly address inadequate dietary intake or disease—the immediate causes of malnutrition. Nutrition-sensitive interventions or development efforts are those that, in the context of sector- specific objectives, also aim to improve the underlying determinants of nutrition (adequate food access, healthy environments, adequate health services, and care practices), or aim at least to avoid harm to the underlying or immediate causes, especially among the most nutritionally vulnerable populations and individuals. 2 Scheme (ICDS) for nutrition services; (b) National Rural Health Mission and District Health Society for health services; (c) Public Health Engineering Department and Rural Development Department for sanitation services; (d) Krishi Vigyan Kendras (farm science centers) and Horticulture Department for agriculture services; and (e) Panchayati Raj Institutions for local governance issues. These committees were scheduled to meet once a month and were expected to provide swift and transparent resolution of supply-side issues; for example, issues related to service provision could be raised at the committee at the appropriate level and escalated through the hierarchy of committees as needed until a solution had been implemented. However, it was found that convening monthly meetings with representatives of so many different departments was time-consuming and occasionally impossible, so that meetings were held very infrequently. Support for the formation of these committees was soon withdrawn. 7. This component also aimed to strengthen existing government platforms. Several platforms—the Bachpan Diwas, the Annaprashan Diwas, and the Village Health Sanitation and Nutrition Day (VHSND), designed to be held monthly at the Anganwadi Center—had already been instituted in all areas but were either nonfunctional or minimally functional before the start of this pilot. Special attention was given to reviving these platforms, which were to be used to reinforce the BCC messages and to promote convergence among the HSC, CMs, and the health and ICDS frontline workers (FLWs). Over time, however, it was found that these events were not being held at the intended frequency, and that the intended services were not being provided. Therefore, the program shifted its focus to using the HSC members to generate awareness and strengthen coordination and provider-community interface. The VO-level HSC was trained to improve convergence and coordination efforts by assisting government FLWs like Anganwadi workers, Accredited Social Health Activists, and Auxiliary Nurse Midwives in their duties. In addition, there was an attempt to revive and strengthen such other institutional mechanisms as the Village Health Sanitation and Nutrition Committee and village-level monitoring committee, HSC meetings, and VO-level problem-redressal meeting. A. Community and Sample Characteristics 8. Among the 599 Indian districts, Saharsa ranks 582 on the District Development and Diversity index, and it is the eighth-lowest district of the 38 districts of Bihar. Our study sample was predominantly Hindu and belonged to either the Other Backward Classes or the Scheduled Caste social groups. The average household size was close to 7 people, and the mean years of schooling for the respondent woman was only 2.2 years. On average, women got married at the age of 17 years and had their first baby at 19.6 years. Households were quite poor, owning only 6 out of a total of 25 possible assets, on average. About 73 percent of the women were not employed. Of the sample, 22 percent reported that their husbands were employed in agriculture, 42 percent in non-agricultural day labor, and 13 percent as migrant laborers. Nearly three- fourths of the women had a bank account, a much higher proportion than the 49 percent of men who had bank accounts. 9. Compared with the average household in Saharsa and Bihar, households in our sample were more likely to be Hindu and to be from either the Scheduled Caste or tribe groups. The households in our sample were larger, the women we surveyed had fewer years of education, on average, and they were less likely to be unemployed. Their husbands were considerably less 3 likely to be engaged in agricultural labor, but also less likely to be either a salaried worker, or unemployed. 10. Less than a quarter of the villages in the sample reported having access to health services: about 23 percent reported having a health subcenter, 5 percent a primary health center, and less than 1 percent a government hospital, dispensary, or nutrition rehabilitation center. About 28 percent of the villages had private clinics run by unqualified practitioners. A large majority of the villages (91-97%) had Anganwadi workers, Accredited Social Health Activists, Auxiliary Nurse Midwives, and JEEViKA CMs. Some villages reported having doctors (13%) and nongovernmental organization (NGO) health workers (7%, increasing to 23% by endline). 11. About 80 percent of the villages had electricity supply for more than six hours a day, and at baseline about 15 percent had irregular electricity supply. By endline, however, close to 96 percent of villages reported more than six hours of electricity supply, and less than 3 percent reported irregular electricity. Less than 30 percent of the villages cited “piped water in the yard” as the main source of drinking water, and about 65 percent reported a tube well as the main source of drinking water—that is, household members had to go to the water source and fetch water for various purposes. Even though this is less ideal than having piped water in the yard, tube well water is safer than other sources of water found in rural India (such as rivers or open wells). Almost all villages reported the presence of open defecation in the village at baseline, although the practice declined, with approximately 88 percent of the villages reporting it at endline. 4 Box 2. Community Mobilizers and their role in the BCC delivery Community Mobilizers (CMs) are a JEEViKA cadre of women recruited from the communities they serve. They are often themselves SHG members, and they need to have passed grade 8. Their main responsibility within the JEEViKA system is to keep the books for the SHGs they work with. Each CM covers about 12-15 SHGs, all located near her own home. She is expected to attend SHG meetings and to record in her ledgers all savings and credit transactions made – who has contributed money to the central pot, who was loaned money, who has repaid a loan, and so on. When the JEEViKA-MC pilot was first designed and its feasibility tested, the role of disseminating health and nutrition BCC was assigned to another cadre of women, JEEViKA Sahelis (saheli means “friend” in Hindi). However, it was deemed infeasible to retain an entire cadre of workers specifically for this purpose, so the CMs were assigned the additional responsibility of the BCC delivery. CMs in the treatment area therefore received training on the BCC content and delivery in addition to their training on their responsibilities as bookkeepers for the SHGs. In addition to their basic salaries for the bookkeeper work, they were to receive an additional monthly incentive for the health and nutrition BCC sessions they conducted, each designed to take about an hour to deliver. Finally, video dissemination was introduced about a year into the intervention. Videos on six different topics were prepared, and each CM was given a pico-projector and training on how to screen the videos. Videos were screened in a public place—for example, on the wall of the school or Anganwadi center—for members of the community. Since this required an additional visit to the villages in the evenings the CM was to receive a 5 III. Evaluation of the Multisectoral Convergence Initiative Pilot 12. The feasibility phase of the program was conducted from March 2014 to December 2015. For the pilot program, we chose from the Saur Bazaar, Sonbarsa Raj, and Pattarghat blocks of Saharsa district, Bihar, 24 Gram Panchayats that had mature JEEViKA groups (those with which work on the core program began in 2011) but did not have Community Health and Nutrition Care Centers. From these 24 Gram Panchayats, 12 were selected at random to receive the JEEViKA-MC pilot treatment interventions (“treatment arm”), with the other 12 Gram Panchayats serving as controls (“control arm” or “comparison arm”). The JEEViKA-MC pilot was then conducted in these 12 Gram Panchayats from June 2016 to August 2018. 13. To assess the impact and effectiveness of the pilot, the International Food Policy Research Institute (IFPRI) and the World Bank designed a randomized controlled trial whose findings could inform the potential scale-up of the program (and of other health and nutrition BCC programs like it) in Bihar and elsewhere. The evaluation aimed to answer the following questions: • Do the JEEViKA-MC interventions lead to improved nutrition outcomes, as measured by the improved body mass index (BMI) of women of reproductive age? • Do the JEEViKA-MC interventions improve the health, hygiene, and nutrition knowledge and practices of SHG members and mothers of young children? • Do the JEEViKA-MC interventions increase the use of government health, nutrition, and sanitation programs and of JEEViKA food-security-related services? 14. Expected outcomes. The primary outcomes assessed in this study were improved BMI for women and greater dietary diversity for children aged 6-23 months. In addition, a number of related secondary outcomes were assessed: • Women o Dietary diversity o Health, hygiene, and nutrition knowledge and practices • Children aged 6-23 months o Anthropometry (height-for-age, weight-for-height, and weight-for-age Z- scores; stunting, wasting, and underweight) o Infant and young child feeding (IYCF) practices o Morbidity • Household o Household food security measured by the Household Food Insecurity Access Scale (HFIAS) o Use of government programs and of JEEViKA food-security-related services o Hygiene and sanitation practices, including handwashing and use of latrines 15. Evaluation design and methods. The impact evaluation consisted of two rounds of panel data—a baseline survey conducted in April-May 2016, and an endline survey conducted in October-November 2018. In addition, a mixed-methods process evaluation was conducted 6 in April–May 2017 to provide insight into implementation challenges and to allow for any necessary midcourse corrections. 16. Survey sampling. From each Gram Panchayat in the study we randomly selected five villages, and from each village we selected 25 households that had a woman who was a member of a JEEViKA SHG and a woman with at least one child aged 6-23 months. 17. The baseline survey was carried out in 131 villages. We interviewed 2,246 households with respondent women who met the sampling criteria—1,164 in the treatment areas and 1,082 in the comparison areas. We measured the height and weight of the respondent woman and all children under 2 years of age in the household. We denoted the youngest child between 6 and 23 months the “index child.” 18. At endline, we revisited all 2,246 baseline households and were able to re-interview 2,119 of them (those with baseline respondent women available), for an attrition rate of only 5.65 percent. We collected anthropometric data for 2,116 respondent women from the baseline, re-interviewed the mothers of 2,084 index children (35 were not alive), and collected anthropometric data for 2,006 index children from the baseline. In addition to the index child, if the mother had given birth to one or more children since the baseline, at endline we collected information on the youngest of those children between the ages of 6 and 23 months. There were 805 such youngest children, and anthropometric data were available for all of them, with no dates of birth missing. 19. Detailed information on the evaluation design and methods is available in the Impact Evaluation report (please refer to the link provided on page 1). 7 IV. Results A. Primary Outcomes 20. This section examines whether the JEEViKA-MC interventions led to improved nutrition outcomes, reporting the effects on women’s BMI and on reported dietary diversity for children. 1. Women’s BMI 21. At baseline, the average BMI among all women in the sample was 19.07 (± 2.3), and there were no significant differences in BMI between women in the treatment and comparison arms. 3 Overall, at baseline, slightly over half the respondent women were of normal weight and 44 percent were underweight. At endline, the proportion of women who were of normal weight increased in both arms, but more in the comparison arm. The proportion who were underweight also declined in both arms, again slightly more in the comparison arm than in the treatment arm. Thus, given the negative but insignificant results, we find no impact of the pilot interventions on women’s BMI or on the probability that a woman is underweight. 2. Children’s Dietary Diversity 22. We examined reported dietary diversity for both the index child and the youngest child. For the index children we report results on the number of food groups consumed in the previous 24 hours, and for the youngest children we report both the number of food groups and whether the child achieved minimum dietary diversity (MDD)—that is, consuming at least four out of seven food groups. 4 23. Index child. At baseline, reported dietary diversity among the index children was quite poor in both arms (Figure 1). A large proportion consumed grains (78% overall) and dairy (61% overall); only a small proportion consumed other food groups such as fruits, vegetables, and pulses (38-39%) and vitamin A-rich fruits and vegetables (24%); and consumption of eggs and flesh foods was negligible. We found that 82 percent of the children were not identified as vegetarian; therefore, the low consumption of eggs and flesh foods is likely more a result of limited resources than of preference. The mean number of food groups consumed by the index children at baseline was low at 2.45 (± 1.46) food groups, and only 25.9 percent of these children satisfied MDD. 3 Underweight = BMI < 18.5 kg/m2), normal = 18.5 < BMI < 25 kg/m2, and overweight = BMI > = 25 kg/m2. 4 The seven food groups for children are grains, pulses, dairy, eggs, flesh foods, vitamin A-rich fruits and vegetables, and other fruits and vegetables. 8 Grains 77 79 Dairy 59 63 Other fruits and vegetables 39 41 Pulses 38 Comparison arm 40 Treatment arm Vitamin A rich fruits and vegetables 23 25 Flesh foods (meat, chicken, fish) 4 3 Eggs 0 0 0 20 40 60 80 100 Figure 1. Proportion of index children consuming each food group at baseline, by treatment arm 24. The pilot succeeded in improving the reported dietary diversity of these index children: there were positive and significant (albeit small) increases in the number of food groups these children consumed. As Figure 2 shows, this effect was seen across the board. The proportions of index children consuming almost all food groups increased. The largest increases were in pulses (both arms), vitamin A-rich fruits and vegetables (treatment arm only), other fruits and vegetables (treatment arm only), and dairy (both arms). The dramatic improvement in consumption of pulses between survey rounds is commendable, though it occurred in both arms and hence cannot be attributed to the treatment alone. An increase in consumption as the child grows older might also have contributed. However, the consumption of animal-sourced foods remained very low. 9 100 Grains 100 77 79 73 Dairy 81 59 63 91 Other fruits and vegetables 93 39 41 84 Pulses 86 38 40 25 Vitamin A rich fruits and vegetables 37 23 25 12 Flesh foods (meat, chicken, fish) 8 4 3 2 Eggs 2 0 0 0 20 40 60 80 100 Endline Comparison arm Endline Treatment arm Baseline Comparison arm Baseline Treatment arm Figure 2. Proportion of index children consuming each food group at baseline and endline, by treatment arm 25. Youngest child. The pilot had a positive and significant impact on the number of food groups consumed by the youngest children—an 8.4 percent increase over the comparison arm mean. (Because these children were born after the start of the pilot, we do not have information on them from the baseline.) At endline, 58.3 percent of children consumed four or more food groups (61.9% in the treatment arm, 54.5% in the comparison arm), a large improvement over baseline, in which only 22.6 percent of all index children achieved MDD (24.1% in the treatment arm, 20.9% in the comparison arm). It is worth noting that the youngest children in the comparison arm at endline were consuming a higher number of food groups than did the index children of similar age in the comparison arm at baseline (3.41 versus 2.23). This improvement in child diets over time may indicate a trend in improved diets or may reflect the impact of seasonality, since the baseline and endline surveys were conducted at different times of the year. B. Secondary Outcomes 26. This section examines whether the JEEViKA interventions improved the health, hygiene, and nutrition knowledge and practices of SHG members and mothers of young children. 10 1. Outcomes for Women 27. Dietary diversity. At both baseline and endline we collected 24-hour recall data on the respondent women’s consumption of 10 food groups—grains; white roots, tubers, and plantains; pulses; nuts and seeds; dairy; meat, poultry, and fish; eggs; dark green leafy vegetables; vitamin A-rich fruits and vegetables; and other fruits and vegetables. MDD for women is defined as consuming 5 of these 10 food groups. At baseline, the proportion of women attaining MDD was 27.4 percent in the treatment arm and 34 percent in the comparison arm (Figure 3). By the time of the endline survey, the proportion of women attaining MDD in the comparison arm did not move much, increasing only to 35.4 percent. However, the proportion in the treatment arm showed an impressive increase of almost two-thirds to 47 percent. Our endline impact estimates corroborate this finding, showing a 10.3 percentage point increase in the likelihood of a woman’s achieving MDD. This is a substantial improvement over the course of the 26 months of the intervention period. Overall, it appears that while the pilot was not successful in moving women’s BMI, it was very successful in improving the quality of their diets. 50 47.1 45 40 35.4 34 35 30 27.4 25 20 15 10 5 0 Treatment arm Comparison arm Baseline Endline Figure 3. Proportion of women attaining MDD at baseline and endline, by treatment arm 28. At baseline, the consumption of various food groups was balanced across arms (Figure 4). All women reported consuming starchy staples (grains, roots, and tubers), depending on cereal calories as a primary source of energy. In both arms, only around one-third of women reported consuming dark green leafy vegetables and vitamin A-rich fruit and vegetables. The food groups in which there was improvement over time are pulses, dairy, other fruits, and other vegetables (Figure 4). The proportion consuming nuts and seeds increased as well, but the overall numbers are very low. There was not much improvement in the consumption of flesh foods or eggs, and the consumption of dark green leafy vegetables declined in both arms. And 11 although 77 percent of the women reported that they were not vegetarians, a much smaller proportion reported actually eating flesh foods in the 24 hours before the survey. This suggests that resource constraints may be more salient for these households than lack of information about these foods. 2 2 Eggs 0 0 5 9 Nuts and Seeds 1 2 15 9 Meat, poultry and fish 11 9 93 93 Other vegetables 70 69 17 26 Dark green leafy vegetables 34 33 10 14 Other vitamin-A rich fruits and vegetables 38 34 65 72 Dairy 55 52 83 84 Pulses 69 62 23 38 Other fruits 10 11 100 99 Grains, white roots and tubers, and plantains 100 100 0 20 40 60 80 100 120 Endline Comparison arm Endline Treatment arm Baseline Comparison arm Baseline Treatment arm Figure 4. Proportion of respondent women consuming each food group at baseline and endline, by treatment arm 29. Improved health, hygiene, and nutrition knowledge. When we administered a knowledge module to respondent women, women in the treatment arm scored higher in knowledge both overall and in all areas. The differences between arms were not large, and even the comparison arm scored a 73 (out of 100) on the overall knowledge test. However, the results indicate that women in the treatment arm significantly improved their knowledge around child feeding, dietary diversity, and kitchen gardens, a key intermediate step in improving diet quality for mother and child. The section on child feeding included questions on the appropriate age to feed a child a range of different foods, and the section on dietary diversity and home cultivation asked about the benefits of various types of foods (for example, green leafy vegetables, vitamin-A rich fruits and vegetables), the components of a tri-colored meal, and the vegetables and fruits that can be grown in the kitchen garden at different times of the year. 30. Improved health practices. To examine the impact of the pilot on health and hygiene practices, we collected information on three indicators for the index child: whether the mother provided oral rehydration salts (ORS) or zinc when the child had diarrhea, and whether mother and child participated in the VHSND in the three months before the survey. Although the treatment had no impact on two of these three indicators, the likelihood that the mother gave 12 ORS when the child had diarrhea was 14.3 percent higher in the treatment group than the endline mean in the comparison arm. 31. For the youngest child, the pilot did not appear to affect the likelihood that the pregnancy was registered with a health worker, or that the birth was in a health facility. However, these two outcomes are highly prevalent even in the comparison arm: of the comparison arm mothers, 99 percent registered their pregnancy and 83 percent gave birth in a private or government health facility. Thus there was little room for improvement in these two indicators. There was also no impact on the number of times antenatal care was provided during pregnancy, on participation in the VHSND in the three months before the survey, or on the likelihood of the child’s having an immunization card—a practice that is also very widespread, even in the comparison arm: 93 percent of the children had a health card. And while there was no impact on the likelihood that the mother would give the child zinc during diarrhea, the likelihood of giving ORS was 26 percent higher among women in the treatment group than among those in the comparison arm. 32. Finally, the pilot treatment had some encouraging impacts on the consumption of iron folic acid (IFA) and receipt of calcium tablets during pregnancy with the youngest child. While there was no impact on the likelihood of consuming IFA, the number of IFA tablets consumed increased significantly. There was also a small increase in the likelihood that the mother received calcium tablets, and a sizable and significant increase in the number of days on which the mother took calcium tablets (90 percent more days of calcium tablet consumption during pregnancy than the comparison arm mean). It is true that increased consumption of IFA tablets and receipt of calcium tablets may also be dictated by the supply of these services and not solely by mothers’ demand, and it is possible that the pilot’s convergence component is responsible for strengthening service delivery in these areas. Even so, it appears clear that given an adequate supply, greater knowledge of the importance of IFA seems to have translated into greater consumption. 33. Hygiene and sanitation practices. The study looked at the extent to which households always treat water to make it safe to drink, use the correct practices to treat water, use correct materials to wash hands, use an improved drinking water source, dispose of children’s stools correctly, and use an improved toilet. As Figure 5 shows, the proportions of households that reported using the right materials to wash hands and having an improved drinking water source were high, above 95 percent, while the proportion that correctly disposed of children’s stools or treated water to make it safe to drink was very low. Of the households that treated water, 70 percent or more used correct practices. The treatment arm did better than the control on several of these indicators, though the differences were small. The high level of knowledge about sanitation in both arms is likely a result of the Bihar government’s concerted push to improve sanitation. 13 97.08 94.53 99.36 98.92 100 90 78.23 80 70.45 70 60 47.45 50 40 35.39 29.15 30 20.44 20 9.69 12.3 10 2.74 4.3 0 Always treat Uses correct Uses correct Uses an Uses an Uses correct Uses correct water to make practice to treat materials to improved improved toilet method to method to it safer to drink water wash hands drinking water (N=1096,1023) dispose of dispose of (N=1096,1023) (N=147,132) (N=1096,1023) source index child's youngest (N=1096,1023) stools child's stools (N=971,910) (N=423,382) Treatment arm Comparison arm Figure 5. Proportion of households that engage in correct sanitation and hygiene practices at endline, by treatment arm 2. Outcomes for Children 34. Infant and young child feeding (IYCF) practices for the youngest child under the age of 2 years. For these children we examined core IYCF indicators—early initiation of breastfeeding; continued breastfeeding at 1 year; introduction of solid, semi-solid, or soft food; MDD; and consumption of iron-rich food—as well as two optional indicators, ever breastfed and continued breastfeeding at 2 years. Figure 6 shows the proportion of youngest children meeting these core and optional indicators at endline. Adherence to four of the IYCF recommendations—continued breastfeeding at 1 year; introduction of solid, semi-solid, or soft food; ever breastfed; and continued breastfeeding at 2 years—was high, but the proportion of children who met MDD or consumed iron-rich food was quite low. The treatment arm appears to have done better on all indicators except continued breastfeeding at 1 year, though the differences were small. For the core practices, the pilot seems to have had no impact on early initiation of breastfeeding or continued breastfeeding at 1 year—both practices were reasonably widespread even among women in the comparison arm—or on the achievement of MDD or the consumption of iron-rich food. There was a large and significant impact on the introduction of solid, semi-solid, or soft food. For the optional practices, the pilot does not appear to have had an impact on the likelihood of the child’s ever being breastfed, but it appears to have increased the likelihood of continued breastfeeding at 2 years. 14 100 95.05 98.21 95.27 92.93 88.89 86.42 90 74.94 77.22 80 69.63 70 63.64 61.94 60 54.45 50 40 30 20 12.29 8.9 10 0 Early initiation Continued Introduction of Minimum Consumption of Ever breastfed Continued of breastfeeding breastfeeding at solid, semi- dietary iron-rich food (N=423,382) breastfeeding at (N=423,382) 1 year solid or soft diversity (N=423,382) 2 years (N=101,112) food (N=36,22) (N=423,382) (N=81,79) Treatment arm Comparison arm Figure 6. Proportion of youngest children meeting core and optional IYCF recommendations at endline, by treatment arm 35. Anthropometry among children. At endline we found no impact of the pilot interventions on child anthropometric Z-scores for the index children or for the youngest children. However, there was an increase in the likelihood of being stunted. 36. Morbidity among children. Decreased morbidity among children was expected to be one of the key secondary outcomes of this pilot. However, the pilot did not have an impact on the likelihood of the index child’s suffering from two conditions of ill health, fast/short breathing and cough, and there was a 4.5 percentage point increase in the likelihood of the index child’s suffering from diarrhea. For the youngest child, the likelihood of suffering from a cough increased by 9 percentage points, and the likelihood of suffering from fever by 5.7 percentage points. 3. Outcomes for Households 37. A key determinant of health outcomes is food security within the household. To assess this, we administered the Household Food Insecurity Access Scale (HFIAS), which was developed and validated by the Food and Agriculture Organization. We used the responses to this series of nine questions to calculate both an overall food insecurity score and indicators for three separate HFIAS domains—anxiety and uncertainty, insufficient food quality, and insufficient food quantity. As Figure 7 shows, at baseline more than 40 percent of the households in both arms experienced food insecurity as measured by the three HFIAS domains: this population is highly food-insecure. Surprisingly, the proportions experiencing anxiety and uncertainty and insufficient quality actually increased in both arms by the endline. However, the most extreme form of food insecurity—insufficient food intake or quantity—declined substantially in both arms by the endline. There do not appear to be large differences across arms at either baseline or endline. Therefore, we find no impact of the pilot on the experience of food insecurity by the treatment arm in any domain. 15 100 90 80 74 72 70 66 64 60 52 48 51 50 45 40 41 40 30 20 22 20 10 0 Anxiety and uncertainty Insufficient food quality Insufficient food intake Baseline Treatment arm Baseline Comparison arm Endline Treatment arm Endline Comparison arm Figure 7. Proportion of households experiencing the HFIAS conditions at baseline and endline, by treatment arm C. Use of Government Schemes and JEEViKA services 38. This section examines whether the JEEViKA-MC interventions increased the use of government health, nutrition, and sanitation programs and of JEEViKA food-security-related services. 39. Government schemes. One of the aims of the pilot was to increase participants’ awareness and use of government services. To assess the impact of the pilot in this area, we asked respondent women about their use of the services provided by Anganwadi Centers under the ICDS program, specifically for the index child. The JEEViKA-MC pilot had no impact on the likelihood of the child’s being enrolled in the Anganwadi Center, of the child’s attending the Anganwadi Center in the past 12 months or currently attending the Anganwadi Center, or of the mother’s receiving the Take Home Ration meant for feeding herself or the child. The average receipt of these services varied. While 84 percent of comparison arm mothers whose child was enrolled in the Anganwadi Center reported that their child had attended the Anganwadi Center in the past 12 months, only 50 percent or less of mothers received the Take Home Ration for themselves or their child. The intervention had no impact on awareness of the JSY, the Janani Shishu Suraksha Karyakram (JSSK), and the Pradhan Mantri Matritva Suraksha Yojana (PMSMA) or the Public Distribution System (PDS). Households in the comparison groups were slightly more likely to use the Public Distribution System (PDS) than households in the treatment areas. 40. Self-help group (SHG) meetings and loans. Part of the focus of the MC pilot was on strengthening awareness and use of JEEViKA’s core platforms, especially the funds intended for health and food-security-related purposes. We asked respondents about services and 16 platforms that were available in both treatment and comparison areas as part of the standard JEEViKA model, as well as about services that were provided or strengthened only in the treatment panchayats. 41. In both arms, around 73 percent of the women were part of an SHG at baseline and endline and had been SHG members for a little over four years. Most SHGs have savings and credit activities, and most women participated in these activities. Thus the pilot had no impact on a woman’s being an SHG member or on the SHG’s having savings and credit activities. However, women in the treatment areas were significantly less likely than the women in the comparison areas to have never attended an SHG meeting. At endline, more than half of the women had taken a loan from the SHG, with an average loan amount of slightly more than 10,000 Indian rupees (INR). 5 In the treatment arm, about 37 percent of those that took out a loan reported using it for medical expenses and 14 percent reported using it for consumption needs. The proportion who took a loan for their family’s consumption needs was significantly higher in the comparison arm. 42. We asked the women who were SHG members at endline about topics discussed in the SHG meetings. Most women, close to 68 percent, reported discussing savings and credit issues—topics that, along with personal issues, were discussed equally in the SHGs in both arms. About 49 percent of the women reported discussing issues related to water, sanitation, and hygiene (WASH). Among the other topics commonly discussed during these meetings were the use of the Food Security Fund (FSF) to achieve food security (41.5%), kitchen garden cultivation (39.8%), the importance of food and dietary diversity (37.9%), and the use of the Health Risk Fund (HRF) for healthcare (37.1%)—and the treatment arm SHGs consistently discussed these topics with greater frequency. 43. Health Risk Fund. Although a significantly higher proportion of women in the treatment arm were aware of the HRF (55.02% vs. 46.14%), the use of the fund was low: only 19.2 percent and 25.3 percent of women in the treatment and comparison arms, respectively, used the fund for health emergencies. Most of the women did not use the HRF because they did not have a health emergency. Of the women who did request loans, some in both arms had had their request denied, either because the HRF lacked sufficient money or because a loan was given to someone in greater need. 44. Food Security Fund. Approximately 73 percent of treatment arm respondents reported being aware of the FSF, and nearly three-quarters of these respondents had used the fund and received food from the VO. In both of these areas the treatment arm numbers were significantly higher than the comparison arm numbers. In addition, a higher percentage of women in the treatment arm reported that other SHG members had received food from the VO’s FSF. Some women had not requested food because they did not need it, and some had had their request denied—often because of lack of money in the FSF. 45. Impact of pilot. Overall, then, the pilot had no impact on the likelihood that a woman would take a loan from her SHG. There was a small (7 percentage point) reduction in the likelihood of her taking a loan from the HRF—a result that could indicate improved health or earning capacity as a result of the intervention and, hence, a lower dependence on this fund. 5 At the time of writing, 1 US$ was approximately equal to INR 70. 17 Finally, there was an increase of 11 percentage points in the likelihood of someone in the household using the FSF to purchase food. Since using the FSF was specifically encouraged in the treatment arm as a way to improve household access to food, this is a very encouraging result. 46. Kitchen gardens. Kitchen gardens are fairly common in these areas of India, but the model being encouraged under the MC pilot was one in which the garden was cultivated year- round, to aid in household food security and increase the quantity of fruits and vegetables consumed. Nearly two-thirds (62%) of the respondent women in the treatment arm reported ever having a kitchen garden, significantly more than the one-half of the respondent women in the comparison arm who had ever had such a garden. The proportion who currently had a kitchen garden was also higher in the treatment arm (76%) than in the comparison arm (67%). However, the average time for which these gardens had been cultivated was longer in the comparison arm (42.1 months vs. 35.5 months). The primary reason respondents gave for not having a kitchen garden was lack of space or land, followed by lack of time and lack of water. Around 36 percent of the respondent women, and 50 percent of the comparison arm women, reported not having received any information about kitchen gardens. The main source of advice on kitchen gardens was the CM in the treatment arm, and family and friends in the comparison arm. A higher proportion of respondents in the treatment arm received advice on what crops to grow (70.3% vs. 54.5%), how to plant seeds (63.9% vs. 50.7%), how to tend to the plants (60.6% vs. 46.1%), and what inputs to add (57.5% vs. 43.9%). Across both the arms, only a small percentage of respondents had received inputs for a kitchen garden—mostly seeds. 47. Other JEEViKA-MC intervention activities. As part of the JEEViKA-MC pilot, the CMs screened—primarily in their homes—video messages on health and nutrition topics. Exposure to these video messages was much higher among the households in the treatment areas than in comparison areas. In addition, during home visits under the pilot, the HSC, CM, or newer cadres of CRPs and Community Nutrition Resource Persons (CNRPs) provided advice on child healthcare, dietary, and feeding practices, and demonstrated feeding practices and recipes. The prevalence of home visits was almost twice as high in the treatment arm as in the comparison arm. And finally, several community-level events were organized, including nukkad nataks and camps for child checkups/mother counseling. About 12 percent of the respondent women in the treatment arm, and 7 percent of the women in the comparison arm, reported that JEEViKA had organized a community event in the past 6 months. Clearly, the pilot increased the likelihood that women would receive health and nutrition messages by one of these means. D. Exposure to, Trial of, and Adoption of Key Messages 48. This section examines the extent to which the women under the pilot were exposed to specific health and nutrition messages, tried the recommended behavior, and adopted that behavior into regular practice. • Exposure. More than 70 percent of the women in the treatment arm had heard messages on handwashing practices, washing vegetables before cutting, and exclusive breastfeeding. Other major messages they heard included care during pregnancy, child morbidity assistance, and colostrum feeding. About 20-40 percent 18 of women reported awareness about other key health and nutrition messages. Apart from a few messages (e.g., those around feeding colostrum, care of the mother during pregnancy, and exclusive breastfeeding until 6 months), the exposure among the women in the treatment arm was significantly higher than in the comparison arm. • Trial. For 9 of the 12 key messages, more than 90 percent of the women who had heard the message also practiced it at home—and in general, a higher proportion of women in the treatment arm than in the comparison arm. In particular, the women in the treatment arm were more likely than those in the comparison arm to have tried the behaviors of feeding children aged 6–23 months tri-colored food and of growing vegetables in the kitchen garden. • Adoption. Practices for a few messages, like handwashing and washing vegetables before cutting, had been adopted (with prevalence rates significantly higher in the treatment arm). For 5 of the 12 key messages—mother and child eating tri-colored food, adding oil to the child’s food, handwashing, and vegetable-washing practices—more than 80 percent of women who tried the recommended behavior continued to practice it. 49. Figure 8 summarizes exposure to, trial of, and adoption of the key messages disseminated by the JEEViKA-MC pilot. Exposure, trial, and adoption are all higher in the treatment arm than in the comparison arm. Overall, it appears that once a woman is exposed to the message, the trial rates are quite high, although there seem to be barriers to the longer-term adoption of various recommended practices. 19 100 90 80 70 60 50 40 30 20 10 0 Still Practising Ever heard Ever tried Ever heard Ever tried Still Practising Still Practising Still Practising Ever heard Ever tried Ever heard Ever tried Ever heard Ever tried Still Practising Ever tried Still Practising Ever tried Still Practising Ever tried Still Practising Ever tried Still Practising Ever tried Still Practising Ever tried Still Practising Ever tried Still Practising Ever heard Ever heard Ever heard Ever heard Ever heard Ever heard Ever heard Household Children b/w Family One should Vegetables One should It is very A child should A child should One should It is important Pregnant Members 6m-2yrs members give a child should be grow important for be exclusively be fed the add oil to the for the women need should eat tri- should eat tri- should wash ORS when washed before vegetables in a the pregnant breastfed till colostrum (the food of pregnant to eat more coloured food coloured food their hands they are they are cut home/kitchen woman to take the age of 6 initial thick children mother to get food than non- before suffering from garden sufficient rest months yellow milk) between the at least 4 pregnant preparing diarrhoea during ages of 6 antenatal women food, before pregnancy months and 5 check-ups feeding the years done during child and after pregnancy defecation Comparison arm Treatment arm Figure 8. Exposure to, trial of, and adoption of message at endline, by treatment arm (N = 2,119) 20 V. Costing Study 50. This section estimates the costs of providing the JEEViKA-MC interventions. In brief, we identified the different cost categories or line items that may be part of the budget for such a pilot—staff time, material development, training costs, travel costs, management costs, and overheads—and used focus group discussions and key informant interviews with program staff, implementers, and managers to gain a thorough understanding of how the interventions were implemented. 6 We also collected data on the total number of SHGs, women within these SHGs, and the target beneficiaries reached. We distinguished between the two components— the behavior change communication (BCC) Component 1, and the coordination and convergence (CC) Component 2. The intervention had two phases: a feasibility phase of about two years, and the rollout and implementation phase of slightly more than two years (27 months). 51. The total cumulative cost of implementing the pilot was approximately US$420,354. Of the total costs, about 25 percent was spent on overall project activities (those that cannot be attributed to only BCC or CC). The BCC component accounted for 71 percent of the total cost and the CC for 5 percent. A total of 1,591 SHGs were covered in the intervention, and a total of 3,823 target beneficiaries (women and children) were reached. 52. Figure 9 shows the allocation of the total cumulative cost across the feasibility phase, material development and replication, training, implementation, and monitoring. The feasibility phase was quite long and expensive, accounting for over 44 percent of the total cost of the intervention. This is a large fraction of total costs, given the short window of implementation of the pilot, but if implementation were continued over a longer horizon this fraction would decrease. Programs have to invest large amounts of resources in the feasibility phase to design an intervention that can be scaled up; so, in that sense, many of the costs incurred in the feasibility phase can be considered as up-front fixed costs or sunk costs. 53. Table 1 provides estimates of the per-unit costs under different assumptions about the financial cost of the feasibility phase. When we account for 50 percent of the financial cost of the feasibility phase, the per-SHG cost of implementing the pilot drops from US$264 to US$206, and the per-SHG member per-meeting cost for implementing the pilot drops from US$0.49 to US$0.38. However, when we do not include any of the costs incurred in the feasibility phase (treating them as sunk costs), we have a per-SHG implementing cost of US$148, per-beneficiary cost of US$62, and per-SHG meeting cost of US$2.75. 54. Since this is one of the first studies to estimate the cost of implementing a nutrition BCC intervention using SHGs as the delivery platform, we do not have access to cost estimates from other studies to compare the estimated implementation costs of the JEEViKA-MC pilot. 6 Details about the methodology and steps we used in calculating the program costs are available here: https://www.ifpri.org/publication/engaging-womens-groups-improve-nutrition-findings-evaluation. 21 Feasibility phase 8% 5% Material development and replication 44% Training 25% Implementation & Implementation support-BCC Implementation & Implementation support-C&C 12% 6% Monitoring Figure 9. Allocation of total costs by activities Table 1. Estimates of per-unit costs under different scenarios (US$) Scenario Cost per Cost per Cost per SHG target SHG beneficiary meeting Accounting for full 264 110 4.89 feasibility phase costs Accounting for one- 206 86 3.82 half of the feasibility phase costs Accounting for one- 177 74 3.28 quarter of the feasibility phase costs Not accounting for the 148 62 2.75 feasibility phase costs Source: Authors’ calculations. 22 VI. Discussion: Impact, Implementation, and Financial Cost 55. In this chapter we reflect on the performance of the pilot in terms of impact, implementation, and financial cost. We also relate our findings to findings from similar interventions and make some recommendations for programs of this type. A. Impact: Evidence from Health and Nutrition Outcomes 56. The key findings of the impact evaluation are that the JEEViKA-MC pilot had small but significant impacts on reported dietary diversity for both women and children. The intervention appears to have worked through the anticipated pathways, delivering higher exposure to key messages on nutrition through the SHG platform, and contributing to improved knowledge about nutrition and some improved practices among women covered by the pilot program. 57. The health and nutrition BCC component of the pilot seems to have worked relatively well. There were positive impacts on knowledge of health, hygiene, and nutrition practices among women in the treatment arm, as well as positive change in several practices, such as dietary diversity (among women and children), continued breastfeeding at 2 years of age, timely introduction of complementary foods, and provision of ORS to children with diarrhea. Awareness of the HRF and FSF increased, and women in the treatment arm were significantly more likely to use the FSF to purchase food. There was also a small positive impact on cultivating a kitchen garden. There was no impact on preventive/curative health-seeking behaviors (except giving ORS) or sanitation and hygiene practices. 58. However, improvements in dietary diversity and greater uptake of the FSF did not ultimately lead to improvements in anthropometric outcomes, morbidity among children, and household food security. One possible explanation is that the duration of the pilot was too short to reasonably expect impact on anthropometric measures, which are a result of many interrelated factors. A second possible explanation is that, despite improvements, the quantity and quality of food being consumed was not enough to reduce the prevalence of underweight among women and children. Gaps still remain: for example, the consumption of eggs and flesh foods (e.g., meat, fish, or chicken) was very low even at endline, even in families that self- identified as non-vegetarian. The fact that more than 50 percent of women did not meet MDD even in the treatment arm suggests that more effort is needed to close barriers to consumption and to improve dietary diversity and consumption of iron-rich foods in such settings. 59. The findings of our evaluation are largely consistent with what other such studies have found: although several studies found an impact of group-based interventions on dietary diversity either for mothers or for children, very few found an impact on anthropometric outcomes or on morbidity. 7 These various studies suggest that although improvements in 7 See, for example, Kumar, N., Raghunathan, K., and Menon, P. (2017), An Impact Evaluation of the JEEViKA Multisectoral Convergence Pilot in Bihar. SIEF Baseline Validation Report. Unpublished Baseline Report. Washington, DC: International Food Policy Research Institute. See also Nair, N., et al., Effect of participatory women's groups and counselling through home visits on children's linear growth in rural eastern India (CARING trial): A cluster-randomised controlled trial. Lancet Glob. Health 5, e1004–e1016. http://dx.doi.org/10.1016/S2214-109X(17)30339-X. See also Ruel, Marie T., Quisumbing, Agnes R., and 23 dietary diversity can be achieved through diverse efforts to support behavior change, this may not be translating into sustained daily consumption of foods that promote linear growth. Trials in Bangladesh that combined food or cash supplements with behavior change efforts were more successful in supporting improvements in child growth outcomes. 8 60. The full set of results from the impact evaluation is presented in the Impact Evaluation Report (please refer to the link provided on page 1). B. Implementation: What Worked, What Didn’t, and Why 61. To place these impact results in the program framework, we draw on the lessons learned during both the midcourse process evaluation study conducted in 2017 (see Box 3) and the impact evaluation. The JEEViKA platforms seem to have functioned well, with women reporting being active members of these groups at both baseline and endline. Those in the treatment group were more likely to have discussed topics related to care during pregnancy, care and feeding of newborns and young children, dietary diversity, kitchen gardens, core JEEViKA funds, and Government schemes—topics being promoted as part of the pilot—and to have been exposed to the video messages. Thus the health and nutrition BCC component of the pilot seems to have worked well despite several challenges. 62. Unfortunately, the CC component did not work so well. This second component underwent considerable transformation during the pilot. To begin with, the formation of convergence committees and the activation of community events were delayed, and then the committees were discontinued soon after their formation because of difficulties in scheduling regular meetings. Instead, greater emphasis was placed on ICDS-organized community events like the Annaprashan and Bachpan Diwas—until support to these events was also discontinued and the HSC was tasked with the work of convergence and coordination. The HSCs were still being trained at the pilot midpoint, so that the effective period of their integration into the pilot—and hence of the implementation of certain components like home visits and participation in ICDS events—was fairly short. A new cadre, the CNRP, was introduced in June–July 2017, with several roles: they were expected to visit the VHSND, Annaprashan Diwas, and Bachpan Diwas; monitor the rollout of BCC; visit VO meetings to facilitate health and nutrition discussions there; and support the HSC in their training, CRP drives, and home visits. 9 63. While women in the treatment arm were more likely to have heard messages related to dietary diversity, cultivating a kitchen garden, sanitation practices, giving ORS, and care during pregnancy, exposure was still quite low—under 50 percent of the women heard these messages. Exposure to several other messages was similar across the treatment and the comparison arms. Balagamwala, Mysbah (2018), Nutrition-sensitive agriculture: What have we learned so far? Global Food Security 17 (June 2018): 128-153. https://doi.org/10.1016/j.gfs.2018.01.002. 8 See Ahmed, A., et al. (2016), Which kinds of social safety net transfers work best for the ultra-poor in Bangladesh? Operation and Impacts of the Transfer Modality Research Initiative. Dhaka: International Food Policy Research Institute and World Food Programme. 9 There were also challenges with the JEEViKA structure, staff shortages, and staff ownership and buy-in that were identified in the process evaluation report, summarized in Box 3. 24 Part of the reason for this could be the generic nature of the messages—that is, they were likely to be transmitted through several mechanisms. 64. The pilot depended heavily on the effectiveness of one worker, the CM, on whose shoulders the bulk of the implementation rested. During our process evaluation, we found that many CMs had not been paid for long periods of time, sometimes more than a year—a fact that affected their motivation and willingness to work. Given the CMs’ unique position in the system and their close access to the women, their time was in great demand, and the responsibilities of other programs also fell on them. For example, for two schemes, the sanitation drive and the life insurance scheme, CMs were asked to complete additional surveys or help members register during normal SHG meeting hours. Competing demands on the CMs’ time may have diluted implementation intensity. 65. In addition, the training of the CMs could have benefitted from streamlining and reinforcement. CMs in the treatment arm were provided two sets of BCC training (one by JEEViKA Technical Support Program staff and the second by World Bank consultants) and were thus given two sets of instructions that might have been confusing. CMs’ knowledge of the BCC content was not always up to the mark; 10 however, the real shortfall was in their ability to relate existing JEEViKA platforms to the BCC content being delivered—for example, how to use the FSF to purchase food, or how to build and maintain kitchen gardens year-round. Refresher trainings that specifically laid out the JEEViKA interventions and their link to health and nutrition (in addition to reiterating the BCC content) might have improved the functioning of the CM. 66. Finally, the structure of the JEEViKA hierarchy, with a very top-down approach to the program, made implementation of the pilot difficult. Many upper-level staff were tasked with allocating and supervising tasks without any real appreciation of the amount of time and effort it took to implement them on the ground. With orders coming from the state or district teams and little involvement of block-level teams in trainings, dissemination of findings, or discussion of next steps, local buy-in was minimal, and the pilot was not given priority. This aspect of implementation is one that should be addressed as the program is scaled up. 10 While CM knowledge was better in the treatment arm than in the control arm, there was considerable room for improvement, and indeed in our process evaluation report we noted some areas of knowledge in which CM performance appeared to have deteriorated since baseline. 25 Box 3. Findings of the Process Evaluation of the • These responsibilities distracted the staff from their JEEViKA-MC Pilot main tasks under the pilot and took up time that could IFPRI conducted a midterm process evaluation in April- otherwise be spent on the nutrition intervention during May 2017 (about 14 months before the endline survey), SHG meetings. examining the performance of the JEEViKA-MC pilot • CM workload and payments. Besides being tasked along five broad domains: implementation platforms, with multiple responsibilities (several unrelated to the training and awareness of roles, implementation processes, exposure of SHG households to key pilot), many CMs had not received their salaries for messages, and utilization of the intervention. An many months and had also not received the incentives important finding of the process evaluation was that for displaying the Digital Green videos and providing very few differences in exposure were observed across the nutrition BCC. Thus it is perhaps unsurprising that households in the treatment and comparison arms. In CM motivation was low. addition, differences in knowledge and utilization • CM training. CMs’ knowledge of the BCC content between households in the treatment and comparison suggests that their training may have been inadequate, arms were minimal, and few improvements in and we recommended, for both CMs and their knowledge were seen in the treatment arm between supervisors, refresher trainings that included content baseline and midterm. Several other findings that knowledge assessments. We also pointed out the emerged from the process evaluation are useful to keep importance of linking the health and nutrition BCC to in mind while evaluating the impact of the MC pilot the existing JEEViKA funds and interventions, like presented in the current report. the FSF and the kitchen gardens, during the trainings. Things that were working well Besides content training, CMs also needed additional • The implementation platforms (the SHGs, VOs, training in soft skills, including retaining the attention procurement committees) were in place and of the women and delivering content. functioning. • JEEViKA staff and key cadres’ knowledge of the • Staff shortages. A shortage of JEEViKA staff and aim of the project and of their specific roles and cadres (especially Community Coordinators, Village responsibilities was good. Resource Persons, and CMs) made it hard to • The content of the BCC was accurate and implement the program. comprehensive. • SHG meetings.The provision of BCC through SHG • The CMs in the treatment arm had better meetings posed another set of challenges. Topics other knowledge about the BCC topics and the HRF, than health and nutrition often took precedence in FSF, and kitchen gardens. SHG meetings, and the health- and nutrition-related • The majority of the SHGs were discussing health topics were discussed either not at all or only briefly. and nutrition topics during their meetings. Only a few CMs reported using picture cards, games, • Women in treatment areas were more likely than or flipcharts to disseminate the information. In a those in the control areas to report discussing health population with limited education, such aids increase and nutrition topics. understanding and retention and generate greater Things that were not working so well interest in the content. Finally, as mentioned above, no • Changes to and delays in the intervention. Repeated links were made between the information alterations to not only the CC component but also the disseminated and the resources available to help overall intervention design, with ever-changing women put the suggestions into practice. During SHG priorities, meant that many parts of this component meeting observations, only in one meeting did a CM functioned for less than the full term of the pilot. In discuss kitchen gardens with the members. addition, the long delays in addressing • Staff ownership and buy-in. Formal training was not implementation issues further weakened the provided to JEEViKA staff members (Block Project intervention. Managers, Area Coordinators, Community • BCC content. Messages being delivered by the CM Coordinators, and others) until almost nine months were generic and could have reached the comparison after the rollout of the intervention. Thus they did not arm households through several other channels, feel a sense of ownership about the project and its diluting measures of impact of the intervention. outcomes, and may not have been able to adequately • JEEViKA structure. The JEEViKA structure—with understand or monitor the progress of the BCC many layers of individuals, a complex reporting component. In addition, their limited training on health system, and multiple requests of the same FLW or and nutrition might have affected their feelings of cadre (the CM/Community Coordinators)—presented ownership of the work under the pilot. a key challenge. Several of the higher-level staff at _________________________ the district and state levels supervised multiple Note: The full findings of the process evaluation are available JEEViKA activities but did not work directly on the in Raghunathan et al. (2017a), An Impact Evaluation of the pilot. Thus they had a limited appreciation of the JEEViKA Multisectoral Convergence Pilot in Bihar – A multiple requests being made of lower-level staff, and Process Evaluation Report. Unpublished Baseline Report. of the additional responsibilities that had been given Washington, DC: International Food Policy Research Institute. to the Community Coordinators and CMs. 26 C. Costs: How Much did it Cost to Implement the JEEViKA-MC Pilot? 67. We estimate the total cumulative cost of implementing the pilot to be approximately US$420,354. About 25 percent of this total was spent on overall project activities (those that cannot be attributed to only BCC or CC), the BCC component accounted for 71 percent, and the CC accounted for 5 percent. The cost per target beneficiary was estimated to be US$110, and the cost per SHG member per meeting was US$0.49. 68. We found that the feasibility phase accounted for about 44 percent of total costs. Since these were up-front fixed costs, the per-unit cost of implementing the intervention would decline as the duration of implementation increased. When we exclude 50 percent of the financial cost of the feasibility phase, the per-SHG cost of implementing the pilot drops from US$264 to US$206, and the per-SHG member per-meeting cost for implementing the pilot drops from US$0.49 to US$0.38. When we exclude all the costs incurred in the feasibility phase (treating them as sunk costs), we have a per- SHG implementing cost of US$148, per-beneficiary cost of US$62, and per-SHG member per meeting cost of US$0.27. Because these figures do not include any of the costs incurred in the feasibility phase, they are the relevant cost estimates for a potential scale-up of the intervention. Assuming two meetings per month on nutrition, this works out to about $6 per SHG member per year, which is comparable to the cost of the intervention implemented in the trial by Nair et al. (2017, op. cit.) in Odisha. Other studies that estimated the cost of implementing nutrition education programs do not present estimates that are comparable to ours; in general, information on the costs of nutrition-sensitive programs is very limited, particularly for the types of interventions studied here (Ruel, Quisumbing, and Balagamwala 2018, op. cit.). D. Recommendations 69. The impact evaluation findings show that the intervention was successful in changing some key knowledge and practice indicators, although our process evaluation had highlighted several factors that could have limited the magnitude of impact or the impact on even more outcomes. On the basis of these findings, we offer recommendations on some key features of intervention design and implementation and outline some implications for future research. 70. Leverage the SHGs for impact. The basic platform—the SHG—seems to work effectively. The fact that women in the treatment arm are discussing nutrition-related topics at SHG meetings suggests that the pilot intervention was effective in integrating this content into the ongoing SHG meetings. Additional pilot-specific activities, such as community events and videos, also appear to have been implemented. As a result, exposure to key messages is significantly higher among women in the treatment arm than in the comparison arm. This implies that the basic approach and design have the potential to be successfully leveraged for impact. 71. Streamline the BCC component and increase its relevance for SHG audiences and for impact. Although the content of the BCC material was comprehensive, positive change occurred for only a few knowledge indicators and behaviors. In addition, the BCC content—which was heavily targeted to pregnant women and mothers of young children—was not always appropriate for the average SHG woman. The BCC might have been more effective if the content had focused on a few key areas that were relevant to all women who attend SHG meetings—for example, dietary diversity and food security. More tightly focused content would likely have helped to reinforce key behaviors. If the intent of the program was to target and improve outcomes only for pregnant and lactating women, then the more general SHG platforms might not be the most appropriate setting, given their mixed age and reproductive stage composition. Mothers’ groups, focused on women in their child’s first 1000 days, might be a more effective means of reaching those target beneficiaries. 72. In addition, focusing on both dietary quality and diversity might do more to affect growth outcomes. Although the pilot did have impacts on reported dietary diversity, this improvement came about through increased consumption of plant-based foods (including pulses) and dairy, but not of animal-sourced foods, even though most households self-identified as non-vegetarian. Recent studies emphasize the importance of animal-source proteins to support better child growth in poor settings. 11 73. Limit changes in the intervention components and ensure that staff at all levels are motivated to facilitate implementation. The JEEViKA-MC interventions and components within them were continuously adjusting, adapting, and changing. In the initial period of any pilot, flexibility and the ability to adapt to program and community conditions are important to develop a good mix of intervention components that will work well and will be acceptable to the target population. However, substantial changes occurred well into the second year of implementation—notably several reassignments of responsibility for the CC component and late rollout of the video-based BCC component. These frequent changes may have contributed to confusion among those implementing on the ground and to delays in the rollout of specific components, which in turn may have limited these components’ potential for impact. In addition, it is imperative to ensure as early as possible in the course of programs that program staff at all critical operational levels are trained and motivated to support implementation. 74. Understand and address the multiple barriers to behavior change. Our findings reinforce that knowledge about optimal behavior is a necessary—but not sufficient—condition for the adoption of optimal practices. For example, while overall diet diversity improved, there was no increase in the consumption of animal-sourced foods, even though most households self-identified as non-vegetarian. Other barriers—such as availability, access (which includes affordability and agency), and, perhaps, buy-in from other household members (especially male members)—could all be important. Future research should examine these additional barriers to behavior change through questions such as the following: What are household members’ perceptions of the messages delivered? What is the credibility of the person delivering them? What are the social barriers preventing adoption—such as community sanction/acceptance—even if the household agrees with the message? What kind of access do households have to markets, and what foods can they buy there? What are the economic and other structural barriers (such as time and workload) to adopting better dietary practices? 75. Addressing these barriers will require complementary investments and increased uptake of Government services. For instance, the finding that food insecurity remains high clearly suggests that 11 See, for example, Headey, D., K. Hirvonen, and J. Hoddinott. 2018. Animal sourced food and child stunting. American Journal of Agricultural Economics 100(5): 1302-1319; Kim, S.S., Nguyen, P.H., Yohannes, Y., Abebe, Y., Tharaney, M., Drummond, E., Frongillo, E.A., Ruel, M.T. and Menon, P., 2019. Behavior Change Interventions Delivered through Interpersonal Communication, Agricultural Activities, Community Mobilization, and Mass Media Increase Complementary Feeding Practices and Reduce Child Stunting in Ethiopia. The Journal of nutrition. 28 resources are a key constraint to further improvements in diet quality. Animal-sourced foods are more expensive, so—even if they are preferred—households may need to limit their consumption of them in favor of a larger quantity of cheaper foods. A BCC intervention with information on dietary diversity is likely to be far more effective in improving diets when bundled with in-kind or cash transfers, or when implemented where household resource availability and basic food security create favorable conditions for the consumption of more preferred foods. 29 VII. Conclusions 76. This report presents the endline findings of an impact evaluation of the JEEViKA Multisectoral Convergence pilot, designed as an effectiveness trial, in one district in Bihar, India. JEEViKA, a rural livelihoods project, supports SHGs—savings and credit-based groups of about 15-20 women, mostly targeted toward those from poor households—with the aim of improving their livelihoods and enhancing household incomes. Two complementary sets of interventions—health and nutrition behavior change communication (BCC) to improve women’s knowledge and household practices, and efforts to improve service access through convergence—were layered onto the existing core package of JEEViKA activities and were targeted to women who were members of the SHGs already formed by JEEViKA. Within this target population, households with young children, mothers of young children, and pregnant women were the primary focus of the JEEViKA-MC pilot. 77. The evaluation, designed as a randomized controlled trial using a panel survey of women with children 6–23 months of age at baseline, finds that the JEEViKA-MC pilot had small but significant impacts on women’s and children’s reported dietary diversity, but not on anthropometric outcomes for either women or children. The intervention worked through the anticipated pathways, delivering higher exposure to key messages on nutrition through the SHG platform, and contributing to improved knowledge about nutrition and improvement of some practices among women covered by the pilot program. There were no impacts on the use of Government programs. 78. The interventions tested in this study were delivered at a cost of $6 per SHG member per year, which are comparable with those of another intervention in Odisha, and they yielded similar but smaller impacts than that research trial identified. 79. As women’s SHG platforms are being leveraged across Bihar and, indeed, across India to scale up BCC in India’s national nutrition mission, our evaluation brings rigorous evidence to the knowledge base on the potential for these platforms to have impact. Our findings also highlight some of the key challenges and emphasize that efforts to use women’s group platforms to deliver nutrition behavior change interventions should consider the relevance of the behaviors that are being promoted, the broad-based nature of the target audiences reached through SHGs, and the role of other factors that support nutrition-related behavior change so that behavior change will be translated into biological impact. 80. We conclude that women’s self-help platforms like JEEViKA have some potential to improve nutrition-related behaviors, but we recommend further efforts to address the challenges to implementation and to behavior change that have been identified in this evaluation. 30