from EVIDENCE to POLICY Learning what works for better programs and policies July 2019 COLOMBIA: Can a successful parenting program be implemented at scale? EARLY CHILDHOOD DEVELOPMENT Children everywhere need enough nutritious food and stimu- childhood program for poor children in semi-rural areas of lation to grow and develop to their full potential. Yet many Colombia. The 18-month program was modeled on a highly disadvantaged children in low-income countries do not receive successful program evaluated in Jamaica (see Evidence to Policy the support they need in the first years of life, negatively af- note, May 2014). However, while the Jamaica program relied fecting their future health, education, and earnings. The num- on specially-trained paraprofessionals, the Colombia program bers reveal the scale of the problem: an estimated 250 million used an existing network of beneficiaries of a conditional cash children under transfer program for the country’s poorest families. These age five in de- women, who were paid for their involvement in the parenting veloping coun- program, were trained and tasked with visiting mothers with tries are failing young children weekly, delivering messages about the impor- to reach their tance of early stimulation and demonstrating play activities us- developmen- ing low cost or homemade toys and picture books. At the end tal potential. of the 18-month intervention, children whose mothers had re- Parenting pro- ceived the home visits showed gains in cognitive development grams seek to and in their ability to understand and process words. There improve young were more play materials in the home and children’s primary children’s devel- caregivers reported engaging in more play activities at home. opment by helping parents learn to stimulate their children’s However, when researchers returned about two years later to brain and body development through talk, play, and frequent retest the children, who were now around five years old, the im- high-quality interactions. When tested on a small scale, such pacts of the program had faded out. Children whose mothers programs have had remarkable benefits––not only in improv- had received the training no longer showed any gains compared ing cognitive development and social-emotional skills during with children whose mothers hadn’t been part of the program. childhood, but also in positively affecting children’s future edu- Parenting practices also no longer were any different than what cation, wages, and well-being. The challenge is how to imple- could be observed in the control group. Researchers believe ment quality programs at scale, while keeping the cost afford- that children’s initial cognitive gains may have been too small able for governments facing numerous financial demands. One to be sustained after the program ended and that the women approach is to integrate a parenting program into an already delivering the stimulation program might not have had suf- existing government program – such as a cash transfer or health ficient coaching or supervision to be as effective as they could program – and use the existing structure for delivery. have been. The findings point to the challenges of scaling up To help build evidence on how to do this successfully, the early childhood programs for long-term impact and the need World Bank’s Strategic Impact Evaluation Fund supported a to identify the right strategies for implementing stimulation 2-year follow-up of an impact evaluation of an early programs. Context While Colombia has made large strides in terms of eco- that 31 percent of children from the lowest socioeconomic cat- nomic development and reducing poverty in recent years, ex- egory were anemic. treme poverty remains a major challenge. World Bank data for To help tackle poverty, the Colombian government started 2016 shows that 2.2 million people – or 4.5 percent of the a conditional cash transfer program in 2001 called Familias en population – lived on $1.90 or less a day. Acción. Through the program, the poorest 20 percent of Co- Children living in the poorest households often experience lombian households are entitled to receive monthly transfers of substantial developmental delays. In Colombia, about 15 per- between US$8 and US$16 per child, conditional on children EARLY CHILDHOOD DEVELOPMENT cent of children ages 12-24 months are chronically malnour- under age 7 going for health checkups and children ages 6-17 ished. Early childhood anemia is also prevalent in Colombia attending school. among the poorest children: a 2010 national survey estimated Evaluation A clustered randomized evaluation of the early stimulation pro- maternal-child interactions and to help mothers participate gram described above took place from 2010-2013 in 96 towns in developmentally appropriate learning activities, many cen- in three regions of Colombia. In each of the three regions, 32 tered on daily routines. Home visits were planned to take place towns were randomly assigned to one of three treatment groups weekly over 18 months, so about 73 visits in total. The aver- or the control group. The intervention in the three treatment age number of visits per fam- groups used the infrastructure of Familias en Acción to iden- ily turned out to be 63. The tify eligible families and deliver the programs. The research second treatment group re- team paid a small weekly stipend to employ locally elected ceived the parental stimula- representatives of the Familias en Acción program, known as tion program as well as daily Madres Líderes, or Mother Leaders, to deliver both micronu- micronutrient supplements trient supplements and the parental stimulation program dur- containing iron, zinc, vita- ing visits to families’ homes. Supervisors, who had experience min A, vitamin C, and folic with child development and/or an undergraduate degree in a acid for all children under related field, received six weeks of initial training, following age 6 in the household, as a which they trained the Madres Líderes over a two-week period. means of reducing anemia. The Madres Líderes received an additional one-week follow- The third treatment group received only the micronutrient up training session from supervisors a month or two into the supplements. program. Once the program was up and running, supervisors The baseline survey took place in 2010, prior to the ran- were expected to meet with the Madres Líderes every six weeks. domization, among 1,419 children who were aged 12 to 24 The first treatment group received the parental stimulation months at the start of the study. Children were observed at program, which was modeled on the successful home-visiting baseline, then 18 months later when children were two-and- model from Jamaica, where parents met weekly with trained a-half years old to three-and-a-half years old, and then again workers who showed them how to talk and play with their two years later. By the time of the second follow-up in 2013, children to stimulate development. Similar to the program in children were four-and-a-half to five-and-a-half years old. Tests Jamaica, the Madres Líderes in Colombia made weekly home focused on cognition, language, behavior, and school readiness. visits and provided low-cost and homemade toys, form boards, The surveyors also observed what was happening in the home and other materials that were left in the home and changed and asked caregivers about their play practices with the child. weekly. The visits were designed to improve the quality of This policy note is based on “Impacts 2 years after a scalable early childhood development intervention to increase psychosocial stimulation in the home: A follow-up of a cluster randomised controlled trial in Colombia,” PLOS Medicine (2018) Findings At the end of the 18-month program, children whose weren’t any more likely to have engaged in play with their mothers received the weekly visits did better in caregivers, according to parents’ reports. terms of cognitive development and receptive language skills – the ability to understand and Micronutrient supplements didn’t have a positive im- process what one hears—than children whose pact on children’s growth and development, whether mothers didn’t receive the visits. delivered on their own or in conjunction with home visits. When measured 18 months later, children in families as- signed to the parental stimulation group experienced a signifi- Even in the short run, children in families that received the Findings cant improvement in cognitive scores and language scores of micronutrient supplements didn’t experience any reductions 0.26 and 0.22 standard deviations, respectively, as compared in anemia or other positive developmental impacts relative with children whose mothers didn’t receive the home visits. to children in the control group, despite mothers’ report- This amounts to almost a third of the gap in cognition be- ing that they gave the supplements regularly. Researchers tween children living in the 25% poorest and the 25% richest hypothesize that either mothers weren’t giving the supple- households in a sample of low and low-middle income fami- ments as regularly as they said they were, children’s anemia lies in Bogota.* However, the children didn’t show any sta- wasn’t linked to a micronutrient deficiency but to something tistically significant gains in expressive language nor in their else, or the timing of delivery of the micronutrients was mis- motor skills, relative to kids in the control group. targeted given the evolution of anemia in this population. Overall, the micronutrient supplements didn’t provide any When parents received home visits, they played more added positive impact on children’s development. with their children and had more play materials in the home at the time of the first follow-up. The study demonstrates that an early childhood education program can be delivered effectively Using UNICEF’s family care indicators, the research team at scale as part of an existing welfare program, found that in homes visited by the Madres Líderes, children however the fade-out of impacts after two years had a larger variety of play materials and they were more points to the need to strengthen aspects of the likely to have played with an adult in the three days prior to home visit program. the first follow-up interview. Given the impacts did not last, the evaluation team suggests But the initial improvements in children’s some potential areas for improvement. One challenge was development and their home environment faded two supervision. The only requirement to be hired as Madres years after the program ended. Líderes was basic literacy, time availability, and interest in the job. The supervisors had little support and they At the time of the second follow up, there were no statistically were responsible for training and supervising 24 Madres significant differences in children’s cognitive development or Líderes across eight different municipalities. Although they receptive language. There also weren’t any new gains in terms were required to meet with each Madres Líderes every 6 of school readiness, executive function skills such as impulse weeks --compared to weekly meetings in the successful but control and working memory, or behavioral development small-scale trial in Jamaica-- in practice meetings averaged such as getting along with other children. Compared to the once every 9 weeks. Improving supervision, along with control group, the amount of play materials in homes of chil- recruitment and support for those carrying out the home dren in the group that received the program was no longer visits, may help ensure programs like this are implemented higher. Children whose mothers received the home visits also properly and successfully. * Marta Rubio-Codina, Orazio Attanasio, Costas Meghir, Natalia Varela and Sally Grantham-McGregor, “The Socioeconomic Gradient of Child Develop- ment: Cross-Sectional Evidence from Children 6–42 Months in Bogota” The Journal of Human Resources, vol. 50 no. 2 (2015), 464-483 EARLY CHILDHOOD DEVELOPMENT Conclusion This research shows that it is possible to deliver a model program and for future scale-ups of other programs that have of early childhood education at scale and through existing worked in small-scale efficacy trials. It is possible that effects government services. It also shows that this program can on education, earnings, and social outcomes may appear at have an impact on child development within an 18-month a later age, as had happened in studies in the United States period. Two years after the program ended, however, the im- and Jamaica. Challenges in implementation in Colombia provements in child development and parenting practices also suggest that further understanding how to strengthen observed in the short-term had faded out. Unraveling the programs at scale will be key for turning promising early reasons why the program didn’t achieve a larger or longer- childhood development interventions into effective tools for lasting impact will be important for improvement of this policymakers. The Strategic Impact Evaluation Fund, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help alleviate poverty. The goal is to collect and build empirical evidence that can help governments and development organizations design and implement the most appropriate and effective policies for better educational, health, and job opportunities for people in low and middle income countries. For more information about who we are and what we do, go to: http://www.worldbank.org/sief. The Evidence to Policy note series is produced by SIEF with generous support from the British government’s Department for International Development and the London-based Children’s Investment Fund Foundation (CIFF). THE WORLD BANK, STRATEGIC IMPACT EVALUATION FUND 1818 H STREET, NW, WASHINGTON, DC 20433 Series editor: Aliza Marcus