DIREC TIONS IN DE VELOPMENT Human Development Taking Stock of Programs to Develop Socioemotional Skills A Systematic Review of Program Evidence Maria Laura Sánchez Puerta, Alexandria Valerio, and Marcela Gutiérrez Bernal Taking Stock of Programs to Develop Socioemotional Skills Direc tions in De velopment Human Development Taking Stock of Programs to Develop Socioemotional Skills A Systematic Review of Program Evidence Maria Laura Sánchez Puerta, Alexandria Valerio, and Marcela Gutiérrez Bernal © 2016 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved 1 2 3 4 19 18 17 16 This work is a product of the staff of The World Bank with external contributions. The findings, interpreta- tions, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) http:// creativecommons.org/licenses/by/3.0/igo. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: Sánchez Puerta, Maria Laura, Alexandria Valerio, and Marcela Gutiérrez Bernal. 2016. Taking Stock of Programs to Develop Socioemotional Skills: A Systematic Review of Program Evidence. Directions in Development. Washington, DC: World Bank. doi:10.1596/978-1-4648- 0872-2. License: Creative Commons Attribution CC BY 3.0 IGO Translations—If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations—If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content—The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any ­ ­ third-party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to reuse a component of the work, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. All queries on rights and licenses should be addressed to the Publishing and Knowledge Division, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights​ @­worldbank.org. ISBN (paper): 978-1-4648-0872-2 ISBN (electronic): 978-1-4648-0873-9 DOI:10.1596/978-1-4648-0872-2 Cover photo: © Susanne Riber Christensen/Getty Images. Used with permission. Further permission required for reuse. Cover design: Debra Naylor, Naylor Design, Inc. Library of Congress Cataloging-in-Publication Data has been requested Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Contents Acknowledgments ix About the Authors xi Executive Summary xiii Abbreviations xvii Chapter 1 Motivation and Objectives 1 Notes 3 References 3 Chapter 2 Literature Review 7 The Relationship between Socioemotional Skills and Life Outcomes 7 Evidence of Programs That Modify Socioemotional Skills 8 Notes 11 References 12 Chapter 3 Definitions: What Are Socioemotional Skills? 15 Socioemotional Skills 16 Soft Skills 17 Noncognitive Skills 17 Character Skills 18 Personality Traits and Temperament 18 Twenty-First Century Skills or Competencies 19 Life Skills 19 Notes 21 References 21 Chapter 4 Conceptual Framework 25 Program Characteristics 26 Participants’ Characteristics 28 Outcomes 28 Note 29 References 29 Taking Stock of Programs to Develop Socioemotional Skills   v   http://dx.doi.org/10.1596/978-1-4648-0872-2 vi Contents Chapter 5 Search Methodology 31 Search Phase 1: Programs before Formal Education 32 Search Phase 2: School-Based Programs 32 Search Phase 3: Out-of-School Programs 33 Notes 34 References 34 Chapter 6 Program Analysis throughout the Life Cycle 37 Before-School Programs 37 Outcomes of Before-School Programs 44 School-Based Programs 50 Outcomes of School-Based Programs 60 Out-of-School Programs 67 Outcomes of Out-of-School Programs 71 Notes 78 References 79 Chapter 7 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? 89 Targeting 89 Focus 90 Impacts 90 Replicability 91 Evaluation 92 Direction for Future Research 93 References 93 Appendix A Before-School Program Descriptions 95 References 115 Appendix B School-Based Program Descriptions 121 References 156 Appendix C Out-of-School Program Descriptions 161 References 177 Boxes 6.1 Home Visiting Program: The Jamaican Study 41 6.2 Home Visiting Program: Nurse-Family Partnership, United States 42 6.3 Center-Based Program: Save the Children’s Early Childhood Development Programme, Mozambique 43 6.4 Center-Based Program: HighScope Perry Preschool Program, United States 43 6.5 School-Based Program: Al’s Pals 56 Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Contents vii 6.6 School-Based Program: I Can Problem Solve, United States 56 6.7 School-Based Program: 4Rs Program, New York City 57 6.8 After-School Program: Big Brothers Big Sisters of America, United States 58 6.9 Impact Evaluations That Include an Analysis of an Intervention’s Components: Project SAFE and a School-Based Intervention in Colombia 63 6.10 Out-of-School Program: Juventud y Empleo, Dominican Republic 71 Figures 3.1 Defining Socioemotional Skills 16 4.1 Conceptual Framework 26 4.2 Basic Elements Analyzed in Each Program 27 5.1 Literature Review Process to Identify Socioemotional Programs 34 6.1 Outcomes of Before-School Programs 45 6.2 Selected Program Characteristics That Affect the Outcomes of Before-School Programs 47 6.3 Participant Characteristics That Affect the Outcomes of Before-School Programs 49 6.4 Program Components in the Sample of Programs 59 6.5 Outcomes of School-Based Programs 60 6.6 Selected Program Characteristics That Affect the Outcomes of School-Based Programs 63 6.7 Selected Participant Characteristics That Affect the Outcomes of School-Based Programs 66 6.8 Plausible Relationship between Risk Level and Program Effects 66 6.9 Program Components in Sample of Out-of-School Programs 70 6.10 Outcomes of Out-of-School Programs 73 6.11 Selected Program Characteristics That Affect the Outcomes of Out-of-School Programs 75 6.12 Selected Participant Characteristics That Affect the Outcomes of Out-of-School Programs 77 Tables 3.1 Examples of the Constructs Measured under Each Concept 20 6.1 Distribution of Programs by Age Covered 38 6.2 Salient Characteristics of Home Visiting Programs before Formal Education 39 6.3 Salient Characteristics of Center-Based Programs before Formal Education 40 6.4 Distribution of Programs by School Grade–Range Covered 52 6.5 Distribution of Programs by School Grade Tested 54 6.6 Salient Characteristics of School-Based Programs 59 6.7 Distribution of Out-of-School Programs by Age 68 Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 viii Contents 6.8 Main Characteristics of Socioemotional Skills Components in Out-of-School Programs 69 6.9 Salient Characteristics of Out-of-School Programs 72 A.1 Before-School Programs—Home Visiting Programs 96 A.2 Before-School Programs—Child Care Centers 108 B.1 School-Based Program Descriptions 122 B.2 School-Based Programs by Component 155 C.1 Out-of-School Program Descriptions 162 C.2 Out-of-School Programs by Component 177 Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Acknowledgments This report was prepared by a team composed of Maria Laura Sánchez Puerta, Alexandria Valerio, and Marcela Gutiérrez Bernal from the World Bank. Claire Miller provided technical and research support, and Sergio Guerra prepared a background paper that was instrumental in the creation of the report. The team appreciates the strategic guidance and overall support received from Claudia Costin (senior director, Education Global Practice), Amit Dar (director, Education Global Practice), and Luis Benveniste (practice manager, Global Engagements, and Education Global Practice). Helpful peer review comments were received from Margo Hoftijzer (senior economist), Ines Kudo (senior education specialist), and Victoria Levin (senior economist) from the World Bank and Laura Ripani (lead economist) from the Inter-American Development Bank. The report also incorporates inputs received from Omar Arias (lead economist, World Bank) and from participants in the technical seminars sponsored by the World Bank’s Skills Global Solutions Group held in December 2015. The team appreciates the overall assistance of Lorelei Lacdao, Fahma Nur, and Marie Madeleine Ndaw from the World Bank. The written pieces contained within this review were edited by Marc DeFrancis. The report received financial support from the Skills and Information, Communication, and Technology (ICT) Trust Fund of the Government of the Republic of Korea. Taking Stock of Programs to Develop Socioemotional Skills   ix   http://dx.doi.org/10.1596/978-1-4648-0872-2 About the Authors Maria Laura Sánchez Puerta is a senior economist in the Jobs Group of the Social Protection and Labor Global Practice at the World Bank, where she specializes in the intersection of labor and development economics. She cur- ­ rently leads the jobs and skills agenda and coleads the global STEP initiative, which includes household and employer surveys measuring adult skills in 17 countries. She prepared one of the first job diagnostics at the country level and contributed to an innovative, multisector work program on jobs in Kenya. Maria Laura’s research includes cognitive and noncognitive skills and labor ­ outcomes; design, implementation, and evaluation of active labor market programs; income mobility in Latin America; informality and labor market ­ segmentation; and the effects of globalization on working conditions. Maria Laura ­ has also supported analytical and operational work in Argentina, Brazil, Cambodia, Colombia, El Salvador, Indonesia, Kenya, Lebanon, Rwanda, and Tunisia. Maria Laura holds a PhD in economics from Cornell University and joined the Institute for the Study of Labor (IZA) as a research fellow in 2007. Alexandria Valerio has over 20 years of experience leading and managing large-scale research projects, multidisciplinary teams, and senior-level client ­ relationships, with a policy focus on education reform (early, primary, and tertiary ­ education), entrepreneurship, skills, and training in diverse country contexts. She has led multidisciplinary teams in the analysis, design, implementation, and evalu- ation of investment operations. Alexandria is currently leading global research agendas focused on measuring adult skills using large-scale household and employer surveys in 17 countries, analyzing the impact of different types of educa- tion and skill sets on employment and development outcomes, and identifying the characteristics of effective entrepreneurship education and training programs. Prior to joining the Global Engagement and Knowledge unit in the Education Global Practice, she was responsible for the World Bank’s education policy dia- logue and lending portfolios in the Latin America and the Caribbean Region (Argentina, Brazil, Chile, Nicaragua, Paraguay and Panama), as well as in Angola and Mozambique. Taking Stock of Programs to Develop Socioemotional Skills   xi   http://dx.doi.org/10.1596/978-1-4648-0872-2 xii About the Authors Alexandria`s work extends beyond the education sector, covering a wide range of issues including social protection and labor, jobs, growth and competitiveness, child development, and school health. Her published work includes peer- reviewed books and papers on workforce development policy, technical and vocational training, entrepreneurship training, tools to measure skills in adult populations, cost and financing of early childhood education, social impact analy- sis of school fees, and school health programs to prevent HIV/AIDS in school- age populations. She is currently a global lead for the World Bank’s Skills Global Solutions Group and a core member of the global interagency group on Technical Vocational Education and Training/Skills and the technical working group on Human Resource Development for the G-20. Alexandria holds a PhD in comparative education and economics of education from Columbia University and a master’s degree in public administration in economic development policy from the Maxwell School at Syracuse University. Marcela Gutiérrez Bernal has five years of experience designing, implementing, and evaluating social programs in more than 10 countries in Latin America, Asia, the Middle East, and the Western Balkans. Her areas of work include poverty reduction and social protection systems, conditional and emergency cash transfer programs, financial inclusion initiatives, and early childhood development strate- gies. She worked with the World Bank Group as a project coordinator of the STEP Skills Measurement Program, the first-ever initiative to measure cognitive and socioemotional skills in more than 16 developing countries. She also worked as a senior adviser to the Ministry of Social Protection and Inclusion in Peru. Previously, she was employed at the Inter-American Development Bank, where she assisted in the ideation, execution, assessment, and continuous improvement of government programs and strategies in Colombia and Panama. Marcela is currently pursuing a master’s degree in public administration and international development at Harvard University. She holds MA and BA degrees in economics from Universidad de los Andes (Bogotá, Colombia) and a BA in business administration. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Executive Summary Relevance Possessing a range of cognitive, socioemotional, and technical skills is important for individuals to maximize their chances of success in many aspects of life. In particular, a growing body of research highlights the effects that socioemotional skills have on a variety of outcomes, from wages and academic performance to health. Programs to help participants develop such skills continue to expand in both high-income and lower-income countries, targeting individuals of almost all ages and life stages. Socioemotional skills development is embedded in programs as diverse as early-childhood nutrition programs for adolescent mothers, K–12 academic curricula, and workforce training programs for vulnerable adults. However, the characteristics that make some programs more successful than others—or even what types of outcomes programs use to measure “success”—are less clear in the literature. This analysis seeks to fill this knowledge gap through a systemic review of socioemotional skills development programs. It uses a new conceptual framework to examine diverse programs that have been rigorously evaluated to answer the following research questions: • What is the existing landscape of socioemotional skills development programs? • How do these programs measure success? • What do we know about programs that work? Conceptual Framework To categorize the wide range of socioemotional program objectives, this analysis breaks down programs by where they occur, program characteristics, partici- pant characteristics, and outcomes measured. Only programs with randomized or quasi-randomized evaluations are included. The programs in this review are divided into three categories, on the basis of when they are implemented in the life cycle: (a) before-school programs (infants and young children), (b) school- based programs (preschool to secondary education), and (c) out-of-school programs (usually targeted, vulnerable populations). Next, the review assesses programs by their specific characteristics: objectives, components and comple- mentary elements, setting, and quality. It also classifies programs by participant Taking Stock of Programs to Develop Socioemotional Skills   xiii   http://dx.doi.org/10.1596/978-1-4648-0872-2 xiv Executive Summary profiles, including ages of participants and participant-targeting systems. Importantly, the review further assesses programs by their outcomes. Although all programs emphasize socioemotional skills development, they present a wide variety of outcome indicators used to measure success. Reported outcomes are classified into four broad categories: (a) health related, (b) risk factor related, (c) academic/cognitive related, and (d) economic related. Methodology Within each of the three life-cycle stages, we identified potential studies and programs from relevant databases, online resources, and experts; screened them in order to include only studies that were written in English after the year 2000, that included randomized or quasi-randomized control trials, that had sufficient sample sizes, and that reported information on standard errors; and determined which papers and follow-ups on the same program were most relevant and were analyzed using the most rigorous approaches. For programs that targeted children before they entered school, the review includes 21 programs, 42 percent of which were conducted in low- or middle- income countries. They included follow-up periods ranging from three months to 37 years. For school-based programs, the review includes 45 programs, 10 percent of which came from low- and middle-income countries, with follow-up from nine weeks to six years. For out-of-school programs, the review includes 20 programs, 75 percent of which took place in low- and middle-income coun- tries; follow-up periods ranged from less than two weeks to four years (most less than one year). Key Findings Before-school programs typically target children younger than age five and tend to focus not only on children but also on their families. Most before-school programs measure outcomes related to academics (such as academic perfor- ­ mance and graduation rates) or risk factors (such as criminal activity); smaller percentages measured health and economic outcomes. School-based programs target a broad range of grades (prekindergarten through 12th grade), often through classroom curricula that usually follow a yearly sequence, that have grade-specific content, and that tend to include all children attending an institution. Of the 45 sample programs, few included curricula for an entire system of prekindergarten through 12th grade, and most focused on lower grades, rather than explicitly targeting adolescents. Most of the school-based pro- grams (59 ­ percent) measured outcomes related to risk factors (such as behavioral variables like aggression toward peers and cooperation); 32 percent measured ­ academic-related outcomes. Few evaluated economic and health outcomes. Out-of-school programs typically promote skills formation in teenage and adult populations who are not enrolled in, and may not have completed, formal educa- tion. Out-of-school programs with socioemotional skills components have a Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Executive Summary xv range of objectives, such as helping individuals find jobs or decreasing gender-based violence. Socioemotional skills development is usually only one ­ part of a larger set of outcomes. The programs surveyed targeted ages ranging from 10 to 55 years, but most focused on working-age teenagers and adults, especially those between ages 16 and 30. Labor market outcomes were the goal of about 75 percent of the programs included in this review. Most successful programs teach socioemotional skills as an embedded compo- nent of a broader curriculum that includes active pedagogical, classroom, and training programs: • Among before-school programs in particular, programs with multiple compo- nents that target health, cognitive development, and emotional development tend to yield greater and longer-term results. In addition, the type of curriculum fol- lowed affects the program’s effectiveness; most desirable are curricula where chil- dren plan, carry out, and review their own activities while engaging in active learning, or where teachers respond to children’s self-initiated play in loosely structured, socially supportive settings. Programs that have more qualified staff and that are implemented with greater intensity and fidelity exhibit greater effects. • Successful school-based programs tend to follow the SAFE approach, that is, they are sequenced, active, focused, and explicit. Programs that take a whole-school approach and prioritize implementation fidelity tend to be ­ more successful. Programs that were integrated into the school day tended to demonstrate more success than after-school programs. ­ • Such findings stress the value of developing socioemotional skills in coordi- nation with other types of skills, as part of a comprehensive, intertwined curriculum—rather than offering separate, stand-alone socioemotional skills “modules.” Programs are particularly effective when they target vulnerable populations and, in particular, young children: • Before-school programs appear to have a greater impact than those that take place later in life. However, these early-childhood programs also tend to be more intensive, more targeted toward vulnerable populations, more likely to involve family members, and have longer follow-up periods. • Most school-based programs tend to be universal and offered to all children; whereas evidence on the impact of these programs is more mixed, the greatest effects appear to be on risk factors (such as externalizing or internalizing behavior or aggression toward peers) among vulnerable populations. • A number of out-of-school programs observed small but statistically signifi- cant impacts on economic outcomes, such as employment, participation in the formal job sector, number of hours worked weekly, and earnings. However, overall evidence on the programs’ effect on employment levels and quality is mixed. Most programs appear to work better for younger participants, for females, and when implemented in urban areas. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 xvi Executive Summary School-based and out-of-school programs need longer follow-up periods and more clarity on proxies used to evaluate socioemotional skills as outcomes: • Whereas over one-third of before-school programs included a follow-up period of 10 years or more, only 3 percent of school-based programs and no out-of-school programs did so. Thus, any potential effects of these programs can be observed only over a short time, and the longevity of any positive ­ outcomes is uncertain. • Although programs explicitly target socioemotional skills development, few explicitly state exactly what socioemotional skills they intend to improve. In addition, most impact evaluations do not measure the skills that the pro- gram seeks to alter. Future research should include skills assessment in order to measure these outcomes more directly. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Abbreviations 4Rs reading, writing, respect, and resolution (program, United States) ASQ Ages and Stages Questionnaire CARE character actualization requires education (program, United States) CSE cognitive-social-emotional DAS developmental assessment session DPII Developmental Profile II ECD early childhood development ELA Empowerment and Livelihood for Adolescents (program, Uganda) EPAG Economic Empowerment of Adolescent Girls (program, Liberia) ERIC Education Resources Information Center GED General Educational Development GPA grade point average HIPPY Home Instruction Program for Preschool Youngsters (program, United States) HIV/AIDS human immunodeficiency virus/ acquired immune deficiency syndrome ICPS I Can Problem Solve (program, United States) ICT information and communication technology IQ intelligence quotient ISFP Iowa Strengthening Families Program (United States) KIPP Knowledge Is Power Program (United States) LEAD Leadership Education Through Athletic Development program, United States) (­ LIFT Linking the Interests of Families and Teachers (program, United States) MAPs Mindful Awareness Practices (program, United States) NICHHD National Institute of Child Health and Human Development Early Child Care Research Network NOW New Opportunities for Women (program, Jordon) Taking Stock of Programs to Develop Socioemotional Skills   xvii   http://dx.doi.org/10.1596/978-1-4648-0872-2 xviii Abbreviations OECD Organisation for Economic Co-operation and Development PALS Play and Learning Strategies (program, United States) PAT Parents as Teachers (program, United States) PATHS Promoting Alternative Thinking Strategies (program, United States) PDFY Preparing for the Drug Free Years (program, United States) PEF Peace Education Foundation PIDI Proyecto Integral de Desarrollo Infantil (program, Bolivia) PPVT Peabody Picture Vocabulary Test RCT randomized control trial SAFE sequenced, active, focused, and explicit; Strategies Aimed at Family Empowerment (program, United States) SEAL Social and Emotional Aspects of Learning (program, United Kingdom) SEL social and emotional learning SET Social and Emotional Training (program, Sweden) SF Strengthening Families (program, United States) STAR Student/Teacher Achievement Ratio (program, United States) TEEP Turkish Early Enrichment Program UNESCO United Nations Educational, Scientific, and Cultural Organization UNICEF United Nations Children’s Fund WHO World Health Organization Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 chapter 1 Motivation and Objectives Recent literature has shown that a combination of cognitive, socioemotional and technical or job-related skills is a stepping-stone for success. According to the Organisation for Economic Co-operation and Development (OECD 2012), poorly skilled individuals are at higher risk of economic disadvantages, unemployment, and long-term reliance on social benefits. Those findings are consistent with the research of Heckman, Pinto, and Savelyev (2013), who identify cognitive, socioemotional and job-related skills as having an important effect on life outcomes. Similarly, Cunha and others (2005) find that wages, schooling, criminality, and teenage pregnancy are affected by cognitive ability, perseverance, motivation, self-control, self-esteem, and risk and time preferences. A robust literature documents the effects of cognitive skills on a range of economic and life outcomes, yet comparatively less information is available on the effects of socioemotional skills on similar outcomes. The lack of information has prompted a new line of academic inquiry that examines the role of ­ socioemotional skills in a range of outcomes. Interestingly, recent literature indi- cates that noncognitive or social skills mediate cognitive performance, educa- tional attainment, behavior, health, and labor market outcomes (Carneiro, Crawford, and Goodman 2007; Heckman, Stixrud, and Urzua 2006).1 Research from Durlak and others (2011) shows that the lack of socioemotional competen- cies, together with the lack of connection to the schooling environment, can harm academic performance, behavior, and health.2 Finally, research has also found that noncognitive or personality skills rival IQ in predicting educational attainment, labor market success, health, and criminality.3 From a public policy perspective, socioemotional skills have gained impor- tance as they might offer policy makers an unexplored opportunity to improve individual life outcomes and strengthen workforce productivity. More important, these skills deserve careful attention for several reasons. First, unlike cognition,4 they may be malleable throughout the life cycle (Almlund and others 2011; Boyatzis 2008; Cherniss and others 1998; Goleman 2000). The exact duration of that malleability is debated. For example, Heckman and Carneiro (2003) Taking Stock of Programs to Develop Socioemotional Skills   1   http://dx.doi.org/10.1596/978-1-4648-0872-2 2 Motivation and Objectives indicate that although cognitive skills are set by age 8, noncognitive skills can be modified until the late teenage years; others contend that personality traits are set by age 30 (Costa and McCrae 1990, 1994, 2006; James 1890; McCrae and Costa 2003). Yet other researchers, including Walsh (2005), indicate that, given the slow development of the prefrontal cortex, socioemotional skills remain mal- leable even after age 30. Second, investments in socioemotional skills can foster equity and promote social mobility, economic growth, social cohesion, and general well-being. Research indicates that given the positive associations found between ­ socioemotional skills and labor market outcomes, improving these skills could lead to greater equity (Hartas 2011). Since skill formation promotes employ- ability and employment, skills can pave the way for expanding economic growth. Ultimately, skill formation can promote social cohesion, that is, the capacity of societies to manage collective decision making (World Bank 2012). Finally, as the following chapters will show, socioemotional skills increase the well-being of the population, as individuals increase their optimism levels, improve their self- concepts, and form healthier relationships, among other outcomes. Despite the increasing importance of the role of socioemotional skills, little systematized and rigorous (experimental) evidence exists documenting the effectiveness of programs that seek to develop these skills throughout the life cycle. The lack of information constrains the policy dialogue with client countries seeking evidence and programs to foster socioemotional skills. This systematic review5 aims to fill an important knowledge gap by distilling existing evidence and offering a menu of program approaches to develop key socioemotional skills to influence important life outcomes. This review contrib- utes to the existing literature on socioemotional skills by identifying and organiz- ing programs using a new conceptual framework. The review includes a number of diverse programs that have been rigorously evaluated and that seek to effect different outcomes in multiple contexts, including in developing countries. The inclusion of developing countries is important, as most of the accumulated evidence comes from programs implemented in developed countries where con- ­ straints and context may be different from those in developing countries and, thus, could potentially mediate outcomes differently. In an effort to generate robust policy advice from proven programs, the review casts a narrow net by considering evidence only from programs that had random- ized or quasi-randomized evaluations. All of the evaluations analyzed in this review include the information needed to calculate the statistical significance of the effect sizes. The program descriptions (such as components, target popula- tions, objectives, intensity, and costs) were collected from published or publicly available evaluation documents, as an analysis of administrative data was beyond the scope of this review. The review focuses on programs that are preventive in nature. They include programs that target universal and primarily young populations, as well as programs that aim to prevent negative behaviors largely among at- risk youth. Other important programs, such as those that fight recidivism Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Motivation and Objectives 3 among adults with criminal histories, substance dependence, or other clinical conditions, are beyond the scope of this review and are excluded from the analysis even if they have important components of socioemotional skill formation. Notes 1. Carneiro, Crawford, and Goodman (2007) refer to noncognitive or social skills and measure them using the Bristol Social Adjustment Guides, which capture social mal- adjustment. Heckman, Stixrud, and Urzua (2006) discuss noncognitive skills and measure them using Rotter’s locus of control scale and Rosenberg’s self-esteem scale. 2. In their meta-analysis of socioemotional learning programs, the authors include six different outcomes: (a) social and emotional skills, which include “different types of cognitive, affective, and social skills related to such areas as identifying emo- tions from social cues, goal setting, perspective taking, interpersonal problem solving, conflict resolution and decision making” (Durlak and others 2011, 410); (b) attitudes toward self, school, and social topics; (c) positive social daily behav- iors, including getting along with others; (d) conduct problems, such as disrup- tive class behavior, noncompliance, aggression, bullying, and delinquency, among ­ others; (e) emotional distress, including depression and anxiety; and (f) academic performance. 3. Heckman and Kautz (2012) measure noncognitive skills, including self-reports, teacher reports, or behaviors that relate directly to these skills (externalizing and internalizing behavior, dysregulated aggression, creativity, verbal intelligence, hostility, beliefs and attitudes, impulse control, dependency, self-esteem, and self-efficacy, among others). Almlund and others (2011) measure personality and traits, such as trust, reciprocity, risk aversion, and the Big Five (openness to experience, conscien- tiousness, extraversion, agreeableness, and neuroticism). Borghans and others (2008) measure personality using the Big Five factors. ­ 4. Almlund and others (2011) state that cognition becomes stable around age 10. 5. We adopt the definition of systematic review coined by Waddington and others (2012, 360): “studies which synthesise all the existing high-quality evidence using transparent methods to give the best possible, generalisable statements about what is known. ... [A] systematic review has a clear protocol for systematically searching defined databases over a defined time period, with transparent criteria for the inclu- sion or exclusion of studies.” References Almlund, Mathilde, Angela Lee Duckworth, James J. Heckman, and Tim D. Kautz. 2011. “Personality Psychology and Economics.” NBER Working Paper 16822, National Bureau of Economic Research, Cambridge, MA. Borghans, Lex, Angela Lee Duckworth, James J. Heckman, and Bas ter Weel. 2008. “The Economics and Psychology of Personality Traits.” Journal of Human Resources 43 (4): 972–1059. Boyatzis, Richard E. 2008. “Leadership Development from a Complexity Perspective.” Consulting Psychology Journal: Practice and Research 60 (4): 298–313. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 4 Motivation and Objectives Carneiro, Pedro, Claire Crawford, and Alissa Goodman. 2007. “The Impact of Early Cognitive and Non-Cognitive Skills on Later Outcomes.” Discussion Paper 0092, Centre for the Economics of Education, London School of Economics. Cherniss, Cary, Daniel Goleman, Robert Emmerling, Kim Cowan, and Mitchel Adler. 1998. “Bringing Emotional Intelligence to the Workplace.” Consortium for Research on Emotional Intelligence in Organizations, Rutgers University, New Brunswick, NJ. Costa, Paul T. Jr., and Robert R. McCrae. 1990. “Personality Disorders and the Five-Factor Model of Personality.” Journal of Personality Disorders 4 (4): 362–71. ———. 1994. “Set Like Plaster? Evidence for the Stability of Adult Personality.” In Can Personality Change?, edited by Todd F. Heatherton and Joel Lee Weinberger, 21–40. Washington, DC: American Psychological Association. ———. 2006. “Age Changes in Personality and Their Origins: Comment on Roberts, Walton, and Viechtbauer.” Psychological Bulletin 132 (1): 26–28. Cunha, Flavio, James J. Heckman, Lance Lochner, and Dimitriy V. Masterov. 2005. “Interpreting the Evidence on Life Cycle Skill Formation.” In Handbook of the Economics of Education, vol. 1, edited by Eric A. Hanushek and Finis Welch, 697–812. Amsterdam and New York: North-Holland. Durlak, Joseph A., Roger Weissberg, Allison Dymnicki, Rebecca Taylor, and Kriston Schellinger. 2011. “The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Programs.” Child Development 82 (1): 405–32. Goleman, Daniel. 2000. “Leadership That Gets Results.” Harvard Business Review March–April: 78–90. Hartas, Dimitra. 2011. “Families’ Social Backgrounds Matter: Socio-Economic Factors, Home Learning and Young Children’s Language, Literacy and Social Outcomes.” British Educational Research Journal 37 (6): 893–914. Heckman, James, J., and Pedro Carneiro. 2003. “Human Capital Policy.” NBER Working Paper 9495, National Bureau of Economic Research, Cambridge, MA. Heckman, James J., and Tim Kautz. 2012. “Hard Evidence on Soft Skills.” Labour Economics 19 (4): 451–64. Heckman, James J., Rodrigo Pinto, and Peter A. Savelyev. 2013. “Understanding the Mechanisms through Which an Influential Early Childhood Program Boosted Adult Outcomes.” American Economic Review 103 (6): 1–35. Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24 (3): 411–48. James, William. 1890. “The Consciousness of Self.” Chap. 10 in The Principles of Psychology. New York: Henry Holt and Company. McCrae, Robert R., and Paul T. Costa Jr. 2003. Personality in Adulthood: A Five-Factor Theory Perspective. New York: Guilford Press. OECD (Organisation for Economic Co-operation and Development). 2012. “Better Skills. Better Jobs. Better Lives: The OECD Skills Strategy.” Paris, OECD. http://www.oecd​ .org/general/50452749.pdf. Waddington, Hugh, Howard White, Birte Snilstveit, Jorge Garcia Hombrados, Martina Vojtkova, Philip Davies, Ami Bhavsar, John Eyers, Tracey Perez Koehlmoos, Mark Petticrew, Jeffrey C. Valentine, and Peter Tugwell. 2012. “How to Do a Good Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Motivation and Objectives 5 Systematic Review of Effects in International Development: A Tool Kit.” Journal of Development Effectiveness 4 (3): 359–87. Walsh, David. 2005. Why Do They Act That Way? A Survival Guide to the Adolescent Brain for You and Your Teen. New York: Free Press. World Bank. 2012. World Development Report 2013: Jobs. Washington, DC: World Bank. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Chapter 2 Literature Review The Relationship between Socioemotional Skills and Life Outcomes As mentioned previously, growing evidence suggests that socioemotional skills predict a range of important life outcomes. Research from Heckman, Stixrud, and Urzua (2006) uses data from the National Longitudinal Survey of Youth in the United States and indicates that noncognitive skills (measured through Rotter’s locus of control scale and Rosenberg’s self-esteem scale) have a positive and strong influence on schooling decisions, employment, work experience, occupational choice, and wages, while simultaneously minimizing risky behaviors (such as smoking, participation in illegal activities, and unplanned pregnancy). Carneiro, Crawford, and Goodman (2007) find similar evidence for the United Kingdom using information from the National Child Development Survey. Their analysis shows that noncognitive or social skills (measured using the Bristol Social Adjustment Guides, which capture social maladjustment) are positively associated with important education and labor market outcomes at different follow-up periods (at ages 16, 23, and 42). For instance, individuals with higher social skills experience longer school trajectories (remaining in school beyond age 16), higher graduation rates from tertiary education (obtaining a degree by age 42), better employment opportunities and higher wages (by age 42), a greater number of months of accumulated work experience between ages 23 and 42, lower smoking rates at age 16, lower teenage pregnancy and criminality rates at ages 16 and 42, and better health at age 42. In particular, studies that focus on the role of socioemotional skills in labor market outcomes report statistically significant effects. Using information on 5,025 graduates from Wisconsin high schools in the United States, Muller and Plug (2006) report that personality (measured using the Big Five scale) in high school had a statistically significant effect on earnings later in life. They also report that the magnitude of the effect was comparable with effects commonly generated by cognitive ability.1 Also, Kuhn and Weinberger (2002) document that leadership skills produce higher wages for adults in the United States, even when cognitive skills are held constant. For their part, using data from Germany, Taking Stock of Programs to Develop Socioemotional Skills   7   http://dx.doi.org/10.1596/978-1-4648-0872-2 8 Literature Review Heineck and Anger (2010) find that personality traits, and especially locus of control, have an important effect on wages.2 Employers acknowledge the importance of socioemotional skills in the workplace. For example, using the STEP (Skills Toward Employment and Productivity) Survey in Vietnam, Bodewig and Badiani-Magnusson (2014) find that the most important skills that employers seek in blue-collar workers are job-specific technical skills, fol- lowed by behavioral and cognitive skills, such as teamwork and problem solving. With regard to white-collar workers, employers seek individuals with critical think- ing, problem-­ solving, and communication skills. Guerra, Modecki, and Cunningham (2014) identify the following eight skill areas that employers demand (summarized as PRACTICE): (a) problem solving, (b) resilience, (c) achievement motivation, (d) control, (e) teamwork, (f) initiative, (g) confidence, and (h) ethics. According to Job Outlook 2015 (NACE 2015), the five main attributes that employers look for on candidates’ résumés are (a) leadership, (b) ability to work in a team, (c) written com- munication skills, (d) problem-solving skills, and (e) work ethic, all of which can fall under the realm of socioemotional skills. These are followed by analytical and quan- titative skills and technical skills, which can be considered cognitive. Evidence of Programs That Modify Socioemotional Skills Given the accumulating evidence on the role of socioemotional skills in shaping key life outcomes, there is growing interest in gathering information on the pro- grams that can modify these skills. For instance, Almlund and others (2011) analyze 15 programs in developed countries. Their results show that some programs can modify personality and behavior and, as such, they may be consid- ­ ered promising mechanisms to address poverty and disadvantage.3 Guerra, Modecki, and Cunningham (2014) distill findings from 53 successful programs; a key finding from their analysis is that the socioemotional skills valued by employers and commonly targeted by programs that aim to improve labor mar- ket outcomes can be taught successfully when aligned with the optimal stage for skills development. Cost-effectiveness and effects tend to be greatest in programs targeting young children. In a systematic review of 84 research reports, mostly from the United States, Lösel and Beelmann (2003) find that socioemotional skills t ­ raining for individuals under age 18 can help prevent antisocial behavior, especially among at-risk students.4 In a review of 27 programs, mostly from developed coun- tries and covering primarily programs for young children, Kautz and others (2014) find that although programs aimed at adolescents can work, programs at early stages in life appear to be the most successful.5 This finding is supported by research from Cunha, Heckman, and Schennach (2010), who find that the opti- mal stage to invest in noncognitive skills development is during early childhood.6 To date, most of the evidence available is for comprehensive programs that aim to develop socioemotional skills along with other types of skills and behaviors, making it sometimes difficult to disentangle results. Since most systematic reviews Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Literature Review 9 and meta-analyses are not limited to stand-alone programs on socioemotional skills, it is important to interpret results carefully. Such caution is especially impor- tant for programs that have not been evaluated using rigorous methodologies. Impacts of Early Childhood Development Programs Evaluations of early childhood programs to develop socioemotional skills, which are more readily available than programs later in life, show positive effects. For example, Baker-Henningham and López Bóo (2010) aim to identify the effec- tiveness of early childhood stimulation in developing countries and report that, regardless of nutritional supplementation, early stimulation benefits cognitive development. Their study is not limited to experimental or quasi-experimental evaluations and does not yield conclusive evidence on the effects of early child- hood programs in developing countries on schooling, behavior, or maternal out- comes. Finally, after analyzing experimental evidence from 31 home-visiting programs for pregnant women and parents of young children, Olds and Kitzman (1993) find that not all programs work; the programs more likely to succeed are those that are comprehensive in focus, involve frequent visits and well-trained professionals, and serve at-risk families. Impacts of School-Based Programs School-based programs can also prove effective. Heckman, Stixrud, and Urzua (2006) and Kautz and others (2014) find that schooling generates cognitive and noncognitive skills even after controlling for reverse causality (the fact that stronger skills may lead to more schooling). Durlak and others (2011) analyze the results from 213 broad school-based learning programs to develop social and emotional skills, mostly in developed countries, and find that such programs improve social and emotional skills, attitudes, behavior, and academic performance.7 Lösel and Beelmann (2003) report findings from a meta-analysis of 84 ran- domized evaluations of skills-training programs to prevent antisocial behaviors in children of all educational levels. Their analysis shows greater effects in studies with smaller samples. They also find that the majority of evaluations reviewed have higher positive effects on social and cognitive outcomes than on antisocial behavior.8 Finally, for a study published by the Collaborative for Academic, Social, and Emotional Learning, Payton and others (2008) summarize the results from 317 studies of children in kindergarten through eighth grade and find that pro- grams can be effective whether they are implemented during or after school, whether they target students with or without behavioral problems, whether they are implemented in different grades or locations (urban, rural, or suburban), and whether they include diverse racial and ethnic characteristics.9 These findings are consistent with a more recent study from Domitrovich and others (2013), which focuses on 11 middle school and high school programs in the United States where preliminary findings show promising effects. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 10 Literature Review After-school programs, primarily from the United States, can also show prom- ising results. Durlak (1997) studied 69 after-school programs implemented from kindergarten through high school in the United States with the objective of fostering socioemotional learning. The review shows that some programs signifi- cantly affected behaviors and academic achievement. Those effects were particu- larly strong for programs that followed the “SAFE approach,” that is, they were appropriately sequenced, active, focused, and explicit.10 A report from the Afterschool Alliance (2014) analyzes more than 70 evaluations of after-school programs in the United States and concludes that those programs generate important gains for children in improving academic performance, safety, disci- pline, school attendance, and avoidance of risky behavior.11 Impacts of Training Programs Out-of-school initiatives with socioemotional skills components are usually training programs that seek to improve job-related outcomes. Based on evidence from developed countries, Heckman, Lalonde, and Smith (1999) and Kluve and others (2007) find inconclusive evidence of the effects of training programs on labor market–related outcomes. In contrast, Martin and Grubb (2001) state that formal classroom training and on-the-job training appear to help female reentrants. Dar and Tzannatos (1999) examine nearly 100 evaluations of active ­ labor market programs and find similar results: training for the long-term unem- ployed can help when the economy is improving, and youth training generally has no positive effect on employment or earnings. Programs that foster labor market–related outcomes in the developing world are rarely analyzed, but existing evidence appears promising. Urzúa and Puentes (2010) broaden this literature to include studies of programs around the world. In their analysis of 215 job-training programs, they find that effective programs tend to be intensive and integrated, involve the public sector, and start early. However, only 30 of these programs had experimental evaluations (23 of them in the United States), and those showed positive or neutral results. Similarly, Betcherman and others (2007) gather evidence from 289 studies from 84 countries (42 percent of them in member countries of the Organisation for Economic Co-operation and Development) and observe that evidence in developing countries is lacking and that the level of program evaluation is weak (close to 40 percent of all of the programs included had no evaluation information on outcomes or effects). With the existing information, they report that most programs appear to have positive labor market impacts, but just over half of the programs that had cost–benefit analyses were cost-effective (only 25 had such analyses, and 14 were found to be cost-effective). Further, the impact on youth employment appears to be more favorable in developing and transition countries than in developed countries. Card, Kluve, and Weber (2015) analyze 200 studies, finding that effects are significant two to three years after program completion. They also find greater effects for programs that focus on human capital accumulation and for those that are implemented during a recession. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Literature Review 11 Both developed and developing countries have many programs for out-of- school youth focusing on improving employability and employment prospects. However, only the new wave of employability programs tends to include socioemotional skills as part of their comprehensive packages. For example, Honorati and McArdle (2013) identify successful programs in developing coun- tries as those that are demand driven, have a sound governance structure, and take a comprehensive approach (combining different types of skills training with support services). Ibarrarán and Rosas-Shady (2009) find that in contrast to evi- dence from developed countries, the results from seven comprehensive programs in Latin America and the Caribbean range from modest to meaningful. González- Velosa, Ripani, and Rosas-Shady (2012) analyze programs in Latin America and point out that, because of low levels of investment, only modest (but sustained) effects could be expected. Notes 1. “The Big Five taxonomy of personality traits is now widely accepted as the organiza- tional structure of personality traits … consists of conscientiousness, openness to experience, neuroticism, agreeableness, and extraversion” (Pierre and others 2014, 28). Refer to page 17 of their article for further details. 2. The authors measure locus of control, reciprocity, and the Five Factor Personality Inventory. 3. Personality is measured using Rotter’s locus of control scale, Rosenberg’s self-esteem scale, personal behavior (absences and truancies, lying and cheating, stealing, and swearing or use of obscene words), externalizing behavior, the Big Five scale, effort, initiative, disruptive behavior, and so forth. 4. Social skills were measured through reported antisocial behavior; social skills such as social interaction skills and prosocial behavior; or social cognitive skills like self-control and social problem-solving skills. 5. The authors measure noncognitive skills, including externalizing and internalizing behavior, creativity, verbal intelligence, hostility, beliefs and attitudes, impulse control, and self-esteem. 6. The explanation behind this finding is that higher initial levels of skills increase the productivity of future investments in skill generation (investment in socioemotional skills early in life builds the base for subsequent investment). 7. For a list of the outcomes included in this research, please refer to footnote 7 in their article. 8. For more information, refer to footnote 13 in their article. 9. Payton and others (2008) group social and emotional competencies in self-awareness, self-management, social awareness, relationship skills, and responsible decision making. 10. The author measures social and emotional skills, including “identifying emotions from social cues, goal setting, perspective taking, interpersonal problem solving, conflict resolution, and decision making” (Durlak 1997, 410). 11. These variables were measured through school attendance, class participation, home- work completion, attitudes toward school, self-concept, and decision making, among others. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 12 Literature Review References Afterschool Alliance. 2014. “Taking a Deeper Dive into Afterschool: Positive Outcomes and Promising Practices.” Afterschool Alliance, Washington, D.C. Almlund, Mathilde, Angela Lee Duckworth, James J. Heckman, and Tim D. Kautz. 2011. “Personality Psychology and Economics.” NBER Working Paper 16822, National Bureau of Economic Research, Cambridge, MA. Baker-Henningham, Helen, and Florencia López Bóo. 2010. “Early Childhood Stimulation Programs in Developing Countries: A Comprehensive Literature Review.” Inter- American Development Bank, Washington, DC. Betcherman, Gordon, Martin Godfrey, Susana Puerto, Friederike Rother, and Antoneta Stavreska. 2007. “A Review of Interventions to Support Young Workers: Findings of the Youth Employment Inventory.” Social Protection Discussion Paper 715, World Bank, Washington, DC. Bodewig, Christian, and Reena Badiani-Magnusson, with Kevin Macdonald, David Newhouse, and Jan Rutkowski. 2014. Skilling Up Vietnam: Preparing the Workforce for a Modern Market Economy. Washington, DC: World Bank. Card, David Edward, Jochen Kluve, and Andrea Weber. 2015. “What Works? A Meta- Analysis of Recent Active Labor Market Program Evaluations.” NBER Working Paper 21431, National Bureau of Economic Research, Cambridge, MA. Carneiro, Pedro, Claire Crawford, and Alissa Goodman. 2007. “The Impact of Early Cognitive and Non-Cognitive Skills on Later Outcomes.” Discussion Paper 0092, Centre for the Economics of Education, London School of Economics. Cunha, Flavio, James J. Heckman, and Susanne M. Schennach. 2010. “Estimating the Technology of Cognitive and Noncognitive Skill Formation.” Econometrica 78 (3): 883–931. Dar, Amit, and Zafiris Tzannatos. 1999. Active Labor Market Programs: A Review of the Evidence from Evaluations. Washington, DC: Social Protection, World Bank. Domitrovich, Celene, Joseph A. Durlak, Paul Goren, and Roger P. Weissberg. 2013. 2013 CASEL Guide: Effective Social and Emotional Learning Programs—Preschool and Elementary School Edition. Chicago: Collaborative for Academic,Social, and Emotional Learning. Durlak, Joseph A. 1997. Successful Prevention Programs for Children and Adolescents. New York: Plenum Press. Durlak, Joseph A., Roger Weissberg, Allison Dymnicki, Rebecca Taylor, and Kriston Schellinger. 2011. “The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Programs.” Child Development 82 (1): 405–32. González-Velosa, Carolina, Laura Ripani, and David Rosas-Shady. 2012. “How Can Job Opportunities for Young People in Latin America Be Improved?” Technical Note IDB-TN-345, Inter-American Development Bank, Washington, DC. Guerra, Nancy, Katherine Modecki, and Wendy Cunningham. 2014. “Developing Social- Emotional Skills for the Labor Market: The PRACTICE Model.” Policy Research Working Paper WPS 7123, World Bank, Washington, DC. Heckman, James J., Robert J. Lalonde, and Jeffrey A. Smith. 1999. “The Economics and Econometrics of Active Labor Market Programs.” In Handbook of Labor Economics, vol. 3, edited by Orley Ashenfelter and David Card, 1865–2097. Amsterdam and New York: North-Holland. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Literature Review 13 Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24 (3): 411–48. Heineck, Guido, and Silke Anger. 2010. “The Returns to Cognitive Abilities and Personality Traits in Germany.” Labour Economics 17 (3): 535–46 Honorati, Maddalina, and Thomas P. McArdle. 2013. “The Nuts and Bolts of Designing and Implementing Training Programs in Developing Countries.” Working Paper 78980, World Bank, Washington, DC. Ibarrarán, Pablo, and David Rosas-Shady. 2009. “Evaluating the Impact of Job Training Programmes in Latin America: Evidence from IDB Funded Operations.” Journal of Development Effectiveness 1 (2): 195–216. Kautz, Tim, James J. Heckman, Ron Diris, Bas ter Weel, and Lex Borghans. 2014. “Fostering and Measuring Skills: Improving Cognitive and Non-Cognitive Skills to Promote Lifetime Success.” NBER Working Paper 20749, National Bureau of Economic Research, Cambridge, MA. Kluve, Jochen, David Card, Michael Fertig, Marek Góra, Lena Jacobi, Peter Jensen, Reelika Leetmaa, Leonhard Nima, Eleonora Patacchini, Sandra Schaffner, Christoph M. Schmidt, Bas van der Klaauw, and Andrea Weber. 2007. Active Labor Market Policies in Europe: Performance and Perspectives. Berlin and New York: Springer. Kuhn, Peter, and Catherine Weinberger. 2002. “Leadership Skills and Wages.” IZA Discussion Paper 482, Institute for the Study of Labor, Bonn. Lösel, Friedrich, and Andreas Beelmann. 2003. “Effects of Child Skills Training in Preventing Antisocial Behavior: A Systematic Review of Randomized Evaluations.” Annals of the American Academy of Political and Social Science 587 (1): 84–109. Martin, John P., and David Grubb. 2001. “What Works and for Whom: A Review of OECD Countries’ Experiences with Active Labour Market Policies.” Swedish Economic Policy Review 8 (2): 9–56. Muller, Gerrit, and Erik J. S. Plug. 2006. “Estimating the Effect of Personality on Male and Female Earnings.” Industrial and Labor Relations Review 60 (1): 3–22. NACE (National Association of Colleges and Employers). 2015. Job Outlook 2015. Bethlehem, PA: NACE. Olds, David L., and Harriet J. Kitzman. 1993. “Review of Research on Home Visiting for Pregnant Women and Parents of Young Children.” Future of Children 3 (3): 53–92. Payton, John, Roger P. Weissberg, Joseph A. Durlak, Allison B. Dymnicki, Rebecca D. Taylor, Kriston B. Schellinger, and Molly Pachan. 2008. “The Positive Impact of Social and Emotional Learning for Kindergarten to Eighth-Grade Students: Findings from Three Scientific Reviews—Technical Report.” Collaborative for Academic, Social, and Emotional Learning, Chicago. Pierre, Gaëlle, Maria Laura Sanchez Puerta, Alexandria Valerio, and Tania Rajadel. 2014. “STEP Skills Measurement Surveys: Innovative Tools for Assessing Skills.” Social Protection and Labor Discussion Paper 1421, World Bank, Washington, DC. Urzúa, Sergio, and Esteban Puentes. 2010. “La evidencia del impacto de los programas de capacitación en el desempeño en el mercado laboral.” Inter-American Development Bank, Washington, DC. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Chapter 3 Definitions: What Are Socioemotional Skills? In this chapter we present some of the most cited definitions used to refer to the broad concept of socioemotional skills. In trying to define this concept, most authors make reference to a list of personality traits1 that have been found useful in many facets of life. Authors explicitly state that socioemotional skills are dif- ferent from traditional intelligence quotient (IQ) measures or from raw intelli- gence. At the same time, they recognize that socioemotional skills interact with intelligence, which must be taken into consideration when measuring outcomes and estimating causal relationships. As illustrated by figure 3.1, there are diverse definitions that refer to the broad concept of socioemotional skills. However, despite their apparent differ- ences, the definitions presented subsequently describe similar underlying con- cepts. Heckman and Kautz (2013) suggest that all of them refer to the same concept and that they are often used interchangeably. This similarity is observed in several of the papers analyzed here. For example, in the impact evaluation of the program Juventud y Empleo (Youth and Employment Program), Ibarrarán and others (2012) use the terms socioemotional skills, non- cognitive skills, life skills, and soft skills to describe the intervention and its observed effects. Duckworth and Yeager agree with this position and state that “all of the ... terms refer to the same conceptual space, even if connotations differ” (2015). They further mention that all of the attributes are “(a) concep- tually independent from cognitive ability, (b) generally accepted as beneficial to the student and to others in society, (c) relatively rank-order stable over time in the absence of exogenous forces ... (d) potentially responsive to intervention, and (e) dependent on situational factors for their expression” (Duckworth and Yeager 2015). Although the definitions presented here are drawn from the available literature base of socioemotional abilities, we recognize that some important ­ concepts are beyond the scope of this review and, as such, are not included. Taking Stock of Programs to Develop Socioemotional Skills   15   http://dx.doi.org/10.1596/978-1-4648-0872-2 16 Definitions: What Are Socioemotional Skills? Figure 3.1  Defining Socioemotional Skills Socioemotional life skills Personal Soft skills qualities Similar concepts 21st century Noncognitive skills skills Character skills These traits are professional skills, life-coping skills, and performance character, for example. This section is intended to present the concepts typically associated with pro- grams that seek to instill changes in behavior, personality traits, customs or habits, grit, and others. Socioemotional Skills The concept of socioemotional skills is typically found in the psychology litera- ture, particularly in publications by psychologists and psychiatrists whose work is related to education or the learning process. Authors such as Payton and others (2008) and Durlak and others (2011) define socioemotional competencies as skills that enable individuals to (a) recognize and manage their emotions, (b) cope successfully with conflict, (c) navigate interper- sonal problem solving, (d) understand and show empathy for others, (e) establish and maintain positive relationships, (f) make ethical and safe choices, (g) contrib- ute constructively to their community, and (h) set and achieve positive goals. Accordingly, they define socioemotional learning as the process of acquiring and applying this set of competencies effectively. Pierre and others state that socioemotional skills “relate to traits covering mul- tiple domains (such as social, emotional, personality, behavioral, and attitudinal)” Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Definitions: What Are Socioemotional Skills? 17 (2014, 8). The NoVo Foundation, a private foundation that supports initiatives for girls and women, defines social and emotional learning (SEL) as “the process of developing fundamental skills for life success ... SEL’s skills include recognizing and managing emotions, developing caring and concern for others, establishing positive relationships and making responsible decisions.”2 One advantage of the term skill in the previous definitions is that it denotes malleability, which opens the possibility for programs that can generate change. However, not everyone agrees. Duckworth and Yeager contend that “referring to them as skills may implicitly exclude beliefs (for example, growth mindset), val- ues (for example, prosocial motivation), and other rational attitudes (for exam- ple, trust)” (2015). Soft Skills The Oxford Dictionaries website defines soft skills as all “personal attributes that enable someone to interact effectively and harmoniously with other people.”3 Note that the term soft skills is typically found in business and ­ management literature. In fact, more than 80 percent of the papers found in the Education Resources Information Center—or ERIC—database that include the specific term soft skills present findings from analyses of issues in postsecondary education. For example, those papers often analyze programs that target managers and executives or programs that promote leadership skills in business. Bancino and Zevalkink define them as “the cluster of personality traits, social graces, facility with language, personal habits, friendliness, and optimism that mark people to varying degrees” (2007, 22). Kaplan, Sorensen, and Klebanov (2007) and Kaplan, Klebanov, and Sorensen (2012) mention similar characteristics as those embraced by Bancino and Zevalkink, including creativity, listening skills, and team skills. Similarly, Heckman and Kautz use the concept of soft skills to describe “personality traits, goals, motivations, and preferences that are valued in the labor market, in school, and in many other domains” (2012, 451). Although the previous definitions are not entirely overlapping, the concept of soft skills is similar to that of socioemotional skills. Noncognitive Skills Another commonly used term is noncognitive skills, an expression used primarily by economists. They typically use noncognitive skills to refer to the bundle of skills that are not covered in standard definitions of cognitive skills (numeracy, literacy, and so forth). According to Messick, “once the term cognitive is appro- priated to refer to intellective abilities and subject-matter achievement in con- ventional school areas ... the term non-cognitive comes to the fore by default to describe everything else” (1978, 2). However, leading scholars outside of economics find the term unfitting. As pointed out by Duckworth and Yeager (2015), using the term noncognitive Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 18 Definitions: What Are Socioemotional Skills? has a number of shortcomings. First, because the term is broad, it is “difficult to define with precision, often misinterpreted because of lack of consensual ­ definitions, hard to measure ... and representative of heterogeneous rather than homogenous categories” (Duckworth and Yeager 2015). Heckman and Rubinstein suggest that the term is unhelpful and elusive, given that “no single factor has yet emerged to date in the literature on non-cognitive skills, and it is unlikely that one will ever be found, given the diversity of traits subsumed under this category” (2001, 145). Another often cited limitation of using the term noncognitive skills is that it indirectly implies that all that is contained under the noncognitive skills category (for example, personality, self-control, grit, perseverance, dependability, persis- tence, self-esteem, optimism, and time preference) is unrelated to cognition. This implication is inaccurate in psychology, as facets of psychological functioning involve processing information and thus are inherently cognitive (Duckworth and Yeager 2015).4 Character Skills In the psychology literature, Colman defines character skills as “the aggregate or combination of psychological traits that distinguish a person from others” (2009, 125). In the economics literature, Heckman and Kautz refer to the term charac- ter skills “to describe the personal attributes not thought to be measured by IQ tests or achievement tests” (2013, 10). They suggest character skills include “per- severance (“grit”), self-control, trust, attentiveness, self-esteem and self-efficacy, resilience to adversity, openness to experience, empathy, humility, tolerance of diverse opinions, and the ability to engage productively in society” (2013, 6). Detractors of this terminology point out that character and virtue are admirable qualities with a positive connotation and that psychological traits or attributes do not necessarily need to be positive or negative. Personality Traits and Temperament A number of studies use personality and temperament to define socioemotional skills. Psychologists have a long-established tradition of studying the determinants, characteristics, and instruments to measure with a level of precision an individual’s personality traits and temperament. According to the American Psychological Association, “personality refers to individual differences in characteristic patterns of thinking, feeling, and behaving.”5 Sanson, Prior, and Kyrios (1990, 179) state that temperament “refers to early-appearing, intrinsic individual differences in the style of behavior, as distinct from its content or motivation.” Psychologists have made significant advances in personality measurement, and today there is a wide array of taxonomies to measure personality. Some of the most commonly used include the Big Five, the Big Three, the MPQ, and the Big Nine.6 This diversity of instruments should not be interpreted as a lack of agree- ment on the measurement of personality and temperament. Rather, the diversity Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Definitions: What Are Socioemotional Skills? 19 should be seen as an ability to measure finer grades embedded in the measure- ment of personality and temperament (Rothbart, Ahadi, and Evans 2000). Of these measurement instruments, the Big Five is a well-accepted and widely used taxonomy. It measures the following five facets of personality: (a) openness to experience, (b) conscientiousness, (c) extraversion, (d) agreeable- ness, and (e) neuroticism. The five are usually referred to as OCEAN. Whereas there is generally agreement on the measurement of personality, there is a lack of consensus on the extent to which personality can be used to denote the presence or absence of socioemotional skills. A key disagreement in the literature relates to the practice of equating “personality traits” to competen- cies, skills, attitudes, beliefs, or values, given that personality and temperament tend to remain stable over time. However, research has found that personality traits and temperament are mutable and can be altered by experiences (Almlund and others 2011). Twenty-First Century Skills or Competencies The term 21st century skills describes “a comprehensive set of skills that, along with content mastery, are what all sectors can agree are essential for success.”7 The 21st century skills encompass a wide range of competencies, including learning and innovation skills (creativity and innovation, critical thinking and problem solving, and communication and collaboration); information, media, and technology skills (information and media literacy and information and communication technology); and life and career skills (flexibility and adapt- ability, initiative and self-direction, social and cross-cultural skills, productivity and accountability, and leadership and responsibility). Whereas the 21st cen- tury skills are not labeled as socioemotional skills, it could be argued that some of the competencies can fall in the realm of socioemotional skills. Although Duckworth and Yeager (2015) embrace the skills, they contend that referring to the skills as 21st century skills might be misleading. The name implies that such skills have only come to light recently and are a requirement for the needs of this century when, in fact, some of them (for example, self-control and gratitude) have been around and deemed important well before the turn of the 21st century. Life Skills A term that has been used for some time to capture socioemotional abilities is life skills. The United Nations Educational, Scientific, and Cultural Organization (UNESCO), the United Nations Children’s Fund (UNICEF), and the World Health Organization (WHO) have used the term extensively, yet it is interesting that these organizations acknowledge that there is no universally accepted defini- tion of life skills (Singh 2003; WHO 1999). According to UNESCO’s working definition, life skills encompass “a mix of knowledge, behaviour, attitudes, and values and designate the possession of some skill and know-how to do something or Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 20 Definitions: What Are Socioemotional Skills? reach an aim. They include competencies such as critical thinking, creativity, abil- ity to organise, social and communication skills, adaptability, problem solving, and ability to co-operate on a democratic basis to shaping a peaceful future” (Singh 2003, 4; italics in original). UNICEF and WHO define life skills as “psychosocial abilities for adaptive and positive behavior that enable individuals to deal effec- tively with the demands and challenges of everyday life. They are loosely grouped in three broad categories of skills: cognitive skills for analyzing and using informa- tion, personal skills for developing personal agency and managing oneself, and interpersonal skills for communicating effectively with others.”8 These skills are linked to health, peace education, human rights, citizenship education, and other social issues. This review will subsume the various terminologies described in table 3.1 under the term socioemotional skills. This examination is done with the under- standing that these categories include certain levels of overlap and important differences. In fact, 21st century skills include many job-specific skills such as problem solving and critical thinking that are not contained under the socioemotional terminology. Table 3.1 Examples of the Constructs Measured under Each Concept Terminology Contents Socioemotional skills Cover multiple domains such as social, emotional, personality, behavioral, and attitudinal (Pierre and others 2014) Soft skills Creativity, listening skills, problem solving, creative thinking, leadership, teamwork, ability to work independently Noncognitive skills Every skill not captured by cognitive tests Character skills Performance character: “those qualities needed to realize one’s potential for excellence—to develop one’s talents, work hard, and achieve goals” (Character Education Partnership 2008) Moral character: “those qualities needed to be ethical—to develop just and caring relationships, contribute to community, and assume the responsibilities of democratic citizenship” (Character Education Partnership 2008) Personality qualities Openness, conscientiousness, extraversion, agreeableness, and neuroticism or emotional stability 21st century skills Measures of (a) learning and innovation skills, including creativity and innovation, critical thinking and problem solving, and communication and collaboration; (b) information, media, and technology skills including the ability to access, evaluate, use, and manage information; analyze media; create media products; and apply technology effectively; (c) life and career skills such as flexibility and adaptability, initiative and self-direction, social and cross-cultural skills, productivity and accountability, and leadership and responsibility Life skills Loose groups of three broad categories of skills: (a) cognitive skills for analyzing and using information; (b) personal skills for developing personal agency and managing oneself; and (c) interpersonal skills for communicating and interacting effectively with others (These skills are linked to health, peace education, human rights, citizenship education, and other social issues.) Note: This table is for illustration purposes only. It is not intended to be comprehensive. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Definitions: What Are Socioemotional Skills? 21 However, this review does not seek to convey supremacy to the term socioemotional skills or claim that it is the ideal terminology to describe the concepts.9 Notes 1. Borghans and others (2008) define personality traits as patterns of thought, feelings, and behavior. 2. Accessed April 16, 2015, http://novofoundation.org/advancing-social-and-emotional​ -learning/what-is-social-and-emotional-learning/. 3. Accessed January 20, 2014, http://www.oxforddictionaries.com/definition/english​ /­soft-skills?q=soft+skills. 4. Duckworth and Yeager (2015) mention that self-control, for example, is affected by the representation of temptations in the mind—a cognitive event. 5. See http://www.apa.org/topics/personality/. 6. The Big Three inventory captures three high-order dimensions: extraversion, neuroti- cism, and psychoticism. It usually is measured using the Eysenck Personality Questionnaire. The Big Five is a five-factor scheme that measures extraversion, con- scientiousness, agreeableness, neuroticism, and openness to experience. The Big Nine measures affiliation, potency, achievement, dependability, agreeableness, adjustment, intellectance, rugged individualism, and locus of control. Finally, the MPQ measures social closeness, social potency, achievement, control, aggression, stress reaction, and absorption (Blum and Noble 1996). 7. The Partnership for 21st Century Learning (P21) was founded in 2002 bringing together the business community, education leaders, and policy makers to analyze the skills required in the 21st century. They created the 21st century skills, which are widely known in the field. See http://www.p21.org/about-us/our-history. 8. Accessed April 16, 2015, http://www.unicef.org/lifeskills/index_7308.html. 9. This is done for consistency with various other materials and programs currently sponsored by the World Bank, such as the STEP (Skills Toward Employment and Productivity) Skills Measurement Program (Pierre and others 2014) and other reviews (Guerra, Modecki, and Cunningham 2014). We acknowledge that the termi- nology is fluid and subject to change. References Almlund, Mathilde, Angela L. Duckworth, James J. Heckman, and Tim D. Kautz. 2011. “Personality Psychology and Economics.” NBER Working Paper 16822, National Bureau of Economic Research, Cambridge, MA. Bancino, Randy, and Claire Zevalkink. 2007. “Soft Skills: The New Curriculum for Hard-Core Technical Professionals.” Techniques: Connecting Education and Careers 82 (5): 20–22. Blum, Kenneth, and Ernest P. Noble, eds. 1996. Handbook of Psychiatric Genetics.Vol. 236. Boca Raton, FL: CRC Press. Borghans, Lex, Angela Lee Duckworth, James J. Heckman, and Bas ter Weel. 2008. “The Economics and Psychology of Personality Traits.” Journal of Human Resources 43 (4): 972–1059. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 22 Definitions: What Are Socioemotional Skills? Character Education Partnership. 2008. “Performance Values: Why They Matter and What Schools Can Do to Foster Their Development.” Position paper, Character Education Partnership, Washington, DC. Colman, Andrew M. 2009. A Dictionary of Psychology. 3rd ed. Oxford, U.K.: Oxford University Press. Duckworth, Angela L., and David Scott Yeager. 2015. “Measurement Matters: Assessing Personal Qualities other than Cognitive Ability for Educational Purposes.” Educational Researcher 44 (4): 237–51. Durlak, Joseph A., Roger P. Weissberg, Allen B. Dymnicki, Rebecca D. Taylor, and Kriston B. Schellinger. 2011. “The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Programs.” Child Development 82 (1): 405–32. Guerra, Nancy, Kathryn Modecki, and Wendy Cunningham. 2014. “Developing Social- Emotional Skills for the Labor Market: The PRACTICE Model.” World Bank Policy Research Working Paper 7123, World Bank, Washington, DC. Heckman, James J., and Tim D. Kautz. 2012. “Hard Evidence on Soft Skills.” Labour Economics 19 (4): 451–64. ———. 2013. “Fostering and Measuring Skills: Programs That Improve Character and Cognition.” NBER Working Paper 19656, National Bureau of Economic Research, Cambridge, MA. Heckman, James J., and Yona Rubinstein. 2001. “The Importance of Noncognitive Skills: Lessons from the GED Testing.” American Economic Review Papers and Proceedings 91 (2): 145–49. Ibarrarán, Pablo, Laura Ripani, Bibiana Taboada, Joan Miguel Villa, and Brígida García. 2012. “Life Skills, Employability and Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design.” IZA Discussion Paper 6617, Institute for the Study of Labor, Bonn. Kaplan, Steve, Mark Klebanov, and Morten Sorensen. 2012. “Which CEO Characteristics and Abilities Matter?” Journal of Finance 67 (3): 973–1007. Kaplan, Steve, Morten Sorensen, and Mark Klebanov. 2007. “In Leadership, ‘Hard’ Skills Trump ‘Soft’ Skills.” Harvard Business Review HBR.org. https://hbr.org/2007/12​/­in​ -leadership-hard-skills-trum. Messick, Samuel. 1978. “Potential Uses of Noncognitive Measurement in Education.” ETS Research Bulletin 1978 (1): i–25. Payton, John, Roger P. Weissberg, Joseph A. Durlak, Allison B. Dymnicki, Rebecca D. Taylor, Kriston B. Schellinger, and Molly Pachan. 2008. “The Positive Impact of Social and Emotional Learning for Kindergarten to Eighth-Grade Students: Findings from Three Scientific Reviews—Technical Report.” Collaborative for Academic, Social, and Emotional Learning, Chicago. Pierre, Gaëlle, Maria Laura Sánchez Puerta, Alexandria Valerio, and Tania Rajadel. 2014. “STEP Skills Measurement Surveys: Innovative Tools for Assessing Skills.” Social Protection and Labor Discussion Paper 1421, World Bank, Washington, DC. Rothbart, Mary K., Stephan A. Ahadi, and David E. Evans. 2000. “Temperament and Personality: Origins and Outcomes.” Journal of Personality and Social Psychology 78 (1): 122–35. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Definitions: What Are Socioemotional Skills? 23 Sanson, Ann, Margot Prior, and Michael Kyrios. 1990. “Contamination of Measures in Temperament Research.” Merrill-Palmer Quarterly 36 (2): 179–92. Singh, Madhu. 2003. “Understanding Life Skills.” Paper commissioned for the Education for All Global Monitoring Report 2003/04 Gender and Education for All: The Leap to Equality, United Nations Educational, Scientific, and Cultural Organization Institute for Education, Hamburg, Germany. WHO (World Health Organization). 1999. “Partners in Life Skills Education: Conclusions from a United Nations Inter-Agency Meeting.” WHO, Department of Mental Health, Geneva. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Chapter 4 Conceptual Framework A number of programs foster socioemotional skills to achieve diverse objectives, such as child development or labor market–related outcomes. Such programs offer a range of different services and target diverse population groups. For example, some programs offer monthly home visits to provide early stimulation activities for children who live in marginalized households and communities. Other programs focus on providing classroom instruction to develop specific skills for youths or young adults. Those differences, along with where and how a given program is implemented, must be taken into account when determining the program’s overall success. Drawing lessons from diverse program offerings and target groups is complex and calls for a way to organize a program’s charac- teristics, results, and outcomes. The first and most important task in categorizing programs is to identify whether they occur inside or outside the formal education system. Programs that take place in a school tend to have a different set of characteristics than those that happen outside of school. For example, school-based programs typically target universal populations. Out-of-school programs focus on specific groups, with a vulnerability criterion attached to participation. Another important distinction is the age of participants. Programs built around the school system usually target students ranging from ages 4 to 18—those in preschool through grade 12. Out-of-school programs are more likely to target (a) infants and younger children for early childhood development programs or (b) adolescents and young adults ages 11 to 25. The programs included in this review are organized into three categories, according to when they are implemented in the life cycle: (a) before school (infants and young children), (b) school based (preschool to secondary educa- tion), and (c) out of school (usually targeted, vulnerable populations). As a general rule, programs for infants and young children tend to have longer follow-ups on participants that generate more and richer information on out- comes and effects. School-based programs target enrolled students with a sepa- rate curriculum that focuses solely on socioemotional skills. Programs outside the Taking Stock of Programs to Develop Socioemotional Skills   25   http://dx.doi.org/10.1596/978-1-4648-0872-2 26 Conceptual Framework school usually focus on objectives (including labor market–related outcomes) that are above and beyond the development of socioemotional skills. Figure 4.1 presents the conceptual framework that will be used throughout this book. This stylized image organizes the programs in this review according to age of beneficiaries and relationship with the formal education system. To launch the review of programs, we identified what matters most in program features and participant characteristics for achieving program objectives ­ and outcomes (figure 4.2). Every program is immersed in an environment that inevitably mediates its outcomes. An analysis of the broad social, cultural, ­ economic, and political context in which each program takes place is beyond the scope of this review. Program Characteristics The review analyzes evidence from programs with randomized or quasi-­ randomized evaluations that included the information needed to calculate the statistical significance of the effect sizes. We will carefully assess the main features of each program, including (a) objectives, (b) components and complementary elements, (c) program setting, and (d) program quality levels. Although all programs selected for this review aim to foster socioemotional skills, they have a wide range of objectives, including (a) promot- ­ ing young children’s learning potential, (b) improving school readiness, (c) reduc- ing children’s developmental risks associated with poverty, (d) reducing high school dropout rates, and (e) improving the economic prospects (labor participa- tion, earnings, socioeconomic status) of jobless youths. This wide range of objectives translates into different program designs and, ultimately, outcomes and effects. Few of the programs reviewed focus exclusively on the development of Figure 4.1 Conceptual Framework Senior Middle Primary Before school School-based 3 5 8 11 14 17 18 +55 Center based and home visiting Age Out-of-school programs Professional training Mentoring Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Conceptual Framework 27 Figure 4.2  Basic Elements Analyzed in Each Program Program characteristics – Objectives – Components and complementary elements – Setting Outcomes – Health Context – Risk factors – Academic and cognitive – Economic Participant profile – Age – Targeting system socioemotional skills. However, all programs included in this review have socioemotional skills as one of their explicit objectives, and all have a clear com- ­ ponent or activity to develop such skills. The second factor that we will analyze is the program components and complementary elements. Although programs have a commonly shared goal (fostering socioemotional skills), no two programs are alike. Some pro- grams are narrowly constructed with only one component, whereas other programs are broad based and include multiple components. In some cases, components of the broad-based programs have elements that are intertwined. Examples of component activities include family or community training, class- room instruction, and experiential learning. Complementary elements can include, for example, health access stemming from participating in early child- hood services to mentoring and networking opportunities for eligible benefi- ciaries of training. We also study the setting in which program services are provided. The setting is important given that socioemotional skills are usually moderated by diverse environmental characteristics. Bearing this fact in mind, we analyze the different effects of programs that occur in diverse environments. For example, programs that are implemented in institutions (nurseries, schools, or the workplace) occur in controlled, structured, and organized environments compared with programs implemented at home or in multipurpose community or training centers. Another important feature that we analyze relates to the quality of programs. The analysis will show that (a) program content (for example, curricula), Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 28 Conceptual Framework (b) delivery method (for example, pedagogy, duration, intensity, and class or group size), and (c) characteristics of individuals delivering the program (for example, their qualifications, professional training, and experience) are all critical elements of any program and determine the achievement of desired outcomes. Participants’ Characteristics An essential distinction among programs is related to the characteristics of eligible participants (for example, age, socioeconomic background, educational attainment, vulnerability, personality, and previous work experience). Our analy- sis will focus on two main aspects: age group and targeting mechanisms. As stated previously, most of the available research on the process of skills formation is structured around specific ages. The consensus is that the skills-formation process is dynamic, and compelling evidence indicates that the life cycle contains critical or sensitive periods for optimal learning (Cunha and Heckman 2007). This evidence lends support to using age group as an area of interest, since it has important effects on the dynamics of a program and the program’s objectives. Both the literature and our review of programs indicate that program participation early in life usually involves small groups or even one-on​ -one interactions with parents or caregivers and their children. Such programs tend to have a strong health component and aim to foster both cognitive and socioemotional development in ways that differ greatly from school-based programs that focus on children between the ages of 4 and 18. Similarly, our review of programs shows that those aimed at adults most often seek to foster socioemotional skills as a way to shape participants’ immediate labor market outcomes. Our analysis finds that even though programs target specific age groups, most programs use intense screening processes to select their beneficiaries, targeting those with low socioeconomic status, high risk, vulnerability, or unemployment. A program’s targeting mechanism is important insofar as it moderates outcomes and objectives. For example, early childhood programs targeting infants with few environmental resources attempt to diminish inequity in cognitive and socioemotional development; programs that tar- get the unemployed usually develop traits that ease labor market–related outcomes.1 Outcomes Although programs (or their components) share the common goal of fostering socioeconomic skills, each is unique in the outcome indicators that are selected to determine success. Also, the monitoring and evaluation strategies attached to each program and the data collection mechanisms vary considerably. For exam- ple, some programs rely on self-reported beneficiary data, others use information collected by blind observers or specialists, and many use administrative data. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Conceptual Framework 29 To facilitate the review, reported outcomes are classified into the following four broad categories: • Health related: Targeted outcomes include substance use (tobacco, alcohol, and drugs), fertility decisions and HIV (human immunodeficiency virus) risk behaviors, mental and physical health, and mortality. • Risk factor related: Targeted outcomes include number of crimes committed, number of arrests, recidivism, running away, internalizing behavior problems (depression, stress, and so forth), externalizing behavior problems (aggression, cooperation), social competencies, attention, domestic violence, hyperactivity disorders, self-control, bullying, parental practices, marital status, and work- place satisfaction. • Academic/cognitive related: Targeted outcomes include academic performance (grades), IQ (intelligence quotient), language and literacy skills, standardized test results, visual-spatial memory, reading ability, vocabulary development, school enrollment, school attendance, high school completion, postsecondary enrollment, use of school remedial services, dropout rates, grade retentions, progress on training ­programs, and classroom climate and support. • Economic related: Targeted outcomes include migration, employment, earnings, salary, employment status, high-skilled job, reemployment, marital status, cost–benefit analysis and rates, workplace satisfaction, voucher earnings, and progress in training programs. Note 1. Being born into a disadvantaged environment leads to skill gaps between children even before school age (Paxson and Schady 2007; Rubio-Codina and others 2014). References Cunha, Flavio, and James J. Heckman. 2007. “The Technology of Skill Formation.” American Economic Review 97 (2): 31–47. Paxson, Christina, and Norbert Schady. 2007. “Cognitive Development among Young Children in Ecuador: The Roles of Wealth, Health, and Parenting.” Journal of Human Resources 42 (1): 49–84. Rubio-Codina, Marta, Orazio Attanasio, Costas Meghir, Natalia Varela, and Sally Grantham-McGregor. 2014. “The Socio-Economic Gradient of Child Development: Cross-Sectional Evidence from Children 6–42 Months in Bogota.” Journal of Human Resources 50 (2): 464–83. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Chapter 5 Search Methodology In this chapter, we describe the search strategy applied to identify relevant literature and programs seeking to foster socioemotional skills. Given our ­ program classification, a methodologically comprehensive review had to be conducted in three stages according to life cycle. In the first stage, we identified ­ literature and program evaluations of early childhood development programs, that is, those undertaken with children before they enter the formal and compul- sory education system. In the second stage, we focused on school-based programs (from preschool to secondary education). And in the third stage, we reviewed literature on programs that take place outside the formal and compulsory school environment. Segmenting our search strategy was deemed necessary as key defi- nitions, relevant programs, and associated literature vary according to the age of the population and program settings. Within each stage, the search contained three phases: • Identification: During this phase, we identified potential studies by examining specialized and multidisciplinary databases, analyzing the references section of pertinent papers, searching the websites of universities and international devel- opment institutions, and consulting with experts, among others. Papers and programs selected in this phase aimed to affect socioemotional skills. • Screening: The papers selected in the first phase were further analyzed to determine if the complete program file and component details were readily accessible, if they were written in English after the year 2000, if they kept a strict methodological approach (randomized or quasi-randomized control ­ trials), if they had sufficient sample sizes, and if they reported information on standard errors. • Inclusion: The screening phase usually resulted in the identification of more than one paper evaluating the effects of a particular program. When that was the case, we kept the relevant follow-ups and papers that had a wider variety of outcomes analyzed with a rigorous approach. Taking Stock of Programs to Develop Socioemotional Skills   31   http://dx.doi.org/10.1596/978-1-4648-0872-2 32 Search Methodology Search Phase 1: Programs before Formal Education For this segment of the literature, we reviewed four different sources: (a) papers identified using the search engine of the American Economic Association under the sole keyword early childhood, (b) papers and policy documents referenced by international development agencies, (c) programs identified by University of Chicago professor James Heckman,1 and (d) papers referenced in revised meta- analyses or in other documents analyzed. During each of these phases, special effort was made to include credible research that took place in developing countries. That effort resulted in the identification of 118 potential studies that aimed to affect socioemotional skills. Of those studies, 111 were accessible, written in English, and methodically strict. After eliminating multiple follow- ups for the same program, 82 studies were included in the early childhood stage. Search Phase 2: School-Based Programs For the school-based portion of the literature, a set of relevant keywords and search terms were used to scan a short list of online newspapers and websites to identify relevant programs, researchers, and institutions working on socioemotional skills at a school level.2 Three sources were useful (Gelles 2015; ­ Kahn 2013; Kimball and Smith 2013). All were written for a general audience, referenced academic institutions, and cited researchers working on the topic. From those articles, we identified nine relevant institutions, six researchers who were repeatedly mentioned, and four programs directly related to our topic of interest. Using this information, we searched for relevant academic literature in peer-reviewed journals and electronic databases. In particular, we extended our initial list of keywords with the names of the programs previously identi- fied and conducted systematic searches on electronic databases and policy discussion websites. The following electronic databases and institutions were reviewed: • Databases: ERIC, EBSCOhost, EconPapers, JSTOR, Web of Science, Web of Knowledge, ProQuest, and Google Scholar. • Institutions: American Psychological Association; Collaborative for Academic Social, and Emotional Learning; Office of Juvenile Justice and Delinquency Prevention of the U.S. Department of Justice; and the Inter-American Development Bank. We analyzed 247 reports and found 98 academic articles evaluating the effects of different programs aimed at fostering socioemotional skills. We kept only those that were written after the year 2000 and targeted universal population groups. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Search Methodology 33 Excluded from our search strategy were programs that targeted specific condi- tions or groups of individuals. We acknowledge the limitation of excluding pro- grams that target specific conditions or groups of individuals, but the available alternatives were not time efficient and cost-effective. However, we decided to include some insights from targeted programs that were mentioned in other compilations or meta-analyses. In cases where more than one paper was written about a specific program, we decided to keep only those that were relevant follow-ups and those that used a wider variety of outcomes. We ended up managing a sample of 48 different studies and 45 different ­ programs. Most of those 45 programs were conducted in a school environment and targeted populations between the ages of 3 and 18. Search Phase 3: Out-of-School Programs Our search strategy for out-of-school programs was broader than in the first two phases. The reason was that no unified pattern existed to organize the literature by a small set of words, concepts, or ideas,3 and therefore there was no unique place to compile all programs. Our approach was as follows: • We identified papers and policy documents that most international devel- opment agencies highlighted in their reports on youth employability. We were particularly careful to include evidence regarding developing countries. • We reviewed the programs examined by Heckman and Kautz (2013). • We created a database of effective out-of-school violence-prevention programs as suggested by the Office of Juvenile Justice and Delinquency Prevention4 and the National Registry of Evidence-Based Programs and Practices of the U.S. Substance Abuse and Mental Health Services Administration.5 Of the 130 documents initially identified, 42 records were screened. All others were neither up-to-date nor high-quality research (randomized control trial or quasi-experiment) written in English. Once we eliminated the multiple evaluations of the same program, the resulting list of out-of-school, labor market–related programs included 20 different programs and eight compilations. Figure 5.1 illustrates the flow of the literature review process that we conducted to identify studies and literature of socioemotional skills. As a result, the analysis includes more programs than previous systematic reviews. Further, it includes as much evidence from developing countries as possible. Finally, it does not concentrate on one particular socioemotional skill or one particular life stage. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 34 Search Methodology Figure 5.1 Literature Review Process to Identify Socioemotional Programs Before school School-based Out of school 78 records 20 reports identified from 167 records 52 records Identification 98 records 80 records through database identified identified identified identified database search and through through through through search and international database other references references international development searching sources development institutions institutions Screening 98 full texts 111 records 42 records assessed for screened screened eligibility Included 82 studies included 48 studies included 29 studies included 21 programs 45 programs 20 programs 7 compilations 3 compilations 8 compilations Notes 1. See http://heckman.uchicago.edu/page/early-childhood-interventions-0. 2. The following set of keywords were searched: noncognitive (with and without a hyphen), socioemotional, social-emotional learning, self-restraint, persistence, self-­ awareness, perseverance, self-esteem, locus of control, internalities, Big Five personality traits, youth behavior problems, moral capital, consistency, dependability, persistence, emotional intelligence, emotional skills, and socioemotional learning. The online papers and websites that were used are the New York Times, Washington Post, Boston Globe, Huffington Post, Financial Times, and Quartz. 3. To our knowledge, Heckman and Kautz (2013) provide the only unified framework in the literature. 4. National Institute of Justice, Crime Solutions web page, http://www.crimesolutions​ .gov. 5. Substance Abuse and Mental Health Services Administration website, http://nrepp​ .samhsa.gov. References Gelles, David. 2015. Mindful Work: How Meditation Is Changing Business from the Inside Out. Boston: Eamon Dolan/Houghton Mifflin Harcourt. Heckman, James J., and Tim Kautz. 2013. “Fostering and Measuring Skills: Programs That Improve Character and Cognition.” NBER Working Paper 19656, National Bureau of Economic Research, Cambridge, MA. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Search Methodology 35 Kahn, Jennifer. 2013. “Can Emotional Intelligence Be Taught?” New York Times, September 11. http://www.nytimes.com/2013/09/15/magazine/can-emotional​ -intelligence​-­be-taught.html?pagewanted=all&_r=0. Kimball, Miles, and Noah Smith. 2013. “There’s One Key Difference between Kids Who Excel at Math and Those Who Don’t.” Quartz, October 27. http://Qz.Com/139453​ /­Theres-One-Key-Difference-Between-Kids-Who-Excel-At-Math-And-Those-Who​ -Dont/. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Chapter 6 Program Analysis throughout the Life Cycle Before-School Programs The early stages of life are critical for an individual’s healthy development. However, the unequal opportunities faced by the most vulnerable at-risk infants lead to an achievement gap that emerges even before preschool (Melmed 2008; Rubio-Codina and others 2014; Shady and Paxson 2007). Early childhood pro- grams that improve nutrition, cognitive skills, and socioemotional skills have gained the attention of scholars and policy makers, as these programs have the potential to prevent this achievement gap from forming early in children’s lives and, therefore, to promote equality. Additionally, such programs have a high rate of return and yield greater results for vulnerable children than do equal invest- ments made during a later stage in life (Heckman 2008; Heckman, Stixrud, and Urzua 2006; Heckman and others 2009; Rolnick and Grunewald 2007). Because initial levels of skills affect skill acquisition (a phenomenon described as “skills beget skills”), investments early in life provide the foundation to increase the efficiency of further investments (Cunha and others 2005). Seeking to capture the existing evidence regarding the effectiveness of early childhood programs on socioemotional skills, we analyze the effects of 21 before-school programs, 10 of which take place outside the United States.1 All of the programs that were included were rigorously evaluated through random- ized control trials or quasi-experimental techniques, with follow-up periods that range from two months to 37 years. Almost half of the analyzed programs mea- sure results more than five years after program implementation. Programs that begin before the formal schooling begins typically target children younger than 5 years old, and some begin even before birth (see tables 6.1, 6.2, and 6.3). These programs focus not only on young children or newborns but also on their families. Outside the United States, all these pro- grams target at-risk families (in the United States, seven programs have univer- sal coverage). Taking Stock of Programs to Develop Socioemotional Skills   37   http://dx.doi.org/10.1596/978-1-4648-0872-2 38 Program Analysis throughout the Life Cycle Table 6.1  Distribution of Programs by Age Covered Months Years Program name Country Pregnancy 1 2 3 4 5 6 7 8 9 10 11 12 2 3 4 5 6 Home visiting programs Nurse-Family United States Partnership Healthy Families United States Parenting Canada Practices Parents as United States Teachers (PAT) Attachment and United States Biobehavioral Catch-Up Intervention (ABC) Comprehensive United States Child Development Program Durham United States Connects Early Childhood Bangladesh Parenting Programme Positive Action United States Play and Learning United States Strategies Jamaican Study Jamaica Integrated Early Colombia Childhood Development Intervention Parent-Child United States Home Program Child-Parent United States Psychotherapy HIPPY United States Center-based programs Early Childhood Indonesia Education and Development program Early Childhood Philippines Development Program table continues next page Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 39 Table 6.1  Distribution of Programs by Age Covered (continued) Months Years Program name Country Pregnancy 1 2 3 4 5 6 7 8 9 10 11 12 2 3 4 5 6 Abecedarian United States PIDI Bolivia ECD intervention Mozambique Early Enrichment Turkey Project Total 7 8 7 7 7 7 10 9 9 10 10 10 11 11 10 9 6 3 Note: Shading indicates that the program includes that particular age group. ECD = early childhood development; HIPPY = Home Instruction Program for Preschool Youngsters; PIDI = Proyecto Integral de Desarrollo Infantil. Table 6.2 Salient Characteristics of Home Visiting Programs before Formal Education Program characteristics Participant profile Outcomes Objectives Age Health: Substance abuse by both Teach parents responsive parenting skills to 0 to 5 years old mothers and children; risky sexual support their child’s socioemotional and behavior; childhood mortality; cognitive development; provide early children’s injuries; mothers’ childhood care subsequent live births Components and complementary elements Targeting mechanisms Risk factors: Parent–child cooperation; Parental training Socioeconomic status, marital emotional support from parents; Health assistance status (single mothers), quality of language inputs; social young age (teenagers), engagement; arrest and conviction; Setting educational attainment of aggression; emotional and behavioral Home visiting program parents, nutritional status problems; child abuse and neglect; (stunted), unemployment risky behavior Quality Personnel: Coaches, nurses, community status Academics: Verbal communication; health workers, paraprofessional trainers, mathematical grade point average; trained parent educators, college achievement tests; enrollment in graduates school care; graduation rates; expulsion from school; classroom Intensity: 1.5 hours per week for 3 months; adaptation 1 hour per week for 2 years; 46 twice- weekly visits Economic-related factors: Use of welfare; employment; earnings Cost: US$500–US$13,600 per child Source: Based on 21 rigorously evaluated programs described in detail in appendix A. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 40 Program Analysis throughout the Life Cycle Table 6.3 Salient Characteristics of Center-Based Programs before Formal Education Program characteristics Participant profile Outcomes Objectives Age Guarantee school readiness; foster a child’s 0 to 5 years old Health: Mortality; emergency care inclination to learn; support cognitive, episodes; maternal anxiety social, physical, and emotional disorder; overnight hospital stay; development; improve health and substance abuse; teenage well-being; improve parent–child parenting; child insurance relationship coverage; health status; weight and height changes; anemia; Components and complementary elements Targeting mechanisms hemoglobin; wasted condition Classroom curriculum Socioeconomic status, marital status Health assistance (single mothers), young age Risk factors: Arrests; externalizing Parental training (teenagers), educational behavior; risky behavior; attainment of parents, nutritional relationships between children Setting status (stunted), IQ levels and parents; hyperactivity; social Center-based program skills at age three; problem solving through fairness; positive Quality peer play; positive parenting Personnel: Teachers, trained women from behaviors; self-help; self- the community, pediatric nurses with development; self-concept; master’s degrees aggression Intensity: 2.5-hour preschool program plus weekly home visits; 3 to 10 hours/day of Academics: IQ; grade point average; day care; 90-minute weekly education vocabulary; oral comprehension; sessions for parents; four to seven reading and math achievement; sessions for families grade repetition; assignment to Cost: US$516–US$67,000 special education; school enrollment; high school graduation; college attendance; educational attainment; school dropout rate; school adjustment Economic-related factors: Maternal and child earnings; homeownership; high-skilled job; asset ownership; age at first employment; parental labor engagement Source: Based on 21 rigorously evaluated programs described in detail in appendix A. Note: IQ = intelligence quotient. Overall, before-school programs can be categorized into two broad groups: (a) home visiting programs and (b) center-based programs. Home Visiting Programs Home visiting programs occur in the home, where children’s most relevant inter- actions take place. Most of these programs aim to foster child development by improving parenting practices, increasing sensitive responses, and raising the level of stimulation (language and cognitive opportunities) at home. Basic health ser- vices and the provision of micronutrients complement some programs. Program services are delivered by a wide variety of individuals, ranging from nurses to trained parent educators. They also vary in their intensity, with some Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 41 lasting three months and others up to two years. The associated cost also varies considerably: some programs cost US$500 per child per year, whereas others cost US$13,600 per participant over a three-year period. Home visiting programs have the advantage of allowing for ample participa- tion, since providers can reach difficult-to-access locations and can adapt the services to families’ schedules. In addition, when entering a home the provider gains a comprehensive view of the child and his or her family, which facilitates tailoring the service to meet the child’s specific needs. (For examples of home visiting programs, see boxes 6.1 and 6.2.) Center-Based Programs Center-based programs typically provide a combination of education and stimu- lation services with nutrition at a child care center. As these programs take place before formal schooling, they can be strongly oriented toward promoting school readiness or more heavily oriented toward providing care and guaranteeing child well-being, depending on the curriculum. They sometimes include health ­ checkups and a parental education component. Although they promote early childhood care and stimulation, these programs also facilitate the ability of the child’s main caregiver to get a job. (For examples of home visiting programs, see boxes 6.3 and 6.4.) Box 6.1 Home Visiting Program: The Jamaican Study The Jamaican Study was a home visiting program that targeted growth-stunted children ages nine months to 24 months living in poor neighborhoods in Jamaica. Participants were randomly assigned into one of four groups: (a) psychosocial stimulation, consisting of two years of weekly home visits with trained community health aides (paraprofession- als) with a curriculum that promoted language development (for example, mother–child conversations, labeling things and actions), parenting skills, educational games, the use of praise, and improvement in the self-esteem of both the child and the mother; (b) nutri- tional supplementation, consisting of one kilogram of milk formula provided weekly for  two years; (c) both psychosocial stimulation and nutritional supplementation; and (d) a control group. Short-term outcomes suggested that all three groups improved early cognitive development, but in general, psychosocial stimulation outperformed nutritional supplementation. In the long term, only the psychosocial stimulation group improved ­ cognitive and character skills, increasing the average earnings of participants at age 22 by 42 percent relative to the control group. Findings suggest psychosocial stimula- tion can have substantial effects on labor market outcomes and might reduce inequality in later life. Source: Gertler and others 2013. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 42 Program Analysis throughout the Life Cycle Box 6.2 Home Visiting Program: Nurse-Family Partnership, United States The Nurse-Family Partnership is a home visiting program that targets first-time, unmarried, low-income mothers (usually adolescents) with the objective of improving prenatal condi- tions and early parenting. Trained nurses do the following: (a) encourage mothers to improve their health-related behaviors, (b) improve children’s health and development by helping parents provide more competent care, and (c) improve families’ economic self-sufficiency by ­ helping parents make appropriate choices regarding family planning, finishing their educa- tion, and finding work. The program has been replicated in several locations and is one of the few programs with long-term follow-ups. Treated families receive between 0 and 16 home visits during preg- nancy and between 0 and 59 home visits from birth through the child’s second birthday.a During the first month, prenatal visits are weekly, after which they taper off to biweekly visits until the child is born. After birth, weekly visits resume for the first six weeks, and then biweekly visits continue until the child is approximately 20 months old. The final four visits, leading up to the child’s second birthday, occur monthly. Treated children improved their vocabulary skills, their grade point averages, and their results in achievement tests relative to the control group. Furthermore, they had a lower rate of behavioral problems in the borderline or clinical range 9 years later, and they had a lower probability of arrest and conviction 15 years later. Girls in the program had fewer children and used less Medicaid than the comparison group. Effects were stronger for girls. Sources: Eckenrode and others 2010; Kitzman and others 2010; Olds 2006; Olds, Henderson, Cole, and others 1998; Olds, Henderson, and Kitzman 1994; Olds, Henderson, Tatelbaum, and others 1986; Olds and others 1986; Olds and others 1997; Olds and others 2007; Olds and others 2010; Olds and others 2014; Zielinski, Eckenrode, and Olds 2009; Nurse-Family Partnership website, http://www.nursefamilypartnership.org. a. The evaluations of the program measure intention to treat. Therefore, if families who were assigned to the treatment group were not visited, their outcomes are still captured in the treatment group. The characteristics of center-based programs vary immensely. For example, the service providers of Proyecto Integrado de Desarrollo Infantil in Bolivia are mothers in the community, whereas the service providers of the Parenting Practices program in Canada are pediatric nurses with master’s degrees. Additionally, some of the programs involve four to seven counseling sessions for parents, whereas others involve 10 hours of daily care. These differences are reflected in the costs, which range from US$500 for 18 months (Integrated Early Childhood Development Intervention in Colombia) to US$67,000 for five years of the program (Abecedarian). Outcomes in the literature usually include child abuse and neglect, child health and safety, home environment, parental responsibility and sensitivity, parental harshness, depression, parental stress, child cognition, and child internal- izing and externalizing behavior problems. Programs usually exhibit short-term positive results, and outcomes vary from modest to multiple, positive, and persis- tent among treated children (see boxes 6.3 and 6.4). Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 43 Box 6.3 Center-Based Program: Save the Children’s Early Childhood Development Programme, Mozambique Save the Children’s Early Childhood Development Programme, carried out in rural areas of Gaza Province in Mozambique, seeks to improve children’s cognitive, social, emotional, and physical development. It has two main components: (a) high-quality early stimulation, psycho- social support, and literacy and numeracy instruction in community-based preschool centers that take care of children for 3 hours and 15 minutes every day; and (b) monthly parenting meetings oriented toward strengthening positive parenting practices. Communities are ultimately responsible for managing and sustaining the centers, which are staffed with volunteer teachers who are mentored, trained, and supervised. The program costs about US$2.50 per month per child. Among other effects, an evaluation of the program found significant increases in primary school enrollment, improvements in cognitive and problem-solving abilities, and progress in motor skills and socioemotional and behavioral outcomes. However, language, stunting, and wasting did not show significant improvements. Source: Martinez, Nadeau, and Pereira 2012. Box 6.4 Center-Based Program: HighScope Perry Preschool Program, United States The HighScope Perry Preschool Program had two components: a 2.5-hour preschool curricu- lum on weekdays during the school year and weekly home visits by teachers. The latter com- ponent was designed to involve parents with the classroom curriculum and to improve the household environment. The program targeted low-income African American children who were at high risk of school failure, and it is well-known for the outcomes it was found to have during long-term follow-up (participants were followed until they were age 40). Impact evaluations found that participants faced less grade repetition than nonpartici- pants; scored better on intellectual and language tests from ages 3 to 7; scored better on school achievement tests at ages 9, 10, and 14; and scored better on literacy tests at ages 19  and 27. Furthermore, they achieved higher levels of schooling and greater high school graduation rates. At ages 15 and 19, participants had significantly better attitudes toward school than did nonparticipants. With regard to economic performance, at age 40 participants had higher employment rates and greater median annual earnings. They were also significantly more likely to own a house or a car and reported having more in savings accounts than did nonparticipants. Importantly, participants had significantly fewer lifetime arrests and arrests for violent crimes than did nonparticipants, as well as fewer arrests for property and drug crimes and fewer subsequent prison or jail sentences. Sources: Heckman and others 2010; “HighScope Perry Preschool Study” web page, http://www.highscope.org/content​ .­asp?contentid=219. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 44 Program Analysis throughout the Life Cycle Outcomes of Before-School Programs The discussion that follows presents the main findings of programs that occur before formal schooling and that have among their components improvement in participants’ socioemotional skills. The task is particularly challenging because, as Heckman and Kautz (2013) suggest, there are more differences among evalu- ations than there are evaluations, making it difficult to understand exactly why some programs are more successful than others. In particular, (a) programs target different populations; (b) specific program components are not usually tested or evaluated; (c) even within a particular population, samples are heterogeneous in many aspects; (d) identification prob- lems might be present in some evaluations; (e) small samples decrease the statis- tical power and make it hard to detect significant impacts even when they exist; (f) not all programs measure the same outcomes; and (g) most programs have only short-term follow-ups. In other words, because of external validity consid- erations, caution was necessary when giving general conclusions. We will face this challenge using two basic inputs: our own compilation of program characteristics, outcomes, and results (see appendixes); and (b) a group of selected reports and compilations available in the literature. Findings and con- clusions will be presented using the same framework that we used in the previ- ous section, that is, organizing programs according to their relation with the formal education system and the ages of the participants involved. This section starts by presenting the primary outcomes documented in the programs analyzed and the available meta-analysis. It will then provide an over- view of the program characteristics that mediate the achievement of those par- ticular results, and, finally, it will describe consistent findings regarding the effect of the participant’s profile over the results of the programs. To get an approximate idea of the kinds of results that were captured by pro- gram evaluations at each stage of the life cycle, we first grouped the statistically significant and unique outcomes of each program by whether they relate to health, risk factors, academics, or economic aspects. For example, for Proyecto Integral de Desarrollo Infantil in Bolivia, we counted one result for health (incre- ments in weight and height for children), two results related to academics (motor skills and language skills), one result for risk factors (statistically significant effect in psychosocial skills), and another result related to economics (future earnings). Then, we added the number of outcomes in each category across programs. Finally, we calculated the percentages by dividing the outcomes measured in each category by the total number of results (measured in the sum of all of the categories).2 Most of the outcomes documented by our sample of studies of before-school programs are related to academics or risk factors. As mentioned before, programs that target individuals before formal schooling tend to have longer follow-up periods than do other types of programs. In particular, (a) 38 percent of the effects were related to academic activities, such as schooling, literacy, math skills, intelli- gence quotient (IQ), school performance, and graduation rates; (b) 31 percent of Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 45 the effects were related to risk factors, such as criminal activity, arrests, and exter- nalizing and internalizing behavior; (c) 18 percent were related to health vari- ables, such as drug use, fertility, and physical health; and (d) 13 percent were related to economic variables, such as welfare use, employment status, earnings, and marital status (see figure 6.1). In the short term, before-school programs foster cognitive skills. In the long term, they improve behaviors and reduce welfare use, which indicates that they affect socioemotional skills. A careful analysis of the studies shows that child care and preschool education programs implemented in different contexts and with different methodologies appear to have consistent short-term effects on cognitive and academic performance. Long-term studies find that the cognitive gains diminish over time, but the programs yield increased rates of high school graduation and lower rates of teen pregnancy, arrests, and substance use and abuse. As suggested by Heckman and Kautz (2013), these findings are indicative of the benefits that early prevention can bring by promoting socioemotional skills during early childhood.3 For example, for the Project Student/Teacher Achievement Ratio (STAR) pro- gram in the United States, the effects of class quality on test scores fade by eighth grade, but gains in noncognitive measures persist. A relationship exists between cognitive and noncognitive skills, and, for example, being more persistent leads to individuals’ studying for longer periods, which in turn yields higher test scores. Both the HighScope Perry Preschool Program and the Abecedarian program (also in the United States) find lasting effects on adult outcomes as well, including criminal activity and graduation, despite the fact that effects on test scores fade. Figure 6.1 Outcomes of Before-School Programs Program characteristics 38% Academic 31% Risk factors Context Outcomes 13% Economic 18% Health Participant profile Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 46 Program Analysis throughout the Life Cycle Finally, combined data from Abecedarian and Project CARE (character actualiza- tion requires education) in the United States show that the effect of early child- hood programs on the IQs of eight-year-old children is mediated by improvements in infants’ responsiveness to people and objects in their surroundings. Reported rates of return for early childhood programs are extremely high. For example, Heckman and others (2010) and Heckman and Kautz (2013) estimate that the rate of return of the HighScope Perry Preschool Program is about 7 percent to 10 percent a year. Behrman, Cheng, and Todd (2004) estimate that Proyecto Integrado de Desarrollo Infantil in Bolivia has a ratio of earning benefits to cost of 1.7 to 3.7. Additionally, the Abecedarian program was found to have benefits of US$2.50 for every US$1.00 invested. Finally, evaluations of the Nurse-Family Partnership find that the cost of the program was recovered from low-socioeconomic families before the children reached age four (Eckenrode and others 2010; Olds, Henderson, Cole, and others 1998; Olds, Henderson, and Kitzman 1994; Olds, Henderson, Tatelbaum, and others 1986; Olds and others 1986; Olds and others 1997; Zielinski, Eckenrode, and Olds 2009. However, some early childhood programs do not work as envisioned and even have negative results. For example, Loeb and others (2007) find that although exposure to a center-based program before kindergarten had positive results in prereading and math skills for children, it had similar-sized but nega- tive effects on a teacher-reported behavioral measure that captured self-control and a variety of interpersonal skills. Similarly, Magnuson, Ruhm, and Waldfogel (2007) find that prekindergarten intervention increases reading and mathemat- ics skills at school entry but also increases behavioral problems and reduces self-control. Finally, Bouguen and others (2014) find that a preschool program in Cambodia had negative or insignificant effects on some indicators of child development. The extensive differences in results might derive from the heterogeneity of the program characteristics (the dissimilar design elements and implementation realities behind each intervention). As illustrated in figure 6.2, six elements that the literature finds alter the effects of before-school programs include (a) a pro- gram’s components, (b) its curriculum, (c) its intensity, (d) the qualifications of its personnel, (e) its teacher-to-child ratio, and (f) the fidelity of its implementa- tion (including context-specific responses to the intervention). The effects of each of these six elements are reviewed next. First, programs with multiple components that target health, cognitive devel- opment, and emotional development tend to produce greater and longer-lasting effects. These conclusions may suffer from confounding factors, since multiple- component programs tend to be more intense. Additionally, the Jamaican Study and the Integrated Early Childhood Development Intervention show that nutri- tional programs tend to be less effective than stimulation programs. Furthermore, as mentioned by Kautz and others (2014), programs that involve parents are usually more effective. That finding can be explained by the fact that if the parent changes his or her practices, the home environment is transformed. After the program ends, the child still benefits from a more Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 47 Figure 6.2 Selected Program Characteristics That Affect the Outcomes of Before-School Programs Components Curriculum type Intervention intensity Program characteristics Personnel qualifications Teacher-to-child ratio Outcomes Context Implementation fidelity Participant profile s ­upportive or stimulating environment, thereby increasing the effect and the persistence of the programs. Second, as expected, the type of curriculum a program follows affects the program’s effectiveness. The HighScope Preschool Curriculum Comparison Study (Schweinhart and Weikart 1997) in the United States measured the effect of implementing three different curricula: (a) the HighScope model, where children planned, carried out, and reviewed their own activities while engaging in active learning; (b) a traditional nursery school model in which teachers responded to children’s self-initiated play in a loosely structured, socially supportive setting; and (c) a direct instruction model that involved teachers following a script to directly teach children academic skills. This study found that people born in poverty experienced fewer emotional prob- lems and felony arrests if they attended a preschool program that used either the HighScope model or a traditional nursery school model than if they attended a preschool with the direct instruction model. No significant differ- ences were found between the effects of the HighScope and the nursery school models when participants were 23 years old. When compared with the direct instruction model, beneficiaries of the HighScope and the nursery school models required less treatment for ­ emotional impairment or distur- bance, had engaged in fewer acts of misconduct, had experienced fewer arrests for felonies, had committed fewer property crimes, were more toler- ant, were more involved in volunteer work, and were more likely to plan to Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 48 Program Analysis throughout the Life Cycle graduate from college. The three models had similar effects on IQ, school achievement, and high school graduation rates. Howard and Brooks-Gunn (2009) find that for home visiting programs, using a clearly defined curriculum with a formal structure is key to inducing and pro- moting optimal program results. Programs where the content of the visits varied dramatically from family to family had inconsistent outcomes. Third, intense programs, which have greater coverage with regard to hours per day, days per week, or even years of coverage, result in greater effects. For example, in the Nurse-Family Partnership trial in New York, the group that received home visits from public health nurses during both pregnancy and infancy experienced greater benefits in most child-related outcomes than the groups that received the intervention in either pregnancy or infancy but not both (Eckenrode and others 2010; Olds, Henderson, Cole, and others 1998; Olds, Henderson, and Kitzman 1994; Olds and others 1986; Olds and others 1997; Zielinski, Eckenrode, and Olds 2009. Fourth, programs with more qualified personnel tend to be more effec- tive. In the Nurse-Family Partnership in the United States, having a parapro- fessional in charge of the visits instead of a nurse led to the program losing its effect for the general population (effects were found only for high-risk children) (Kitzman and others 2010; Olds and others 2007; Olds and oth- ers 2010). In the STAR program in the United States, students with more experienced teachers in kindergarten had higher earnings (Chetty and others 2010). Howard and Brooks-Gunn (2009) state that home visitors’ creden- tials matter, but their importance depends on the objectives or goals. In general, professional home visitors such as nurses are preferred, but for those programs that seek primarily to improve mother–infant attachments (as opposed to promoting healthy behaviors), social workers or trained para- professionals can also be beneficial. Fifth, programs with greater teacher-to-child ratios are more effective. In the STAR program in the United States, students in small classes (13 to 17 students) performed better on standardized tests, such as the SAT, and were more likely to attend college than were those in larger classes (22 to 25 students) (Chetty and others 2010; Krueger and Whitmore 2001). These results were stronger when the small class size occurred during the child’s earliest school years and dimin- ished when small class size occurred after first grade. Being in a small class did not affect student self-concept and motivation (Word and others 1990). According to Epstein (1993), the HighScope Perry Preschool Program in the United States showed this positive effect of small class size with up to two adults teaching in a class of up to 20 students. Sixth, critically, having a well-designed program is insufficient: interven- tions that do not follow the designed standards and practices have fewer or even negative effects. The fidelity of implementation (understood as faithfulness to a program’s standards and practices) can be low because of administrative or operational limitations. Countries should ensure that they have the capacity to implement the programs that they design successfully. In Cambodia, Bouguen Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 49 and others (2014) attribute the negative effects of the program they evaluated to implementation constraints, low take-up rates, and context-specific behavioral responses to the intervention.4 In many early childhood programs, implementation fidelity is low. Araujo, López-Bóo, and Puyana (2013) find that the quality standards stipulated by early childhood programs in Latin America do not translate into the daily prac- tices of the operators of these services. In particular, they identify that (a) cen- ter-based programs usually involve more children and fewer adults than those reported by program directors, forcing a child-to-caregiver ratio that might hinder beneficial outcomes; (b) although programs are de jure completely funded by local authorities, parents are usually “encouraged” to pay a fee for the services they receive from the centers; and (c) great heterogeneity exists in the daily schedule of activities. Further, they suggest that ensuring positive effects requires ensuring fidelity or compliance with implementation standards. Another important finding is that the effects of before-school programs tend to yield superior benefits among the most vulnerable populations (see figure 6.3). This finding strongly supports the use of early childhood programs as ­ mechanisms to decrease countries’ inequality levels. Even when effects are widespread, they tend to be greater for children born in high-risk environments (with younger, single, or uneducated parents; families with few socioeconomic resources; or with low birth weight). For in ­ Figure 6.3 Participant Characteristics That Affect the Outcomes of Before-School Programs Program characteristics Context Children Outcomes Participant Age, marital status, profile and educational level Low socioeconomic Mothers status High-risk environment Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 50 Program Analysis throughout the Life Cycle example, for the Nurse-Family Partnership in Tennessee (United States), sig- nificant effects were found for children at risk in the areas of grade point aver- age at age 12, achievement tests, arithmetic achievement, and aggression, among others (Kitzman and others 2010; Olds and others 2007; Olds and others 2010). The Parents as Teachers (PAT) program in the United States also achieved greater benefits for very low-income parents and their children, as well as for Latino groups who were less educated (Wagner and Clayton 1999; Wagner, Spiker, and Linn 2002). For the Parent-Child Home Program (United States), infants that started with the lowest IQs also obtained the most bene- fits (Levenstein and others 1998). Further, effects tend to be greater for low-income, first-time adolescent mothers and for those who are psychologically vulnerable. Howard and Brooks-Gunn (2009) find that programs targeting adolescent mothers tended to differ in effectiveness from programs that enrolled mothers with other backgrounds. The greatest effects were usually found among low-income, first-time adolescent mothers. Another important group that benefited most from this type of intervention was psychologically vulnerable mothers (that is, mothers who were rated high in depressive symptoms and low in self-mastery). The greater effect of programs on vulnerable beneficiaries could be attribut- able to early childhood programs leveling the playing field. That is, it could be that these programs level out the preexisting differences in stimulation and even in nutrition among children and parents of varying socioeconomic strata. Before-school programs offer a stimulating environment for infants and toddlers of all backgrounds. In their absence, children from low-resource households would face a harsher environment, so the programs provide a setting that is more productive than their homes. In contrast, infants with plentiful resources would have been highly stimulated at home even if the programs did not exist, so they benefit less from them. School-Based Programs Programs that foster socioemotional skills during the school year are both abun- dant and strategic. On the one hand, Heckman and others (2010) and Heckman and Kautz (2012) have found that important periods of brain malleability occur during the school years. Additionally, since vital personality traits—such as ­conscientiousness5—increase monotonically from childhood into late adulthood, school environments allow for continuous skills development (Borghans and oth- ers 2008). For instance, school-based programs analyzed in this book, such as Montessori and Family Check-Up, target a broad range of school grades (kinder- garten through 12) as potential beneficiaries with classroom curricula that follow a yearly sequence, have grade-specific content, and adjust program length and intensity to children’s school progression. Finally, these programs multiply the impacts of investments made in disadvantaged children during early childhood.6 Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 51 Operational or practical advantages result from working with groups of ben- eficiaries who are already present in an organized school atmosphere. According to Rutter (1987), the structured and supportive environment developed in classroom settings eases the implementation of socioemotional programs. Furthermore, since elementary school participation is widespread in most coun- tries, school-based socioemotional programs have higher enrollment and reten- tion rates. Besides, since schools are already established and working, programs that are built on top of school curricula can be implemented and scaled up more quickly. Finally, as Lösel and Beelmann (2003) state, school-based programs may have relatively low costs as they can involve group training delivered by regular teachers. Even though myriad programs aim to develop socioemotional skills in schools, in this book we analyze 45 programs that are widely mentioned in the literature and that have been rigorously evaluated with follow-up periods that range from nine weeks to 23 years.7 Ten of these programs are located outside the United States. Unlike most early childhood development programs, school-based programs tend to be universal at a school or classroom level, benefiting all children and adolescents attending the institution. When targeting exists, it occurs mostly at a geographical level. For example, the Linking the Interests of Families and Teachers program covers all first- and fifth-grade elementary school children and their families living in neighborhoods characterized by high rates of juvenile delinquency. These programs use different nomenclatures to refer to their objective popu- lation: some define specific age groups, others particular school grades or taxono- mies, and others explicit life periods. To facilitate the analysis, we refer mostly to the grade taxonomy (from prekindergarten to 12th grade, for children ages 3 to 18). When analyzing the school grades covered by the programs, two basic con- clusions emerge. First, curricula for the entire system from prekindergarten to 12th grade are scant. Second, the availability of socioemotional curricula is greater for early childhood and childhood age cohorts. In our sample of 45 pro- grams, only 2 covered the whole schooling range. Additionally, whereas 7 pro- grams exclusively target prekindergarten and kindergarten students, only 2 programs focus explicitly on adolescents.8 This outcome can be easily observed in table 6.4, which illustrates the distri- bution of the programs according to the school grades offered. It is organized so that programs that include higher grades are located at the top of the list. Horizontally, it shows the grade range covered by each program in our sample. Vertically, it shows the supply of curricula by grade. Table 6.5 presents the school grades that were included in the impact evalua- tions. The pattern is similar to that shown in table 6.4 in the sense that most of the empirical evidence is available for early childhood and childhood ages, pre- cisely where the supply of programs is greater. Information is limited regarding the impact on adolescent populations. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 52 Program Analysis throughout the Life Cycle Table 6.4  Distribution of Programs by School Grade–Range Covered Grade Program Country Pre-K K 1 2 3 4 5 6 7 8 9 10 11 12 Montessori United States Family Check-Up United States Positive Action United States SEAL United Kingdom Raising Healthy United Children States Michigan Model United for Health States KIPP United States Mato-Oput5 Uganda Learn and Serve United America States Teen Outreach United States Becoming a Man United States Second Step United States Open Circle United States RULER program United States Social and Sweden Emotional Training Bal Sabha India Aerobic Running United States ISFP and PDFY United States MindUp Canada Fast Track PATHS United States Responsive United Classroom States Project CARE United Child States Development Project Resolving United Conflict States Creatively Steps to Respect United States table continues next page Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 53 Table 6.4  Distribution of Programs by School Grade–Range Covered (continued) Grade Program Country Pre-K K 1 2 3 4 5 6 7 8 9 10 11 12 Chicago United Child-Parent States Center I Can Problem United Solve States Incredible Years United States LEAD United States 4Rs Program United States Unique Minds United States Project SAFE United States Intervention on Istanbul Grit Drama in Finland Finland LIFT United States MAPs United States Attention United Academy States Program STAR United States Al’s Pals United States School-based Colombia programs HighScope Perry United Preschool States Program Head Start United States Tools of the Mind United States Child Health Mauritius Project Preschool Cambodia program Peace Works United States Total 15 27 28 28 27 26 27 23 16 15 12 11 10 10 Note: Darker areas represent higher school grades. 4Rs = reading, writing, respect, and resolution; CARE = character actualization requires education; ISFP = Iowa Strengthening Families Program; KIPP = Knowledge Is Power Program; LEAD = Leadership Education Through Athletic Development; LIFT = Linking the Interests of Families and Teachers; MAPs = Mindful Awareness Practices; PATHS = Promoting Alternative Thinking Strategies; PDFY = Preparing for the Drug Free Years; RULER = recognizing, understanding, labeling, expressing, and regulating emotions; SAFE = Strategies Aimed at Family Empowerment; SEAL = Social and Emotional Aspects of Learning; STAR = Student/Teacher Achievement Ratio. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 54 Program Analysis throughout the Life Cycle Table 6.5  Distribution of Programs by School Grade Tested Grade Program Country Pre-K K 1 2 3 4 5 6 7 8 9 10 11 12 Learn and Serve United States America Teen Outreach United States Becoming a Man United States Bal Sabha India Aerobic Running United States Positive Action United States MindUp Canada Intervention on Istanbul Grit KIPP United States SEAL United Kingdom Social and Sweden Emotional Training Resolving Conflict United States Creatively RULER program United States Second Step United States Family Check-Up United States Montessori United States ISFP and PDFY United States LEAD United States Project CARE United States Child Development Project Responsive United States Classroom Steps to Respect United States Michigan Model United States for Health Drama in Finland Finland LIFT United States Mato-Oput5 Uganda Open Circle United States Unique Minds United States STAR United States Fast Track PATHS United States Attention United States Academy Program MAPs United States 4Rs Program United States table continues next page Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 55 Table 6.5  Distribution of Programs by School Grade Tested (continued) Grade Program Country Pre-K K 1 2 3 4 5 6 7 8 9 10 11 12 Raising Healthy United States Children School-based Colombia programs Al’s Pals United States Project SAFE United States I Can Problem United States Solve Incredible Years United States HighScope Perry United States Preschool Program Chicago United States Child-Parent Center Head Start United States Tools of the Mind United States Preschool Cambodia program Peace Works United States Child Health Mauritius Project Total 10 10 12 9 12 12 13 12 6 4 2 3 2 2 Note: Darker areas represent higher school grades. 4Rs = reading, writing, respect, and resolution; CARE = character actualization requires education; ISFP = Iowa Strengthening Families Program; KIPP = Knowledge Is Power Program; LEAD = Leadership Education Through Athletic Development; LIFT = Linking the Interests of Families and Teachers; MAPs = Mindful Awareness Practices; PATHS = Promoting Alternative Thinking Strategies; PDFY = Preparing for the Drug Free Years; RULER = recognizing, understanding, labeling, expressing, and regulating emotions; SAFE = Strategies Aimed at Family Empowerment; SEAL = Social and Emotional Aspects of Learning; STAR = Student/Teacher Achievement Ratio. Analysis of the impact evaluations shows that knowledge of these programs is limited by the fact that the majority of the evaluations focus on a subgroup of the academic grades covered by each intervention (see boxes 6.5 and 6.6). Since the evaluations cover a smaller grade range than the p ­ rograms themselves, they leave important gaps in the longitudinal understanding of these types of programs. That is, questions remain unanswered regarding cumulative impacts, variations of intensity along different years, and the effects of a particular sequence of curricu- lum content over the process of skills development, among other matters. The short follow-up periods in the evaluations do little to increase the under- standing of the real impact of these programs. Among the 45 programs that were studied, only 8 analyzed the situation of the beneficiaries five or more years later, and only 2 of those (Family Check-Up and Positive Action) targeted children older than age five. Furthermore, close to half of them followed participants for a year or less after the beginning of implementation.9 Consequently, questions remain regarding whether or how the results will persist over time. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 56 Program Analysis throughout the Life Cycle Box 6.5 School-Based Program: Al’s Pals Lynch, Geller, and Schmidt (2004) evaluated the effectiveness of the Al’s Pals early childhood program’s outcomes associated with socioemotional learning. The program combines a two- day teacher training component, in-class curriculum, and parent awareness. Thirty-three class- rooms were preselected (unclear criteria), 17 were randomly assigned to receive the Al’s Pals curriculum (218 children), and 16 remained as control groups (181 children). Classrooms that received the Al’s Pals intervention showed significant positive changes in socioemotional competence and prosocial skills as measured by the Preschool and Kindergarten Behavior Scales and Child Behavior Rating Scale–30. They also showed improve- ment in positive measures of coping. Control classrooms showed no significant changes in any of these measures. Sources: Lynch 1998; Lynch, Geller, and Hunt 1998; Lynch, Geller, and Schmidt 2004. Box 6.6 School-Based Program: I Can Problem Solve, United States Boyle and Hassett-Walker (2008) present the results of a two-year evaluation of the I Can Problem Solve (ICPS) program. The goal of ICPS is to teach children to think about solutions, anticipate consequences, and solve problems. It comprises a specific curriculum of activities, a two-day teacher training component, and instructional sheets for parents. The intervention was implemented in kindergarten and first-grade classrooms in a racially and ethnically diverse urban school district (city and state not disclosed). The district’s schools were matched according to (a) the number of students and kindergar- ten classrooms, (b) racial and ethnic student body composition, (c) socioeconomic status, and (d) language proficiency of students. Treatment was randomly assigned at the school level (226 students) to one of three groups: a two-year ICPS group (n = 96), a one-year ICPS group (n = 106), and a control group (n = 24). (None of the control group participants were African American.) Teachers were required to assess baseline and postevaluation student behaviors using two different behavior rating scales. The findings suggest that the results were positive and mono- tonically increasing with years of implementation. That is, the authors claimed the effective- ness of ICPS in increasing prosocial behaviors and in reducing aggressive behaviors. (Those outcomes are not directly related to the program’s objective.) Sources: Boyle and Hassett-Walker 2008; Kumpfer and others 2002. The objective of most programs that tackle socioemotional skills in a school environment is to enhance prosocial characteristics in participants, such as the ability to solve conflicts and be assertive.10 Some also have the intention of decreasing risk factors related to early aggressive, violent, or antisocial behaviors or related to substance abuse. Others seek to improve the classroom climate and conduct as well. The importance of improving the classroom climate is explained Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 57 by Brown and others (2010), who suggest that a positive classroom climate has been associated with greater self-esteem, perceived cognitive competence, inter- nal locus of control, mastery motivation, school satisfaction, academic perfor- mance, and less acting-out behavior. Moreover, positive teacher–child relationships have also been associated with classroom quality and the process of learning (NICHHD 2003). To teach all students in a classroom or school to identify and effectively handle emotions or social situations that could generate problems or conflicts in the absence of proper training or awareness, the programs that were studied have three main components. First and foremost, they are often based on a socioemotional syllabus that is integrated into the educational curriculum. ­ Second, since teachers usually deliver the curriculum, the programs usually include teacher training on implementing the socioemotional component (incor- porating specific dynamics or exercises into their existing school classes), as well as on class and group management. Third, the programs usually have a set of household activities that reinforce concepts learned at the school and that involve parents in their children’s education. For example, the 4Rs (reading, writing, respect, and resolution) Program is a literacy-based curriculum in conflict resolution and socioemotional learning, composed of 21 to 35 grade-specific lessons. To teach this curriculum and achieve positive rules and norms and safe and secure classroom environments, teachers receive 25 hours of training and ongoing coaching. Finally, a compo- nent called 4Rs Family Connections consists of activities for children and par- ents at home (see box 6.7). Box 6.7 School-Based Program: 4Rs Program, New York City Brown and others (2010) used a novel measurement tool and a clustered, randomized control trial evaluation strategy to assess the short-term effectiveness of the 4Rs (reading, writing, respect, and resolution) Program on classroom quality. The 4Rs Program includes a compre- hensive class curriculum, teacher training, and take-home activities to develop social and emotional skills. Participants were 82 third-grade teachers in 18 urban public elementary schools in the New York City area. Classroom quality was measured using the CLASS assessment tool, which combines three different dimensions of the class environment: (a) classroom organization, (b) classroom emo- tional support, and (c) classroom instructional support. Results showed positive improve- ments on their measure of classroom quality, and effects were robust to differences in teacher’s socioemotional factors. Among the study’s limitations pointed out by the authors were (a) the small sample, which limited the ability to control observable factors in the estimation; and (b) the nature of the intervention and the design of the evaluation, which made it impossible to map specific components of the intervention to classroom quality. Sources: Brown and others 2010; Jones and others 2010; Jones, Brown, and Aber 2011. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 58 Program Analysis throughout the Life Cycle By implementing each of these components, programs alter various environ- ments where people learn and interact. Curricular modifications and teacher trainings alter classroom norms or school-wide practices (or both) to induce specific behaviors indirectly. Household activities alter parenting practices and modify the family environment. Other programs, not included in this review, encourage extracurricular activities (usually sports or music) as a way to create an environment that requires sharing responsibilities with others and to boost the learning of socioemotional skills (see box 6.8).11 Most programs foster skills directly, through a school curriculum, and indirectly, through parenting and the household environment. An analysis of the frequency of each of these components shows that class socioemotional curriculum and teacher training are the most recurrent features of ­ programs in school environments. Figure 6.4 illustrates how many programs in our sample include each of these four components. Since programs usually tar- get one or more components, the number of programs for some components is more than 30. Box 6.8 After-School Program: Big Brothers Big Sisters of America, United States Some programs, like Big Brothers Big Sisters of America, are extracurricular and might involve only a portion of all students. In this intervention, children attending school who were referred by school staff were matched to a trained volunteer mentor, who spent 45–60 minutes with the child about once a week. Matches often chose how they spent their time together, although all of the programs had some degree of structure (that is, the activities from which matches could choose are at least partly outlined by the program). In a few cases, the activities in which matches engaged were predetermined. More often, programs offered suggestions for the meetings. Most of the con- tents did not focus on academics but instead emphasized creative activities (such as drawing and arts and crafts), playing games, and talking about various topics (such as friends, family, academic issues, and the importance of staying in school). A randomized control trial involving 1,139 youths in grades four through nine in 71 schools nationwide found positive and significant effects of the program on (a) academic performance ­ umber (overall, in science and in written and oral language), (b) the quality of classwork and the n of assignments turned in by students, (c) scholastic efficacy (feeling more competent academi- cally), (d) students’ college expectations, (e) the level of serious school misconduct, (f) the num- ber of absences without an excuse, and (g) children’s willingness to start skipping school. The program found no benefits in the out-of-school areas that were examined, including substance use, misconduct outside of school, relationships with parents and peers, social acceptance, self- esteem, and assertiveness. The program cost is an estimated US$1,000 per student per year. Source: Herrera and others 2008. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 59 Figure 6.4 Program Components in the Sample of Programs 40 35 Number of programs 30 25 20 15 10 5 0 Teacher training Class curriculum Household activities Extracurricular Table 6.6 Salient Characteristics of School-Based Programs Program characteristics Participant profile Outcomes Objectives Age Health: Risk of initiating substance use; Develop socioemotional skills; improve 5 to 18 years old substance abuse; sexual behavior; classroom climate; increase protective depression; anxiety factors; and decrease risk factors related Risk factors: Hostile attribution bias; to aggressive, violent, or antisocial attention/hyperactivity symptoms; behaviors and substance abuse problem or antisocial behavior; disruptive Components and complementary Targeting mechanisms behaviors; conduct problems; elements Primarily universal programs oppositional behavior; bullying behaviors; Classroom curriculum at a school level. aggressive fantasies; physical aggression; Teacher training Geographic targeting may verbal abuse; arrests; delinquent behavior; Household activities occur for select schools in victimization; executive control; attention riskier or poorer areas. seeking; autonomy; interpersonal Setting negotiation and coping strategies; social School withdrawal; prosocial behavior; social and interpersonal skills; social cooperation; Quality social independence; self-management Personnel: Primarily schoolteachers; some skills; behavioral and emotional programs also use volunteers with self-regulation; self-esteem; optimism; some level of education positive affect; authority acceptance; Intensity: From 20-minute lessons parenting skills; emotional support delivered three to five times a week to Academics: Teacher–student relations; four to nine classroom sessions classroom quality; classroom Cost: Almost never reported management; school bonding; educational aspirations and expectations; grade point average; achievement test scores; reading; math; writing; metacognition; academic engagement; effort and discipline in class; expected graduation rates; school absences; suspensions; disciplinary referrals Source: Based on 45 rigorously evaluated programs described in detail in appendix B. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 60 Program Analysis throughout the Life Cycle Unfortunately, little evidence illustrates the relative effectiveness of each of these components. The reason for the lack of evidence is that school-based pro- grams that seek to promote socioemotional skills do not normally test the effec- tiveness or intensity of their specific components. Out of the 45 programs included in the analysis, only the evaluations of Project SAFE, school-based programs in Colombia, and Becoming a Man evaluate the relative effect of the different components. Furthermore, only the evaluation of I Can Problem Solve incorporates the level of exposure in the evaluation design, allowing for an analy- sis of the persistence of the results (in particular, one can compare a group that has been in a program for two years with a group that was treated only during the first year, along with a control group for the two years). The remaining programs capture the effect of exposure time by analyzing the impacts of the program after the first year and comparing them with the impacts in the second year. However, this method does not allow one to see how long the effects persist once the program is no longer delivered. Outcomes of School-Based Programs This section focuses on the main findings from evaluations of school-based pro- grams that foster socioemotional skills. The primary outcomes of these types of programs will be discussed, followed by important program and participant characteristics. Most of the impacts of school-based programs are related to risk factors ­ (figure 6.5). In fact, our list of statistically significant outcomes for school-based Figure 6.5 Outcomes of School-Based Programs Program characteristics 32% Academic 59% Risk factors Context Outcomes 2% Economic 7% Health Participant profile Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 61 programs was distributed as follows: 59 percent related to behavioral variables, such as internalizing and externalizing behavior, aggression to peers, and coopera- tion; 32 percent were academic-related outcomes, such as grades, standardized test results, and classroom climate; and 7 percent related to health variables, such as drug use and high-risk sexual behavior. The minimal evidence related to important economic variables indicates the lack of longer follow-up periods in the evaluations of this part of the literature. Durlak and others (2011) conducted a meta-analysis with 213 studies and conclude that those programs significantly improved students’ socioemotional skills and attitudes toward self, school, and others. Both internalizing behav- iors (such as self-esteem, depression, and emotional distress) and externalizing behavior problems (such as aggression toward others and conduct problems) were also effectively addressed with these types of programs. In a systematic review of 87 rigorous research reports, Lösel and Beelmann (2003) find that a large number of programs revealed positive and statistically significant outcomes with regard to antisocial behavior, social skills, and social-cognitive skills. In a meta-analysis of only programs that sought to reduce aggressive and disruptive behavior, Wilson and Lipsey (2007) find that all outcomes were positive and statistically significant.12 The impact of these programs on academic perfor- mance (standardized test scores and grades) is also significant, and as docu- mented by Payton and others (2008, 6), there is “an average gain on achievement test scores of 11 to 17 percentile points.”13 Finally, the programs have effects on the health of participants. The positive effects of these programs are generalizable for both in-school and after-school settings and in diverse geographical contexts (urban, suburban, and rural areas). However, after-school programs tend to have smaller outcomes than school-based programs (Durlak and others 2011; Payton and others 2008). When measured, these programs also seem to have positive rates of return. Belfield and others (2015) conducted a benefit–cost analysis of six prominent socioemotional learning programs: (a) 4Rs, (b) Positive Action, (c) Life Skills Training, (d) Student Success through Prevention (Second Step), (e) Responsive Classroom, and (f) Social and Emotional Training (the first five programs are in the United States; the last one was implemented in Sweden). They find that all of the programs have a positive return, as their benefits exceed their costs. In particular, the average benefit–cost ratio is almost 11 to 1, which means that for every dollar invested in the programs, there is a return of US$11. However, even though school-based programs appear to have positive impacts on risk factors, they have certain limitations on their effectiveness. For example, • Programs might be effective in inducing changes only in evident and not complex risk factors. The evaluation of Second Step (United States) shows that pro- grams might be more effective at reducing evident forms of disruptive and aggressive behaviors, such as physical aggression, rather than more complex forms of aggression, such as homophobic teasing and sexual violence (Espelage and others 2013). Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 62 Program Analysis throughout the Life Cycle • The effectiveness of some programs might depend on the incorporation of school- wide or community-wide components. After analyzing programs aimed at fight- ing bullying in school, Vreeman and Carroll (2007) find that programs that aim to change behavior based strictly on social cognitive principles are not effective. They suggest that effective programs need to address systemic issues and social environments. Thus, effective programs tend to incorporate a whole- school approach that involves the entire school community. • The Cambridge-Somerville Program (United States) had negative effects on its ­participants. This program provided medical assistance, tutoring, summer camps, and parent–teacher meetings for 13-year-old children who had behavioral prob- lems (Kautz and others 2014). A randomized control trial that measured the impact of the program found increases in the drinking habits, serious mental diseases, heart problems, blood pressure, crime rates, and mortality rates of its participants. According to McCord (1978), the program did not create a sense of autonomy among participants. Once removed, it led to the original unfavorable behaviors while generating rejection and resentment for the lack of support. Further research is needed to evaluate whether the documented effects are permanent or transitory. Many studies have not included postprogram follow-up ­ assessments, and existing evidence related to the subject is mixed. Using 24 studies with a median follow-up period of 52 weeks, Payton and others (2008) find that the effects of the programs on socioemotional skills, positive attitudes and social behaviors, conduct problems, and academic performance persist over time. However, they also find that those long-term effects are less strong than the effects immediately after the intervention. By contrast, upon evaluating Across Ages, Aseltine, Dupre, and Lamlein (2000) find that the impacts of the program (designed to reduce substance use in high-risk youth) disappeared in a six-month follow-up. Additionally, the question of what works better and under which circumstances has not been properly answered in the literature. As mentioned before, evalua- tions vary profoundly, which increases the difficulty of pinpointing the character- istics that drive program success (figure 6.6). Hahn and others (2007) argue that although some school programs are likely to have greater impacts than others, the characteristics (or the setting) that make programs more effective are not clearly understood. Jones and Bouffard (2012) note that research linking specific program components to outcomes is rarely found in the literature. Wilson and Lipsey (2007) find that multicomponent programs do not have a greater impact on the reduction of problematic behaviors than single-component school-based programs. Payton and others (2008) and Durlak and others (2011) find that this observation can derive from the challenges associated with the implementation of multicomponent programs. Furthermore, the programs in our sample that analyze the effect of comple- menting the school-based component with a parental-home component have mixed results. As box 6.9 describes, the Project SAFE program found greater results when targeting both the school and the home environments. However, the Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 63 Figure 6.6 Selected Program Characteristics That Affect the Outcomes of School-Based Programs Parental component SAFE Program characteristics Length and intensity of intervention Fidelity of implementation Context Outcomes Participant profile Note: SAFE = sequenced, active, focused, and explicit. Box 6.9 Impact Evaluations That Include an Analysis of an Intervention’s Components: Project SAFE and a School-Based Intervention in Colombia To evaluate the impact of Project SAFE (Strategies Aimed at Family Empowerment), Kumpfer and others (2002) compare four groups of students: (a) children receiving the I Can Problem Solve (ICPS) intervention, a school-based universal primary prevention curriculum delivered by trained teachers in classes three to five times a week in 20-minute lessons; (b) students receiving the ICPS intervention combined with the Strengthening Families (SF) Program (a 14-session family skills training program that c ­ onsists of skills training for the parents, skills training for their children, and skills training for the family); (c) students receiving the ICPS intervention and only the parenting component of the SF Program; and (d) control students not participating in any of the programs. The authors find that the most effective program delivery was the simultaneous adminis- tration of ICPS and SF. The ICPS intervention alone affected school bonding and self-­ regulation. The ICPS with the parenting component of the SF had significant effects on social competence and self-regulation. Finally, the implementation of ICPS and the full SF Program affected school bonding, parenting skills, family relationships, social competency, and behav- ioral self-regulation, all of which are associated with the risk of substance abuse. For the impact evaluation of a school-based intervention in Colombia (Klevens and others  2009), students were divided into three groups: (a) some received the teacher-only ­ intervention, where teachers received 10 four-hour weekly workshops on standard classroom management techniques and strategies for shaping children’s behaviors; (b) some received a box continues next page Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 64 Program Analysis throughout the Life Cycle Box 6.9  Impact Evaluations That Include an Analysis of an Intervention’s Components: Project SAFE and a School-Based Intervention in Colombia (continued) teacher/parent combined intervention that included the same teacher’s training as well as a one-hour weekly parenting intervention that covered almost the same topics as the teacher’s training; and (c) a third group served as a control. The results of the intervention were surprising, as the teacher/parent combined interven- tion had a smaller effect on prosocial behavior. Sources: Klevens and others 2009; Kumpfer and others 2002. results of the school-based programs in Colombia found that involving schools alone had greater effects than combining parent and teacher intervention. Several authors (Durlak and others 2011; Payton and others 2008) agree that the most successful socioemotional programs in school settings are sequenced, active, focused, and explicit (grouped together under the acronym SAFE). Sequenced step-by-step approach. programs have a planned set of activities that develop skills in a ­ Active programs use active forms of learning like role playing, which provide the opportunity to use the recently acquired skills. Focused ­ programs devote a suffi- cient amount of time and attention to socioemotional skills training. Finally, explicit programs clearly identify and specify the ­ socioemotional skills that they aim to develop as a result of the intervention (Arthur and others 1998; Bond and Hauf 2004; Durlak 1997; Durlak, Weissberg, and Pachan 2010; Dusenbury and Falco 1995; Gresham 1995). These authors find that those four characteristics reinforce one another; an intervention that is focused but not sequenced or active will not be as effective as one that is sequenced, active, focused, and explicit. Durlak, Weissberg, and Pachan (2010) explain that the presence or absence of those characteristics in different programs moderates the programs’ effects. Complementing the finding regarding SAFE programs, Weissberg and Greenberg (1998) report that programs will most likely promote and sustain socioemotional skills formation when they are relevant to the challenges faced by children; when they have longer implementation periods (teaching skills over several years); and when they accompany socioemotional skills training with the opportunity to use the skills learned, a characteristic captured under the SAFE approach as “active.” The length and intensity of the programs also appear to have important effects. For example, the evaluation of the STEPS to Respect program finds that children who are part of the program for two years experience greater improve- ments in problem behavior on the playground than children who participate for only one year (Frey and others 2009). Something similar is found with the Unique Minds program, where the level of program exposure directly influenced the results for cognitive, social, and emotional skills and grade gains; self-efficacy; and problem solving. Finally, Boyle and Hassett-Walker (2008) find that students who participated in the I Can Problem Solve program for two years showed less Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 65 relational aggression than both students who had only one year of the program and students in the control group. As with programs focused on early childhood development, implementation fidelity appears to affect a program’s outcomes. Humphrey, Lendrum, and Wigelsworth (2010) argue that the lack of significant effects of the Social and Emotional Aspects of Learning program results from implementation problems. Chang and Muñoz (2006) find that varying degrees of implementation fidelity of the Child Development Project implemented by Project CARE led to differ- ent results: when including all schools, there were no differences between all treated schools and controls. However, when comparing high-quality treatment and control schools, statistically significant effects were found in variables that included reading and number of student referrals. Finally, Payton and others (2008) and Durlak and others (2011) find that the absence of a careful record of implementation problems might hinder the promotion of better results in school-based programs. Further, they state that programs with reported imple- mentation problems yielded fewer positive outcomes than those with no reported problems. ­ ocioemotional Multiple evaluations find that school-based programs that target s skills development are effective both for the general population and for particu- lar ethnic groups. For example, the evaluation of Fast Track PATHS (Promoting Alternative Thinking Strategies) documents effects both for students of multiple ethnicities in elementary schools located in neighborhoods where delinquency and juvenile arrests are high and for students in more typical American public schools (CPPRG 2010; Domitrovich, Cortes, and Greenberg 2007). Similarly, significant outcomes are achieved by Family Check-Up, a program implemented both among children in urban schools and among ethnic-minority adolescents (Connell and Dishion 2008; Connell, Klostermann, and Dishion 2012; Fosco and others 2013; Stormshak, Connell, and Dishion 2009; Stormshak and others 2011; Van Ryzin, Stormshak, and Dishion 2012; Fosco and Dishion 2012). Further, the evaluation of Positive Action also documents impacts for racially diverse students in both Hawaii and Chicago (Lewis and others 2013; Snyder and others 2013). However, the effect of school-based programs on socioemotional skills is greater for students who exhibit higher initial levels of problem behavior or more risk factors (see figure 6.7). Some programs—such as 4Rs, Linking the Interests of Families and Teachers, and Mindful Awareness Practices—find effects on the entire population but report effects of greater magnitude on highly vulnerable students (those with the greatest levels of initial aggression or with the lowest level of executive functions). The greater effects of these pro- grams on groups with multiple risk factors are also documented by Lösel and Beelmann (2003). They explain that universal programs have lesser impacts on behavior because they affect the learning process of children who would not develop serious behavioral problems even in the absence of the program. Thus, in the long run, this group exhibits no major differences in behavior from those exhibited by a control group. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 66 Program Analysis throughout the Life Cycle Lösel (2001) warns against generalizing this finding to extremely high-risk youngsters, stating that an inverted U-shaped relationship between risk level and program effect appears to be most plausible. Following this theory, the most effective school-based programs would target those youngsters who are at risk, but not at extreme risk (see figure 6.8). For their part, Hahn and others (2007) suggest that program effectiveness might decrease with the age of participants. Payton and others (2008) and Durlak and others (2011) also report that a student’s mean age is usually signifi- cantly and negatively related to skill outcomes. However, they do not provide tentative channels or explanations for this result. Figure 6.7 Selected Participant Characteristics That Affect the Outcomes of School-Based Programs Program characteristics Context Outcomes Students with high risk/ high levels of initial problem Participant profile Age Figure 6.8 Plausible Relationship between Risk Level and Program Effects Effectiveness Risk of student Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 67 Out-of-School Programs This section characterizes a set of programs that aim to promote socioemotional skills outside the school environment. As Kuckulenz (2007, 1) states, “Post- school learning is an important source of skill formation that accounts for as much as one-third to one-half of all skill formation in a modern economy.” That conclu- sion is driven not only by the fact that most of the world’s population is out of school,14 but also by the documented productivity of skill investment in teenage and adult populations (Kautz and others 2014). Socioemotional skills are particu- larly relevant in this life cycle, since as Kautz and others (2014, 78) mention, “If the early years have been compromised, it is more effective in the adolescent years to focus on developing non-cognitive skills rather than cognitive skills.” Having said this, out-of-school programs with socioemotional skill compo- nents have an ample range of objectives. For example, some seek to promote job placement for the unemployed, others intend to reduce gender violence or to improve stress-related coping strategies, whereas still others aim to decrease recidivism among individuals with substance dependence or conduct disorders. Although all of these programs play important roles in social protection systems around the world, this book will not cover programs specifically designed for people who have expressed addictions or conduct disorders. Additionally, of the out-of-school programs analyzed in this section, 75 percent are in countries other than the United States. Most (75 percent) of the programs analyzed in this section seek to improve labor market outcomes. These programs usually include the promotion of ­ socioemotional skills as part of a larger set of components, such as work-related skills training, mentoring, and on-the-job training. Of the programs studied, 10 percent aim to prevent gender-based violence by fostering socioemotional skills and providing educational activities and communication campaigns. Another 5 ­ percent are oriented toward the reduction of risky sexual behaviors among Latino immigrants in the United States, and, finally, 5 percent provide intensive remedial education, social skills training, and personal development. The group of individuals targeted by out-of-school programs range from age 10 to over age 55. However, since most of the programs included in the sample have job placement as an objective, they focus on potentially economically active populations (adolescents and young adults). Furthermore, as can be seen in table 6.7, which illustrates the distribution of programs in our database orga- nized by targeted age, most programs focus on younger cohorts. Although all of the programs in our sample include youths ages 16 to 30 as part of their target audiences, only 30 percent include individuals over age 30. A careful analysis of the participant profiles of these programs shows that most focus on unemployed or underemployed youths with a low socioeconomic background. Some programs, such as National Guard Youth ChalleNGe (United States), Apprenticeship Training for Vulnerable Youth (Malawi), Juventud y Empleo (Dominican Republic), and Kingston Young Men’s Christian Association (YMCA) Youth Development Programme (Jamaica), explicitly benefit Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 68 Program Analysis throughout the Life Cycle Table 6.7  Distribution of Out-of-School Programs by Age Targeted age Program Country ≤ 15 ≤ 20 ≤ 25 ≤ 30 ≥ 31 Team Awareness United States JOBS program United States Jordan NOW Jordan Ninaweza Kenya Apprenticeship Training Malawi for Vulnerable Youth Galpão Aplauso Brazil Procajoven Panama Juventud y Empleo Dominican Republic entra21 Argentina EPAG program Liberia Questscope Non-Formal Jordan Education Program H Brazil Jóvenes en Acción Colombia Year Up United States Job Corps United States ELA program Uganda Involucrando Hombres Chile Jóvenes en el Fin de la Violencia de Género Kingston YMCA Youth Jamaica Development Programme Joven Noble United States National Guard Youth United States ChalleNGe Program Total 6 18 14 11 6 Note: Darker areas represent older age groups. ELA = Empowerment and Livelihood for Adolescents; EPAG = Economic Empowerment of Adolescent Girls and Young Women; NOW = New Opportunities for Women; YMCA = Young Men’s Christian Association. adolescents who have dropped out of school. These programs have the intention of reintegrating youths into the education system or equipping them with the necessary skills to enter the labor market. A wide variety of programs benefit women; 20 percent of them were exclusively aimed at women, contributing to the amelioration of gender inequality in the labor market.15 The following components are included in most programs that seek to improve labor market outcomes for vulnerable groups: • Classroom instruction: Instruction normally includes technical and theoretical training in a particular field, as well as academic education in math, language, and writing. Professional training services are usually provided only for low-skill occupations like bakery work, bartending, hair styling, tailoring, and carpentry. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 69 • Counseling or mentoring: Programs often provide their beneficiaries with the opportunity to contact a professional, role model, or more experienced indi- vidual to advise them on how to interview, where to seek jobs, and how to pursue additional training. • Internships or other workplace-based training activities: Many programs offer hands-on training opportunities in the workplace, including internships or apprenticeships. Internship programs are more common in developing coun- tries and consist of less-intensive interactions that usually last from three to six months and that do not involve classroom training. Apprenticeship programs, which are very common in Europe, last for three to four years and involve highly structured training that combines in-class knowledge (that relates to the particular occupation) with workplace practice (Kautz and others 2014). • Socioemotional training: Among others, this component fosters self-esteem, work habits, social harmony, team-building and teamwork skills, negotiation and conflict resolution, time management and punctuality, dress and body language, positive thinking, sense of control, and stress management (see ­ table 6.8). Occasionally, socioemotional training is delivered through an Table 6.8 Main Characteristics of Socioemotional Skills Components in Out-of-School Programs Duration Program name Country (hours) Contents Ninaweza Kenya 40 Emotional intelligence, self-care, decision making, motivation, communication (listening, body language), cooperation and competition, teamwork, personal planning and goal setting, work ethics and perspectives, job searching Juventud y Dominican 75 Values, self-esteem, group integration, conflict Empleo Republic resolution, social risk management, communication, work planning and organization, life project development and a plan for job acquisition, job-search orientation Galpão Brazil 120 Basic principles of “social harmony,” which emphasize Aplauso civics and certain shared values along with socioemotional development, for example, concepts, principles, and values such as ethics; civic responsibility; respect; environmental education; solidarity; health; and honesty Jordan NOW Jordan 45 Effective communication and business writing skills (for example, making presentations, writing business reports and different types of correspondence), team-building and team work skills (for example, characteristics of a successful team, how to work in different roles within a team), time management, positive thinking and how to use it in business situations, excellence in providing customer service, and résumé writing and interviewing skills entra21 Argentina 64 Life skills Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 70 Program Analysis throughout the Life Cycle explicit classroom curriculum. Other times, it is incorporated in the classroom dynamic and taught simultaneously with the particular field or the academic subject. Some programs also transmit socioemotional skills during the appren- ticeship, internship, or job-training periods. Seeking to offset some of the opportunity costs associated with leaving the labor force while training, programs like Jóvenes en Acción in Colombia and Juventud y Empleo in the Dominican Republic also provide stipends to their participants. Additionally, programs like Jordan New Opportunities for Women (NOW) provide subsidies to employers that hire the programs’ graduates. Previous evaluations of such wage subsidies have found limited results, which are normally attributed to low usage rates of subsidies and to possible stigma effects (Groh and others 2012). Figure 6.9 illustrates the components found in the set of programs gathered in our literature review and their frequency. Classroom training is by far the most commonly recurring feature of the programs analyzed. It is followed by counsel- ing and mentoring, and by workplace-based job training. Few programs include a subsidy component. The average length of the programs in our database is 10 months (median of 6 months). However, programs vary substantially: whereas Team Awareness lasts only five hours, the Kingston YMCA Youth Development Programme can take up to four years to complete. Sixty percent of the programs are full time and last more than a week. When present, the on-the-job training component lasts from one to six months. Only two programs (Job Corps and the National Guard Youth ChalleNGe Program) involve high-intensity arrangements where beneficiaries live on the program’s premises. As Honorati and McArdle (2013) document, life skills training can last 45 hours (as in Jordan NOW), 75 to 80 hours (as in Juventud y Empleo in the Dominican Republic and the Youth Empowerment Program in Kenya), or even 200 to 400 hours, combining life skills and technical training (as in the Jóvenes programs for youth training in Latin America). Figure 6.9 Program Components in Sample of Out-of-School Programs 20 Number of programs 15 10 5 0 Classroom training Counseling/ On-the-job training/ Stipend/subsidy mentoring internship/ apprenticeship Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 71 Information on costs was available for 45 percent of the programs. Cost information normally includes the entire program, not just the socioemo- tional skills formation module or component. For these programs, invest- ments ranged from US$85 to US$16,500 per participant. Taking into account that the higher costs are associated with the two residency-based programs in our sample, the costs of out-of-school programs are low when compared with programs in the other stages of the life cycle. On the basis of an analysis of job-oriented programs in Latin America, González-Velosa, Ripani, and Rosas- Shady (2012) suggest that the small per capita investment should be associ- ated with modest expected results. The programs they analyzed cost between US$330 and US$750. Outcomes of Out-of-School Programs This section synthesizes the central findings regarding out-of-school programs with components that strengthen socioemotional skills. Following a description of the main outcomes of these types of programs is a discussion on the aspects of program design and implementation that alter a program’s impacts. Finally, the ways that various participant characteristics can hinder or facilitate the attain- ment of positive results are summarized. The majority of the out-of-school programs analyzed in this book are ori- ented toward improving labor market outcomes (see box 6.10 and table 6.9). Consequently, most of the documented program outcomes are related to job status and earnings and to behavior. In fact, 38 percent of the outcomes fall under the economic-related category and a similar percentage of the out- comes (35 percent) are related to behavioral aspects, such as leadership skills, self-esteem, conflict resolution abilities, confidence, and arrests and convic- tions. Furthermore, 17 percent of the outcomes are related to health (such as obesity, and drug and alcohol consumption), and 10 percent are related to academic aspects (such as enrollment, hours in education, and certification). See figure 6.10. Box 6.10 Out-of-School Program: Juventud y Empleo, Dominican Republic Juventud y Empleo is a workplace-based intervention that targets Dominican adolescents between ages 16 and 29 who live in poor neighborhoods; are not attending school; lack a high school diploma; and are unemployed, underemployed, or inactive. The program pro- vides them with 150 hours of vocational training, 75 hours of socioemotional (life skills) training, and a three-month internship to increase the likelihood of their finding a job, to ­ reintegrate them productively into society, and to prevent them from participating in delinquent activities. While participating in the training, participants receive a stipend of close to US$3 per day. box continues next page Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 72 Program Analysis throughout the Life Cycle Box 6.10  Out-of-School Program: Juventud y Empleo, Dominican Republic (continued) At 18 to 24 months after the intervention, overall employment showed few effects, but job quality for males was affected significantly (17 percent increase in formal employ- ment, 7  ­percent increase in monthly earnings for those employed), especially in Santo Domingo. The program also had a positive effect on teenage pregnancy, and in women’s expectations and perceptions about the future. With regard to socioemotional skills, the program improved leadership skills, persistency of effort, and conflict resolution. Six years after the intervention started, employment and earnings continued to show no  significant effects. However, formal employment for men had sustained positive effects (25  percent more men in the treatment group have a job with health insurance) and for both genders in Santo Domingo (when comparing the participants and the control group, 30 percent more individuals participating in the intervention have jobs with health insurance). In Santo Domingo, the program also increases earnings by 25 percent for women participants over the control group. Sources: Fazio 2011; Ibarrarán and others 2012; Ibarrarán and others 2015. Table 6.9 Salient Characteristics of Out-of-School Programs Program characteristics Participant profile Outcomes Objectives Age Health: Use of tobacco and other drugs; Improve the prospects (labor capacity, 10 to over 55 years old teenage pregnancy; mental health; earning levels, socioeconomic status) of depressive symptoms; overweight; sexually jobless youths and disadvantaged groups transmitted diseases; HIV risk knowledge; by making youths more responsible, condom and birth control use; safe sexual employable, and productive activity; delay in marriage and births; health insurance Integrate school dropouts into society Targeting mechanisms Risk factors: Arrests and conviction rates; Empower women along economic and Socioeconomic and violent incidents; stalking incidents; reproductive health dimensions employment status, aggressive behavior; attitudes about Prevent and reduce the negative effects gender roles; expectations and anxiety age, educational associated with unemployment and about the future; leadership skills; attainment, job-seeking stress. self-esteem; confidence and geographical location, Reduce violence against women race, gender empowerment; prosocial behavior; Prevent risk-related sexual behaviors positive unwinding; conflict resolution; Components and complementary elements self-organization; persistency and effort; Counseling happiness and life satisfaction; stress; social Classroom curriculum support; collective civic efficacy; cultural On-the-job training/ internship, subsidy esteem Setting Academics: Knowledge of ICT and life skills; Community training centers, firms enrollment; hours in education; degrees, Quality certificates, or diplomas Personnel: Trained facilitators Economic-related factors: Employment Intensity: 5 hours to 4 years (formal and informal); hours worked per Costs: US$85 to US$16,500 week; earnings; tenure; savings; perception of entrepreneurship; credit standing; welfare dependency; retirement benefits; spending Source: Based on 20 rigorously evaluated programs that have socioemotional skills formation as one of their components. Note: See appendix C for program descriptions. HIV = human immunodeficiency virus; ICT = information and communication technology. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 73 Figure 6.10 Outcomes of Out-of-School Programs Program characteristics 10% Academic 35% Risk factors Context Outcomes 38% Economic 17% Health Participant profile The programs’ effects on employment, formality levels, number of hours worked per week, and wages or earnings tend to be small but statistically signifi- cant. However, evidence regarding program effects on employment levels and quality of employment is mixed. For example, the impact evaluations of Juventud y Empleo (Dominican Republic), Jóvenes en Acción (Colombia), and entra21 (Argentina) found no significant effects on employment levels, but they found significant improvements in formality (employment with health insur- ance, a written contract, formal wages) and earnings for those employed. Similarly, the Year Up program (United States) significantly increased the annual earnings of beneficiaries, primarily through higher hourly wages and better job placements, but with no significant changes in employment rates. In contrast, programs like Job Corps (United States) and Galpão Aplauso (Brazil) increased employment rates but had no effect on the quality of the jobs offered to participants.16 When measured, job-oriented programs had mixed effects on risk factors. For example, after 75 hours of basic or life skills training that aimed to strengthen trainees’ self-esteem and work habits, Juventud y Empleo had an effect on par- ticipants’ aspirations, conflict resolution skills, leadership, self-esteem, and order and self-organization.17 It also affected persistence of effort among women and leadership skills among men. In Jordan, Questscope Non-Formal Education was found to have had significant effects on conduct problems (as measured by the Strengths and Difficulties Questionnaire, which screens for behavioral issues in five domains of psychological attributes),18 men’s prosocial behaviors, connect- edness, and hyperactivity, although the effects on self-efficacy and social skills Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 74 Program Analysis throughout the Life Cycle were not statistically significant. Further, programs targeting women, such as Ninaweza in Kenya, EPAG (Economic Empowerment of Adolescent Girls and Young Women) in Liberia, and ELA (Empowerment and Livelihood for Adolescents) in Uganda significantly bolstered the confidence levels and empow- erment of teenage girls.19 By contrast, Galpão Aplauso in Brazil found no signifi- cant effects on grit, on participants’ score on the competencies covered by the Social and Personal Competencies Scale, on their social activities, or on risky behavior.20 Interestingly, the two programs that offer residential services in our sample (Job Corps and National Youth Guard ChalleNGe) have short-term effects on arrests and conviction rates. However, the effects of both programs on criminal behavior fade over time. Kautz and others (2014) attribute this phenomenon to the fact that since youths are housed in the program, they are “incapacitated” from com- mitting any crime in the short term. Once they return to their original environ- ment, they revert to their usual behaviors and criminality. That, however, does not mean that modifying aggression should not be considered a viable program out- come for programs that target extremely disadvantaged youth: the Kingston YMCA Youth Development Programme in Jamaica finds statistically significant effects on the aggressive behavior and aggressive propensity of its participants. Finally, some of the programs have statistically significant effects on academics-related variables, such as knowledge of information and communica- ­ tion technology; knowledge of life skills; enrollment in educational institutions; number of hours spent on education; and attainment of degrees, certificates, or diplomas. Those effects were expected, as most of the programs have a compo- nent that trains participants on particular job-related skills. Cost–benefit analyses find that out-of-school programs that target labor market outcomes and are not residential appear to be cost-effective (see figure 6.11). As Card, Kluve, and Weber (2010) mention, few studies include ­ the information required to conduct a cost–benefit analysis. However, the impact evaluations that do capture cost-effectiveness have found positive results. For example, the evaluation of Procajoven (Panama) found that the overall costs of the program were recovered in 12.6 months. Additionally, Attanasio, Kugler, and Meghir (2011) estimate that Jóvenes en Acción (Colombia) yielded internal rates of return of 35 percent overall and 21.6 per- cent for women. Further, Alzúa, Cruces, and Lopez Erazo (2013) calculate that entra21 (Argentina) had an internal rate of return of 19 percent, which, although positive, was lower than the rate found for similar programs in the region. Finally, several studies of apprenticeship training in European countries find high rates of return for participants, in the range of 9 percent to 15 percent (Adda and others 2013; Bougheas and Georgellis 2004; Clark and Fahr 2002; Winkelmann 1996). Consistent with those findings, Hollenbeck (2008) esti- mates that in the state of Washington (United States), social and governmental returns (for example, taxes received on earnings and reductions in spending on unemployment insurance, Temporary Assistance for Needy Families, food stamps, and Medicaid benefits) to apprenticeship programs administered by the Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 75 Figure 6.11 Selected Program Characteristics That Affect the Outcomes of Out-of-School Programs Socioemotional skills component Apprenticeship and on the job Program characteristics Nonresidential service Quality/ delity of implementation Context Outcomes Participant profile state’s Department of Labor and Industries are greater than 20 percent within the first 2.5 years after apprentices leave the program. By contrast, residential programs that foster job placement do not appear to be cost-effective. Programs such as Job Corps and National Guard Youth ChalleNGe have very high costs for trainee housing and meals and for teaching costs. Even with important benefits, those costs lead to net earnings effects of close to zero (Bloom, Gardenhire-Crooks, and Mandsager 2009; Millenky, Bloom, and Dillon 2010; Millenky and others 2011; Perez-Arce and others 2012; Schochet, Burghardt, and McConnell 2008). These results are limited by the fact that out-of-school programs have short follow-up periods. In fact, only 25 percent of the programs analyzed in this book have a follow-up period that exceeds two years. Shorter follow-up peri- ods may lead to an underestimation of the effects of active labor market pro- grams21 and may limit the number of outcomes that can be measured. For example, evaluations that cover only two months after an intervention cannot reasonably expect to find effects on fertility outcomes. This limitation restricts the accuracy of the rates of return calculated for these programs. In addition, it also increases the difficulty of making claims regarding the longevity of the results. For example, the evaluation of Jordan NOW found extremely large immediate effects from providing a subsidy to employers as an incentive to hire graduates of the program. Nevertheless, the effects disappeared 14 months after the start of the intervention. This finding is consistent with reports from Calero and others (2014), who report that the impact of wage subsidies peaks at 12 months and then fades. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 76 Program Analysis throughout the Life Cycle Kautz and others (2014) argue that this lack of long-term evaluation might be incorrectly fueling the perception that programs targeting adolescents are not as effective as programs that target earlier life cycles. For instance, if labor market– oriented programs have low impacts over the short run but significant impacts over the long run, calculated estimations might underestimate their effects. This problem is observable in the evaluation of the United States’ Workforce Investment Act, which appeared to have negative impacts in the short run but actually resulted in higher earnings in the long run (Heinrich and others 2013). Caliendo, Schmidl, and Uhlendorff (2011) observe a similar phenomenon in Germany, where short-term negative impacts are followed by positive significant impacts after 10 months. Card, Kluve, and Weber (2015) analyze 200 economet- ric evaluations of active labor market programs and find that long-term evalua- tions seem more favorable: even though average impacts are close to zero in the short-term, impacts become positive and significant two or three years after the intervention. Because of the diversity of out-of-school programs, identifying their effective elements is difficult. However, evidence suggests that program components mat- ter, and that “workplace-based programs that teach non-cognitive skills appear to be the most effective remediation programs for adolescents” (Kautz and others 2014, 82). In particular, • There appears to be a greater benefit from including socioemotional skills compared with having mainly cognition and academic learning (Kautz and others 2014). In the evaluation of Jordan NOW, Groh and others (2012) find that although life skills training had only slight effects on employment (no effects in the short term and a slight effect one year after program completion but only for those outside of Amman), it improved life outlook and reduced depression. Additionally, in the evaluation of the Ninaweza program (Kenya), Alvarez de Azevedo, Davis, and Charles (2013) find that the inclusion of a socioemotional skills component had a significant effect on knowledge acqui- sition, life skills, and the probability of obtaining a job. Oddly, the group in this program that went without life skills training experienced greater increases in their weekly income. • Furthermore, on-the-job training appears to be a constant component of effective out-of-school programs (Calero and others 2014; Urzua and Puentes 2010). One reason could be that this type of program involves the transmission of field-specific job-related skills, as well as noncognitive skills, such as punctuality, personal presentation, and teamwork. Furthermore, workplace-oriented training eases matching of workers and firms. The prospect of future employment may also serve as an important incentive for participating youth, increasing their effort to meet the requirements of the particular profession. However, because these programs integrate cog- nitive and socioemotional ­ education with particular incentives for satisfac- tory performance, they do not allow one to discern the relative effectiveness of each of these elements. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 77 Our limited evidence on the use of residential services to boost labor market outcomes is not promising. Only two programs in our sample of preventive pro- grams offer residential services: Job Corps and National Guard Youth ChalleNGe. These programs take place in the United States, target at-risk youth and high- school dropouts, usually take more than a year to complete, and include training and remedial education as well as mentoring. Over the short term, they have substantial impacts on employment, educational attainment, and crime. However, the effects of Job Corps on earnings and the effects of National Guard Youth ChalleNGe on high school graduation and criminality fade over time. Further analysis of the programs included in this book shows that, as expected, the quality of the programs mediates the magnitude of their effects. For example, the impact evaluation of Questscope Non-Formal Education (Jordan) found both significant negative effects in low-quality centers and important positive effects in high-quality centers. Also, the low-intensity socioemotional interven- tion called Jordan NOW was found to have a limited effect on women’s gradua- tion from public community colleges.22 Most programs appear to work better for younger individuals and females, and when implemented in cities (see figure 6.12). The impact of both Procajoven (Panama) and Jóvenes en Acción (Colombia) on the probability of employment, number of hours worked weekly, and labor earnings is limited to women. Procajoven also has greater effects in Panama City. Ibarrarán and others (2012) also find that Juventud y Empleo (Dominican Republic) has greater benefits for younger individuals and for those living in Santo Domingo, but more in men’s formal employment. entra21 (Argentina), Questscope Non-Formal Education Figure 6.12 Selected Participant Characteristics That Affect the Outcomes of Out-of-School Programs Program characteristics Context Outcomes Age Participant Gender profile Location Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 78 Program Analysis throughout the Life Cycle (Jordan), and Job Corps (United States) also have greater impacts on younger cohorts. In contrast, the effects of entra21 and Team Awareness (United States) are greater for men. Additionally, Card, Kluve, and Weber (2010) report that youth programs lead to fewer and less positive results in labor outcomes. In an analysis of more than 200 econometric evaluations, Card, Kluve, and Weber (2015) find larger gains for females and participants that enter the program after long-term unemployment. With regard to location, multiple authors argue that programs aimed at improving labor market outcomes appear to have greater effects in developing countries. For example, Urzua and Puentes (2010) and Ñopo and Saavedra (2003) find more positive evidence in Latin America and the Caribbean than in more industrialized countries. Betcherman, Olivas, and Dar (2004) document similar results after reviewing 69 impact evaluations of unemployed and youth training; the impacts they found in the 19 programs implemented in developing countries were greater than those found in the United States and Europe. Notes 1. Appendix A describes the before-school programs that were analyzed. 2. The calculation focused on statistically significant outcomes only. Including both sig- nificant and nonsignificant outcomes would have been ideal, as that would have provided information on the objectives of the programs and their effectiveness. However, most evaluations do not report nonsignificant outcomes, resulting in no knowledge regarding further program objectives. 3. Only the Nurse-Family Partnership and the Jamaican Study find effects on cognitive abilities that persist over time. 4. Wealthier children who first attended primary school switched to preschool, but chil- dren from poorer families who were attending primary school left the formal educa- tion system when the official age of primary school was enforced. 5. Conscientiousness is defined as the degree to which a person is willing to comply with conventional rules, norms, and standards. 6. Currie and Thomas (1995) document that Head Start’s impact on vulnerable minori- ties decreases significantly when these children stop attending the program and return to their initial environment. 7. Appendix B describes the school-based programs that were analyzed. 8. To avoid redundancy, prekindergarten and kindergarten programs were included in the preceding section concerning early childhood programs in an institutional setting. 9. See appendix B for further details. 10. According to Eisenberg and Mussen (1989, 3), prosocial behavior refers to “volun- tary actions that are intended to help or benefit another individual or group of ­individuals.” 11. Because of space and time limitations, this book does not analyze programs that take place during after-school hours or in the summer. 12. Standardized mean difference is defined as the difference between the treatment and control group means on an outcome variable divided by their pooled standard deviation. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 79 13. See, for example, Battistich, Schaps, and Wilson (2004) and Brock and others (2008). 14. According to the World Development Indicators, 73.6 percent of the world’s popula- tion was over age 15 in 2013. 15. These programs are Ninaweza, Jordan NOW (New Opportunities for Women), EPAG (Economic Empowerment of Adolescent Girls and Young Women), and ELA (Empowerment and Livelihood for Adolescents). 16. In the case of Galpão Aplauso, Brazil faced high levels of formality (com carteira assi- nada), and the program did not rely on subsidized internships with a formal job (Calero and others 2014). 17. The impact evaluation of Juventud y Empleo captured socioemotional skills in two ways. First, it developed the Social and Personal Competencies Scale to capture lead- ership, behavior in conflict situations, self-esteem, abilities to relate to others, order, empathy, and communication skills. Second, it used Rosenberg’s self-esteem scale and Duckworth’s Grit scale. 18. For more information, see Goodman, Meltzer, and Bailey (1998). 19. Ninaweza provides 40 hours of life skills training, including attention to emotional intelligence, self-care, decision making, motivation, communication, cooperation and competition, goal setting, and teamwork. ELA provided life skills training that included materials on sexual and reproductive health, menstruation and menstrual disorders, pregnancy, sexually transmitted diseases, HIV/AIDS (human immunodefi- ciency virus/acquired immune deficiency syndrome) awareness, family planning, rape, management skills, negotiation and conflict resolution, leadership among adolescents, bride price, child marriage, and violence against women. 20. The impact evaluation of Galpão Aplauso uses the Social and Personal Competencies Scale developed for the evaluation of Juventud y Empleo in the Dominican Republic. 21. When analyzing more than 200 studies of active labor market programs, Card, Kluve, and Weber (2015) find that average impacts may be close to zero in the short run, but become more positive two to three years after program completion. 22. Jordan NOW provides 45 hours of socioemotional training over a nine-day period, with a maximum of 30 participants in each training group. The course covers effective com- munication and business writing skills (for example, making presentations and writing business reports and different types of correspondence), team building and teamwork skills (for example, characteristics of a successful team, how to work in different roles within a team), time management, positive thinking and how to use it in business situ- ations, excellence in providing customer service, and résumé and interviewing skills. References Adda, Jerome, Christian Dustmann, Costas Meghir, and Jean-Marc Robin. 2013. “Career Progression, Economic Downturns, and Skills.” NBER Working Paper 18832, National Bureau of Economic Research Cambridge, MA. Alvarez de Azevedo, Thomaz, Jeff Davis, and Munene Charles. 2013. “Testing What Works in Youth Employment: Evaluating Kenya’s Ninaweza Program.” Global Partnership for Youth Employment, Washington, DC. Alzúa, María Laura, Guillermo Cruces, and Carolina Lopez Erazo. 2013. “Youth Training Programs beyond Employment: Evidence from a Randomized Controlled Trial.” https://www2.unine.ch/files/content/sites/irene/files/shared/documents/seminaires​ /­Alzua.pdf. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 80 Program Analysis throughout the Life Cycle Araujo, María Caridad, Florencia López-Bóo, Juan Manuel Puyana. 2013. Overview of Early Childhood Development Services in Latin America and the Caribbean. Washington, DC: Inter-American Development Bank, Social Protection and Health Division. Arthur, Winfred Jr., Winston Bennett Jr., Pamela L. Stanush, and Theresa L. McNelly. 1998. “Factors That Influence Skill Decay and Retention: A Quantitative Review and Analysis.” Human Performance 11 (1): 57–101. Aseltine, Robert H. Jr., Matthew Dupre, and Pamela Lamlein. 2000. “Mentoring as a Drug Prevention Strategy: An Evaluation of ‘Across Ages.”’ Adolescent and Family Health 1 (1): 11–20. Attanasio, Orazio, Adriana Kugler, and Costas Meghir. 2011. “Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial.” American Economic Journal: Applied Economics 3 (3): 188–220. Battistich, Victor, Eric Schaps, and Nance Wilson. 2004. “Effects of an Elementary School Intervention on Students’ ‘Connectedness’ to School and Social Adjustment during Middle School.” Journal of Primary Prevention 24 (3): 243–62. Behrman, Jere R., Yingmei Cheng, and Petra E. Todd. 2004. “Evaluating Preschool Programs When Length of Exposure to the Program Varies: A Nonparametric Approach.” Review of Economics and Statistics 86 (1): 108–32. Belfield, Clive, Brooks Bowden, Alli Klapp, Henry Levin, Robert Shand, and Sabine Zander. 2015. “The Economic Value of Social and Emotional Learning.” Center for Benefit-Cost Studies in Education, Teachers College, Columbia University, New York. Betcherman, Gordon, Karina Olivas, and Amit Dar. 2004. “Impacts of Active Labor Market Programs: New Evidence from Evaluations with Particular Attention to Developing and Transition Countries.” Social Protection Discussion Paper 0402, World Bank, Washington, DC. Bloom, Dan, Alissa Gardenhire-Crooks, and Conrad Mandsager. 2009. “Reengaging High School Dropouts: Early Results of the National Guard Youth Challenge Program Evaluation.” Report, MDRC, New York. Bond, Lynne A., and Amy M. Carmola Hauf. 2004. “Taking Stock and Putting Stock in Primary Prevention: Characteristics of Effective Programs.” Journal of Primary Prevention 24 (3): 199–221. Borghans, Lex, Angela Lee Duckworth, James J. Heckman, and Bas ter Weel. 2008. “The Economics and Psychology of Personality Traits.” Journal of Human Resources 43 (4): 972–1059. Bougheas, Spiros, and Yannis Georgellis. 2004. “Early Career Mobility and Earnings Profiles of German Apprentices: Theory and Empirical Evidence.” Labour 18 (2): 233–63. Bouguen, Adrien, Deon Filmer, Karen Macours, and Sophie Naudeau. 2014. “Preschools and Early Childhood Development in a Second Best World: Evidence from a Scaled-Up Experiment in Cambodia.” CEPR Discussion Paper 10170, Center for Economic Policy Research, Washington, DC. Boyle, Douglas, and Connie Hassett-Walker. 2008. “Reducing Overt and Relational Aggression among Young Children: The Results from a Two-Year Outcome Study.” Journal of School Violence 7 (1): 27–42. Brock, Laura L., Tracy K. Nishida, Cynthia Chiong, Kevin J. Grimm, and Sara E. Rimm- Kaurman. 2008. “Children’s Perceptions of the Classroom Environment and Social Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 81 and Academic Performance: A Longitudinal Analysis of the Contribution of the Responsive Classroom Approach.” Journal of School Psychology 46 (2): 129–49. Brown, Joshua L., Stephanie M. Jones, Maria D. LaRusso, and J. Lawrence Aber. 2010. “Improving Classroom Quality: Teacher Influences and Experimental Impacts of the 4Rs Program.” Journal of Educational Psychology 102 (1): 153–67. Calero, Carla, Carlos Henrique Corseuil, Veronica Gonzales, Jochen Kluve, and Yuri Soares. 2014. “Can Arts-Based Interventions Enhance Labor Market Outcomes among Youth? Evidence from a Randomized Trial in Rio de Janeiro.” IZA Discussion Paper 8210, Institute for the Study of Labor, Bonn. Caliendo, Marco, Ricarda Schmidl, and Arne Uhlendorff. 2011. “Social Networks, Job Search Methods and Reservation Wages: Evidence for Germany.” International Journal of Manpower 32 (7): 796–824. Card, David, Jochen Kluve, and Andrea Weber. 2010. “Active Labour Market Policy Evaluations: A Meta-Analysis.” Economic Journal 120 (548): F452–F477. ———. 2015. “What Works? A Meta-Analysis of Recent Active Labor Market Program Evaluations.” NBER Working Paper 21431, National Bureau of Economic Research, Cambridge, MA. Chang, Florence, and Marco A. Muñoz. 2006. “School Personnel Educating the Whole Child: Impact of Character Education on Teachers’ Self-Assessment and Student Development.” Journal of Personality Evaluation in Education 19: 35–49. Chetty, Raj, John N. Friedman, Nathaniel Hilger, Emmanuel Saez, Diane W. Schanzenbach, and Danny Yagan. 2010. “How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project STAR.” NBER Working Paper 16381, National Bureau of Economic Research, Cambridge, MA. Clark, Damon, and René Fahr. 2002. “The Promise of Workplace Training for Non- College-Bound Youth: Theory and Evidence from German Apprenticeship.” Discussion Paper 518, Centre for Economic Performance, London School of Economics. Connell, Arin M., and Thomas J. Dishion. 2008. “Reducing Depression among At-Risk Early Adolescents: Three-Year Effects of a Family-Centered Intervention Embedded within Schools.” Journal of Family Psychology 22 (4): 574–85. Connell, Arin M., Susan Klostermann, and Thomas J. Dishion. 2012. “Family Checkup Effects on Adolescent Arrest Trajectories: Variation by Developmental Subtype.” Journal of Research on Adolescence 22 (2): 367–80. CPPRG (Conduct Problems Prevention Research Group). 2010. “The Effects of a Multiyear Universal Socio-Emotional Learning Program: The Role of Student and School Characteristics.” Journal of Consulting and Clinical Psychology 78 (2): 156–68. Cunha, Flavio, James J. Heckman, Lance Lochner, and Dimitriy V. Masterov. 2005. “Interpreting the Evidence on Life Cycle Skill Formation.” In Handbook of the Economics of Education, vol. 1, edited by Eric A. Hanushek and Finis Welch, 697–812. Amsterdam and New York: North-Holland. Currie, Janet, and Duncan Thomas. 1995. “Does Head Start Make a Difference?” American Economic Review 85 (3): 341–64. Domitrovich, Celene E., Rebecca C. Cortes, and Mark T. Greenberg. 2007. “Improving Young Children’s Social and Emotional Competence: A Randomized Trial of the Preschool PATHS Curriculum.” Journal of Primary Prevention 28 (2): 67–91. Durlak, Joseph A. 1997. Successful Prevention Programs for Children and Adolescents. New York: Plenum Press. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 82 Program Analysis throughout the Life Cycle Durlak, Joseph A., Roger Weissberg, Allison Dymnicki, Rebecca Taylor, and Kriston Schellinger. 2011. “The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Programs.” Child Development 82 (1): 405–32. Durlak, Joseph A., Roger Weissberg, and Molly Pachan. 2010. “A Meta-Analysis of After- School Programs That Seek to Promote Personal and Social Skills in Children and Adolescents.” American Journal of Community Psychology 45 (3–4): 294–309. Dusenbury, Linda, and Mathea Falco. 1995. “Eleven Components of Effective Drug Abuse Prevention Curricula.” Journal of School Health 65 (10): 420–25. Eckenrode, John, Mary Campa, Dennis W. Luckey, Charles R. Henderson, Robert Cole, Harriet J. Kitzman, Elizabeth Anson, Kimberly Sidora-Arcoleo, and David L. Olds. 2010. “Long-Term Effects of Prenatal and Infancy Nurse Home Visitation on the Life Course of Youths: 19-Year Follow-Up of a Randomized Trial.” Journal of the American Medical Association 164 (1): 9–15. Eisenberg, Nancy, and Paul Henry Mussen, eds. 1989. The Roots of Prosocial Behavior in Children. Cambridge: Cambridge University Press. Epstein, Ann S. 1993. Training for Quality: Improving Early Childhood Programs through Systematic In-Service Training. Ypsilanti, MI: HighScope Press. Espelage, Dorothy L., Sabina Low, Joshua R. Polanin, and Eric C. Brown. 2013. “The Impact of a Middle School Program to Reduce Aggression, Victimization, and Sexual Violence.” Journal of Adolescent Health 53 (2): 180–86. Fazio, Maria Victoria. 2011. “Programa Juventud y Empleo en República Dominicana: Análisis de la percepción de los empleadores acerca ee las pasantías y ee las perspec- tivas ee inserción laboral de los jóvenes.” Técnical Note IDB-TN-240, Inter-American Development Bank, Washington, DC. Fosco, Gregory M., Jennifer L. Frank, Elizabeth A. Stormshak, and Thomas J. Dishion. 2013. “Opening the ‘Black Box’: Family Check-Up Intervention Effects on Self- Regulation That Prevents Growth in Problem Behavior and Substance Use.” Journal of School Psychology 51 (4): 455–68. Frey, Karin S., Miriam K. Hirschstein, Leihua Van Schoiack Edstrom, and Jennie L Snell. 2009. “Observed Reductions in School Bullying, Non-Bullying Aggression, and Destructive Bystander Behavior: A Longitudinal Evaluation.” Journal of Educational Psychology 101: 466–81. Gertler, Paul, James Heckman, Rodrigo Pinto, Arianna Zanolini, Christel Vermeerch, Susan Walker, Susan Chang-Lopez, and Sally Grantham-McGregor. 2013. “Labor Market Returns to Early Childhood Stimulation: A 20-Year Follow Up to an Experimental Intervention in Jamaica.” Policy Research Working Paper 6529, World Bank, Washington, DC. González-Velosa, Carolina, Laura Ripani, and David Rosas-Shady. 2012. “How Can Job Opportunities for Young People in Latin America Be Improved?” Technical Note IDB-TN-345, Inter-American Development Bank, Washington, DC. Goodman, Robert, Howard Meltzer, and V. Bailey. 1998. “The Strengths and Difficulties Questionnaire: A Pilot Study on the Validity of the Self-Report Version.” European Child and Adolescent Psychiatry 7 (3): 125–30. Gresham, Frank M. 1995. “Best Practices in Social Skills Training.” In Best Practices in School Psychology III, edited by Alex Thomas and Jeff Grimes, 1021–30. Washington, DC: National Association of School Psychologists. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 83 Groh, Matthew, Nandini Krishnan, David McKenzie, and Tara Vishwanath. 2012. “Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment in Jordan.” Policy Research Working Paper 6141, World Bank, Washington, DC. Hahn, Robert, Dawna Fuqua-Whitley, Holly Wethington, Jessica Lowy, Alex Crosby, Mindi Fullilove, Robert Johnson, Akiva Liberman, Eve Moscicki, LeShawndra Price, Susan R. Snyder, Farris Tuma, Stella Cory, Glenda Stone, Kaushik Mukhopadhaya, Sajal Chattopadhyay, and Linda Dahlberg. 2007. “Effectiveness of Universal School- Based Programs to Prevent Violent and Aggressive Behavior: A Systematic Review.” American Journal of Preventive Medicine 33 (2 Suppl.): S114–S129. Heckman, James J. 2008. “Schools, Skills and Synapses.” Economic Inquiry 46 (3): 289–324. Heckman, James J., Lena Malofeeva, Rodrigo Pinto, and Peter A. Savelyev. 2009. “The Powerful Role of Noncognitive Capabilities in Explaining the Effects of the Perry Preschool Program.” Unpublished manuscript, University of Chicago, Department of Economics. Heckman, James J., and Tim Kautz. 2012. “Hard Evidence on Soft Skills.” Labour Economics 19 (4): 451–64. ———. 2013. “Fostering and Measuring Skills: Programs That Improve Character and Cognition.” NBER Working Paper 19656, National Bureau of Economic Research, Cambridge, MA. Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24 (3): 411–48. Heinrich, Carolyn J., Peter R. Mueser, Kenneth R. Troske, Kyung-Seong Jeon, and Daver C. Kahvecioglu. 2013. “Do Public Employment and Training Programs Work?” IZA Journal of Labor Economics 2 (1): 1–23. Herrera, Carla, Tina J. Kauh, Siobhan M. Cooney, Jean Baldwin Grossman, and Jennifer McMaken. 2008. “High School Students as Mentors: Findings from the Big Brothers Big Sisters School-Based Mentoring Impact Study.” Public/Private Ventures, Philadelphia. http://www.childtrends.org/?programs=big-brothersbig-sisters-school​ -based-mentoring#sthash.NrUcSLm1.dpuf. Hollenbeck, Kevin. 2008. “Is There a Role for Public Support of Incumbent Worker On-the-Job Training?” Working Paper 08–138, W. E. Upjohn Institute for Employment Research, Kalamazoo, MI. Honorati, Maddalina, and Thomas P. McArdle. 2013. “The Nuts and Bolts of Designing and Implementing Training Programs in Developing Countries.” Working Paper 78980, World Bank, Washington, DC. Howard, Kimberly S., and Jeanne Brooks-Gunn. 2009. “The Role of Home-Visiting Programs in Preventing Child Abuse and Neglect.” Future of Children 19 (2): 119–46. Humphrey, Neal, Ann Lendrum, and Michael Wigelsworth. 2010. “Social and Emotional Aspects of Learning (SEAL) Programme in Secondary Schools: National Evaluation.” Research Report DFE-RR049, Department for Education, London. Ibarrarán, Pablo, Jochen Kluve, Laura Ripani, and David Rosas-Shady. 2015. “Experimental Evidence on the Long-Term Impacts of a Youth Training Program.” IZA Discussion Paper 9136, Institute for the Study of Labor, Bonn. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 84 Program Analysis throughout the Life Cycle Ibarrarán, Pablo, Laura Ripani, Bibiana Taboada, Juan Miguel Villa, and Brigida Garcia. 2012. “Life Skills, Employability and Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design.” IZA Discussion Paper 6617, Institute for the Study of Labor, Bonn. Jones, Stephanie M., and Suzanne M. Bouffard. 2012. “Social and Emotional Learning in Schools: From Programs to Strategies.” Social Policy Report 26 (4): 1–22. Jones, Stephanie M., Joshua L. Brown, and J. Lawrence Aber. 2011. “Two-Year Impacts of a Universal School-Based Social-Emotional and Literacy Intervention: An Experiment in Translational Developmental Research.” Child Development 82: 533–54. Jones, Stephanie M., Joshua L. Brown, Wendy Hoglund, and J. Lawrence Aber. 2010. “A School-Randomized Clinical Trial of an Integrated Social-Emotional Learning and Literacy Intervention: Impacts after One School Year.” Journal of Consulting and Clinical Psychology 78 (6): 829–42. Kautz, Tim, James J. Heckman, Ron Diris, Bas ter Weel, and Lex Borghans. 2014. “Fostering and Measuring Skills: Improving Cognitive and Non-Cognitive Skills to Promote Lifetime Success.” NBER Working Paper 20749, National Bureau of Economic Research, Cambridge, MA. Kitzman, Harriet J., David L. Olds, Robert E. Cole, Carole A. Hanks, Elizabeth A. Anson, Kimberly J. Arcoleo, Dennis W. Luckey, Michael D. Knudtson, Charles R. Henderson Jr., and John R. Holmberg. 2010. “Enduring Effects of Prenatal and Infancy Home Visiting by Nurses on Children: Follow-Up of a Randomized Trial among Children at Age 12 Years.” Archives of Pediatrics and Adolescent Medicine 164 (5): 412–18. Klevens, Joanne, José William Martínez, Brenda Le, Carlos Rojas, Adriana Duque, and Rafael Tovar. 2009. “Evaluation of Two Interventions to Reduce Aggressive and Antisocial Behavior in First and Second Graders in a Resource-Poor Setting.” International Journal of Educational Research 48 (5): 307–19. Krueger, Alan B., and Diane M. Whitmore. 2001. “The Effect of Attending a Small Class in the Early Grades on College‐Test Taking and Middle School Test Results: Evidence from Project STAR.” Economic Journal 111 (468): 1–28. Kuckulenz, Anja. 2007. Studies on Continuing Vocational Training in Germany: An Empirical Assessment. Vol. 37, ZEW Economic Studies Series. Heidelberg, Germany: Physica-Verlag. Kumpfer, Karol L., Rose Alvarado, Connie Tait, and Charles Turner. 2002. “Effectiveness of School-Based Family and Children’s Skills Training for Substance Abuse Prevention among 6–8-Year-Old Rural Children.” Psychology of Addictive Behaviors 16 (45): S65–S71. Levenstein, Phyllis, Susan Levenstein, James A. Shiminski, and Judith E. Stolzberg. 1998. “Long-Term Impact of a Verbal Interaction Program for At-Risk Toddlers: An Exploratory Study of High School Outcomes in a Replication of the Mother–Child Home Program.” Journal of Applied Developmental Psychology 19 (2): 267–85. Lewis, Kendra M., Marc B. Schure, Niloofar Bavarian, David L. Dubois, Joseph Day, Peter Ji, Naida Silverthorn, Alan Acock, Samuel Vuchinich, and Brian R. Flay. 2013. “Problem Behavior and Urban, Low-Income Youth: A Randomized Controlled Trial of Positive Action in Chicago.” American Journal of Preventive Medicine 4 (6): 622–30. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 85 Loeb, Susanna, Margaret Bridges, Daphna Bassok, Bruce Fuller, and Russell W. Rumberger. 2007. “How Much Is Too Much? The Influence of Preschool Centers on Children’s Social and Cognitive Development.” Economics of Education Review 26 (1): 52–66. Lösel, Friedrich. 2001. “Evaluating the Effectiveness of Correctional Programs: Bridging the Gap between Research and Practice.” In Offender Rehabilitation in Practice: Implementing and Evaluating Effective Programs, edited by Gary A. Bernfeld, David P. Farrington, and Alan W. Leschied, 67–92. Chichester, U.K.: Wiley. Lösel, Friedrich, and Andreas Beelmann. 2003. “Effects of Child Skills Training in Preventing Antisocial Behavior: A Systematic Review of Randomized Evaluations.” Annals of the American Academy of Political and Social Science 587 (1): 84–109. Lynch, Kathleen Bodisch. 1998. “Results of Michigan Replication Study 1996–97: Child Outcomes—Al’s Pals: Kids Making Healthy Choices.” Institute for Development Disabilities, Virginia Commonwealth University, Richmond. Lynch, Kathleen Bodisch, Susan R. Geller, and Denise R. Hunt. 1998.“Successful Program Development Using Implementation Evaluation.” Journal of Prevention and Intervention in the Community 17 (2): 51–64. Lynch, Kathleen Bodisch, Susan R. Geller, and Melinda G. Schmidt. 2004. “Multi-Year Evaluation of the Effectiveness of a Resilience-Based Prevention Program for Young Children.” Journal of Primary Prevention 24 (3): 335–53. Magnuson, Katherine A., Christopher Ruhm, and Jane Waldfogel. 2007. “Does Prekindergarten Improve School Preparation and Performance?” Economics of Education Review 26 (1): 33–51. Martinez, Sebastian, Sophie Nadeau, and Vitor Pereira. 2012. “The Promise of Preschool in Africa: A Randomized Impact Evaluation of Early Childhood Development in Rural Mozambique.” enGender Impact: The World Bank’s Gender Impact Evaluation Database, World Bank, Washington, DC. McCord, Joan. 1978. “A Thirty-Year Follow-Up of Treatment Effects.” American Psychologist 33 (3): 284–89. Melmed, Matthew. 2008. Testimony on “Investing in Early Education: Paths to Improving Children’s Success” before the House Committee on Education and Labor, 110th Cong., 2nd sess., January 23. Millenky, Megan, Dan Bloom, and Colleen Dillon. 2010. “Making the Transition: Interim Results of the National Guard Youth Challenge Evaluation.” MDRC, New York. Millenky, Megan, Dan Bloom, Sara Muller-Ravett, and Joseph Broadus. 2011. “Staying on Course: Three-Year Results of the National Guard Youth Challenge Evaluation.” MDRC, New York. NICHHD (National Institute of Child Health and Human Development Early Child Care Research Network). 2003. “Social Functioning in First Grade: Associations with Earlier Home and Child Care Predictors and with Current Classroom Experiences.” Child Development 74 (6): 1639–62. Ñopo, Hugo, and Jaime Saavedra. 2003. “Recomendaciones para la mejora del levanta- miento de la linea de base de projoven y sugerencias para la construcción de una linea de base aleatorizada como parte de un disen o experimental” (Recommendations to Improve Baseline Data Collection for Projoven and to Construct a Baseline Using Random Assignment as Part of an Experimental Design). Grupo de Análisis para el Desarrollo, Lima. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 86 Program Analysis throughout the Life Cycle Olds, David L. 2006. “The Nurse-Family Partnership: An Evidence-Based Preventive Intervention.” Infant Mental Health Journal 27 (1): 5–25. Olds, David L., John Eckenrode, Charles R. Henderson Jr., Harriet J. Kitzman, Jane Powers, Robert Cole, Kimberly Sidora, Pamela Morris, Lisa M. Pettitt, and Dennis W. Luckey. 1997. “Long-Term Effects of Home Visitation on Maternal Life Course and Child Abuse and Neglect: Fifteen-Year Follow-Up of a Randomized Trial.” Journal of the American Medical Association 278 (8): 637–43. Olds, David L., Charles R. Henderson Jr., Robert Chamberlin, and Robert Tatelbaum. 1986. “Preventing Child Abuse and Neglect: A Randomized Trial of Nurse Home Visitation.” Pediatrics 78 (1): 65–78. Olds, David L., Charles R. Henderson Jr., Robert Cole, John Eckenrode, Harriet J. Kitzman, Dennis W. Luckey, and Jane Powers. 1986. “Long-Term Effects of Nurse Home Visitation on Children’s Criminal and Antisocial Behavior: 15-Year Follow-Up of a Randomized Controlled Trial.” Journal of the American Medical Association 280 (14): 1238–44. Olds, David L., Charles R. Henderson Jr., and Harriet J. Kitzman. 1994. “Does Prenatal and Infancy Nurse Home Visitation Have Enduring Effects on Qualities of Parental Caregiving and Child Health at 25 to 50 Months of Life?” Pediatrics 93 (1): 89–98. Olds, David L., John R. Holmberg, Nancy Donelan-McCall, Dennis W. Luckey, Michael D. Knudtson, and JoAnn Robinson. 2014. “Effects of Home Visits by Paraprofessionals and by Nurses on Children: Follow-Up of a Randomized Trial at Ages 6 and 9 Years.” JAMA Pediatrics 168 (2): 114–21. Olds, David L., Harriet J. Kitzman, Robert Cole, Carole Hanks, Kimberly Arcoleo, Elizabeth Anson, Dennis W. Luckey, Michael D. Knudtson, Charles R. Henderson Jr., Jessica Bondy, and Amanda J. Stevenson. 2010. “Enduring Effects of Prenatal and Infancy Home Visiting by Nurses on Maternal Life Course and Government Spending: Follow-Up of a Randomized Trial among Children at Age 12 Years.” Journal of the American Medical Association 164 (5): 419–24. Olds, David L., Harriet J. Kitzman, Carole Hanks, Robert Cole, Elizabeth Anson, Kimberly Sidora-Arcoleo, and Jessica Bondy. 2007. “Effects of Nurse Home Visiting on Maternal and Child Functioning: Age-9 Follow-Up of a Randomized Trial.” Pediatrics 120 (4): e832–e845. Payton, John, Roger P. Weissberg, Joseph A. Durlak, Allison B. Dymnicki, Rebecca D. Taylor, Kriston B. Schellinger, and Molly Pachan. 2008. “The Positive Impact of Social and Emotional Learning for Kindergarten to Eighth-Grade Students: Findings from Three Scientific Reviews—Technical Report.” Collaborative for Academic, Social, and Emotional Learning, Chicago. Perez-Arce, Francisco, Louay Constant, David S. Loughran, and Lynn A. Karoly. 2012. “A Cost–Benefit Analysis of the National Guard Youth ChalleNGe Program.” RAND Research Monograph TR1193, RAND Corporation, Santa Monica, CA. Rolnick, Arthur J., and Rob Grunewald. 2007. “The Economics of Early Childhood Development as Seen by Two Fed Economists.” Community Investments 19 (2): 13–14, 30. Rubio-Codina, Marta, Orazio Attanasio, Costas Meghir, Natalia Varela, and Sally Grantham-McGregor. 2014. “The Socio-Economic Gradient of Child Development: Cross-Sectional Evidence from Children 6–42 Months in Bogota.” Journal of Human Resources 50 (2): 464–83. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Analysis throughout the Life Cycle 87 Rutter, Michael, 1987. “Psychosocial Resilience and Protective Mechanisms.” American Journal of Orthopsychiatry 57 (3): 316–31. Schochet, Peter Z., John Burghardt, and Sheena McConnell. 2008. “Does Job Corps Work? Impact Findings from the National Job Corps Study.” American Economic Review 98 (5): 1864–86. Schweinhart, Lawrence J., and David P. Weikart. 1997. “The High/Scope Preschool Curriculum Comparison Study through Age 23.” Early Childhood Research Quarterly 12 (2): 117–43. Shady, Norbert, and Christina Paxson. 2007. “Does Money Matter? The Effects of Cash Transfers on Child Health and Development in Rural Ecuador.” Policy Research Working Paper 4226, World Bank, Washington, DC. Snyder, Frank J., Alan Acock, Sam Vuchinich, Michael W. Beets, Isaac J. Washburn, and Brian R. Flay. 2013. “Preventing Negative Behaviors among Elementary-School Students through Enhancing Students’ Social-Emotional and Character Development.” American Journal of Health Promotion 28 (1): 50–58. Stormshak, Elizabeth A., Arin M. Connell, and Thomas J. Dishion. 2009. “An Adaptive Approach to Family-Centered Intervention in Schools: Linking Intervention Engagement to Academic Outcomes in Middle and High School.” Prevention Science 10 (3): 221–35. Stormshak, Elizabeth A., Arin M. Connell, Marie Hélène Véronneau, Michael W. Myers, Thomas J. Dishion, Kathryn Kavanagh, and Allison S. Caruthers. 2011. “An Ecological Approach to Promoting Early Adolescent Mental Health and Social Adaptation: Family-Centered Intervention in Public Middle Schools.” Child Development 82 (1): 209–25. Urzua, Sergio, and Esteban Puentes. 2010. “La evidencia del impacto de los programas de capacitación en el desempeño en el mercado laboral.” Inter-American Development Bank, Washington, DC. Van Ryzin, Mark. J., Gregory M. Fosco, and Thomas J. Dishion. 2012. “Family and Peer Predictors of Substance Use from Early Adolescence to Early Adulthood: An 11-Year Prospective Analysis.” Addictive Behaviors 37 (12): 1314–24. Van Ryzin, Mark. J., Elizabeth A. Stormshak, and Thomas J. Dishion. 2012. “Engaging Parents in the Family Check-Up in Middle School: Longitudinal Effects on Family Conflict and Problem Behavior through the High School Transition.” Journal of Adolescent Health 50 (6): 627–33. Vreeman, Rachel C., and Aaron E. Carroll. 2007. “A Systematic Review of School-Based Programs to Prevent Bullying.” Archives of Pediatrics and Adolescent Medicine 161 (1): 78–88. Wagner, Mary M., and Serena L. Clayton. 1999. “The Parents as Teachers Program: Results from Two Demonstrations.” Future of Children 9 (1): 91–115. Wagner, Mary M., Donna Spiker, and Margaret Inman Linn. 2002. “The Effectiveness of the Parents as Teachers Program with Low-Income Parents and Children.” Topics in Early Childhood Special Education 22 (2): 67–81. Weissberg, Roger P., and Mark T. Greenberg. 1998. “School and Community Competence- Enhancement and Prevention Programs.” In Handbook of Child Psychology, 5th ed., vol. 4, edited by William Damon, Richard M. Lerner, Irving E. Sigel, K. Ann Renninger, 877–954. Hoboken, NJ: Wiley. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 88 Program Analysis throughout the Life Cycle Wilson, Sandra Jo, and Mark W. Lipsey. 2007. “School-Based Programs for Aggressive and Disruptive Behavior: Update of a Meta-Analysis.” American Journal of Preventive Medicine 33 (Suppl. 2): S130–S143. Winkelmann, Rainer. 1996. “Employment Prospects and Skill Acquisition of Apprenticeship-Trained Workers in Germany.” Industrial and Labor Relations Review 49 (4): 658–72. Word, Elizabeth, John Johnson, Helen Pate Bain, B. DeWayne Fulton, Jan Boyd Zaharies, Martha Nannette Lintz, Charles M. Achiles, John Folger, and Carolyn Breda. 1990. “Student/Teacher Achievement Ratio (STAR) Tennessee’s K–3 Class Size Study: Final Summary Report, 1985–1990.” Tennessee State Department of Education, Nashville. Zielinski, David S., John Eckenrode, and David L. Olds. 2009. “Nurse Home Visitation and the Prevention of Child Maltreatment: Impact on the Timing of Official Reports.” Development and Psychopathology 21 (2): 441–53. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Chapter 7 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? This chapter presents our best effort to synthesize the most important findings of the extensive literature on the rigorous evaluations of programs that seek to promote socioemotional skills, usually alongside fostering several other skills or behaviors. The findings below summarize results along the following dimensions: targeting, focus, impacts, replicability, and evaluation. Targeting Most programs reviewed are oriented toward individuals with high vulnerability. Before-school programs tend to benefit (a) children in low socioeconomic strata or who belong to minority groups or (b) children whose parents are poorly educated, have low occupational status, have low cognitive skills, or are in their teenage years. School-based programs are often carried out in institutions located in high-risk areas, including regions with low income, considerable school absenteeism, high mobility, or elevated crime rates. Finally, out-of-school programs focus primarily on low-income unemployed individuals, those who have dropped out of the education system, or women. With regard to age, gender, and location, programs tend to target all age ranges, are mostly gender ­ neutral (although a few out-of-school programs focus on women), and focus on urban areas. Because of the differences in the age groups covered, the process for entry into the different programs varies greatly throughout the life cycle: whereas before- school programs require parental consent and interest, school-based programs normally require consent only from school officials, though some programs do seek parental consent. Finally, out-of-school programs are sought out by youths and adults who make their own decisions about whether to join. Taking Stock of Programs to Develop Socioemotional Skills   89   http://dx.doi.org/10.1596/978-1-4648-0872-2 90 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? Focus Another important finding is that few programs focus exclusively on the development of socioemotional skills. For example, most before-school pro- grams also seek to provide care or to support cognitive development or school readiness; school-based programs seek to improve classroom climate and to decrease aggressive, violent, or antisocial behaviors and substance abuse; and out- of-school programs tend to have labor market–related components. This finding is expected, because socioemotional skills development is normally part of a broader curriculum. Impacts Most of the before-school programs focus on highly vulnerable children and their families. They have important effects on cognitive skills and academic outcomes in the short run, whereas in the long run they improve behavior and risk factors, which indicates that they can modify socioemotional skills. These programs appear to be particularly effective when targeting vulnerable populations, as well as when they have multiple components, involve parents, and are intense with regard to instruction time and the teacher-to-child ratio. Unlike most early childhood development programs, school-based programs tend to be universal at a school or classroom level, benefiting all children and adolescents in the classroom or attending the institution. The evidence regarding these programs is less widespread, but available results show a particularly strong effect on such risk factors as internalizing and externalizing behavior, aggression toward peers, and cooperation. Long and intense programs tend to be more effective, as do those that target vulnerable or at-risk populations. Programs for out-of-school children and youths are usually designed to achieve immediate labor market outcomes (for example, job placement, formal employment, and higher wages). Although some of these programs show positive and statistically significant effects on socioemotional skills, the impacts tend to be small. These programs seem to be cost-effective, but long-term evaluation is lacking. Early childhood programs appear to have a greater impact than those that take place later in life. Kautz and others (2014) state that only very early childhood programs (before age three) improve IQ (intelligence quotient) in a lasting way. Further, they suggest that the most promising adolescent programs integrate aspects of work into traditional education, breaking down the rigid separation between school and employment. Kautz and oth- ers (2014) explain that early childhood is a sensitive period for cognitive and noncognitive skills development. Furthermore, through dynamic com- plementarity, investments in the early years make later investments more productive. They also state that the most successful programs target pre- schoolers (after age three) and primary school children, improving later-life outcomes by developing socioemotional skills. However, as will be explained Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? 91 below, these results could partially reflect the fact that early childhood pro- grams have longer follow-up periods. Throughout the life cycle, socioemotional skills acquisition appears to explain important life outcomes. Several programs throughout the life cycle find long- term effects on earnings despite no indication of cognitive gain. This finding sug- ­ ocioemotional gests that the long-term effects result from a change in beneficiaries’ s skills. Among these programs are the HighScope Perry Preschool Program, the Abecedarian Program, and the Career Academies Program (before- and during- school programs). Replicability Finally, there are questions regarding replicability, as small-scale programs appear to be more effective than massive ones, and external research on scaling-up pro- grams frequently shows that positive effects diminish with dissemination (Durlak and DuPre 2008). Kautz and others (2014) find that the reason behind this outcome in early childhood development programs is that large-scale programs are less intensive and involve parents less frequently. Others argue that this out- come is the result of higher levels of program fidelity in impact evaluations of small-scale programs: normally, the researcher or highly trained university stu- dents implement the intervention, whereas the program itself is normally run by individuals with lower education attainment levels, less training, or less commit- ment to the project. The diminishing positive effects with dissemination might be explained by, among other causes, the following: the programs are seldom designed for universal coverage; and when they are scaled up, they experience lower par- ­ ticipation rates and lower retention, have lower levels of quality and fidelity, and are unable to meet families’ needs (Dodge and others 2014; Howard and Brooks-Gunn 2009). Lösel and Beelmann find greater effects in rigorous impact evaluations of school-based programs that have smaller sample sizes and state, “in large studies, difficulties in maintaining program integrity and homo- geneity of samples or treatments may reduce design sensitivity” (2003, 99). They also find that when small evaluations are taking place, programs tend to be delivered by the study authors, research staff, or other highly qualified individuals—rather than by more typical staff—leading to greater effects than one finds when programs are implemented widely. Hahn and others (2007) also point to the difficulty of providing accurate conclusions because of reporting implementation problems. However, evidence also shows that some large-scale, massive programs can be effective. For example, the school-based Chicago Child-Parent Center program in the United States had significant effects on high school completion, years of edu- cation, juvenile and violent arrests, school dropout rates, grade retention, and use of special-education services. The replicability of the results relies on maintaining such features as expenditure per child, training for providers, and parent involve- ment. Furthermore, the external validity of the results can be compromised when Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 92 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? a program is replicated in a different context or country with dissimilar condi- tions. In some developing countries, for example, where malnutrition levels are higher, programs that effectively foster socioemotional skills in more developed countries or settings cannot be expected to achieve the same results. Evaluation Programs that target early stages in life have longer follow-up periods. Participant outcomes were recorded for 10 or more years after the intervention for 36 percent of the before-school programs, 3 percent of the school-based programs, and none of the out-of-school programs. The length of the follow-up directly affects the results captured by each program, for the following reasons. First, the outcomes that can be measured in programs with short follow-up periods are more constrained. Before-school programs often capture effects that can be observed only with the evolution and growth of participants. For example, they measure behavioral effects with outcomes that evolve over time, from play- ground practices and school suspension to arrests, substance abuse and use, early sexual activity, and even reproductive and childbearing practices. This measure is profoundly richer than what is measured in out-of-school programs, which focus on immediate job-related outcomes. Second, the persistence of any intervention’s effects can be measured only through observations made over an extended period. All evaluations that are restricted to a few months might leave the reader questioning whether the find- ings might vanish after a certain time frame. Furthermore, this lack of informa- tion could mislead those responsible for policy design and implementation. For example, without the 40-year follow-up applied in the Jamaican Study, policy makers could not have learned that providing stimulation was more effective than providing nutrition with regard to cognitive and character achievements for stunted children living in poor, disadvantaged neighborhoods. Third, programs with extensive follow-up periods provide rich information that allows researchers to understand the process of skills acquisition behind the observed impacts (Kautz and others 2014). For example, one could hypothesize that the HighScope Perry Preschool Program’s long-term positive effects on criminality are the result of socioemotional skills acquisition rather than cogni- tive gains, because the follow-up 37 years later found that the effects on test scores faded, whereas behavioral results did not. An additional finding is that although the programs explicitly target socioemotional skills development, few explicitly indicate what socioemotional skills they intend to improve. Further, most impact evaluations do not directly measure the skills that the program intends to alter. As discussed throughout this book, the effects of the different programs are captured in the behavior of the beneficiaries, yet rarely do researchers look to see if an intervention ­ modified personality traits, risk behaviors, or mind-sets. Future research should include skills assessment in order to measure whether or not such changes occur as a result of the intervention. The evaluation of Juventud y Empleo and Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? 93 Galpão Aplauso are exceptions, because they use Rosenberg’s self-esteem scale, Duckworth’s Grit scale, and the Social and Personal Competencies Scale. The Juventud y Empleo intervention had a statistically significant effect on ­ socioemotional skills, whereas Galpão Aplauso had no effects on this area. Furthermore, the literature might contain blind spots on socioemotional skills formation, as the effect of environmental investments to improve socioemotional learning appears underexplored. Only one intervention in our ­ sample (Program H in Brazil) evaluated the impact of a community-wide “lifestyle” social marketing campaign to promote condom use and change gender norms and behavior. Direction for Future Research Although evidence regarding socioemotional skills is promising, important questions still remain unanswered. Future studies should define and measure socioemotional skills directly so as to identify the triggers that lead to behavioral alterations. As mentioned before, a key area for future research is related to the identification of longer-term effects, particularly for school-age and out-of-school populations. It would also be important to measure the relative effectiveness of program intensity versus program duration as well as the way in which the over- all intervention is structured and delivered. Further, impact evaluations of multicomponent programs should capture the cost-effectiveness of the different components. An important area of research lies in the identification of the ideal variations within programs that achieve maximum effects within dissimilar target populations. Other areas of future research include the documentation of the externalities of the programs, which could be achieved by assessing the effects of school-wide versus classroom-wide programs. Much could be gained by documenting the costs of the programs, by including longer follow-up periods, and by collecting direct measures of socioemotional skills. References Dodge, Kenneth A., W. Benjamin Goodman, Robert A. Murphy, Karen O’Donnell, Jeannine Sato, and Susan Guptill. 2014. “Implementation and Randomized Controlled Trial Evaluation of Universal Postnatal Nurse Home Visiting.” American Journal of Public Health 104 (S1): S136–S143. Durlak, Joseph A., and Emily P. DuPre. 2008. “Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation.” American Journal of Community Psychology 41 (3–4): 327–50. Hahn, Robert, Dawna Fuqua-Whitley, Holly Wethington, Jessica Lowy, Alex Crosby, Mindi Fullilove, Robert Johnson, Akiva Liberman, Eve Moscicki, LeShawndra Price, Susan R. Snyder, Farris Tuma, Stella Cory, Glenda Stone, Kaushik Mukhopadhaya, Sajal Chattopadhyay, and Linda Dahlberg. 2007. “Effectiveness of Universal School-Based Programs to Prevent Violent and Aggressive Behavior: A Systematic ­ Review.” American Journal of Preventive Medicine 33 (2 Suppl.): S114–S129. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 94 Program Findings: What Works (or Doesn’t Work) in Fostering Socioemotional Skills? Howard, Kimberly S., and Jeanne Brooks-Gunn. 2009. “The Role of Home-Visiting Programs in Preventing Child Abuse and Neglect.” Future of Children 19 (2): 119–46. Kautz, Tim, James J. Heckman, Ron Diris, Bas ter Weel, and Lex Borghans. 2014. “Fostering and Measuring Skills: Improving Cognitive and Non-Cognitive Skills to Promote Lifetime Success.” NBER Working Paper 20749, National Bureau of Economic Research, Cambridge, MA. Lösel, Friedrich, and Andreas Beelmann. 2003. “Effects of Child Skills Training in Preventing Antisocial Behavior: A Systematic Review of Randomized Evaluations.” Annals of the American Academy of Political and Social Science 587 (1): 84–109. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Appendix A Before-School Program Descriptions Taking Stock of Programs to Develop Socioemotional Skills   95   http://dx.doi.org/10.1596/978-1-4648-0872-2 Table A.1  Before-School Programs—Home Visiting Programs 96 Program Evaluation Name Description Author/description Results Attachment and Beneficiaries: Children younger than age 2 Authors: Dozier and others (2009); Bick and Dozier (2013) Behavior related: Avoidance (–)**; Maternal Biobehavioral Objective: Develop children’s regulatory Type: RCT sensitivity (+)* Catch-up capabilities. Location: United States Intervention Description: The program consists of 10 hour-long Follow-up: 1 month after the intervention (ABC) weekly sessions, where experienced professional Sample: Children ages 1 to 22 months, randomly social workers or psychologists teach caregivers assigned to receive the ABC intervention or the how to interpret children’s alienating behaviors, Developmental Education for Families intervention how to over-ride the issues that interfere with (which has the same intensity, but is designed to nurturing care, and how to provide an enhance cognitive development). Dozier and others environment that fosters child development. (2009) evaluate 46 children and Bick and Dozier Cost: n/a (2013) include 96 children. Term used: Emotional functioning Comprehensive Beneficiaries: Young children (from pregnancy Authors: Pierre and others (1997); Pierre and Layzer The program had no significant effects. The Child until the child enters the school system) from (1989) variables that were measured were Development poor families who need intensive and Type: RCT mother’s employment status, level of Program comprehensive support services Location: United States employment (number of hours worked) of Objective: “Enhance the physical, social, emotional, Follow-up: 5 years later parents, income, level of dependence on and intellectual development of children in Sample: 4,410 families were randomly assigned to public assistance, mother’s educational low-income families; provide support to their either treatment (2,213 families) or control (2,197 status, parental attitudes linked to abusive parents and other family members; and assist families). or neglectful behavior, quality of the home families in becoming economically self- Term used: Social and emotional development environment, parent-child interaction, and sufficient” (Pierre and others 1997, 1). health behaviors of mothers during Description: The program provides comprehensive subsequent pregnancies. services for families. The services targeted at Note: The program was not implemented parents include prenatal care; parental according to design. Only 33% of the education in infant and child development, families were enrolled for five or more healthcare, nutrition, and parenting; and referral years. to education, vocational training, and labor counseling. The services targeted at children include early childhood education, health screening treatment and referral, and nutritional services. Cost: The average total cost per year was $14,984 per family. table continues next page Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results Child-Parent Beneficiaries: Children ages 3 to 5 years who Authors: Lieberman, Ippen, and Van Horn (2006) Behavior related: Total behavior problems Psychotherapy witnessed marital violence Type: RCT (–)***; Mother’s general distress (–)*** Objective: Alleviate symptoms and restore child’s Location: United States development toward healthy functioning Follow-up: 6 months later Description: Child-parent psychotherapy (CPP) is a Sample: 75 preschool-age child-mother dyads were relationship-based treatment that integrates randomly assigned to CPP (42) or case modalities from psychodynamic, attachment, management plus community referral (33). trauma, cognitive-behavioral, and social learning Term used: Emotional functioning theories. The treatment consisted of 55 weekly sessions that lasted approximately 60 minutes each, distributed over a 1-year period. Cost: n/a Play and Beneficiaries: PALS I—mothers with infants ages 6 Authors: Landry, Smith, and Swank (2006) Behavior related: Cooperation in the child Learning to 12 months; PALS II—mothers with toddlers Type: RCT (+)*; mother’s contingent responsiveness Strategies ages 24 to 36 months Location: United States: Houston, Texas, and (+)***; improvements in emotional support (PALS) Objective: Teach parents responsive parenting Galveston, Texas from mothers (warm sensitivity (+)***; program skills to support their child’s socioemotional Follow-up: 3 months later harshness of voice tone (−)**; physical cognitive, and language development. Sample: 264 upper-lower-class and lower-middle- intrusiveness (−)**; support of infant foci of Description: Each week for three months, families class mothers with children ages 6 to 10 months. attention from mothers [maintaining (+)*; are visited by coaches who implement 1.5-hour Sample was divided into two groups: (a) 133 redirecting (+)***]; and quality of language sessions that include the discussion of the mothers receiving 10 meetings (in developmental input from mothers (verbal scaffolding, mother’s practice during the preceding week, a laboratories) with the PALS program content, and labeling objects, labeling actions, verbal description of the behavior targeted in the visit, (b) 131 mothers receiving 10 meetings (in encouragement—all (+)*** an observation and discussion of educational developmental laboratories) where they discussed Academic/cognitive related: Early videos demonstrating the skill, videotaping of new infant skills observed in the previous week, communication (+)*** mothers practicing the skill with their child, measured the children’s development with reviewing the videotaped practice, and planning developmental assessment sessions (DASs), and ways to integrate responsive behavior during the received feedback. upcoming week. Term used: Affective-emotional behavior table continues next page 97 98 Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results PALS I (the infant curriculum) consists of 10 sessions Authors: Landry and others (2008) Infant—behavior related: Cooperation for and is appropriate for parents of infants from Type: RCT children receiving PALS II (+)**; social about ages five months to one year. PALS II (the Location: United States: Houston, Texas, and engagement for children who received toddler curriculum) consists of 12 sessions and is Galveston, Texas PALS II (+)*** appropriate for parents of toddlers from about Follow-up: 3 months later Infant—academic/cognitive related: Use of ages 18 months to three years. Both versions of Sample: 166 upper-lower-class and lower-middle- words for children with PALS II (+)***; the program emphasize similar skills but at an class mothers with children ages 6 to 10 months. coordinating attention for children who age-appropriate level. Sample was divided into four groups: (a) 34 received either PALS I or PALS II (+)**; Cost: According to the U.S. Department of Health mothers who had received PALS I and were to composite language skills for children who and Human Services, approximately US$2,500 receive PALS II, (b) 33 mothers who had received received PALS II (+)** per family (2011 dollars) PALS I and were to receive DASs, (c) 50 mothers who Mother—behavior related: Contingent had received DASs and were to receive PALS II, and responsiveness in group that received PALS I (d) 49 mothers who had received DASs and would and PALS II (+)**; positive affect for mothers continue to receive DASs. who received PALS I regardless of the Term used: Affective-emotional behavior intervention condition during toddler- preschool stage (+)***; improvements in emotional support [warm sensitivity for mothers who received PALS I regardless of the intervention condition during toddler-preschool stage (+)**]; support of infant foci of attention [maintaining for mothers who received PALS I regardless of the intervention condition during toddler-preschool stage (+)**]; redirecting for mothers who received PALS I and PALS II (+)***; significant but smaller effect for mothers who received PALS and DASs in any moment; quality of language input table continues next page Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results [maternal use of rich language for children who received PALS I and PALS II (+)**]; verbal encouragement for children who received PALS II (+)**. Smaller effects were found with PALS I, even though they were greater than those found with DASs. Nurse-Family Beneficiaries: Nulliparous women who were Authors: Eckenrode and others (2010); Luckey and Health related: Cigarette consumption Partnership pregnant less than 29 weeks. Most were (a) low others (2008); Olds, Henderson, and Kitzman (1994); 15 years later (−)*; alcohol consumption income, (b) unmarried, and (c) teenagers. Olds and others 1986; Olds and others 1988; Olds 15 years later (−)**; lifetime sex partners Objective: Provide maternal and early childhood and others 1997; Olds and others 1998; Zielinsky, 15 years later (−)**; number of children for health programs to families most in need. Eckenrode, and Olds (2009) girls born to high-risk mothers 19 years Description: Families received sensory and Type: RCT later (−)**; condom use for girls born to developmental screening, as well as Location: United States: Elmira, New York high-risk mothers 19 years later (+)**; transportation services for prenatal and well-child Follow-up: 19 years later sexual partners for boys 19 years later (+)** care. Additionally, families received monthly Sample: More than 300 youths from the 400 families Behavior related: Instances of running away prenatal and infancy home visits by public health enrolled in the Elmira Nurse-Family Partnership for children in high-risk families 15 years nurses (during pregnancy and the first two program. Approximately 90% of the women were later (−)**; probability of arrest (−) [15 years years of their children’s life). The nurses tried to white, 60% were low income, and 60% were later (−)**; 19 years later, girls only (−)**]; (a) improve pregnancy outcomes by helping unmarried. Their average age was 19. The follow-up probability of being convicted (−) [15 years women improve their health-related behaviors, 19 years later had 352 youths (164 control, 86 receiving later (−)**; 19 years later, girls only (−)**]; (b) improve children’s health and development a mean of 9 home visits during pregnancy; 102 number of lifetime arrests for girls 19 years by helping parents provide more competent care, receiving a mean of 23 home visits during pregnancy, later (−)**; number of lifetime convictions and (c) improve families’ economic self-sufficiency birth, and through the child’s second birthday) and for girls 19 years later (−)** by helping parents make appropriate choices 315 youths for the follow-up 15 years later (144 control, Economic related: Use of Medicaid for girls regarding family planning, finishing their 77 receiving home visits during pregnancy, and 94 born to high-risk mothers 19 years later education, and finding work, and by linking receiving home visits during pregnancy and infancy). (−)**; use of food stamps and receiving Aid them to appropriate services. Term used: Socioemotional development to Families with Dependent Children (−)** table continues next page 99 100 Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results Cost: US$13,600 per woman (2014 dollars), to Other: Child abuse and neglect: during the deliver program services (that is, three years of program, 2 years after the program ended home visits by a trained nurse). This cost was (−); 15 years later (−)*** offset in two of the trials by reduced Note: (a) High-risk group defined as being government spending on mothers’ use of born to an unmarried, low-income mother; welfare and other public assistance (for example, (b) impacts in the 19-years-later evaluation US$14,500 in lower spending per woman over based on the 95% confidence interval 12 years in Memphis, Tennessee). Both the reported. Elmira, New York, and Memphis trials found such lower welfare spending, but the Denver, Colorado, trial did not. Authors: Kitzman and others (2010); Olds and others Health related: Infant and childhood (2007); Olds and others (2010) mortality from preventable causes 6 Type: RCT years later(−)*; children’s injuries or Location: United States: Memphis, Tennessee ingestions before age 2 (−)**; substance Follow-up: 9 years later use by mothers—cigarettes, alcohol, and Sample: 743 primarily low–income, African American, marijuana (−)**; mother’s subsequent live high-risk women (approximately 90% of the births (−)**; mother’s subsequent women were African American, 85% were low low-birth-weight newborns (−)* income, and almost all were unmarried; their Behavior related: Behavior problems in the average age was 18), who were randomly assigned borderline or clinical range (−)**; to receive (a) 255 free developmental screenings aggression for high-risk children (−)** and referrals for their child before age two, (b) 245 Academic/cognitive related: GPA for screenings offered in treatment 2 plus children in risk group at age 12 (+)*; paraprofessional home visiting during pregnancy results on achievement tests for children and the child’s first two years of life, (c) 235 in high-risk group at age 11 (+)**; screenings offered in treatment 1 plus nurse home enrollment in formal out-of-school care visits during pregnancy and infancy. by ages 2 to 4.5 (+)**; intellectual Term used: Socioemotional development functioning and receptive vocabulary scores at age 8 (+)**; for high-risk children, arithmetic achievement (+)** table continues next page Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results Economic related: Welfare and food stamps for high-risk mothers (−)** Note: High-risk group is defined as children born to women who have two of the following three characteristics: unmarried, fewer than 12 years of education, or unemployed. Authors: Olds and others (2014) Health related: Subsequent pregnancies for Type: RCT mothers (−)**; interval between first and Location: United States: Denver, Colorado second birth for mothers (−)**; health care Follow-up: 9 years later encounters for children’s injuries or Sample: 490 women with different backgrounds and ingestions at age 2 (−)**; says hospitalized low levels of income (almost all low income, 46% for injuries or ingestions at age 2 (−)** were Mexican American, 36% were white, 15% were Behavior related: Emotional/behavioral African American, and 84% were unmarried; problems in children under age 6 for average age was 20), who were randomly assigned children visited by nurses (−)*; internalizing to receive nurse home visits or developmental problems at age 9 for children visited by screening and referral to treatment for their child nurses (−)*; dysfunctional attention at age Term used: Socioemotional development 9 for children visited by nurses (−)**; for high-risk children visited by paraprofessionals: deregulated aggression (−)**; behavioral regulation (+)** Academic/cognitive related: For children in high-risk environment who were visited by nurses: receptive language at ages 2, 4, and 6 (+)**; sustained attention at ages 4, 6, and 9 (+)**. Those receiving visits by paraprofessionals presented differences only in visual attention/task switching at age 9 (+)*. Economic related: Months using welfare (−)** table continues next page 101 102 Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results Note: High-risk group is defined as children born to women who have two of the following three characteristics: unmarried, fewer than 12 years of education, or unemployed. Jamaican Study Beneficiaries: Stunted children ages 9 to 24 Authors: Gertler and others (2013) Behavior related: Internalizing behavior months who lived in poor disadvantaged Type: RCT factor (+)**; expelled from school (−)**; neighborhoods Location: Kingston, Jamaica maternal investment in children (+)*** Objective: Develop children’s cognitive and Follow-up: 20 years later Academic/cognitive related: Cognitive personality skills. Sample: 129 stunted children ages 9 to 24 months development in the stimulation Description: The stimulation group received who lived in poor disadvantaged neighborhoods intervention (+); years completed in school one-hour weekly visits from Jamaican assigned to four groups: (a) 32 received (+)*; any college (+)**; enrollment in school community health workers over two years. Visits psychosocial stimulation, (b) 32 received nutritional (+)*** taught parenting skills and encouraged mothers supplementation, (c) 32 received both nutritional Economic related: Employment (+)*; to interact and play with their children in ways supplementation and stimulation, and (d) 33 were earnings (+)***; other: migration that would develop their children’s cognitive controls and received neither and personality skills. The group that received Term used: Psychosocial development nutritional intervention received weekly nutritional supplements for two years. Cost: n/a Home Beneficiaries: Parents who have doubts about or Authors: Baker, Piotrkowski, and Brooks-Gunn 1999 Behavior related: For cohort I in both sites, Instruction lack confidence in their ability to instruct their Type: RCT in New York and a quasi-experimental study classroom adaptation in second grade, Program for children and prepare them for school. in Arkansas with a nonrandomized comparison (+)** in New York, *** in Arkansas Preschool Frequently, these parents did not graduate from group Academic/cognitive related: For cohort I in Youngsters high school or had only limited formal Location: United States: New York and Arkansas New York, cognitive skills (+)** and (HIPPY) education, limited English proficiency, limited Follow-up: 2 years later standardized reading (+)**; promotion to financial resources, or other risk factors. Sample: In New York, 182 families divided in two first grade among cohort I in Arkansas (+); Objective: Help parents with limited formal cohorts: (a) 69 families were part of cohort I, which for cohort II in Arkansas, school readiness education prepare their 4- and 5-year-old started in 1990 (37 received HIPPY and 32 were (+)* and standardized achievement (−)** children for school. Promote preschoolers’ controls) and (b) 113 families were part of cohort II table continues next page Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results school readiness and support parents as their (47 in HIPPY and 66 as control), which received Note: Results were not replicated when children’s first teacher by providing instruction in services in 1991. Children were randomly assigned compared with nonrandomized children the home. Although the curriculum is primarily to HIPPY or to high-quality preschool. who attended high-quality cognitive based, it also fosters socioemotional In Arkansas, children who attended HIPPY were matched prekindergarten. and physical development. over several characteristics with those who did not. Description: In a two-year home-based early Term used: Social/emotional development education intervention program, Authors: Bradley and Gilkey (2002) Behavior related: School suspension (−)*; paraprofessional trainers visited families Type: Quasi-experimental design (within classroom student’s adjustment to classroom (+); bimonthly for 45–60 minutes, providing matching of HIPPY children with demographically enjoyment of books; listening and paying teaching materials and parenting advice, and similar children who did not participate) attention; task orientation, self-direction in supplemented the visits in alternate weeks by Location: United States: Arkansas learning, initiative, and interest in school (+) group meetings with parents and Follow-up: 4 and 6 years later Academic/cognitive related: Test on reading paraprofessionals led by professional HIPPY Sample: 516 HIPPY children and 516 matched and language arts (+); classroom program coordinators. comparison children adaptation (+) Cost: According to U.S. Department of Health and Term used: Social/emotional development Note: No p-values reported in the paper. Human Services, the average cost per child per However, the aspects mentioned above are year is US$1,500 to US$2,500 (2011 dollars). reported as statistically significant. The following variables were not statistically different between groups: math scores, school attendance, special education, classroom grades, teacher ratings of classroom behavior, curiosity, and using assistance. Integrated Early Beneficiaries: Children ages 12 to 24 months Authors: Attanasio and others (2014) Academic/cognitive related: Cognitive scores Childhood located in 96 small municipalities in Colombia Location: 96 small municipalities in Colombia for stimulated children (+)****; receptive Development Objective: Reduce some of the developmental Follow-up: 18 months language for stimulated children (+)** Intervention risks associated with poverty. Sample: 1,440 children ages 12 to 24 months and their Note: None of the programs had effects on in Colombia Description: During 18 months, female community primary caregivers. These children were divided expressive language, fine motor skills, leaders delivered psychosocial stimulation into four groups: (a) 360 were assigned to a control hemoglobin levels, height, weight, and through weekly home visits with play group, (b) 360 received stimulation, (c) 360 received gross motor scores. Micronutrient demonstrations or micronutrient sprinkles that micronutrients, and (d) 360 received both nutrition supplementation had no significant effect were given daily or both combined. and stimulation. on any outcome. No interaction took place Cost: US$500 per year per child Term used: Psychosocial 103 between the programs. table continues next page 104 Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results Parents as Beneficiaries: Low-income women with children Authors: Wagner and Clayton (1999) Health related: Immunization for children Teachers (PAT) no older than 6 months. The program starts in Type: RCT receiving the case management program pregnancy or infancy and continues through Location: United States: RCT 1, Salinas Valley, intervention (−)**; child abuse or neglect in kindergarten entry. California; RCT 2, Los Angeles, San Bernardino, San Teen PAT, for the group receiving PAT and Objectives: (a) Increase parental knowledge of Diego, and Santa Barbara Counties in California case management (−)** early childhood development, (b) improve Follow-up: When children were 2 years old Behavior related: Self-help development for parenting practices, (c) detect developmental Sample: Two RCTs: RCT 1 was composed of 497 Latino Latino families (+)**; social help for Latino delays and health issues early, (d) prevent child families where parents had limited English families when using multivariate analysis abuse and neglect, and (e) increase children’s proficiency; 298 were assigned to treatment (+)* school readiness and success. In addition to (receiving an average of 20 visits of 28–50 minutes) Academic/cognitive related: Cognitive home visits, the program provides health and and 199 to control (which received age-appropriate development in Teen PAT, for the group developmental screenings, group meetings, and toys and referrals if their children were found with receiving PAT and case management (+)**; referrals to resource networks. delays); and RCT 2 was composed of 704 families when using multivariate analysis, cognitive Description: These monthly (or even more with teenage parents who received (a) 177 PAT development measured by DPII for the frequently if at-risk family) home visitation services alone, (b) 174 who received case Latino PAT, and the groups receiving case services are provided by trained parent management, (c) 175 who received PAT and case management and the combined educators (with at least some college education) management, and (d) 178 untreated. intervention in Teen PAT (+)** who provide information about children’s Term used: Social development Note: No benefit with regard to parenting development, model and involve parents in knowledge or practices. developmentally appropriate activities, and respond to parents’ questions and concerns. Additionally, the program involves parent group meetings to discuss child development and to build informal support networks with other parents, periodic developmental screenings, and referrals to community services as needed. table continues next page Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results Cost: The average monthly cost of serving a family Authors: Wagner, Spiker, and Linn (2002) Behavior related: Parent’s self-reported was determined to be US$170 (1996 dollars). Type: RCT happiness when caring for their children Given that the average length of participation Location: United States, multisite trial (+)*** was 30.1 months, the average cost per family Follow-up: When children were 2 years old Note: No significant effects found for children’s was determined to be US$5,117. Those who Sample: 665 families development, or for parents’ practices or persisted in the program through the children’s Term used: Social development knowledge. third birthdays averaged 35.4 months of participation, for an average per-family cost of US$6,018. Dropouts from the program averaged 17.8 months of participation, for an average per-family cost of US$3,026. Parent-Child Beneficiaries: Low-income families with children Authors: Levenstein and others (1998) Academic/cognitive related: Academic Home ages 2 to 4 Type: RCT achievement, and reading, language, and Program Objective: Improve parent–child interactions to Location: United States: Pittsfield, Massachusetts math scores at eighth grade (+); school strengthen children’s cognitive development Follow-up: 16–20 years later dropout at ages 17–22 (−)**; graduation at and early literacy Sample: 123 students who at age 2 (1976–80) had ages 17–22 (+)*** Description: The program consisted of 46 been recruited for the Parent-Child Home Program. twice-weekly paraprofessional or volunteer The control group is very small (15). home visitation services spread over seven Term: Socioemotional behavior months in each of two years. In this play-filled and nondidactic home-based intervention, the toy demonstrator models verbal interaction with the child centered on toys and books that are permanently assigned to the family and encourages the mother to assume responsibility for the interaction. The program has a cognitive curriculum, a socioemotional behavior curriculum, and a “parenting” curriculum. Cost: US$4,500 per child for two-year program table continues next page 105 Table A.1  Before-School Programs—Home Visiting Programs (continued) 106 Program Evaluation Name Description Author/description Results Healthy Families Beneficiaries: New or expectant parents deemed Authors: Dumont and others (2006); Dumont and Health related: For the subgroup of mothers to be at risk of abusing or neglecting their others (2008); Dumont and others (2010) enrolled by 30 weeks of gestation, children Type: RCT incidence of low-birth-weight newborns (−) Objectives: (a) promote positive parenting skills Location: United States: New York Behavior related: Child abuse/neglect at the and parent–child interaction; (b) prevent child Follow-up: 2 years later end of year 2 (+)**; minor physical abuse and neglect; (c) support optimal prenatal Sample: 1,173 families at risk for child abuse and aggression during the past year for care, and child health and development; and (d) neglect randomly assigned to Healthy Families New mothers who engaged in the program at improve parents’ self-sufficiency York or to a control group that received information 30 weeks of pregnancy or earlier (−)**; Description: Paraprofessionals make home visits and referrals to other services harsh parenting by mothers who engaged starting in pregnancy and continuing until the Term used: Psychosocial risks in the program at 30 weeks of pregnancy or child reaches age 5. earlier (−)**; serious abuse or neglect of Cost: n/a children by age 2 by psychologically vulnerable mothers (−)*** Authors: Caldera and others (2007); Duggan and Behavior related: No overall effect on others (2007) maltreatment or the use of more severe Type: RCT forms of physical discipline. The program Location: United States: Alaska had no effects on parental risk. Follow-up: 2 years after birth Sample: 325 families with women Term used: Psychosocial risks Parenting Beneficiaries: Children at risk for mental health Authors: Letourneau and others (2001) Behavior related: Improved Practices problems attributable to poverty and/or their Type: RCT psychophysiological arousal and orienting Canada parents’ inexperience, lack of educational Location: Canada at age 11. This is seen through skin attainment, and young age Follow-up: 11–13 weeks later conductance amplitudes (+)***; skin Objectives: Improve the parent–child relationship Sample: 52 families divided into two groups: (a) 18 conductance rise times (+)***; skin and indirectly enhance the resilience capacity families received support with extensive and conductance recovery times (+)**; among at-risk children. intense parent training, and (b) 34 families received slow-wave electroencephalogram measure Description: Two pilot studies were conducted to support without extensive and intense parent during rest (−)***; electroencephalogram compare the effects of the following two types training. measure during continuous performance of social support for families on parent–child Term used: Social skills task conditions (−)*** relationships: (a) support with extensive and intense parent training targeted toward adolescent parents in the newborn period table continues next page Table A.1  Before-School Programs—Home Visiting Programs (continued) Program Evaluation Name Description Author/description Results (mothers received six weekly home visits delivered by a master’s-prepared pediatric nurse); and (b) support without extensive and intense parent training targeted toward other at-risk parents (for example, low-income, low educational attainment, and single-parent status) of 3- to 4-year-old children identified as having developmental delays. Cost: n/a Durham Beneficiaries: All resident births in Durham, North Authors: Dodge and others 2014 Health related: Emergency care episodes Connects Carolina, between July 1, 2009, and December Type: RCT (−)***; overnights in hospital (−)***; mother 31, 2010 Location: United States: Durham, North Carolina anxiety disorder (−)** Objective: Improve infant health and well-being. Follow-up: 6 months Behavior related: Positive parenting Description: A four- to seven-session program Sample: All 4,777 resident births in Durham between behaviors (+)* assesses family needs (parenting and child care, July 1, 2009, and December 31, 2010, were Academic/cognitive related: Quality family violence and safety, parental mental randomly assigned to intervention and control. out-of-home child care (+)*** health and well-being, and health care) and A random representative subset of 549 families Other: Safety of home environment (+)**; connects parents with community resources. received blind interviews for impact evaluation. community connections (+)***; mother’s Cost: US$700 per birth Term used: Mental health and well-being knowledge of infant development (+)* Note: DPII = Developmental Profile II; GPA = grade point average; RCT = randomized control trial; n/a = not available. Significance levels of results, positive (+) or negative (–), are reported at 1 percent (*), 5 percent (**), and 10 percent (***). 107 108 Table A.2  Before-School Programs—Child Care Centers Program Evaluation Name Description Author/description Results Early Childhood Beneficiaries: Children ages 0–6 living in vulnerable Authors: Hasan and others (2013) Academic/cognitive related: Enrollment Education villages in Indonesia Type: RCT (+)***; receptive language skills (+)**; and Objective: Improve poor children’s overall Location: Indonesia communication and general knowledge for Development development and readiness for further education Follow-up: 20 months later children in poorest families (+)*; social program in within a sustainable quality early childhood Sample: More than 6,000 children in two age competence for children in poorest Indonesia education and development system. cohorts (10 to 46 months), living in 310 poor communities (+)**; language and cognitive Description: The project had three main components: villages. development for children in poorest families 1. Provision on integrated early childhood education Term used: Social and emotional development (+)*; executive function in children of poor and development services. These services were families (+)* delivered through community-driven mechanisms in Note: Most results were in the correct direction targeted poor communities. The program helped but not statistically significantly. When communities identify early childhood education and looking at the younger cohort, the results are development service needs, prepare a proposal, and mixed. No effects on parenting practices. then implement this proposal. 2. Ideation of a sustainable system to ensure quality in early childhood education and development services. This includes the generation and implementation of standards and quality-assurance systems and the institutionalization of early childhood education and development at the district and provincial levels. 3. Program management, monitoring, and evaluation. Cost: n/a Abecedarian Beneficiaries: Economically disadvantaged children up Authors: Barnett and Masse (2007); Campbell Health related: Mortality rates (−)**; likelihood program to age 5 and others (2002); Heckman and Kautz (2013) of halting work because of illness (−)**; Objective: Improve children’s school readiness by Type: RCT smoking (−)** social-cognitive and linguistic development. Location: United States: North Carolina Behavior related: Arrests (−)***; externalizing Follow-up: 20 years behavior (+)**; engaging in risky behavior (−)*** table continues next page Table A.2  Before-School Programs—Child Care Centers (continued) Program Evaluation Name Description Author/description Results Description: Program provides full day (10 hours/day) Sample: 112 children, mostly African American, Academic/cognitive related: IQ until age 1 child care, nutrition, and transportation year-round, born between 1972 and 1977, who were (+)** then faded; grade repetition (−)**; from infancy to kindergarten (approximately 250 believed to be at risk of retarded intellectual assigned to special education (−)**; high days per year). Teacher–child ratios ranged from 1:3 and social development (51 in control group school graduation by age 19 (+)**; college for infants and toddlers to 1:6 for older children. and 53 in treatment); 104 children in follow-up attendance (+) **; educational attainment at Cost: US$67,000 for the five years of attendance for 20 years age 40 (+)*** ($11,000 in year 1; US$16,000 in each of years 2 and Terms used: Social development; social Economic related: Earnings (+)***; 3; US$12,000 in each of years 4 and 5) adjustment; noncognitive skills homeownership (+)** Note: Annual social rates of return: 7%–16% (US$9–US$13 for every US$1 invested). Montessori Beneficiaries: 3- to 5-year-old children Authors: Lillard and Else-Quest (2006) Behavior related: Social problem solving Objective: Foster a child’s natural inclination to learn. Type: RCT through fairness (+)***; positive peer play Description: Montessori schooling is a 100-year-old Location: United States: Milwaukee, Wisconsin (+)**; rough play ambiguous in intent, like system that naturally incorporates practices that Follow-up: 2 years wrestling without smiling (−)**; false belief align with mindfulness and are suited to very young Sample: 25 control children attending public (+)** children. Montessori education is characterized by inner-city schools and suburban public, Academic/cognitive related: Letter/word multiage classrooms, a special set of educational private, or charter schools; and 30 Montessori identification (+)**; phonological decoding materials, long periods spent in student-chosen 5-year-old children ability (+)**; applied problems/math skills activities, collaboration, the absence of grades and Term used: Social skills (+)**; executive function (+)** tests, and individual and small-group instruction in Note: No difference found on basic vocabulary, both academic and social skills. a test of children’s ability to delay Cost: Because all Montessori schools are operated gratification. independently, tuitions vary widely. According to a 2009–10 NAMTA survey of North American Montessori schools, tuitions range from a low of less than US$999 per year to a high of more than US$14,000 per year. table continues next page 109 Table A.2  Before-School Programs—Child Care Centers (continued) 110 Program Evaluation Name Description Author/description Results Early Child Beneficiaries: Children born in three regions (13 Authors: Armecin and others (2006) Health related: Weight for height (+)**; Development provinces) of the Philippines Type: Intent to treat difference in propensity proportion wasted (−)**; anemia (+)**; Program Objective: Improve survival and developmental score matching hemoglobin (+)** potential of children, particularly the most Location: The Philippines Behavior related: Social-emotional skills (+)**; vulnerable and disadvantaged. Follow-up: 3 years self-help (+)**; social development (+)** Description: Children participated in interdepartmental Sample: 6,693 children ages 0 to 4 years Academic/cognitive related: Cognitive programs on nutrition, health, early education, and Term used: Social-emotional skills development (+)**; expressive language social services programs with the following (+)**; gross motor skills (+)**; fine motor skills components: (a) minimizing the health risks to very (+)**; receptive language (+)** young children, (b) contributing to the knowledge of parents and the community about child development and encouraging their active involvement, (c) advocating for child-friendly policies and legislation, (d) improving the ability and attitude of child-related service providers, and (e) mobilizing resources and establishing viable financing mechanisms for ECD projects. Cost: n/a Proyecto Beneficiaries: Poor children ages 6 to 72 months in Authors: Behrman, Cheng, and Todd (2004) Health related: Height for children up to 36 Integral de urban areas Type: Propensity score matching months (+); weight for children younger than Desarrollo Objectives: Improve child nutrition and provide Location: Bolivia 36 months (−) Infantil (PIDI) environments that are conducive to learning. Follow-up: n/a. They compare children in the Behavior related: Psychosocial skills for those Description: Day care, nutrition, and educational program for short (<2 months) and longer ages 37 to 54 months (+)* services are provided to children who live in poor, durations Academic/cognitive related: Bulk motor skills, predominantly urban areas. Under the program, Sample: 1,227 children for round 1; 2,420 fine motor skills, language skills for those children are cared for in groups of 15 in homes in children for round 2; and 364 children for both ages 37 to 54 months (+)* their own neighborhood. The community selects rounds as controls. Economic related: Earnings (+); earning local women to become home day care mothers. Term used: Psychosocial development benefits/cost: 1.7–3.7 These nonformal, home-based day care centers, with two to three caregivers, provide integrated child development services (play, nutrition, growth screening, and health referrals). The women receive child development training before becoming educators but are usually not highly trained. Cost: US$516 yearly per beneficiary table continues next page Table A.2  Before-School Programs—Child Care Centers (continued) Program Evaluation Name Description Author/description Results Turkish Early Beneficiaries: 4- to 6-year-old children from deprived Authors: Kagitcibasi, Sunar, and Bekman (2001); Behavior related: For children with trained Enrichment backgrounds. These children were at risk for low Kagitcibasi and others (2009) mothers, immediate postprogram positive Project (TEEP) educational achievement and failure to develop to Location: Istanbul, Turkey self-concept (+)*; aggression (−)*; school their full potential because of poverty and their Follow-up: 22 years later. Intervention carried adjustment for children with mother training families’ low educational level. out in 1983–85. (+)**; emotional problems for children in the Objective: Improve overall child development. Sample: 255 participants divided into the home environment immediately Description: Under TEEP, children were sorted into one following groups: (a) 27 received mother postintervention (+)***; child self-confidence of three environments: (a) an educational day care training and educational day care, (b) 40 six years later for mother-trained children center (educational group), (b) a custodial day care received mother training and custodial day (+)**; child self-confidence six years later for center (custodial group), or (c) a home (home group). care, (c) 23 received mother training in a home children in the custodial group (−)** Randomization of children within each environment care environment, (d) 37 did not receive Academic/cognitive related: IQ scores for happened only if there was excess demand for that mother training while in educational day care, children with mother training particular service. (e) 65 did not receive mother training while postintervention (+)***; IQ scores for children Mothers of approximately half of the children in each attending a custodial day care center, and who were in educational or home care environment were randomly assigned to receive (f) 63 did not receive mother training while in environment postintervention (+)*; maternal training emphasizing educational activities a home care environment. The 22-year achievement test scores postintervention with the child plus support for the mother through follow-up gathered information on 131 of the (+)**; school grades for the children in the group meetings and guided discussions. 255 participants (39% attrition). educational environment at age 3 (+)**; Cost: n/a Term used: Personality and social development school grades for the children with mother training at age 5 (+)* [no effect of mother training in grades after primary school]; school enrollment seven years after the program for those with mother training (+)***; failed years in middle school for the custodial group (+)**; college attendance for those with mother training (+)* table continues next page 111 Table A.2  Before-School Programs—Child Care Centers (continued) 112 Program Evaluation Name Description Author/description Results Economic related: Ownership of a credit card for those with mother training (+)**; occupational status of those who attended an educational day care center (+)**; ownership of a computer for those who attended an educational day care center (+)* Other: Indicators of integration into modern urban life, such as owning a computer. For children with trained mothers, immediately postprogram: mother’s attentiveness to and direct interaction with the child (+); involvement with children in cognitive-oriented activities (+); educational aspirations and expectations for the child (+); positive disciplinary strategies and praise (+) Early Childhood Beneficiaries: Poor children under age 3 and their Author: Aboud (2011) Health related: Weight for height (+)**; Parenting mothers Type: Intervention-control posttest design preventive health (+)*** Programme in Objective: Promote physical and mental development Location: Poor rural communities in Bangladesh Behavior related: Knowledge of mother (+)***; Rural of children Follow-up: 2 months amount and quality of stimulation and Bangladesh Description: In this program, 90-minute weekly Sample: 170 mothers attended a year of support provided to a child in the family education sessions are offered by trained women, educational sessions; 159 mothers and setting—home inventory (+)***; stimulation known as facilitators, to groups of about 20 mothers. children from neighboring villages that did (+)*** The topics include common diseases and oral not receive the program served as controls Note: The parenting mothers did not rehydration solutions, hygiene, sanitation, Term used: Psychosocial stimulation communicate differently with their children breastfeeding, weaning foods, micronutrient while doing a picture-talking task, and deficiencies, stages of cognitive and language children did not show benefits in nutritional development, how parents can help children learn, status or language comprehension. how to encourage language development, positive discipline, gender equality, and child rights. The facilitators had some secondary education; to deliver the program, they received 17 additional days of basic training with a manual of 40 topics, four days a month of supervision, and monthly refresher courses. Cost: n/a table continues next page Table A.2  Before-School Programs—Child Care Centers (continued) Program Evaluation Name Description Author/description Results ECD Beneficiaries: Families with children ages 36 to Authors: Martinez, Nadeau, and Pereira (2012) Behavior related: Socioemotional indicator intervention 59 months Type: RCT (ASQ) (+)**; caregivers’ belief that physical in Objectives: Improve children’s cognitive, social, Location: Gaza Province in Mozambique punishment is appropriate (−); daily routines Mozambique emotional, and physical development. Specifically, Follow-up: 2 years between parents and children (+)***; the project aims to (a) deliver quality early Sample: 76 communities self-sufficiency activities between parents stimulation, psychosocial support, and emergent Term used: Socioemotional and behavioral and children (+)** literacy and numeracy instruction; (b) strengthen outcomes Academic/cognitive related: Primary school positive parenting practices and decrease harmful enrollment rates (+)***; enrollment at ones; and (c) facilitate children’s transition to primary appropriate age in primary school (+)**; school. hours spent on schooling and homework- Description: In a center-based community-driven related activities (+)***; cognitive and preschool model, young children “learn by doing” problem-solving abilities (+)**; fine motor under the care of supportive adults. Thirty-five skills (+)*; school attendance of older siblings children per classroom receive three hours of care. of beneficiary children (+)*** Parents participate in monthly meetings. Teachers Economic related: Caregivers working in the are selected by the community and then receive a 30 days before the interview (+)* five-day component, refresher training sessions, and Other: Caregivers report a significant increase hands-on mentoring and supervision. in satisfaction with their children’s Cost: US$29.74 per child, average estimated cost for a preparation for future school. 12-month program Note: Some of the principal measures of communication and language development are not significantly different between groups and remain alarmingly low. No differences are noted in rates of stunting and wasting, which was expected since a child’s growth potential is largely determined by age 3. The study finds mixed impacts in children’s health (less diarrhea and fewer skin problems, more illnesses). table continues next page 113 114 Table A.2  Before-School Programs—Child Care Centers (continued) Program Evaluation Name Description Author/description Results Promoting Beneficiaries: Preschool-age children in Head Start Authors: Domitrovich, Cortes, and Behavior related: Anger attribution bias (−)***; Alternative Objectives: Improve children’s social competence and Greenberg (2007) emotion knowledge skills (+)*; social Thinking reduce problem behavior Type: RCT competence (+)***; internalizing behavior Strategies Description: The PATHS program uses a universal, Location: United States: Pennsylvania (+)**; anxiety (−)***; socially withdrawn (−)**; (PATHS) teacher-taught social-emotional curriculum. Thirty Follow-up: 9 months parental perceptions of children as socially curriculum lessons that were delivered once a week included Sample: 20 classrooms with 246 children and emotionally competent (+)*** the topics of compliments, basic and advanced Term used: Socioemotional Note: No significant differences in inhibitory feelings, a self-control strategy, and problem solving. control, attention, or problem solving. In addition to the lessons, teachers generalized the concepts of the curriculum through detailed extension activities (such as group games, art projects, and books) that were integrated into the existing typical preschool programs (such as language arts, music, and art). Cost: n/a Note: ASQ = Ages and Stages Questionnaire; ECD = early childhood development; IQ = intelligence quotient; NAMTA = North American Montessori Teachers’ Association; PEF = Peace Education Foundation; PPVT = Peabody Picture Vocabulary Test; RCT = randomized control trial; n/a = not available. Significance levels of results, positive (+) or negative (–), are reported at 1 percent (*), 5 percent (**), and 10 percent (***). Before-School Program Descriptions 115 References Aboud, Frances E. 2011. “Evaluation of an Early Childhood Parenting Programme in Rural Bangladesh.” Journal of Health, Population and Nutrition 25 (1): 3–13. Armecin, Graeme, Jere R. Behrman, Paulita Duazo, Sharon Ghuman, Socorro Gultiano, Elizabeth M. King, and Nanette Lee. 2006. “Early Childhood Development through an Integrated Program: Evidence from the Philippines.” Policy Research Working Paper 3922, World Bank, Washington, DC. Attanasio, Orazio P., Camila Fernández, Emla O. A. Fitzsimons, Sally M. Grantham- McGregor, Costas Meghir, and Marta Rubio-Codina. 2014. “Using the Infrastructure of a Conditional Cash Transfer Program to Deliver a Scalable Integrated Early Child Development Program in Colombia: Cluster Randomized Controlled Trial.” BMJ 349: g5785. Baker, Amy J. L., Chaya S. Piotrkowski, and Jeanne Brooks-Gunn. 1999. “The Home Instruction Program for Preschool Youngsters (HIPPY).” Future of Children 9 (1): 116–33. Barnett, W. Steven, Kwanghee Jung, Donald J. Yarosz, Jessica Thomas, Amy Hornbeck, Robert Stechuk, and Susan Burns. 2008. “Educational Effects of the Tools of the Mind Curriculum: A Randomized Trial.” Early Childhood Research Quarterly 23 (3): 299–313. Barnett, W. Steven, and Leonard N. Masse. 2007. “Comparative Benefit–Cost Analysis of the Abecedarian Program and Its Policy Implications.” Economics of Education Review 26 (1): 113–25. Behrman, Jere R., Yingmei Cheng, and Petra E. Todd. 2004. “Evaluating Preschool Programs When Length of Exposure to the Program Varies: A Nonparametric Approach.” Review of Economics and Statistics 86 (1): 108–32. Bick, J., and M. Dozier. 2013. “The Effectiveness of an Attachment-Based Intervention in Promoting Foster Mothers’ Sensitivity toward Foster Infants.” Infant Mental Health Journal 34 (2): 95–103. Bouguen, Adrien, Deon Filmer, Karen Macours, and Sophie Naudeau. 2014. “Preschools and Early Childhood Development in a Second Best World: Evidence from a Scaled-Up Experiment in Cambodia.” CEPR Discussion Paper 10170, Center for Economic Policy Research, Washington, DC. Bradley, Robert H., and Barbara Gilkey. 2002. “The Impact of the Home Instructional Program for Preschool Youngsters (HIPPY) on School Performance in 3rd and 6th Grades.” Early Education and Development 13 (3): 301–12. Caldera, Debra, Lori Burrell, Kira Rodriguez, Sarah Shea Crowne, Charles Rohde, and Anne Duggan. 2007. “Impact of a Statewide Home Visiting Program on Parenting and on Child Health and Development.” Child Abuse and Neglect 31 (8): 829–52. Campbell, Frances A., Craig T. Ramey, Elizabeth Pungello, Joseph Sparling, and Shari Miller–Johnson. 2002. “Early Childhood Education: Young Adult Outcomes from the Abecedarian Project.” Applied Developmental Science 6 (1): 42–57. Chetty, Raj, John N. Friedman, Nathaniel Hilger, Emmanuel Saez, Diane Whitmore Schanzenbach, and Danny Yagan. 2010. “How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project STAR.” NBER Working Paper 16381, National Bureau of Economic Research, Cambridge, MA. Diamond, Adele, W. Steven Barnett, Jessica Thomas, and Sarah Munro. 2007. “Preschool Program Improves Cognitive Control.” Science 318 (5855): 1387–88. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 116 Before-School Program Descriptions Dodge, Kenneth A., W. Benjamin Goodman, Robert A. Murphy, Karen O’Donnell, Jeannine Sato, and Susan Guptill. 2014. “Implementation and Randomized Controlled Trial Evaluation of Universal Postnatal Nurse Home Visiting.” American Journal of Public Health 104 (Suppl. 1): S136–S143. Domitrovich, Celene E., Rebecca C. Cortes, and Mark T. Greenberg. 2007. “Improving Young Children’s Social and Emotional Competence: A Randomized Trial of the Preschool PATHS Curriculum.” Journal of Primary Prevention 28 (2): 67–91. Dozier, M., O. Lindhiem, E. Lewis, J. Bick, K. Bernard, and E. Peloso. 2009. “Effects of a Foster Parent Training Program on Young Children’s Attachment Behaviors: Preliminary Evidence from a Randomized Clinical Trial.” Child and Adolescent Social Work Journal 26 (4): 321–32. Duggan, Anne, Debra Caldera, Kira Rodriguez, Lori Burrell, Charles Rohde, and Sarah Shea Crowne. 2007. “Impact of a Statewide Home Visiting Program to Prevent Child Abuse.” Child Abuse and Neglect 31 (8): 801–27. DuMont, Kimberly, Kristen Kirkland, Susan Mitchell-Herzfeld, Susan Ehrhard-Dietzel, Monica Rodriguez, Eunju Lee, China Layne and Rose Greene. 2010. “A Randomized Trial of Healthy Families New York (HFNY): Does Home Visiting Prevent Child Maltreatment.” National Institute of Justice, Washington, DC. https://www.ncjrs.gov​ /­pdffiles1/nij/grants/232945.pdf. DuMont, Kimberly A., Susan Mitchell-Herzfeld, Rose Greene, Eunju Lee, Ann Lowenfels, and Monica Rodriguez. 2006. “Healthy Families New York (HFNY) Randomized Trial: Impacts on Parenting after the First Two Years.” OCFS Working Paper No. 1, New York State Office of Children and Family Services, Albany, NY. DuMont, Kimberly, Susan Mitchell-Herzfeld, Rose Greene, Eunju Lee, Ann Lowenfels, Monica Rodriguez, and Vajeera Dorabawila. 2008. “Healthy Families New York (HFNY) Randomized Trial: Effects on Early Child Abuse and Neglect.” Child Abuse and Neglect 32 (3): 295–315. Eckenrode, John, Mary Campa, Dennis W. Luckey, Charles R. Henderson, Robert Cole, Harriet J. Kitzman, Elizabeth Anson, Kimberly Sidora-Arcoleo, and David L. Olds. 2010. “Long-Term Effects of Prenatal and Infancy Nurse Home Visitation on the Life Course of Youths: 19-Year Follow-Up of a Randomized Trial.” Journal of the American Medical Association 164 (1): 9–15. Epstein, Ann S. 1993. Training for Quality: Improving Early Childhood Programs through Systematic In-Service Training. Ypsilanti, MI: HighScope Press. Gertler, Paul, James Heckman, Rodrigo Pinto, Arianna Zanolini, Christel Vermeerch, Susan Walker, Susan Chang-Lopez, and Sally Grantham-McGregor. 2013. “Labor Market Returns to Early Childhood Stimulation: A 20-Year Follow-Up to an Experimental Intervention in Jamaica.” Policy Research Working Paper 6529, World Bank, Washington, DC. Hasan, Amer, Marilou Hyson, and Mae Chu Chang. 2013. Early Childhood Education and Development in Poor Villages of Indonesia: Strong Foundations, Later Success. Directions in Development. Washington, DC: World Bank. Heckman, James J., and Tim Kautz. 2013. “Fostering and Measuring Skills: Programs That Improve Character and Cognition.” NBER Working Paper 19656, National Bureau of Economic Research, Cambridge, MA. Heckman, James J., Seong Hyeok Moon, Rodrigo Pinto, Peter A. Savelyev, and Adam Q. Yavitz. 2010. “The Rate of Return to the HighScope Perry Preschool Program.” Journal of Public Economics 94 (1–2): 114–28. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Before-School Program Descriptions 117 Kagitcibasi, Cigdem, Diane Sunar, and Sevda Bekman. 2001. “Long-Term Effects of Early Intervention: Turkish Low-Income Mothers and Children.” Journal of Applied Developmental Psychology 22 (4): 333–61. Kagitcibasi, Cigdem, Diane Sunar, Sevda Bekman, Nazli Baydar, and Zeynep Cemalcilar. 2009. “Continuing Effects of Early Enrichment in Adult Life: The Turkish Early Enrichment Project 22 Years Later.” Journal of Applied Developmental Psychology 30 (6): 764–79. Kitzman, Harriet J., David L. Olds, Robert E. Cole, Carole A. Hanks, Elizabeth A. Anson, Kimberly J. Arcoleo, Dennis W. Luckey, Michael D. Knudtson, Charles R. Henderson Jr., and John R. Holmberg. 2010. “Enduring Effects of Prenatal and Infancy Home Visiting by Nurses on Children: Follow-Up of a Randomized Trial among Children at Age 12 Years.” Archives of Pediatrics and Adolescent Medicine 164 (5): 412–18. Krueger, Alan B., and Diane M. Whitmore. 2001. “The Effect of Attending a Small Class in the Early Grades on College-Test Taking and Middle School Test Results: Evidence from Project STAR.” Economic Journal 111 (468): 1–28. Landry, Susan H., Karen E. Smith, and Paul R. Swank. 2006. “Responsive Parenting: Establishing Early Foundations for Social, Communication, and Independent Problem- Solving Skills.” Developmental Psychology 42 (4): 627–42. Landry, Susan H., Karen E. Smith, Paul R. Swank, and Cathy Guttentag. 2008. “A Responsive Parenting Intervention: The Optimal Timing across Early Childhood for Impacting Maternal Behaviors and Child Outcomes.” Developmental Psychology 44 (5): 1335–53. Letourneau, Nicole, Jane Drummond, Darcy Fleming, Gerard Kysela, Linda McDonald, and Miriam Stewart. 2001. “Supporting Parents: Can Intervention Improve Parent- Child Relationships?” Journal of Family Nursing, 7 (2): 159–87. Levenstein, Phyllis, Susan Levenstein, James A. Shiminski, and Judith E. Stolzberg. 1998. “Long-Term Impact of a Verbal Interaction Program for At-Risk Toddlers: An Exploratory Study of High School Outcomes in a Replication of the Mother– Child Home Program.” Journal of Applied Developmental Psychology 19 (2): 267–85. Lieberman, A. F., C. G. Ippen, and P. Van Horn. 2006. “Child-Parent Psychotherapy: 6-month Follow-up of a Randomized Controlled Trial.” Journal of the American Academy of Child & Adolescent Psychiatry 45 (8): 913–18. Lillard, Angeline, and Nicole Else-Quest. 2006. “The Early Years: Evaluating Montessori.” Science 313 (5795): 1893–94. Luckey, Dennis W., David L. Olds, Weiming Zhang, Charles R. Henderson Jr., John Eckenrode, Harriet J. Kitzman, Robert Cole, and Lisa Pettitt. 2008. “Revised Analysis of 15-Year Outcomes in the Elmira Trial of the Nurse-Family Partnership.” Prevention Research Center for Family and Child Health, University of Colorado Department of Pediatrics, Aurora. Martinez, Sebastian, Sophie Nadeau, and Vitor Pereira. 2012. “The Promise of Preschool in Africa: A Randomized Impact Evaluation of Early Childhood Development in Rural Mozambique.” enGender Impact: the World Bank’s Gender Impact Evaluation Database, World Bank, Washington, DC. Muennig, Peter, Lawrence Schweinhart, Jeanne Montie, and Matthew Neidell. 2009. “Effects of a Prekindergarten Educational Intervention on Adult Health: 37-Year Follow-Up Results of a Randomized Controlled Trial.” American Journal of Public Health 99 (8): 1431–37. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 118 Before-School Program Descriptions Olds, David L., John Eckenrode, Charles R. Henderson Jr., Harriet J. Kitzman, Jane Powers, Robert Cole, Kimberly Sidora, Pamela Morris, Lisa M. Pettitt, and Dennis W. Luckey. 1997. “Long-Term Effects of Home Visitation on Maternal Life Course and Child Abuse and Neglect: Fifteen-Year Follow-Up of a Randomized Trial.” Journal of the American Medical Association 278 (8): 637–43. Olds, David L., Charles R. Henderson Jr., Robert Chamberlin, and Robert Tatelbaum. 1986. “Preventing Child Abuse and Neglect: A Randomized Trial of Nurse Home Visitation.” Pediatrics 78 (1): 65–78. Olds, David L., Charles R. Henderson Jr., Robert Cole, John Eckenrode, Harriet J. Kitzman, Dennis W. Luckey, and Jane Powers. 1998. “Long-Term Effects of Nurse Home Visitation on Children’s Criminal and Antisocial Behavior: 15-Year Follow-Up of a Randomized Controlled Trial.” Journal of the American Medical Association 280 (14): 1238–44. Olds, David L., Charles R. Henderson Jr., and Harriet J. Kitzman. 1994. “Does Prenatal and Infancy Nurse Home Visitation Have Enduring Effects on Qualities of Parental Caregiving and Child Health at 25 to 50 Months of Life?” Pediatrics 93 (1): 89–98. Olds, David L., Charles R. Henderson Jr., Robert Tatelbaum, and Robert Chamberlin. 1988. “Improving the Life-Course Development of Socially Disadvantaged Mothers: A Randomized Trial of Nurse Home Visitation.” American Journal of Public Health 78 (11): 1436–45. Olds, David L., John R. Holmberg, Nancy Donelan-McCall, Dennis W. Luckey, Michael D. Knudtson, and JoAnn Robinson. 2014. “Effects of Home Visits by Paraprofessionals and by Nurses on Children: Follow-Up of a Randomized Trial at Ages 6 and 9 Years.” JAMA Pediatrics 168 (2): 114–21. Olds, David L., Harriet J. Kitzman, Robert Cole, Carole Hanks, Kimberly Arcoleo, Elizabeth Anson, Dennis W. Luckey, Michael D. Knudtson, Charles R. Henderson Jr., Jessica Bondy, and Amanda J. Stevenson. 2010. “Enduring Effects of Prenatal and Infancy Home Visiting by Nurses on Maternal Life Course and Government Spending: Follow-Up of a Randomized Trial among Children at Age 12 Years.” Journal of the American Medical Association 164 (5): 419–24. Olds, David L., Harriet J. Kitzman, Carole Hanks, Robert Cole, Elizabeth Anson, Kimberly Sidora-Arcoleo, and Jessica Bondy. 2007. “Effects of Nurse Home Visiting on Maternal and Child Functioning: Age-9 Follow-Up of a Randomized Trial.” Pediatrics 120 (4): e832–e845. Pickens, Jeffrey. 2009. “Socio-Emotional Programme Promotes Positive Behavior in Preschoolers.” Child Care in Practice 15 (4): 261–78. Pierre, R. G. S., J. I. Layzer, B. D. Goodson, and L. S. Berstein. 1997. “National Impact Evaluation of the Comprehensive Child Development Program.” Abt Associates, Cambridge, MA. Pierre, R. G. S, and J. I Layzer, 1989. “Using Home Visits for Multiple Purposes: The Comprehensive Child Development Program.” Future of Children 9 (1): 134–51. Raine, Adrian, Kjetil Mellingen, Jianghong Liu, Peter Venables, and Sarnoff A. Mednick. 2003. “Effects of Environmental Enrichment at Ages 3–5 Years on Schizotypal Personality and Antisocial Behavior at Ages 17 and 23 Years.” American Journal of Psychiatry 160 (9): 1627–35. Raine, Adrian, Peter Venables, Cyril Dalais, Kjetil Mellingen, Chandra Reynolds, and Sarnoff A. Mednick. 2001. “Early Educational and Health Enrichment at Age 3–5 Years Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Before-School Program Descriptions 119 Is Associated with Increased Autonomic and Central Nervous System Arousal and Orienting at Age 11 Years: Evidence from the Mauritius Child Health Project.” Psychophysiology 38 (2): 254–66. Reynolds, Arthur J., Judy A. Temple, Dylan L. Robertson, and Emily A. Mann. 2001. “Long-Term Effects of an Early Childhood Intervention of Educational Achievement and Juvenile Arrest: A 15-Year Follow-Up of Low-Income Children in Public Schools.” Journal of the American Medical Association 285 (18): 2339–46. Schweinhart, Lawrence J., Jeanne Montie, Zongping Xiang, W. Steven Barnett, Clive R. Belfield, and Milagros Nores. 2005. “The High/Scope Perry Preschool Study Through Age 40: Summary, Conclusions, and Frequently Asked Questions.” HighScope Press, Ypsilanti, MI. Venables, Peter. 1978. “Psychophysiology and Psychometrics.” Psychophysiology 15 (4): 302–15. Wagner, Mary M., and Serena L. Clayton. 1999. “The Parents as Teachers Program: Results from Two Demonstrations.” Future of Children 9 (1): 91–115. Wagner, Mary M., Donna Spiker, and Margaret Inman Linn. 2002. “The Effectiveness of the Parents as Teachers Program with Low-Income Parents and Children.” Topics in Early Childhood Special Education 22 (2): 67–81. Westat. 2010. “Head Start Impact Study: Final Report.” U.S. Department of Health and Human Services, Washington, DC. WHO (World Health Organization). 1968. “Neurophysiological and Behavioral Research in Psychiatry.” Technical Report No. 381, WHO, Geneva. ———. 1975. “Working Group on Primary Prevention of Schizophrenia in High Risk Groups: Summary Report.” WHO, Geneva. Zielinski, David S., John Eckenrode, and David L. Olds. 2009. “Nurse Home Visitation and the Prevention of Child Maltreatment: Impact on the Timing of Official Reports.” Development and Psychopathology 21 (2): 441–53. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Appendix B School-Based Program Descriptions Taking Stock of Programs to Develop Socioemotional Skills   121   http://dx.doi.org/10.1596/978-1-4648-0872-2 122 Table B.1 School-Based Program Descriptions Program Evaluation Name Description Author/description Results 4Rs Program Beneficiaries: Students from kindergarten to grade 5 Authors: Brown and others (2010); Jones, Behavior related: Hostile attribution bias 1.5 years Objective: Develop social and emotional skills Brown, and Aber (2011); Jones and others later (−)**; depressive symptoms 1.5 years later related to understanding and managing feelings, (2010) (−)**; aggressive interpersonal negotiation 1.5 years listening and developing empathy, being Type: RCT later (−)***; for those with the highest level of assertive, solving conflict creatively and Location: United States: New York, New York aggression, 1 year after the program aggressive nonviolently, honoring diversity, and standing Follow-up: 2 years fantasies (−)*; attention/hyperactivity symptoms up to teasing and bullying. Sample: 82 third-grade teachers and 82 (−)*** [especially for those with higher levels of Description: The 4Rs Program is a universal classrooms in 18 New York City public initial aggression]; hyperactivity (−)*; social school-based preventive intervention in literacy elementary schools: (a) 45 classrooms competence (+)** [especially for those with higher development, conflict resolution, and intergroup (690 children) received the intervention, levels of initial aggression]; prosocial behavior (+)**; understanding that trains and supports teachers and (b) 37 classrooms (554 children) aggression (−)** [especially for those with higher from kindergarten through grade 5 on how to served as controls levels of initial aggression] integrate the teaching of social and emotional Term used: Socioemotional Academic/cognitive related: Teacher’s report of skills into the language arts curriculum and to academic skills for those with the highest level of achieve positive rules and norms, as well as safe initial aggression 1 and 1.5 years after (+)*; attendance and secure environments. The program has three for those with the highest level of initial aggression primary components: (a) a comprehensive (+)*; standardized reading achievement for those seven-unit, 21–35 lesson, literacy-based with the highest level of initial aggression (+)*; math curriculum in conflict resolution and social- 1.5 years later for those with the highest level of initial emotional learning (provided to teachers in a aggression (+)**; classroom quality (+)**; teacher’s standardized, grade-specific teaching guide); (b) emotional ability (+)**; classroom’s emotional support 25 hours of training followed by ongoing (+)*; classroom’s instructional support (+)** coaching of teachers to support them in teaching Note: No effect on children’s literacy, attention the 4Rs curriculum with a minimum of 12 contacts problems, social competence, academic in one school year; and (c) a parent component achievement. (4Rs Family Connections) that consists of activities for children to do with their parents at home. Cost: US$90 per child per year (2011 dollars) table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Al’s Pals: Kids Beneficiaries: Prekindergarten to second-grade Authors: Lynch, Geller, and Schmidt (2004) Health related: Anxiety, somatic problems (−)*** Making Healthy students (3- to 8-year-old children) Type: RCT Behavior related: For the 1994/95 and 1995/96 Choices Objectives: (a) Increase the protective factor of Location: United States: Virginia (1994/95); Virginia and the 1996/97 Michigan programs, Child social-emotional competence in young children, Virginia (1995/96); Lansing, Michigan Behavior Rating Scale (+)***; in the 1994/95 Virginia and (b) decrease the risk factor of early and (1996/97) pilot, positive coping strategies (+)***, negative persistent antisocial or aggressive behavior by Follow-up: 7 months coping strategies (−)**; for the 1996/97 Michigan preventing the development of increased Sample: 173 intervention and 48 comparison group, problem behavior (−)**; self-centered, aggression and antisocial behaviors in young children in 1994/95 Virginia group; explosive/attention problems/antisocial, children over the course of a typical school year. In 230 intervention and 103 comparison aggressive/social withdrawal (−)**; positive coping particular, Al’s Pals lessons are designed to help children in the 1995/96 Virginia group; (+)*; distract/avoid (+)* young children develop specific skills related to the 218 intervention and 181 comparison Note: The children in the Michigan trial did not following four components of resiliency: (a) social children in the 1996/97 Michigan group. experience any changes in the problem behavior competence, (b) problem solving, (c) autonomy, and Thirty-seven classrooms were selected scale. (d) a sense of purpose and belief in a bright future. randomly. Seventeen classrooms (218 Description: Al’s Pals is a universal early childhood children) were randomly assigned to the prevention initiative that involves teacher training, intervention group and 16 classrooms a yearlong classroom curriculum, and a (181 children) were randomly assigned to companion parent education program. the control group. Specifically, it involves a 46-lesson resiliency-based Term used: Socioemotional competence prevention curriculum implemented over 23 weeks (15–20 minutes/lesson) by trained teachers in a variety of settings, including preschools, child care centers, other early childhood classrooms, and after-school programs. It also involves materials and music, including 14 parent letters that explain what the curriculum is teaching and that offer activities for parents to do at home. Cost: Over five years, implementing the program costs approximately US$9–US$15 per child, depending on the number of children in the classroom. Start-up costs include training on the intervention (core, refresher, and advanced online training cost about US$495) and the purchase of materials (each curriculum kit costs US$685). 123 table continues next page 124 Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Project CARE Child Beneficiaries: Students from kindergarten to grade 6 Authors: Chang and Muñoz (2006) Behavior related: Student autonomy and influence in Development Objective: Promote the growth of the whole child Type: Quasi-experiment the classroom scale (+)***; classroom Project through character, social, ethical, and academic Location: United States: Louisville, Kentucky supportiveness scale (+)*** development. The program aimed to enhance Follow-up: 1 year Academic/cognitive related: Reading for high-quality prosocial characteristics by affecting participants’ Sample: From 87 eligible elementary treatment schools (+)**; number of referrals (−)** attitudes, motives, and behaviors. schools, 8 schools were chosen for Other: Principal supportiveness (+)*; faculty Description: This character education program treatment and 8 for control. Participants collegiality (+)***; trust in students (+)*** promotes academic and social growth in teachers included 390 teachers across 16 schools Note: No significant differences between control and and students through the following components: (190 in control group and 200 in treatment treatment in academic outcomes, attendance, or (a) School Wide Activities—cooperative activities group) and 3,908 students from grades 3, suspension. that encouraged students working together to 4, and 5 (2,025 control group; 1,883 in the solve problems and build relationships and create treatment group) across the 16 schools. caring classroom communities; (b) Developmental Term used: Character Discipline—an approach to classroom management that fosters respectful and caring relationships among students and teachers; (c) Making Meaning—a literature-based language arts/reading component that explicitly integrates ethical and moral conversation into the curriculum by using a diverse set of reading materials with various cultures, issues, and themes; (d) KidzLit— after-school enrichment activities to support reading; and (e) parent activities—Second Step training provided to parents to support prosocial parenting skills. (The impact evaluation analyzes the effect of components a–c). Cost: n/a Authors: Battistich, Schaps, and Wilson Behavior related: Global self-esteem (+)*; (2004) victimization at school (−)*; misconduct at school Type: Quasi-experiment (−)*; involvement in positive youth activities (+)***; Location: Six school districts across the gets along well with others, is sought out by his/her United States fellow students, and has many friends (+)* Follow-up: 4 years table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Sample: 700 beneficiaries and 546 controls In high implementation subsample: sense of efficacy belonging to six elementary schools, (+)**; delinquent behaviors (−)**; friends’ drug use serving at-risk students (+)*; friends’ delinquent behaviors (+)**; friends’ Term used: Social behavior positive involvement in school (+)**; gets along well with others, is sought out by his/her fellow students, and has many friends (+)**; does not hesitate to state opinions, even when others disagree with his/her views (+)*; is engaged in class—participates in discussions, stays on the topic, and generally takes an active part in whatever the class is doing (+)**; appears to be socially awkward and inept—tends to say the “wrong thing” and to be rebuffed or ridiculed by other students (+)**; considers others’ feelings, treats them with respect, and offers and gives help to those who need it (+)* Academic/cognitive related: Sense of efficacy (+)***. In high-implementation subsample: trust in and respect for teachers (+)**; positive teacher–student relations (+)***; educational aspirations (+)***; educational expectations (+)*; GPA (+)***; achievement test scores (+)**; comes to class and completes assignments on time, tries to learn the material, and does the best work he/she can (+)*; task orientation toward learning (+)* Other: Sense of school (+)*; positive teacher–student relations (+)**; liking for school (+)**; attendance at religious services (+)* Note: No significant differences between control and treatment in academic outcomes, attendance, or suspension. table continues next page 125 Table B.1  School-Based Program Descriptions (continued) 126 Program Evaluation Name Description Author/description Results I Can Problem Beneficiaries: Kindergarten and first-grade students Authors: Boyle and Hassett-Walker (2008) Behavior related: Preschool Social Behavior Scale for Solve Objectives: Increase prosocial behavior and reduce Type: RCT those with two years of instruction (+)***; aggressive behavior in children in kindergarten Location: United States Hahnemann behavior rating scale for those with and first grade in a large, ethnically diverse, urban Follow-up: 2 years one or two years of instruction (+)***; rational school district. Sample: 226 students were randomly aggression for those with two years of instruction Description: The Interpersonal Cognitive Problem- assigned in three groups: (a) 96 students (−)**; overt aggression of those with two years of Solving curriculum (also known as I Can Problem received instruction for two consecutive instruction (−)****; prosocial behavior for those Solve) is a school-based universal/primary years; (b) 106 students received only one with two years of instruction (+)***; passivity for prevention program that focuses on children’s year of program instruction, either in those receiving two years of instruction (+)**; cognitive processes and problem-solving skills. kindergarten or first grade; and (c) 24 prosocial behavior for those who received either It uses various techniques, including games, students were assigned to the control one or two years of instruction (+)*** stories, puppets, and role playing. It consists of 83 group for two consecutive years. Note: There is a modest additive effect of the program, 20-minute lessons delivered by trained teachers. Term used: Prosocial behaviors with students who received two years of instruction The lessons include a stated purpose of the lesson, becoming more prosocial and less aggressive than a list of suggested materials, and a teacher’s script students who received one year of instruction. that explains the basic steps in conducting the lessons. I Can Problem Solve is optimally taught three to five times per week. Lessons focus on training children to (a) generate a variety of solutions to interpersonal problems; (b) think through the consequences of each potential solution; and (c) identify thoughts, feelings, and motives that can generate problem situations. Teachers are highly trained (they receive a manual and subsequent booster training, as well as follow-up training when needed). Cost: n/a Project SAFE Beneficiaries: Students from kindergarten to grade 5 Authors: Kumpfer and others (2002) Behavior related: Parenting skills for the group (Strategies Objectives: Prevent substance abuse. In particular, Type: RCT receiving ICPS+SF1 (+)***; family relationships for Aimed at Family the I Can Problem Solve program (ICPS) is directed Location: United States ICPS+SF1 (+)**; social competency for ICPS+SF1 Empowerment) at enhancing problem-solving and critical Follow-up: 9 months (+)** and ICPS+SF2 (+)***; behavioral self-regulation thinking skills. The Strengthening Families (SF) Sample: 655 first-grade students from for all groups (+)*** program was originally designed for children of 12 rural schools randomly divided into Academic/cognitive related: School bonding for all drug abusers. the following groups: (a) 256 received the programs except for ICPS+SF2 (+)*** table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Description: The I Can Problem Solve program is ICPS, (b) 56 received the ICPS combined Note: The variables measured are associated with the described above. The SF program is a 14-session with the SF program (referred to as risk for substance abuse. family skills training program that consists of three ICPS+SF1), (c) 21 received the ICPS and courses: (a) parent skills training, (b) children skills the parenting training of the SF program training, and (c) family life skills training. To maintain (referred to as ICPS+SF2), and (d) 322 were the intervention’s gains, it has booster sessions at 6 in the control group. and 12 months. Four trainers (two parent and two Term used: Social skills children cotrainers) conduct the programs at the schools in weekly scheduled sessions. Incredible Cost: n/a Incredible Years Beneficiaries: Children from prekindergarten to Authors: Webster-Stratton, Reid, and Behavior related: School readiness and conduct grade 5 Stoolmiller (2008) problems (−)**; social competence; child Objective: Promote children’s social skills, self- Type: RCT disengagement (−)**; number of positive feelings regulation, and classroom behavior. Location: United States: Seattle, Washington identified by children (+)***; emotional self- Description: The Incredible Years Child Training Follow-up: 2 years regulation; conduct problems (−)** Curriculum (Dinosaur School) consisted of 30 Sample: 120 classrooms from Seattle-area Academic/cognitive related: Effective discipline by classroom lessons per year (at least 2 per week, Head Starts and 14 elementary schools Head Start teachers (+)**; positive classroom lasting 35–40 minutes each), with preschool and were involved in the project. These management strategies primary grade versions. The content is broken schools were matched on variables such Other: Involvement with parents (+)** into seven units: (a) learning school rules; (b) how to as size, geographic location, and Note: Overall, children who were initially most at risk be successful in school; (c) emotional literacy, demographics of the children. The sample benefited most from the intervention. Furthermore, empathy, and perspective taking; (d) interpersonal consisted of 1,746 kindergarten or all of the student behavioral outcomes showed problem solving; (e) anger management; (f) social first-grade students who were nested in strong teacher-level effects (the teacher played an skills; and (g) communication skills. Materials 160 classrooms, which in turn were nested important role in the effects). include over 300 small-group activities, over 100 under 119 teachers (40 teachers had 2 or videotaped models of children demonstrating more classrooms). Classrooms were in social skills and conflict management strategies, life- culturally diverse schools with high rates size puppets, Dinosaur homework activities, picture of poverty. cue cards, and games. Teachers were trained for four Term used: Social emotional and self-control days (28 hours) spread out in monthly workshops. skills The training involved explanations of how to deliver the Dinosaur School curriculum, as well how to use effective classroom management strategies. 127 table continues next page Table B.1  School-Based Program Descriptions (continued) 128 Program Evaluation Name Description Author/description Results Cost: US$2,506 for a group with 12 children (2007 dollars) (Edwards and others 2007) Michigan Model Beneficiaries: Students from kindergarten to 12th Authors: O’Neill, Clark, and Jones (2011) Health related: Intentions to smoke cigarettes (−)***; for Health grade Type: RCT intentions to drink alcohol (−)*** (MMH) Objective: Help young people live happier and Location: United States: Michigan and Behavior related: Social and emotional skills (+)***; healthier lives. Indiana interpersonal skills (+)***; drug refusal skills (+)***; Description: The MMH uses a universal prevention Follow-up: 2 years self-management skills (+)*; aggressive behavior approach to facilitate skills-based learning Sample: 2,512 students in 52 schools, who (−)** through 20- to 50-minute lessons that incorporate were randomly assigned to: (a) 1,847 a variety of teaching and learning techniques, children in 29 schools received the skills development and practice, and approaches intervention in fourth grade and (b) 29 for building positive lifestyle behaviors in students schools and 1,536 were the control in and families. The fourth-grade curriculum fourth grade. consisted of 25 lessons on social and emotional The size of the groups changed for fifth health; alcohol, tobacco, and other drugs; safety; grade: (a) on the treatment group, 3 and nutrition and physical activity. In fifth grade, schools closed and 1 declined, therefore, 28 lessons were taught across the same health 1,345 children in 25 schools received the topics. The intervention was implemented in intervention in fifth grade and (b) 2 of the classrooms over 12 weeks in grade 4 and 14 weeks control schools closed, therefore, 27 in grade 5 during a normal class period of 40–50 schools and 1,167 students were the minutes by the classroom or health teacher, who control in fifth grade. received a 12-hour curriculum training course with Terms used: Social and emotional health, follow-up support provided as needed. self-management, interpersonal Cost: n/a communication MindUp: Beneficiaries: Students from grades 4 to 7 Authors: Schonert-Reichl and Lawlor (2010) Behavior related: Optimism (+)**; positive affect (+)*; Mindfulness Objective: Foster children’s positive emotions, Type: Quasi-experimental design aggressive behaviors (−)***; oppositional behavior/ Education self-regulation, and goal setting Location: Canada dysregulation (−)***; attention and concentration Program Description: MindUp is a classroom-based universal Follow-up: 9 weeks (+)***; social-emotional competence (+)*** preventive intervention that focuses on Sample: 139 students in grades 4 to 7 Note: The program demonstrated more positive facilitating the development of social and participated in the program, and 107 benefits for preadolescents than for early emotional competence and positive emotions. students were in control classrooms. adolescents. In the curriculum, the mindful attention training Term used: Social and emotional exercises are practiced three times a day (three competence table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results minutes per practice), along with affirmations and visualizations. Cost: n/a Open Circle Beneficiaries: Students from kindergarten to grade 8 Author: Hennessey (2007) Behavior related: Social skills (+)***; problem Objective: Enhance social skills of elementary school Type: Quasi-experimental design behaviors (−)*** children Location: United States Note: No statistical difference was found in student- Description: Twice a week, children and their Follow-up: Approximately 10 months reported measures or on academic performance. teachers come together to work on 1 of 35 Sample: 154 fourth graders in eight lessons. Verbal cues encouraging class members classrooms in four schools (two in to speak up, calm down, and adopt the “school middle- to upper-middle-class suburban listening look” come out of discussions of areas and two that served more diverse concepts and behaviors drawn from the children’s populations) were sampled. A group of own experiences; students also develop students participated in classrooms agreed-upon nonverbal signals that emphasize headed by teachers well versed in the respectful communication. Open Circle curriculum. The rest of the Cost: n/a students participated in classes that were not implementing a social competence program. Term used: Social skills Fast Track PATHS Beneficiaries: Students from prekindergarten to Authors: CPPRG (2010) Behavior related: Authority acceptance (+)***; (Promoting grade 6 Type: Clustered RCT problem levels (−)***; aggression in boys (−)***; Alternative Objectives: Promote emotional and social Location: United States: Nashville, Tennessee; hyperactivity in boys (−)**; prosocial behavior in Thinking competencies and reduce aggression and Seattle, Washington; central Pennsylvania boys (+)* Strategies) behavior problems Follow-up: 3 years Academic/cognitive related: Cognitive concentration Description: The Fast Track program integrated the Sample: 2,937 children of multiple ethnicities (+)***; academic engagement (+) provision of universal services (all children) and from grades 1, 2, and 3 in elementary Note: Most intervention effects were moderated by selective services (children at some risk) into a schools located in high-risk school environment, with effects stronger in less comprehensive model that involved the child, neighborhoods (high delinquency and disadvantaged schools. Effects on aggression were school, family, and community. Children juvenile arrests) greater in students who showed higher baseline demonstrating the greatest degree of early Term used: Emotional and social levels of aggression. conduct problems were selected for a series of competences programs that included weekly parenting support classes, small-group social skills programs, 129 table continues next page Table B.1  School-Based Program Descriptions (continued) 130 Program Evaluation Name Description Author/description Results academic tutoring, and home visits. The universal Authors: Domitrovich, Cortes, and Behavior related: Emotion knowledge skills (+) intervention was started in the classroom Greenberg (2007) [Kusche Emotional Inventory—emotion vocabulary concurrent with the initiation. The PATHS Type: RCT (+)***; assessment of children’s emotions scales— curriculum model synthesizes the domains of Location: United States: Pennsylvania emotion expression knowledge (+)** and anger self-control, emotional awareness and Follow-up: 1 year bias (−)***; Denham Puppet Interview—affective understanding, peer-related social skills, and Sample: 292 three- to four-year-old children perspective-taking skills (+)**]; teacher’s report on social problem solving to focus on promoting in 20 classrooms. Ten of the classrooms social competence (+)***, social skills (+)***, social social and emotional competence. Of these were treated with 30 weekly lessons and cooperation (+)***, social interaction (+)***, and programs for the high-risk children and families, extension activities (such as group games, social independence (+)***; social withdrawal (−)**; 2 to 3 lessons were presented on a regular basis art projects, and books) for nine months. parents’ report on social competence (+)*** throughout most of the school year, and daily Term used: Emotional and social activities were used to promote generalization competences (57 lessons in grade 1, 46 lessons in grade 2, and 48 lessons in grade 3). Cost: With 500 students participating, the cost per student is US$119. Positive Action Beneficiaries: Children in kindergarten to grade 12 Authors: Lewis and others (2013) Behavior related: Normative beliefs in support of Objective: Positively influence multiple behavioral Type: RCT aggression (−)***; bullying behaviors (−)***; domains, such as student academic achievement Location: United States: Chicago, Illinois disruptive behavior (−)***; parents’ report of and substance use. Follow-up: 6 years bullying(−)** Description: The curriculum has the following Sample: 624 children in 14 low-income, Academic/cognitive related: Disciplinary components: self-concept; social and emotional mostly minority, urban Chicago public referrals(−)***; suspensions (−)*** positive actions for managing oneself responsibly; schools were followed for six years (from and positive actions directed toward physical and grades 3 to 8). Attrition was high, and by mental health, honesty, getting along with others, eighth grade, only 363 students remained and continually improving oneself. Each grade in the study. level includes 140 lessons (15–20 minutes each; Term used: Socioemotional kindergarten to grade 6) or 70 lessons (20 minutes each; grades 7 and up). The program also includes teacher, counselor, family, and community training, as well as activities directed toward school-wide climate development. Cost: Average cost per participant: US$510 (2013 dollars) (Belfield and others 2015) table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Authors: Snyder and others (2013) Health related: Substance use (−)***; sexual Type: RCT activity (−)*** Location: United States: Hawaii Behavior related: Violence (−)*** Follow-up: 3 years Academic/cognitive related: Academic behavior (+)*** Sample: 1,784 elementary-age students who were followed from grades 1 and 2 to grade 5. These children attended 20 racially and ethnically diverse schools in Hawaii: 10 of the schools received the intervention for four to five years, and 10 schools served as control. Term used: Socioemotional and character development Raising Healthy Beneficiaries: Students from kindergarten to Authors: Catalano and others (2003) Behavior related: Social competence (+)***; Children (RHC) grade 12 Type: RCT antisocial behaviors (−)** Objective: Reduce developmental expression of risk Location: United States Academic/cognitive related: Teacher reported factors for problem behaviors while increasing Follow-up: 1.5 years academic performance (+)***; school protective factors. Sample: 938 elementary students from first commitment (+)** Description: A multicomponent, multiyear social or second grade who were enrolled in development intervention is delivered to 10 high-risk (low income, low participants during grades 1 through 12. The standardized achievement test scores, school intervention strategies provided during high absenteeism, and high mobility) elementary and middle school consisted of a schools in the Pacific Northwest and series of teacher and staff development randomly divided into two groups: workshops. Student intervention strategies (a) 497 participated in schools with the during the elementary school years consisted of RHC program, and (b) 441 participated in social, emotional, and cognitive skill training control schools infused into classroom and school-wide activities, Term used: Social skills after-school tutoring sessions and study clubs during grades 4 to 6, and student summer camps during the first four summers of the project. table continues next page 131 Table B.1  School-Based Program Descriptions (continued) 132 Program Evaluation Name Description Author/description Results The family intervention strategies offered during the elementary school years consisted of universal and selected components. Multiple-session parenting workshops (for example, “Raising Healthy Children,”“How to Help Your Child Succeed in School,” and “Preparing for the Drug Free Years”) were offered to all parents at intervention schools. For families of students identified as high risk because of academic or behavioral problems, in-home services tailored to the specific risk factors of these children were provided to reinforce curricula covered in the parenting workshops. Cost: n/a Resolving Conflict Beneficiaries: Children in grades 1 through 6 Authors: Aber, Brown, and Jones (2003) Health related: Depressive symptoms (−)*** Creatively Objective: Change the mental processes and Type: Quasi-experimental design Behavior related: Hostile attribution bias (−)***; interpersonal behavioral strategies that lead Location: United States: New York, New York aggressive interpersonal negotiation strategies children to engage in aggression and violence by Follow-up: 2 years (−)***; competent interpersonal negotiation teaching them constructive conflict resolution Sample: 11,160 public elementary students, strategies (−)***; conduct problems (−)***; strategies and promoting positive intergroup from first to sixth grade in 15 schools in aggressive fantasies (−)***; prosocial behavior (+)*** relations New York City. Description: The universal, school-based intervention Term used: Socioemotional development involves violence prevention and intergroup understanding. The intervention has two major components: (a) training and coaching of teachers to support them in implementing a curriculum in conflict resolution and intergroup understanding (teacher training and coaching), and (b) the delivery of that curriculum through classroom instruction for children provided by the trained teachers (classroom instruction). Additional features of the program include peer mediation, principals’ training, and parent training. Cost: n/a table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Responsive Beneficiaries: Students from kindergarten to sixth Authors: Brock and others (2008); Curby, Behavior related: Teacher-rated social skills (+)**; child Classroom grade Rimm-Kaufman, and Abry (2013); perceptions (+)**; emotional support (+); teacher’s Objectives: Integrate social and academic learning Rimm-Kaufman and others (2014) emotional support (+)*** and bolster classroom social support. Type: RCT Academic/cognitive related: Teacher-rated academics Description: The social and emotional learning Location: United States (+)**; standardized reading (+)**; math for children intervention provides professional development Follow-up: 3 years with low levels (+)***; reading (+)** for elementary teachers in the use of 10 specific Sample: 24 schools: 13 were randomized Other: Classroom organization (+)** classroom practices: Morning Meeting, Rule into the intervention and 11 were Creation, Interactive Modeling, Teacher Language, assigned to a wait list. Logical Consequences, Academic Choice, Brock and others (2008) used a sample of 520 Classroom Organization, Guided Discovery, children and 51 teachers in grades 3 to 5. Collaborative Problem Solving, and Working Term used: Socioemotional with Families. Cost: US$900 (2013 dollars); calculated for schools with high implementation fidelity (Belfield and others 2015) RULER program Beneficiaries: Students from kindergarten to eighth Authors: Rivers and others (2013) Behavior related: Emotional support domain (+)**; grade Type: RCT positive climate (+)***; regard for student Objective: Modify the quality of classroom social Location: United States: Brooklyn and perspectives (+)**; emotion-focused interactions interactions so that the climate becomes more Queens, New York (+)** supportive, empowering, and engaging. Follow-up: 15 months Academic/cognitive related: Classroom emotional Description: The program provides comprehensive Sample: 155 fifth- and sixth-grade climate (+) professional development with a literacy-based, classrooms in 62 schools randomized into skill-building, and social and emotional learning receiving either the RULER curriculum or program for students. It is applied universally and the English Language Arts curriculum includes professional development for school Term used: Social and emotional leaders, teachers, and staff, as well as classroom instruction protocols to enhance skill-building opportunities and characteristics of the learning environment. Training program: schoolteachers attended a day and a half of training on RULER at the beginning of the academic year. Cost: n/a 133 table continues next page Table B.1  School-Based Program Descriptions (continued) 134 Program Evaluation Name Description Author/description Results Second Step Beneficiaries: Students from preschool to Authors: Espelage and others (2013) Behavior related: Physical aggression (−)** (Student 11th grade Type: RCT Note: No significant intervention effects for Success through Objective: Help students stay engaged in school; Location: United States: Illinois and Kansas perpetration of or victimization by bullying, Prevention) make good choices; set goals; and avoid bullying, Follow-up: 3 years homophobic teasing, and sexual violence. cyberbullying, and peer pressure. Sample: 3,616 sixth-grade students within Description: Second Step is a universal curricular 36 middle schools randomly assigned: classroom intervention, which for sixth grade (a) 18 schools and 1,940 students in the involves 15 weekly lessons (50 minutes each) experimental group, and (b) 18 schools focused on socioemotional learning skills, and 1,676 students in the control group including empathy, communication, bullying Term used: Socioemotional learning skills prevention, and problem-solving skills. Lessons are highly interactive, incorporating small group discussions and activities, dyadic exercises, whole-class instruction, and individual work. Lessons are structured and supported through an accompanying DVD that contains rich media content, including topic-focused interviews with students and video demonstrations of skills. Cost: US$440 per student (2013 dollars) (Belfield and others 2015) Steps to Respect Beneficiaries: Upper elementary school years Authors: Frey and others (2009) Behavior related: Bullying (−)***; victimization by (grades 3 to 6) Type: RCT bullying (−)***; destructive bystander behavior Objective: Decrease school bullying problems by Location: United States: Pacific Northwest (−)***; aggression (−)***; argumentative students (a) increasing adult monitoring and intervention Follow-up: 2 years (−)***; self-reports of victimization (−)**; self- in bullying incidents, (b) improving systemic Sample: 624 elementary school students in reports of direct aggression (+)***; peer interaction supports for socially responsible behavior, grades 3 through 5 were surveyed, and skills for fourth-grade teachers (+)**; acceptance of (c) changing student normative beliefs that 360 students were observed on the bullying (−)* support bullying, and (d) addressing student playground. Note: Agreeable interactions did not increase. socioemotional skills that counter bullying and Term used: Social emotional skills Acceptance of bullying/aggression did not change. support social competence. Description: The multilevel program coordinates a school-wide environmental intervention, three sequential classroom curricula, and a selected table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results intervention for students involved in bullying. The program includes manuals, written material, and audiovisual presentations for school administration, staff training, classroom curricula, and parent outreach. In the school-wide environmental intervention, school administrators and staff establish school-wide bullying policies and procedures that protect reporting students, stop problems before they escalate, follow up cases, dispel myths regarding bullying, and so on. The classroom curricula include 10 weeks of biweekly basic lessons followed by 8 to 10 literature-based lessons. The individual intervention consists of brief individual coaching sessions with each participant in bullying incidents, intended to provide solution-oriented responses to immediate and long-term student needs and assess effectiveness. Cost: n/a Unique Minds Beneficiaries: Students from kindergarten to fifth Authors: Linares and others (2005) Behavior related: Student self-efficacy (+)***; grade Type: Quasi-experimental design socioemotional competencies (+)**; attention and Objectives: Promote cognitive-social-emotional Location: United States: New York concentration (+)***; social and emotional (CSE) skills, including student self-efficacy, Follow-up: 2 years competence (+)***; authority and compliance problem solving, socioemotional competence, Sample: 119 students (57 in intervention problems (−)***; aggression (+)*** and a positive classroom climate, with the dual school and 62 in comparison school) Academic/cognitive related: Problem-solving skills goals of preventing youth behavioral problems from 13 fourth-grade classrooms (6 in (+)**; math grades (+)** and promoting academic learning. intervention and 7 in comparison) in Note: No significant effect for classroom climate, Description: A classroom package of concepts, urban public schools standardized reading, or math test. activities, tools, and strategies is designed to Term used: Socioemotional skills involve multiple agents (students, peers, teachers and other school staff, and parents) across different settings (classrooms, cafeterias, playgrounds, and the home). The intervention targets student proximal CSE competencies 135 table continues next page Table B.1  School-Based Program Descriptions (continued) 136 Program Evaluation Name Description Author/description Results (self-efficacy, problem solving, and socioemotional functioning) and distal academic learning (grades and achievement). Cost: n/a Family Check-Up Beneficiaries: Children ages 2 to 17 Authors: Caruthers, Van Ryzin, and Dishion Health related: Depression (−)**; substance use (−); (FCU) Objective: Reduce children’s problem behavior (2014); Connell and Dishion (2008); high-risk sexual behavior in adulthood (−) through changing parent–child interactions. Connell, Klostermann, and Dishion (2012); Behavior related: Arrests (−)**; antisocial behavior at Setting: Public middle schools Dishion and others (2002); Stormshak, age 19 (−)**; family relationship (+); involvement in Description: In this school-based, family-centered Connell, and Dishion (2009); Van Ryzin, deviant peer friendship (−) intervention, a family resource center is Stormshak, and Dishion (2012) Academic/cognitive related: School absences during established at the school site. The resource center Type: RCT middle school and into high school (−)**; GPA (+)** provides general informational and consultation Location: United States: midsize urban city services (such as brochures, parenting materials, in the Pacific Northwest parenting topics nights, community resources, and Follow-up: From ages 12 to 23 feedback to parents on their student’s behavior). Sample: 998 sixth-grade adolescents and Additionally, using LifeSkills Training, it provides six their families: 498 assigned to control and in-class lessons to students on the following 500 assigned to treatment. 80% in topics: (a) school success, (b) health decisions, follow-up. (c) building positive peer groups, (d) the cycle of respect, (e) coping with stress and anger, and (f) solving problems peacefully. Furthermore, the FCU model identifies students with early signs of risk in emotional, behavioral, or academic domains and treats them with more intensive family support services (three family- centered intervention sessions that assess family strengths and weaknesses and motivate parents to improve their parenting practices and engage in intervention services that address the specific needs of their family). Families that participate have the opportunity to select intervention options that are tailored to the unique needs of each family and that are grounded in empirically validated family management strategies and table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results school and community resources that can support family change. The intervention was delivered by highly trained personnel who were continuously trained during the study. Cost: n/a Authors: Stormshak and others 2011; Health related: Cigarette, alcohol, and marijuana Van Ryzin, Stormshak, and Dishion (2012)  use (−)** Type: RCT Behavior related: Effortful control/self-regulation Location: United States: midsized urban city (+)**; antisocial behavior [lying to parents, skipping in the Pacific Northwest school, damaging property, fighting] (−)**; Follow-up: 3 years spending time with deviant peers [peers who Sample: 593 adolescents in sixth grade smoke, get in trouble, steal, fight, use alcohol or that belonged to ethnic minorities: drugs] (−)***; family conflict (−)* 386 families assigned to intervention (students at risk and not at risk were invited to participate) and 207 to control. 86% retained across three years. Aerobic Running Beneficiaries: Fourth, fifth, and sixth graders Authors: Tuckman and Hinkle (1986) Health related: Pulse rates (−)*** Objectives: Compare the effects of running with Type: RCT Academic/cognitive related: Creativity (+)*** a normal physical education program. Location: United States Note: No significant differences in classroom behavior, Setting: Public middle schools. It is a school-based, Follow-up: 6 months self-concept, and Bender-Gestalt or Maze test family-centered intervention. Sample: 154 children in grades 4, 5, and 6 scores. Description: Those assigned to running had to do three 30-minute sessions per week for 12 weeks in lieu of attending regular physical education classes. Cost: n/a Leadership Beneficiaries: Children from kindergarten to grade 5 Authors: Lakes and Hoyt (2004) Behavior related: Cognitive, affective, and physical Education Objectives: Examine the impact of school-based tae Type: RCT self-regulation (+)*; prosocial behavior (+)*; Through Athletic kwon do training on self-regulatory abilities. Location: United States: private lower classroom conduct for girls (+)* Development Setting: Private lower school school in a midsize midwestern city Academic/cognitive related: Performance on a (LEAD) Description: A high-level martial arts teacher (with a Follow-up: 3 months mental math test (+)* curriculum black belt and close to 10 years of instruction experience) gave two to three martial art classes 137 table continues next page Table B.1  School-Based Program Descriptions (continued) 138 Program Evaluation Name Description Author/description Results per week to treated children in an environment Sample: 193 children from kindergarten Note: No effect on freedom from distractibility, characterized by respect, discipline, and self- through grade 5, randomly assigned to self-esteem. control. All classes started with a short meditation treatment (martial arts) or control session. Children in control group participated in a (traditional physical education) standard physical education curriculum that included stretching, running, and sports. Cost: n/a Mindful Awareness Beneficiaries: Second- and third-grade children Authors: Flook and others (2010) Behavior related: Behavioral regulation (+)**; Practices (MAPs) ages 7 to 9 Type: RCT global executive control (+)*** Objective: Improve executive functions. Location: United States: Los Angeles, Academic/cognitive related: Metacognition (+)** Setting: On-campus university elementary school California Note: No effect on freedom from distractibility, Description: Thirty minutes of exercises generate a Follow-up: 3 months self-esteem. state of heightened and receptive attention to Sample: 64 children ages 7 to 9 from moment-by-moment experience, twice per week, second- and third-grade classrooms for eight weeks. The program is modeled after classical mindfulness training for adults and uses secular and age-appropriate exercises and games to promote (a) awareness of self through sensory awareness, attentional regulation, and awareness of thoughts and feelings; (b) awareness of others; and (c) awareness of the environment. Cost: n/a Attention Beneficiaries: Elementary school children (grades 1 Authors: Napoli, Krech, and Holley (2005) Health related: Anxiety (+)* Academy to 3) of nine classrooms within two elementary Type: RCT Behavior related: Social skills (+)*** Program (AAP) schools Location: United States: a southwestern city Academic/cognitive related: Selective attention (+)*** Objectives: Help students learn to (a) increase their Follow-up: 3 months Note: No improvement on sustained attention. attention to the present experience, (b) approach Sample: 194 children in grades 1 to 3 in each experience without judgment, and (c) view nine classrooms within two elementary each experience as novel and new with a schools: 97 receiving treatment “beginner’s eye.” (mindfulness program) and 97 in Setting: Elementary schools control (reading or quiet activities) Description: A 24-week training program employed a series of exercises, including breath work, body table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results scan, movement, and sensorimotor awareness activities. The classes met for 45 minutes bimonthly during students’ regular physical education class. Instructors were highly trained and experienced in mindfulness meditation. Cost: n/a Montessori Beneficiaries: Children from preschool to grade 12 Authors: Lillard and Else-Quest (2006) Behavior related: Positive, direct strategy to social (Montessori) Type: RCT problem solving (+)***; greater sense of community Objectives: Generate the optimal development in Location: United States: Milwaukee, (+)** children. Wisconsin Academic/cognitive related Creativity of narrative Description: Montessori schooling is a 100-year-old Follow-up: 6 years (+)**; sentence sophistication in narrative (+)** system that naturally incorporates practices that Sample: 57 12-year-old children attending a Note: No difference found on spelling, grammar, and align with mindfulness and are suited to very public inner-city school and suburban punctuation or on Woodcock-Johnson tests. young children. Montessori education is public, private, or charter school: (a) 28 characterized by multiage classrooms, a special set control and (b) 29 attending a Montessori of educational materials, and student-chosen work school in large time blocks; collaboration and the absence of grades and tests; and individual and small-group instruction in both academic and social skills. Cost: n/a Social and Beneficiaries: Children from kindergarten to grade 5 Authors: Humphrey, Lendrum, and Note: SEAL failed to significantly affect pupils’ social Emotional Objective: Promote the social and emotional skills Wigelsworth (2010) and emotional skills, general mental health Aspects of that underpin effective learning, positive Type: Quasi-experimental difficulties, prosocial behavior, or behavior Learning (SEAL) behavior, regular attendance, staff effectiveness, Location: United Kingdom problems. and the emotional health and well-being of all Follow-up: 2 years who learn and work in schools. Sample: 8,630 children in grade 7 attending Description: SEAL is a loose enabling framework for 22 SEAL schools and a matched group of school improvement. Schools are encouraged to 19 comparison schools explore different implementation approaches to Term used: Social and emotional skills tailor the program to their specific needs. At the school level, SEAL is characterized by the following principles: (a) SEAL implementation is underpinned by clear planning focused on 139 table continues next page Table B.1  School-Based Program Descriptions (continued) 140 Program Evaluation Name Description Author/description Results improving standards, behavior, and attendance; (b) building a school ethos provides a climate and conditions to promote social and emotional skills; (c) all children are provided with planned opportunities to develop and enhance social and emotional skills; (d) adults are provided with opportunities to enhance their own social and emotional skills; (e) staff members recognize the significance of social and emotional skills to effective learning and to the well-being of pupils; (f) pupils who would benefit from additional support have access to small-group work; (g) there is a strong commitment to involving pupils in all aspects of school life; (h) there is a strong commitment to working positively with parents and careers; and (i) the school engages well with other schools, the local community, wider services, and local agencies. Cost: n/a Drama in Finland Beneficiaries: Children in grades 4 and 5 Authors: Joronen and others (2011) Behavior related: Social relationships (+)* Objectives: Improve social relationships and social Type: Quasi-experimental longitudinal and emotional well-being in the classroom and design reduce bullying. Location: Finland Description: Universal school-based drama program Follow-up: 8 months later aims to enhance social relationships and to Sample: 190 primary school students decrease bullying at school. The drama program (response rate 71%) in grades 4 and 5: included classroom drama sessions, follow-up 78 were treated and 56 acted as controls. activities at home, and three parents’ evenings Term used: Social relationships concerning issues of social well-being. The program was delivered by regular teachers or a teacher and a school nurse as a dyad. The implemented drama program included four to nine classroom drama sessions, one to four follow-up home activities, and three parents’ evenings. Cost: n/a table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results School-based Beneficiaries: First- and second-grade students in Authors: Klevens and others (2009) Behavior related: Aggressive behavior (−)***; programs in public schools Type: RCT with pre- and postassessments antisocial behavior for the teacher-only Colombia Objective: Reduce aggressive and antisocial Location: Pereira, Colombia intervention (−)*** behavior by first- and second-grade students in a Follow-up: 6 months after the program starts resource-poor setting. (just after it finishes) Description: The study tested two school-based Sample: 2,491 children in 12 public schools: programs. The first was a teacher-only (a) 5 schools with 40 first- and second- intervention that consisted of teacher training grade classrooms and 881 students that focused on standard classroom management participated in the teacher-only techniques (arranging the physical environment intervention group, (b) 4 schools with 39 to reduce opportunities for conflict, establishing first- and second-grade classrooms and and consistently enforcing clear rules, and 787 students were assigned to the instituting routines and procedures) and teacher/parent combined intervention strategies for shaping children’s behaviors group, and (c) 3 schools with 41 (modeling and reinforcing appropriate behavior classrooms and 823 students were consistently in the classroom’s daily activities and assigned to a wait-list control group. interactions). Teachers received 10 four-hour Term used: Antisocial/prosocial behavior weekly workshops, a manual, and weekly written and verbal feedback. The second program was a teacher/parent combined intervention that included the same teacher training as well as parenting intervention that covered almost the same topics as the teacher training (the session on redesigning the classroom was replaced by one of prosocial behavior). This session lasted one hour, once a week for 10 weeks. Cost: n/a table continues next page 141 Table B.1  School-Based Program Descriptions (continued) 142 Program Evaluation Name Description Author/description Results Mato-Oput5 Beneficiaries: Children ages 9 to 18 Authors: Mutto and others (2009) Behavior related: Attitude toward forgiving a bully Objectives: Reduce negative attitudes toward Type: Cluster randomized control design (+)**; attitude toward forgiving a friend who returns conflict and violence, and reduce injury and Location: Northern Uganda a book he/she had stolen (+)**; not fighting a friend violence rates. Follow-up: 3 months who lies about one (+)**; self-reporting verbally Description: Mato-Oput5 is a value-based, formalized Sample: Six schools: three assigned to abusing others (−)** curriculum taught by specifically trained teachers. control and three assigned to treatment Note: Statistically significant behavioral effects were Its learning areas include conflict, conscience, Term used: Social development not detected, although a downward trend was seen violence, nonviolence, impulse control, anger in the intervention group. management, kindness, forgiveness, empathy, and reconciliation. At least two 40-minute weekly lessons are taught in class. Cost: n/a Linking the Beneficiaries: All first- and fifth-grade elementary Authors: Reid and others (1999) Behavior related: Teacher rating of peer-preferred Interests of school boys and girls and their families living in Type: RCT behavior (+)**; child physical aggression on the Families and at-risk neighborhoods characterized by high rates Location: United States playground (−)*** Teachers (LIFT) of juvenile delinquency Follow-up: 1 year later Note: Those mothers in the LIFT program exhibiting Objective: Affect a set of interrelated antecedents of Sample: 671 first and fifth graders and their the highest preintervention levels of aversive the conduct disorders of children in high-risk families, from 12 elementary schools: behaviors showed the largest immediate reductions neighborhoods. 382 attended the intervention schools in mother-aversive behavior. Description: Universal preventive intervention and 289 attended the control schools. consists of (a) parent training aimed at teaching Term used: Social skills parents how to create a home environment that is most conducive to the ongoing practice of good discipline and supervision (parents met in groups of 10 to 15 families once each week for six weeks, free child care was provided, and a prize was raffled at the end of every session); (b) a classroom-based social skills program consisting of 20 one-hour sessions that included classroom instruction and discussion on specific social and problem-solving skills, skill practice in small and large groups, free play in the context of a group cooperation game, and review and presentation of daily rewards; (c) a playground behavioral program; and (d) systematic communication between teachers and parents. Cost: n/a table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Authors: DeGarmo and others (2009) Health related: Risk in initiation of tobacco use (−)***; Type: RCT risk in initiation of alcohol use (−)**; risk in initiation Location: United States of illicit drug use (−)*; use of tobacco (−)**; use of Follow-up: 1 year alcohol (−)***; use of illicit drugs (−)** Sample: 351 youths in fifth grade within Behavior related: Playground aggression (−)**; 17 different classrooms in six effective family problem solving (+)** randomized schools. Term used: Social skills Becoming a Man Beneficiaries: Children attending grades 9 and 10, Authors: Cook and others (2014) Academic/cognitive related: Math test cores (+)**; referred by school staff Type: RCT math grades (+)*; nonmath courses failed (−)**; Objective: Provide both academic and nonacademic Location: United States: Chicago, Illinois expected graduation rates (+)**; absence without remediation for disadvantaged youths who are Follow-up: 1 year an excuse (−)* falling behind and at great risk of dropping out. Sample: 106 disadvantaged males in grades Note: The confidence levels are too wide to say Description: The socioemotional intervention called 9 and 10 in public high schools (95% are anything about which of the two intervention arms Becoming a Man includes school programming black and 99% are eligible for free or is more effective. Additionally, there was that exposes youths to prosocial adults and reduced-price lunch) divided into three contamination among the different groups and provides youths with social-cognitive skills groups: (a) 24 children received just the spillover. training using principles of cognitive behavioral nonacademic intervention, (b) 48 received therapy. Youths have the chance to participate in both the nonacademic intervention and up to 27 one-hour, weekly group sessions during the academic intervention, (c) 34 received the school day over the school year (the program status quo services. lasts for three-quarters of an academic year). The Term used: Social-cognitive skills intervention is delivered in groups of fewer than 15 youths. Students skip an academic class in order to participate in the program. The program can be delivered by college-educated people without specialized training in psychology or social work. The curriculum also includes efforts to develop specific social or social cognitive skills, such as generating new solutions to problems, learning new ways of behaving, considering another’s perspective, thinking ahead, and evaluating consequences ahead of time. 143 table continues next page Table B.1  School-Based Program Descriptions (continued) 144 Program Evaluation Name Description Author/description Results The academic intervention is delivered by intensive, individualized two-on-one math tutoring provided for one hour per day every day by well-educated, committed people without formal teacher training. Cost: Approximately US$4,400 per participant (ranging from US$3,000 to US$6,000) Social and Beneficiaries: Children in grades 1 through 9 in Authors: Kimber, Sandell, and Bremberg Health related: Alcohol use (−)**; narcotic drug use Emotional Swedish schools (2008) (−)* Training (SET) Objectives: Train students to improve self-control, Type: Quasi-experimental longitudinal Behavior related: Self-image (+)**; well-being (+)**; social competence, empathy, motivation, and design relations with others (+)**; aggressiveness (−)**; self-awareness. Location: Sweden anxiety (−)***; assertiveness (+)*; bullying (−)**; Description: SET is a classroom-based intervention Follow-up: 5 years attention seeking (−)*** that focuses on developing the following five Sample: Children in grades 1 through 9 in functions in students: self-awareness, emotional four schools: (a) 41 classes and 52 teachers Note: No differential effect on social skills. management, empathy, motivation, and social in two schools received the treatment; competence. The program was delivered by (b) 14 classes did not receive the regular classroom teachers during scheduled treatment hours: to grades 1 through 5 twice a week in two Term used: Socioemotional 45-minute sessions, and to grades 6 through 9 in one 45-minute session per week over the school year. The program is guided by detailed manuals for the teacher, one volume for each grade. It also includes a student workbook for each grade. Altogether, the program consists of 399 concrete exercises, some of which are inspired by similar programs in the United States. Teachers are instructed and supervised monthly. Interaction between school and parents is emphasized. Cost: US$540 per student (or US$140 excluding the teacher instruction time) for the five years of intervention (2013 dollars) (Belfield and others 2015) table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Teen Outreach Beneficiaries: High school students in grades 9 Authors: Allen and others (1997) Health related: Pregnancy rates (−)* through 12 Type: RCT Academic/cognitive related: Failing courses (−)***; Objectives: Prevent adolescent problem behaviors Location: United States academic suspension (−)*** by enhancing normative processes of social Follow-up: 9 months later (at program exit) development in high school students; prevent Sample: 695 high school students at 25 sites teen pregnancy and academic failure. In addition, nationwide, who were randomly assigned the program was designed to empower students, to either a Teen Outreach or control group. as they would be “help givers” rather than “help Term used: Life skills receivers.” Description: The program consists of three interrelated elements: (a) a minimum of 20 hours per year of supervised volunteer community service (activities were selected by students and varied substantially in their nature and in the amount of commitment required from participants), (b) classroom-based discussions of service experiences (discussions were designed to engage students through structured discussions, group exercises, role play, guest speakers, and informational presentations), and (c) classroom- based discussions and activities related to key social-developmental tasks of adolescence (classroom discussions were led by trained facilitators and met at least once weekly during a full academic year). Costs: US$500–US$700 per student per year, for a class of 18–25 students (these figures include the cost for facilitators and site-level coordinator time) table continues next page 145 146 Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Learn and Serve Beneficiaries: Students in grades 6 through 12 Author: Melchior (1998) Health related: Ever pregnant or made someone America Objectives: Help young people (a) develop as Type: Quasi-experimental pregnant (−)* responsible citizens, improve their academic skills, Location: United States Behavior related: Cultural diversity (+)**; service and develop as individuals, while helping to meet Follow-up: 1 year leadership (degree to which students feel they are the unmet needs in the public safety nets of the Sample: 1,052 participants in grades aware of the needs in a community, are able to United States; (b) promote the integration of 6 through 12 attending 17 middle school develop and implement a service project, and are service learning in schools and academic and high school sites across the country committed to service now and later in life) (+)***; curriculum; and (c) promote the delivery of Term used: Life skills total civic attitudes (+)***; volunteered for a needed services in the community. community organization or got involved in other Description: The program helps young people community service in the last six months (+)***; develop as responsible citizens, improve their hours or support to community service initiatives in academic skills, and develop as individuals the last six months (+)*** through involvement in meaningful service linked Academic/cognitive related: Engagement in school to structured learning activities. (+)***; math grades (+)**; science grades (+)*; core Costs: Approximately US$149.12 per participant GPA (+)* (1995–96 dollars) Note: Results are greater for specific subgroups. For example, the impact was statistically significant for the educationally disadvantaged students on English, math, and science grades; overall and core GPA; and a number of at-risk behaviors. For the economically disadvantaged students, impacts were significant for school engagement, days absent, and aggressive behaviors (fighting, hurting someone, or using weapons). For females, the program had statistically significant effects on the use of illegal drugs in the past 30 days, aggressive behavior, and the number of at-risk behaviors. Further, the program had effects on English, math, and social studies grades; GPA; and course failures of minority students. table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Bal Sabha (Girls’ Beneficiaries: Girls from grades 6, 7, and 8 in Authors: Delavallade, Griffith, and Behavior related: Expectations of democratically Parliament) the poorest districts of Rajasthan, India Thornton (2015) elected girls about age at marriage (+); expectations Objective: Create soft skills in girls and their Type: RCT about career prospects (−); educational communities. Location: Rajasthan, India expectations and aspirations for nonelected girls in Description: The program consists of democratic Follow-up: 1 year democratically elected program (−); partition of elections (by both boys and girls) of 13 girls from Sample: 30 schools randomly assigned as friendship groups between those who are grades 6, 7, and 8 to meet after school several follows: (a) 10 received the standard randomly selected and those who are not (+) times per month to participate in life skills games program, (b) 10 received a modified Note: Main effects on the standard program where that stimulate growth and development and program in which girls were randomly girls are elected. build  confidence, leadership, and self-esteem. selected rather than elected, and (c) 10 They practice setting goals for their school or were controls and did not receive the community and are told to pass on the life skills program games to other students. Terms used: Soft skills, life skills Cost: n/a Iowa Beneficiaries: Sixth graders Authors: Spoth, Randall, and Shin (2004) Behavior related: Alcohol use (−) *; lifetime Strengthening Objective: Strengthen parent and child skills that Type: RCT drunkenness (−)*; lifetime cigarette use (−)* Families have been found to delay the onset or reduce the Location: United States: Iowa Note: The PDFY program had no significant effects. Program (ISFP) use of substances. Follow-up: 6 years and Preparing Description: The ISFP consists of seven sets of Sample: 667 families of sixth-grade students for the Drug sessions that are conducted once per week for enrolled in 33 rural public schools were Free Years seven consecutive weeks and are held on randomly assigned to the ISFP (11 schools (PDFY) program weekday evenings in participating schools. and 238 families), the PDFY program (11 The ISFP includes separate, concurrent one-hour schools and 221 families), or a minimal- training sessions for parents and children, contact control condition (11 schools or followed by a joint one-hour family session. 208 families). The PDFY program is delivered in five two-hour Term used: prosocial behavior training sessions held on weekday evenings once per week for five consecutive weeks (four sessions are for parents only, and another session on peer-resistance skills is for parents and children). Cost: n/a table continues next page 147 148 Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Chicago Child- Beneficiaries: Disadvantaged, African American, Authors: Reynolds and others (2001) Behavior related: Juvenile arrest (−)**; violent arrests Parent Center inner-city, 3- and 4-year-old children Type: Matching (−)** Program Objectives: Promote academic success by providing Location: United States: Chicago, Illinois Academic/cognitive related: High school completion a school-stable learning environment during the Follow-up: 15 years at age 20 (+)***; years of education (+)**; school preschool and primary grade years and actively Sample: 1,539 low-income, mostly African dropout (−) **. For those with extended program involve parents in their children’s education. American children born in 1980: 989 participation from preschool to second and third Description: Comprehensive education, family, and attending the Chicago program; 550 grade: grade retention (−)***; special education health services, including half-day preschool at children participating in full-day (−)**. ages 3 to 4, half- or full-day kindergarten, and kindergarten school-age services in linked elementary schools Term used: Social outcomes at ages 6 to 9. Child-to-teacher ratios of 17:2 in preschool and 25:2 in kindergarten. Parental participation includes interaction with other parents in workshops, reading groups, crafts projects, classroom volunteering, school events, and field trips, and completing high school. The program also includes health and nutrition services, such as screening, speech therapy, and nursing Cost: Mean per-child expenditures in 1996 for one year of preschool and one year of school-age participation are US$4,350 and US$1,500, respectively. table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Project Student/ Beneficiaries: Students in Tennessee Authors: Chetty and others (2010); Krueger Behavior related: Effort, initiative, lack of disruptive Teacher Objective: Analyze the effects of class size, teacher and Whitmore (2001) behavior in grades 4 and 8 (+)**; arrests for crime (−) Achievement quality, and peers on student’s results. Type: RCT Academic/cognitive related: Standardized tests (+)**; Ratio (STAR) Description: Students in 79 Tennessee schools were Location: United States: Tennessee finish high school (+); college attendance at age 20 randomly assigned to small classes (15 students) Follow-up: 27 years later (+)* or large classes (22 students on average). Sample: 11,571 students from kindergarten Economic related: Home ownership (+); retirement Cost: The average cost per child of reducing class through third grade were randomly savings (+); earnings (+) ** size by 33% for 2.14 years (the mean treatment assigned to a small class (13–17 students), Note: No significant differences in earnings at age 27. duration for STAR students) is US$9,355 (2009 regular-size class (22–25 students), or No significant difference in student’s self-concept dollars). regular-size class with a teacher’s aide. and motivation. Term used: Noncognitive HighScope Perry Beneficiaries: High-risk African American children. Risk Authors: Heckman and others (2010); Health related: Mortality rates (−)**; likelihood of Preschool is defined as parents with low educational Muennig and others (2009); Schweinhart halting work because of illness (−)**; smoking (−)** Program attainment (high school or less); low occupational and others (2005) Behavior related: Arrests (−)***; externalizing status (unemployed or unskilled); low socioeconomic Type: Quasi-experimental behavior (+)**; engaging in risky behavior (−)*** status; low IQ scores (between 70 and 85) with no Location: United States: Michigan Academic/cognitive related: IQ until age 1 (+)** then organic deficiencies; and at high risk of failing school. Follow-up: 37 years later faded; grade repetition (−)**; assigned to special Objective: Support long-term cognitive and Sample: 123 disadvantaged African education (−)**; high school graduation by age 19 educational benefits in children from American children between ages 3 and 5: (+)**; college attendance (+)**; educational disadvantaged backgrounds. 58 were assigned to treatment (high- attainment at age 40 (+)*** Description: From ages 3 to 5, participants received a quality preschool) and 65 to control (no Economic related: Earnings (+)***; homeownership 2.5-hour preschool program on weekdays during preschool) (+)** the school year (approximately 180 days), and Terms used: Behavioral skills; socioemotional Note: Annual social rates of return: 7%–16% (US$9− weekly home visits by the teachers. The curriculum development; personal and social US$13 for every US$1 invested). fostered active learning (children were encouraged dispositions to plan, carry out, and reflect on their own activities). Teachers were certified to teach in elementary, early childhood, and special education. There were four teachers for 20 to 25 children (in Perry Preschool) and two teachers for 16 children in HighScope preschool. Seems to work with 2 adults and up to 20 children (Epstein 1993). Cost: US$17,759 per student per year (undiscounted 2006 dollars) 149 table continues next page 150 Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Head Start Beneficiaries: Children ages 3 to 5 Authors: Westat (2010) Health related: Child insurance coverage (+)*; health Objective: Guarantee school readiness by Type: RCT status for 3-year-olds (+)* (a) improving the child’s health and physical Location: United States Behavior related: Closer and more positive abilities; (b) improving the family’s attitude Follow-up: 3 years relationship of 3-year-old children to parents(+)*; toward future health care and physical abilities; Sample: 378 centers, 2,783 Head Start hyperactivity in 3-year-olds (−)*; total problem (c) encouraging self-confidence, spontaneity, children (some newly entering 3- and behavior in 3-year-olds (−)*; social skills in curiosity, and self-discipline, which will assist in 4-year-olds) and 1,884 control children 3-year-olds (+)*; positive approaches to learning for the development of the child’s social and Term used: Social-emotional development 3-year-olds (+)* emotional health; (d) enhancing the child’s Academic/cognitive related: PPVT vocabulary for mental processes and skills with particular 4-year-olds (+)*; Woodcock Johnson III oral attention to conceptual and communications comprehension for 3-year-olds (+)* skills; (e) establishing patterns and expectations of success for the child, which will create a climate of confidence for present and future learning efforts and overall development; (f ) increasing the ability of the child and the family to relate to each other and to others; and (g) enhancing the sense of dignity and self-worth within the child and his/her family. Description: Comprehensive services include preschool education; medical, dental, and mental health care; nutrition services; efforts to help parents foster their child’s development Cost: US$7,000 per child on average table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Tools of the Mind Beneficiaries: 3- and 4-year-old children attending a Authors: Barnett and others (2008) Behavior related: Problem Behaviors Scale of the state-financed, full-day “Abbott” preschool Type: RCT Social Skills Rating System (−)** education program located in high-poverty Location: United States: New Jersey, One Academic/cognitive related: Cognitive outcomes as school districts school with a high proportion of children measured by the IDEA Oral Language Proficiency Objective: Build strong foundations for school from low-income and non-English- Test (+)* success by promoting intentional and self- speaking families Other: Curriculum for language and reasoning (+)***; regulated learning in preschool- and Follow-up: 8 months later activities in the curriculum (+)**; interactions in the kindergarten-age children. Sample: 274 children ages 3 and 4: (a) 106 curriculum (+)*; positive classroom climate, teacher Description: A child care program with a curriculum received Tools of the Mind; (b) 168 sensitivity, behavior management techniques, and is focused on self-regulation, while teaching received the district’s curriculum productivity (+)* literacy and math. During pretend play, children Term used: Social development Academic/cognitive related: Performance on must inhibit acting out of character, remember demanding tasks for executive functions (+)** their own and others’ roles, and flexibly adjust as their friends improvise. Cost: According to What Works Clearinghouse, the first year costs about US$3,000 per classroom. Curriculum guides cost an additional US$100. The second year costs about US$1,500 per classroom. Authors: Diamond and others (2007) Type: RCT Location: United States: 18 classrooms in a low-income, urban school with state- financed Abbott full-day preschool education Follow-up: 2 years Sample: 147 preschoolers who received two years of program: 62 in district’s version of balanced literacy curriculum and 85 in Tools of the Mind Term used: Inhibitory control (resisting habits, temptations, or distractions) table continues next page 151 152 Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Child Health Beneficiaries: Children ages 3 to 5 selected for the study Author: Raine and others (2001); Raine and Behavior related: Improved psychophysiological Project on the basis of electrodermal functioning (amplitude, others (2003) arousal and orienting at age 11. This is seen frequency, and half-recovery data) at age 3 Type: RCT through: skin conductance amplitudes (+) ***; skin Objective: Originally, this study had the goal of Location: Two randomly selected conductance rise times (+) ***; skin conductance bringing psychiatrically at-risk children together communities in Mauritius recovery times (+) **; slow-wave in nursery schools where “drugs could be used to Follow-up: At ages 11, 17, and 23 electroencephalogram measure during rest (−) ***; bring their autonomic functions within normal Sample: 200 children: 100 assigned to an electroencephalogram measure during continuous range” (WHO 1968, 381). Ethical considerations enriched nursery school intervention, of performance task conditions (−) ***. Behavioral precluded the use of drugs to change autonomic which 93 were followed at age 11; and 100 problems at age 17 (−) ***, which included conduct functioning (WHO 1975, 39), and instead of drugs, assigned to the control group receiving a disorder (−)***; psychotic behavior (−)**; motor the research team developed nursery schools to normal educational experience, of which excess (−)**; schizotypal personality at age 23 for provide an enriched, stimulating experience for 95 were followed at age 11. This normal those who received enrichment and were not an experimental versus matched control group experience consisted of preschool with malnourished (−)**; criminal offending (−)** containing psychiatrically unselected children untrained childminders, in units of poor drawn from the community (Venables 1978). educational quality that provided Description: The two-year intervention consisted of traditional and very rudimentary preschool education, nutrition (education on education with a teacher-to-pupil ratio of nutrition and meals), physical exercise, health 1:30. screening and referral, parental involvement, Term used: Social and emotional remediation of behavioral and learning problems, development and home visits to the family. With regard to cognitive-behavioral and socialization, an important feature of the experimental schools was the use of time-out in place of the more traditional physical punishments for misbehavior, and the provision of one-to-one explanations for why behavior was inappropriate. Particular efforts were made to ensure that individual children were never left on their own or socially isolated. Conversation sessions included the discussions of emotion concepts, such as sadness and love. table continues next page Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Drama and puppet sessions were used to enhance the child’s emotional sensitivity. Game sessions were used to help the children understand the importance of following rules. These sessions were also used to illustrate life problems and how they may be successfully resolved. Requiring children to wear identical clothing (uniforms were provided to the parents) was done in an attempt to reduce social divisions among the children. Staff was highly trained, and the teacher-to-pupil ratio ranged from 1:5.5 to 1:10. Cost: n/a Preschool program Beneficiaries: 3- to 5-year-old children living in 138 Authors: Bouguen and others (2014) Academic/cognitive related: Preschool attendance in Cambodia villages situated in disadvantaged rural areas Type: RCT (+); primary school attendance (−); ASQ gross motor Objective: Provide preschool education to Location: Cambodia scale (+)*; ASQ problem-solving subscale (−)*; vulnerable children, which was integrated into a Follow-up: 17–20 months later overall development index for 5-year-old children primary school rehabilitation effort. Sample: 26 villages received preschool and (−)*; cognitive development index (−)*** Description: The preschool curriculum was 19 villages served as controls; 1,731 Note: No impacts on receptive vocabulary, composed of singing, drawing (mixing colors, children living in these villages were communication, fine motor skills, emotion, conduct, reproducing signs/geographic figures on a board interviewed. Take-up of preschools was hyperactivity, peer, prosocial behavior, height or or with small sticks), physical activities (such as small (only 36% of the children in weight for age, overall development index, cognitive gymnastics and games), some vocabulary (listing treatment villages attended preschool). development index, motor development index, words), and counting. It was designed for the 3- to Term used: Noncognitive anthropometrics index, noncognitive index. The low 5-year age group and does not explicitly include impacts can be attributed to implementation writing or reading. Teachers often organized social constraints, low take-up rates, and a short duration of games in which children had to recall the names program exposure for participating children. The of other students and add or subtract them from negative impact of preschool participation on the a group of pupils. cognitive development of 5-year-old children may Cost: n/a derive from two facts: (a) some children were attending primary school before and switched to preschool, or (b) some other children who were attending primary school left the formal school system when the official age of primary school was enforced. 153 table continues next page 154 Table B.1  School-Based Program Descriptions (continued) Program Evaluation Name Description Author/description Results Peace Works Beneficiaries: Preschool children ages 4 to 5 years Author: Pickens (2009) Behavior related: Social cooperation (+)***; positive Objectives: Promote preschoolers’ social skills and Type: RCT interaction quality (+)***; social independence reduce behavior problems Location: United States: Miami, Florida (+)***; external and internal problem behaviors Description: The program instructs teachers and Follow-up: 1 school year (−)*** parents in how to use activities and ‘‘I-Care Rules Sample: 246 preschool children in the PEF and Language’’ to encourage empathy and fair treatment classrooms; 50 evaluated from play, express feelings, avoid conflict, manage nontreatment schools anger, and interact more positively with others. Term used: Socioemotional The curriculum models conflict resolution, anger management, and communication skills to promote children’s positive interactions, cooperation, and emotional self-regulation. Cost: n/a Note: CARE = character actualization requires education; 4Rs = reading, writing, respect, and resolution; GPA = grade point average; RCT = randomized control trial; n/a = not available. Significance levels of results, positive (+) or negative (–), are reported at 1 percent (*), 5 percent (**), and 10 percent (***). School-Based Program Descriptions 155 Table B.2 School-Based Programs by Component Program components Class Household Program Teacher training curriculum activities Extracurricular 4Rs Raising Healthy Children Bal Sabha Teen Outreach Resolving Conflict Creatively Al’s Pals Project CARE Child Development Project I Can Problem Solve Project SAFE Incredible Years Drama in Finland LIFT Family Check-Up ISFP and PDFY School-based programs in Colombia Michigan Model for Health MindUp Open Circle Fast Track PATHS Positive Action Responsive Classroom RULER program Second Step Steps to Respect Attention Academy Montessori SEAL Social and Emotional Training Mato-Oput5 Learn and Serve America Unique Minds LEAD MAPs Becoming a Man 28 32 13 5 Note: Shading indicates that the program includes that particular component. 4Rs = reading, writing, respect and resolution; CARE = character actualization requires education; ISFP = Iowa Strengthening Families Program; LEAD = Leadership Education Through Athletic Development; LIFT = Linking the Interests of Families and Teachers; MAPs = Mindful Awareness Practices; PATHS = Promoting Alternative Thinking Strategies; PDFY = Preparing for the Drug Free Years; SAFE = Strategies Aimed at Family Empowerment; SEAL = Social and Emotional Aspects of Learning. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 156 School-Based Program Descriptions References Aber, J. Lawrence, Joshua L. Brown, and Stephanie M. Jones. 2003. “Developmental Trajectories toward Violence in Middle Childhood: Course, Demographic Differences, and Response to School-Based Intervention.” Developmental Psychology 39 (2): 324–48. Allen, Joseph P., Susan Philliber, Scott Herrling, and Gabriel P. Kuperminc. 1997. “Preventing Teen Pregnancy and Academic Failure: Experimental Evaluation of a Developmentally Based Approach.” Child Development 68 (4): 729–42. Battistich, Victor, Eric Schaps, and Nance Wilson. 2004. “Effects of an Elementary School Intervention on Students’ ‘Connectedness’ to School and Social Adjustment during Middle School.” Journal of Primary Prevention 24 (3): 243–62. Belfield, Clive, Brooks Bowden, Alli Klapp, Henry Levin, Robert Shand, and Sabine Zander. 2015. “The Economic Value of Social and Emotional Learning.” Center for Benefit-Cost Studies in Education, Teachers College, Columbia University, New York. Boyle, Douglas, and Connie Hassett-Walker. 2008. “Reducing Overt and Relational Aggression among Young Children: The Results from a Two-Year Outcome Study.” Journal of School Violence 7 (1): 27–42. Brock, Laura L., Tracy K. Nishida, Cynthia Chiong, Kevin J. Grimm, and Sara E. Rimm- Kaurman. 2008. “Children’s Perceptions of the Classroom Environment and Social and Academic Performance: A Longitudinal Analysis of the Contribution of the Responsive Classroom Approach.” Journal of School Psychology 46 (2): 129–49. Brown, Joshua L., Stephanie M. Jones, Maria D. LaRusso, and J. Lawrence Aber. 2010. “Improving Classroom Quality: Teacher Influences and Experimental Impacts of the 4Rs Program.” Journal of Educational Psychology 102 (1): 153–67. Caruthers, Allison S., Mark J. Van Ryzin, and Thomas J. Dishion. 2014. “Preventing High- Risk Sexual Behavior in Early Adulthood with Family Interventions in Adolescence: Outcomes and Developmental Processes.” Prevention Science 15 (1): 59–69. Catalano, Richard F., James J. Mazza, Tracy W. Harachi, Robert D. Abbott, Kevin P. Haggerty, and Charles B. Fleming. 2003. “Raising Healthy Children through Enhancing Social Development in Elementary School: Results after 1.5 Years.” Journal of School Psychology 41 (2): 143–64. Chang, Florence, and Marco A. Muñoz. 2006. “School Personnel Educating the Whole Child: Impact of Character Education on Teachers’ Self-Assessment and Student Development.” Journal of Personality Evaluation in Education 19: 35–49. Connell, Arin M., and Thomas J. Dishion. 2008. “Reducing Depression among At-Risk Early Adolescents: Three-Year Effects of a Family-Centered Intervention Embedded within Schools.” Journal of Family Psychology 22 (4): 574–85. Connell, Arin M., Susan Klostermann, and Thomas J. Dishion. 2012. “Family Checkup Effects on Adolescent Arrest Trajectories: Variation by Developmental Subtype.” Journal of Research on Adolescence 22 (2): 367–80. Cook, Philip J., Kenneth Dodge, George Farkas, Roland G. Fryer Jr., Jonathan Guryan, Jens Ludwig, Susan Mayer, Harold Pollack, and Laurence Steinberg. 2014. “The (Surprising) Efficacy of Academic and Behavioral Intervention with Disadvantaged Youth: Results from a Randomized Experiment in Chicago.” NBER Working Paper 19862, National Bureau of Economic Research, Cambridge, MA. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 School-Based Program Descriptions 157 CPPRG (Conduct Problems Prevention Research Group). 2010. “The Effects of a Multiyear Universal Socio-Emotional Learning Program: The Role of Student and School Characteristics.” Journal of Consulting and Clinical Psychology 78 (2): 156–68. Curby, Timothy W., Sara E. Rimm-Kaufman, and Tashia Abry. 2013. “Do Emotional Support and Classroom Organization Earlier in the Year Set the Stage for Higher Quality Instruction?” Journal of School Psychology 51 (5): 557–69. DeGarmo, David S., J. Mark Eddy, John B. Reid, and Rebecca A. Fetrow. 2009. “Evaluating Mediators of the Impact of the Linking the Interests of Families and Teachers (LIFT) Multimodal Preventive Intervention on Substance Use Initiation and Growth across Adolescence.” Prevention Science 10 (3): 208–20. Delavallade, Clara, Alan Griffith, and Rebecca Thornton. 2015. “Girls’ Education, Aspirations, and Social Networks: Evidence from a Randomized Trial in Rural Rajasthan.” Paper presented at the Ninth Annual PopPov Conference on Population, Reproductive Health, and Economic Development, Addis Ababa, Ethiopia, June 25. Dishion, Thomas J., Kathryn Kavanagh, Alison Schneiger, Sarah Nelson, and Noah K. Kaufman. 2002. “Preventing Early Adolescent Substance Use: A Family-Centered Strategy for the Public Middle School.” Prevention Science 3 (3): 191–201. Domitrovich, Celene E., Rebecca C. Cortes, and Mark T. Greenberg. 2007. “Improving Young Children’s Social and Emotional Competence: A Randomized Trial of the Preschool PATHS Curriculum.” Journal of Primary Prevention 28 (2): 67–91. Edwards, Rhiannon T., Alan Céilleachair, Tracey Bywater, Dyfrig Hughes, and Judy Hutchings. 2007. “Parenting Programme for Parents of Children at Risk of Developing Conduct Disorder: Cost Effectiveness Analysis.” BMJ 334 (7595): 682–85. Espelage, Dorothy L., Sabina Low, Joshua R. Polanin, and Eric C. Brown. 2013. “The Impact of a Middle School Program to Reduce Aggression, Victimization, and Sexual Violence.” Journal of Adolescent Health 53 (2): 180–86. Flook, Lisa, Susan L. Smalley, M. Jennifer Kitil, Brian M. Galla, Susan Kaiser-Greenland, Jill Locke, Eric Ishijima, and Connie Kasari. 2010. “Effects of Mindful Awareness Practices on Executive Functions in Elementary School Children.” Journal of Applied School Psychology 26 (1): 70–95. Frey, Karin S., Miriam K. Hirschstein, Leihua Van Schoiack Edstrom, and Jennie L Snell. 2009. “Observed Reductions in School Bullying, Non-Bullying Aggression, and Destructive Bystander Behavior: A Longitudinal Evaluation.” Journal of Educational Psychology 101: 466–81. Hennessey, Beth A. 2007. “Promoting Social Competence in School-Aged Children: The Effects of the Open Circle Program.” Journal of School Psychology 45 (3): 349–60. Humphrey, Neal, Ann Lendrum, and Michael Wigelsworth. 2010. “Social and Emotional Aspects of Learning (SEAL) Programme in Secondary Schools: National Evaluation.” Research Report DFE-RR049. Department for Education, London. Jones, Stephanie M., Joshua L. Brown, and J. Lawrence Aber. 2011. “Two-Year Impacts of a Universal School-Based Social-Emotional and Literacy Intervention: An Experiment in Translational Developmental Research.” Child Development 82: 533–54. Jones, Stephanie M., Joshua L. Brown, Wendy Hoglund, and J. Lawrence Aber. 2010. “A School-Randomized Clinical Trial of an Integrated Social-Emotional Learning and Literacy Intervention: Impacts after One School Year.” Journal of Consulting and Clinical Psychology 78 (6): 829–42. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 158 School-Based Program Descriptions Joronen, Katja, Anne Konu, H. Sally Rankin, and Päivi Åstedt-Kurki. 2011. “An Evaluation of a Drama Program to Enhance Social Relationships and Anti-Bullying at Elementary School: A Controlled Study.” Health Promotion International 27 (1): 5–14. Kimber, Birgitta, Rolf Sandell, and Sven Bremberg. 2008. “Social and Emotional Training in Swedish Classrooms for the Promotion of Mental Health: Results from an Effectiveness Study in Sweden.” Health Promotion International 23 (2): 134–43. Klevens, Joanne, José William Martínez, Brenda Le, Carlos Rojas, Adriana Duque, and Rafael Tovar. 2009. “Evaluation of Two Interventions to Reduce Aggressive and Antisocial Behavior in First and Second Graders in a Resource-Poor Setting.” International Journal of Educational Research 48 (5): 307–19. Kumpfer, Karol L., Rose Alvarado, Connie Tait, and Charles Turner. 2002. “Effectiveness of School-Based Family and Children’s Skills Training for Substance Abuse Prevention among 6–8-Year-Old Rural Children.” Psychology of Addictive Behaviors 16 (45): S65–S71. Lakes, Kimberley D., and William T. Hoyt. 2004. “Promoting Self-Regulation through School-Based Martial Arts Training.” Journal of Applied Developmental Psychology 25 (3): 283–302. Lewis, Kendra M., Marc B. Schure, Niloofar Bavarian, David L. Dubois, Joseph Day, Peter Ji, Naida Silverthorn, Alan Acock, Samuel Vuchinich, and Brian R. Flay. 2013. “Problem Behavior and Urban, Low-Income Youth: A Randomized Controlled Trial of Positive Action in Chicago.” American Journal of Preventive Medicine 4 (6): 622–30. Lillard, Angeline, and Nicole Else-Quest. 2006. “The Early Years: Evaluating Montessori.” Science 313 (5795): 1893–94. Linares, L. Oriana, Nicole Rosbruch, Marcia B. Stern, Martha E. Edwards, Gillian Walker, Howard B. Abikoff, and Jose M. Alvir. 2005. “Developing Cognitive-Social-Emotional Competencies to Enhance Academic Learning.” Psychology in the Schools 42 (4): 405–17. Lynch, Kathleen Bodisch, Susan R. Geller, and Melinda G. Schmidt. 2004. “Multi-Year Evaluation of the Effectiveness of a Resilience-Based Prevention Program for Young Children.” Journal of Primary Prevention 24 (3): 335–53. Melchior, Alan. 1998. “National Evaluation of Learn and Serve America School and Community-Based Programs: Final Report.” Brandeis University, Center for Human Resources, Waltham, MA. Mutto, Milton, Kathleen Kahn, Ronald Lett, and Stephen Lawoko. 2009. “Piloting an Educational Response to Violence in Uganda: Prospects for a New Curriculum.” African Safety Promotion 7 (2): 37–46. Napoli, Maria, Paul Rock Krech, and Lynn C. Holley. 2005. “Mindfulness Training for Elementary School Students: The Attention Academy.” Journal of Applied School Psychology 21 (1): 99–125. O’Neill, James M., Jeffrey K. Clark, and James A. Jones. 2011. “Promoting Mental Health and Preventing Substance Abuse and Violence in Elementary Students: A Randomized Control Study of the Michigan Model for Health.” Journal of School Health 81 (6): 320–30. Reid, John B., J. Mark Eddy, Rebecca Ann Fetrow, and Mike Stoolmiller. 1999. “Description and Immediate Impacts of a Preventive Intervention for Conduct Problems.” American Journal of Community Psychology 27 (4): 483–518. Rimm-Kaufman, Sara E., Ross A. A. Larsen, Alison E. Baroody, Timothy W. Curby, Michelle Ko, Julia B. Thomas, Eileen G. Merritt, Tashia Abry, and Jamie DeCoster. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 School-Based Program Descriptions 159 2014. “Efficacy of the Responsive Classroom Approach: Results from a 3-Year, Longitudinal Randomized Controlled Trial.” American Educational Research Journal 51 (3): 567–603. Rivers, Susan E., Marc A. Brackett, Maria R. Reyes, Nicole A. Elbertson, and Peter Salovey. 2013. “Improving the Social and Emotional Climate of Classrooms: A Clustered Randomized Controlled Trial Testing the RULER Approach.” Prevention Science 14 (1): 77–87. Schonert-Reichl, Kimberly A., and Molly S. Lawlor. 2010. “The Effects of a Mindfulness- Based Education Program on Pre- and Early Adolescents’ Well-Being and Social and Emotional Competence.” Mindfulness 1 (3): 137–51. Snyder, Frank J., Alan Acock, Sam Vuchinich, Michael W. Beets, Isaac J. Washburn, and Brian R. Flay. 2013. “Preventing Negative Behaviors among Elementary-School Students through Enhancing Students’ Social-Emotional and Character Development.” American Journal of Health Promotion 28 (1): 50–58. Spoth, R., Redmond, C., Shin, C. and Azevedo, K., 2004. “Brief Family Intervention Effects on Adolescent Substance Initiation: School-Level Growth Curve Analyses 6 years Following Baseline.” Journal of Consulting and Clinical Psychology 72 (3), 535. Spoth, Richard L., G. K. Randall, and Chungyeol Shin. 2008. “Increasing School Success through Partnership-Based Family Competency Training: Experimental Study of Long-Term Outcomes.” School Psychology Quarterly 23 (1): 70–89. Stormshak, Elizabeth A., Arin M. Connell, and Thomas J. Dishion. 2009. “An Adaptive Approach to Family-Centered Intervention in Schools: Linking Intervention Engagement to Academic Outcomes in Middle and High School.” Prevention Science 10 (3): 221–35. Stormshak, Elizabeth A., Arin M. Connell, Marie Hélène Véronneau, Michael W. Myers, Thomas J. Dishion, Kathryn Kavanagh, and Allison S. Caruthers. 2011. “An Ecological Approach to Promoting Early Adolescent Mental Health and Social Adaptation: Family-Centered Intervention in Public Middle Schools.” Child Development 82 (1): 209–25. Tuckman, Bruce W., and J. Scott Hinkle. 1986. “An Experimental Study of the Physical and Psychological Effects of Aerobic Exercise on Schoolchildren.” Health Psychology 5 (3): 197–207. Van Ryzin, Mark. J., Elizabeth A. Stormshak, and Thomas J. Dishion. 2012. “Engaging Parents in the Family Check-Up in Middle School: Longitudinal Effects on Family Conflict and Problem Behavior through the High School Transition.” Journal of Adolescent Health 50 (6): 627–33. Webster-Stratton, Carolyn, M. Jamila Reid, and Mike Stoolmiller. 2008. “Preventing Conduct Problems and Improving School Readiness: Evaluation of the Incredible Years Teacher and Child Training Programs in High-Risk Schools.” Journal of Child Psychology and Psychiatry 49 (5): 471–88. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Appendix C Out-of-School Program Descriptions Taking Stock of Programs to Develop Socioemotional Skills   161   http://dx.doi.org/10.1596/978-1-4648-0872-2 Table C.1 Out-of-School Program Descriptions 162 Program Evaluation Name Description Author/description Results Work related Procajoven Beneficiaries: Low-income unemployed 18- to Authors: Ibarrarán and Rosas-Shady (2006) Economic related: Probability of 29-year-olds, and first-time job seekers with Type: Quasi-experiment employment for women in Panama City in complete secondary education Location: Panama: Panama City and other both modalities (+)**; number of weekly Objective: Improve prospects for jobless youths and provinces hours worked by females (+)** in Panama disadvantaged groups. Follow-up: 9 to 20 months after the program (+)* in the insertion modality, and by Description: The program has two modalities. The first, Sample: 766 individuals divided into three females (+)** in provinces outside Panama called the insertion modality, provides short-term groups: (a) 295 controls, (b) 199 receiving City (+)* in the transition modality; labor training for low-income unemployed 18- to the insertion modality, and (c) 272 earnings in the month of the survey for 29-year-olds. Classroom training has two parts, job receiving the transition modality the insertion modality by females (+)* in readiness skills and technical training (120 and Term used: Social capital Panama (+)** 150 hours, respectively), followed by 172 hours of Note: No significant effect in the probability internship in a firm. The second modality, called the of employment at a national level. transition modality, focuses on the transition of the No significant effect on labor income. first-time job seekers with complete secondary education, providing job readiness and a longer internship (344 hours). Cost: The insertion modality costs US$611. The transition modality costs US$375. Both cost calculations include a transfer to participants of US$255. Jóvenes en Acción Beneficiaries: Young people between the ages of 18 and Authors: Attanasio, Kugler, and Meghir Economic related: Probability of 25 in the two lowest socioeconomic strata of the (2008, 2009, 2011); employment for women (+)**; number of population Type: RCT weekly hours worked by women (+)**; Objectives: Mitigate the effects of the economic crisis in Location: Colombia tenure for both genders (+)***; labor vulnerable youths by improving their employability Follow-up: 13 to 15 months after the income for women (+)***; formal conditions and the job and social insertion of program contracts for both genders (+)**; formal unemployed youths. Sample: 4,350 individuals, randomly divided wages for both genders (+) ** Description: Classroom training is offered in technical into two groups: 2,040 in treatment and Note: No significant effect when analyzing and soft skills plus internship. Program provided three 2,310 in control only treated men. months of classroom training and three months of Term used: Soft skills on-the-job training, along with a cash transfer of US$2–US$3 per day. Training was provided for almost table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results eight hours per day for diverse occupations, such as sales, secretarial work, and marketing, as well as such positions as seamstress, cooking assistant, information technology specialist, and data entry clerk. On-the-job training lasted an average of five hours per day. Cost: US$750 per person, including participants’ stipends Juventud y Empleo Beneficiaries: Youths between 16 and 29 years of age Authors: Ibarrarán and others (2014) Health related: Teenage pregnancy (−)**; who did not complete high school Type: RCT considers having very good health (+)** Objective: Improve the labor market entry of youths. Location: Dominican Republic Behavior related: Youth expectations Description: Training is provided in technical and soft Follow-up: 18–24 months after the program regarding having a better educational level skills plus internship. Courses are 225 hours (four Sample: 18–24 months after program for women ages 16 to 24 (−)*; living in a months) and are split into two parts: 75 hours of basic implementation, 10,309 randomly better neighborhood for women (+)**; or life skills training and 150 hours of technical or assigned youths were evaluated. They owning a business for women ages 16 to vocational training. Training courses are broad, were divided into two groups: (a) 5,914 in 19 (+)**; completing professional including administrative assistant, baker, hairstylist, treatment (977 did not participate); aspirations for women ages 16 to 19 (+)**; clerk, auto mechanic, and bartender. Basic skills (b) 4,395 in control (977 served as having a better life in 20 years for women training seeks to strengthen trainees’ self-esteem and replacements and participated in the ages 16 to 19 (+)***; having children with a work habits. Finally, training is followed by a program). better life for women ages 16 to 19 (+)**; two-month internship in a private sector firm. Six years after program implementation, four wealth position in 10 years for women All trainees receive RD$70 per day of class participation. groups were compared: (a) 1,901 older than age 24 (+); leadership skills for Cost: About US$330 per participant individuals who were selected for men (+)**; self-esteem for men (+)**; treatment and effectively participated, (b) conflict resolution (+)***; self-organization 262 individuals who were selected as part (+)***; persistency of effort for women (+)** of the program but dropped out, (c) 438 Economic related: Monthly earnings for individuals who were not selected in the women and for those residing in Santo lottery but participated as replacements, Domingo (+)**; employment with health and (d) 678 controls. insurance for men (+)*; employment with Terms used: Socioemotional skills/ written contract for men (+)**; duration of noncognitive skills; life-skills employment for men (+)** Note: No significant effect on labor market participation or on probability of 163 employment. table continues next page 164 Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results Galpão Aplauso Beneficiaries: Disadvantaged favela youths (a) younger Authors: Calero and others (2014) Economic related: Employment during final than age 29, (b) living in households with total Type: RCT week after four to five months (+)**; monthly income below the minimum wage, and Location: Rio de Janeiro, Brazil employment in salaried job during final (c) not involved in drugs or gang activity Follow-up: 1–2 months for the first cohort; week (+)***; monthly labor income (+)*; Objectives: Improve the socioeconomic situation and 4–5 months for the second cohort; and saving as main expenditure (+)** employability of youths by teaching them basic skills, 2–3 months for the third cohort. life skills, and vocational skills and provide placement Sample: 380 individuals: 162 in the treatment Note: No impact on personality-related traits (measured through Grit and the social services. group, 195 in the control group, and 23 and personal competencies scales, which Description: Galpão Aplauso (or the Sociocultural and nonrandomly assigned treated youths capture leadership, behavior in situations Productive Integration of At-Risk Youth Project) is an (preselected because of high vulnerability of conflict, self-esteem, abilities to relate innovative labor training program that uses and high achievement, high need, or to others, order, and empathy and expressive arts and theater as a pedagogical tool. because participating sibling enrolled). communication skills). No impacts on The program consists of a combination of vocational, Attrition was 77%. social activities or risky behavior. academic, and life skills training, delivered through a Term used: Life skills/emotional skills pedagogic method that uses arts and dance. Program duration is approximately six months, five hours a day, five days a week, delivered in three shifts—morning, afternoon, and evening. It comprises 300 hours of vocational training (mainly construction related, soldering, and woodshop); 180 hours of training in academic and basic skills, including remedial courses in both mathematics and Portuguese; and 120 hours in life skills (social harmony and socioemotional development). Cost: The average cost per youth is R$810 (US$385) a month, or R$4,680 (US$2,225) for the entire curriculum (2014 dollars). table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results Apprenticeship Beneficiaries: HIV/AIDS-vulnerable youths (poor, Authors: Cho and others (2013) Health related: HIV testing among women Training for orphans, or school dropouts) who have dropped out Type: RCT (+)*; delay in marriage and births among Vulnerable Youth of school and are younger than age 24 Location: Malawi women (+)** in Malawi Objective: Improve vulnerable youths’ labor market Follow-up: On average, 4 months after Behavior related: Happy and satisfied with outcomes and welfare. training life (+)***; perception that life has Description: Vocational training apprenticeship is Sample: 1,900 vulnerable youths: two-thirds improved during the past year (+)***; felt combined with entrepreneurial support and life skills were assigned to treatment, and one-third stressed or nervous in past month (−)* training. Participants received apprenticeship training were assigned to control. Academic related: Knowledge of trades in the selected occupations (bricklaying, car repair, Term used: Life skills (+)***; knowledge of how to calculate tailoring, and hairstyling; carpentry and joinery, profits of a business (+)***; knowledge of fabrication, and welding among others in master how to start a business (+)***; time spent craftsperson’s shops) and were encouraged to start on learning and training (+)*** their own business upon completing training. The Economic related: Time spent on traveling duration and quality of training vary by trade and to work opportunities (−)**; perception master craftsperson, as each trade has a different that being an entrepreneur is better than curriculum and length of training. Master craftsperson being an employee (+)***; perception that received one-day training programs. Additionally, half the individual is able to earn money of those trained received an individual “pep talk” that outside farming (+)*** transmitted confidence and positivity. Note: Only limited impact was found on new Cost: n/a business activities and increased earnings. A benign, short “pep-talk” is found to be effective in reinforcing some of the positive results, which could be easily replicated in other programs at low cost. Ninaweza Beneficiaries: Unemployed young women Authors: Alvarez de Azevedo, Davis, and Behavior related: Confidence levels of those living in informal settlements around Nairobi, Charles (2013) who previously were not confident (+)*** Kenya Type: RCT Academic/cognitive related: Knowledge of Objective: Improve the employability and earning Location: Kenya ICT (+)***; knowledge of life skills for capacity of young women living in the informal Follow-up: After training (8 weeks) and after treatment 1 (+)*** settlements of Nairobi. the internship and job placement support Economic related: Probability of obtaining a Description: Ninaweza is a youth employability program (6 months) job for those with treatment 1 (+)**; providing young women with technical training in Sample: 1,510 youths randomly assigned into weekly income (+)*** information and communication technology (ICT), one of three groups: (a) 350 received 165 table continues next page Table C.1  Out-of-School Program Descriptions (continued) 166 Program Evaluation Name Description Author/description Results training in life skills, work experience through treatment 1, which consisted of ICT and Note: Although both treatment groups faced internships, and job placement support. life skills, along with on-the-job experience statistically significant impacts on Cost: n/a through internships and job placement knowledge of ICT and weekly income, support; (b) 350 received treatment 2, treatment 1 had greater impacts with which included everything in treatment regard to ICT knowledge and smaller 1 except for the life skills training; and impacts on weekly income when (c) 810 acted as a control group and compared with treatment 2. received none of the program. Term used: Life skills training Jordan New Beneficiaries: Women graduating from eight public Authors: Groh and others (2012) Behavior related: For group with voucher or Opportunities for community colleges Type: RCT both, life ladder (+)***; mobility index Women (Jordan Objective: Assist female community college graduates Location: Jordan (+)***; life ladder for the future for those NOW) in finding employment. Follow-up: 6 and 14 months after the start of assigned to training (+)**; mental health Description: The project tested two programs: the programs for those assigned to training (+)* (a) a subsidy paid to employers equivalent to the Sample: 1,347 women graduates randomly Economic related: For job voucher group: minimum wage for up to six months if they hired the divided into four groups: (a) 299 were employment (+)***; employed and graduate and (b) 45 hours of employability skills given a voucher that would pay employers registered for social security (+)**; ever training. The training course was 45 hours over nine a subsidy equivalent to the minimum employed (+)***; months employed since days (5 hours per day), with a maximum of 30 wage for up to six months if they hired the graduation (+)***; work income (+)***; participants in each training group. The course graduate, (b) 300 were invited to attend 45 hours worked in past week (+)***; soft covered effective communication and business hours of employability skills training skills training’s effect on employment writing skills (such as making presentations and designed to provide them with the soft outside the capital (+) writing business reports and different types of skills employers say graduates often lack, Note: The impacts on the number of hours correspondence), team-building and teamwork skills (c) 299 were offered both programs, and worked during the previous week (such as characteristics of a successful team, how to (d) 449 formed the control group. occurred mostly in the nonformal sector. work in different roles within a team), time Term used: Soft skills The effect was smaller and not statistically management, positive thinking and how to use it in significant four months after the voucher business situations, excellence in providing customer period. Finally, the impacts were stronger service, and résumé and interviewing skills. outside the capital city. Cost: The cost of the voucher was JD 150 each month, for six months. The cost of the soft skills training was US$400 per assigned graduate. table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results entra21 Beneficiaries: Unemployed youths (ages 18 to 30) who Authors: Alzúa, Cruces, and Lopez Erazo Economic related: Formal employment for have finished high school and have a total family (2013) men (+)*** and younger participants income below the poverty line Type: RCT [18–24] (+)**; earnings (+)**; credit in Objective: Facilitate job placement for unemployed youth. Location: Córdoba, Argentina good standing for men (+)*** and Description: The program has several components: Follow-up: 2 years younger participants [18–24] (+)***; (a) 100 hours of in-class training for an oficio (job), Sample: 407 individuals: 220 randomly welfare dependency [Asignación which results from actual employers’ demands; (b) 64 assigned to treatment (only 178 Universal por Hijo] for women hours of training on ICTs and life skills; (c) 704 hours of participated in the program) and ages 18–24 (−)** internship; and (d) 16 hours that varied from basic 187 as controls skills to extra in-class technical training according to Term used: Life skills each type of course. Cost: About US$1,722 per participant Economic Beneficiaries: Young women who (a) were ages 16 to 27, Authors: Adoho and others (2014) Behavior related: Self-confidence (+)***; Empowerment of (b) possessed basic literacy and numeracy skills, (c) were Type: RCT anxiety about circumstances and the Adolescent Girls not enrolled in school within several months before Location: Liberia future (−)**; gender norms (+)*** and Young Women program initiation, and (d) resided in one of nine target Follow-up: 6 months after the program Economic related: Participation in (EPAG) program communities in and around Monrovia, Liberia. Sample: 2,042 young women randomly income-generating activities (+)*** [for Objective: Increase employment and income of young assigned to one of two treatments: (a) JS trainees (+)**; for BDS trainees (+)***]; Liberian women by providing livelihood and life skills 1,273 assigned to the first round of wage employment for JS trainees (+)***; training and facilitating their transition to productive training and (b) 769 assigned to a control self-employment for BDS trainees (+)***; work. group (who would participate in the intensive employment (+)***; earnings Description: The program combined six months of second round of training). (+)*** [especially for BDS trainees]; access classroom-based technical and life skills training, with Term used: Life skills to money (+)***; food security (+); savings a focus on skills with high market demand, followed (+)*** by six months of follow-up support to acquire wage Note: No impact on fertility or sexual employment or start a business. Upon recruitment, behavior. the participants were assigned to a job skills (JS) track or a business development services (BDS) track. The JS track provided training in six areas: (a) hospitality, (b) professional cleaning/waste management, (c) office/computer skills, (d) professional house/office painting, (e) security guard services, and (f) professional driving. The BDS training taught young 167 table continues next page 168 Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results women how to identify microenterprise opportunities on the basis of an assessment of market needs and how to grow and manage any existing businesses they already had. EPAG trainees were given incentives to participate and to make the most of their training: (a) they signed trainee commitment forms at the start of training, (b) they were paid small stipends and a completion bonus contingent upon attendance, (c) they were offered free child care at every training site, (d) they were assisted in opening a savings account at a local bank for their stipend money, and (e) they were formed into small groups or “EPAG teams,” each with a coach or mentor, to foster support networks and boost attendance. Cost: The unit cost of training in round 1 was about US$1,200 for the BDS track and US$1,650 for the JS track. Questscope Beneficiaries: Out-of-school youth typically in Authors: Morton and Montgomery (2012) Behavior related: Conduct problems (−)**; Non-Formal low-income communities (13- to 18-year-old males Type: RCT prosocial behavior in high-quality Education program and 13- to 21-year-old females). The program also Location: Jordan implementation centers with males (+)**; enrolls large numbers of Iraqi refugees. Follow-up: 4 months after the program local adult connectedness and Objective: Integrate school dropouts into society started (at the end of the first part of the hyperactivity in low-quality (education, job, and so forth). program cycle) implementation centers with males (−)**; Description: Questscope Non-Formal Education is a Sample: 127 Jordanian youths (male and social support of friends for high- two-year program that consists of three eight-month female) with a mean age of 15.91 years, attendance youths (+)* education cycles based on participatory randomly assigned to (a) an Note: No significant effects found for methodology. At the end of the program, graduates empowerment-based nonformal developmental assets (self-efficacy or receive a 10th-grade alternative certificate that education program (67 youths) or social skills). enables them to participate in vocational training and (b) a wait-list comparison (60 youths) receive government business loans. Term used: Social skills The program takes place five days a week outside traditional school hours and has two- to three-hour sessions. table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results Participants need to attend at least two sessions per week in order to receive their certificate. Trained adult facilitators deliver sessions with the following content: educationally, each session includes dialogue-based learning activities (for example, related to literacy, math, science, religion, and English). Cost: n/a Job Corps Beneficiaries: Disadvantaged youth. Applicants must Authors: Schochet, Burghardt, and Health related: Health insurance (+)* meet 11 criteria to be eligible: (a) be 16 to 24 years of McConnell (2008) Behavior related: Arrests and conviction age; (b) be a legal U.S. resident; (c) be economically Type: RCT rates (−)* disadvantaged (receiving welfare or food stamps or Location: United States (nationally Academic/cognitive related: Percentage having income less than 70% of Department of representative study) ever enrolled in an education or training Labor’s “lower living standards income level”); (d) live Follow-up: Survey data collected 4 years after program during the 48 months after in an environment characterized by a disruptive random assignment; tax information random assignment (+)*; average hours home life, high crime rates, or limited job collected 9 years after random assignment ever in education or training (+)*; degrees, opportunities; (e) need additional education, training, Sample: Nearly 81,000 youth randomly diplomas, and certificates received (+)*; or job skills; (f) be free of serious behavioral problems; assigned: (a) 9,409 receiving Job Corps GED certificate (+)*; high school diploma (g) have a clean health history; (h) have an adequate training; (b) 5,977 in control group, not (−)*; vocational, technical, or trade child care plan (for those with children); (i) have receiving Job Corps training but able to certificate (+)* registered with the Selective Service Board (if enroll in other available programs (about Economic related: Percentage employed in applicable); (j) have parental consent (for minors); and 70% of the controls enrolled in other previous three months (+)*; hourly wage (k) be judged to have the capability and aspirations to training programs in the four-year period (+)*; paid vacation (+)*; retirement or participate in Job Corps. after random assignment); and (c) program pension benefits (+)* Objective: Help youths become more responsible, nonresearch group employable, and productive citizens. Term used: Social skills training Description: Participants receive intensive vocational training (in more than 75 trades), academic education (which aims to alleviate deficits in reading, math, and writing skills and to provide a GED certificate) and a wide range of other services, including counseling, social skills training, and health education. table continues next page 169 Table C.1  Out-of-School Program Descriptions (continued) 170 Program Evaluation Name Description Author/description Results Most participants reside at a center while training and receive meals and health and dental care and can participate in student government and recreation activities. Job Corps provides placement services to help participants find jobs or pursue additional training. Cost: About US$16,500 per participant (1995 dollars) Empowerment and Beneficiaries: Adolescent girls ages 14 to 20 Authors: Bandiera and others (2014) Health related: Teenage pregnancy (−)**; Livelihood for Objective: Empower Ugandan women along economic Type: RCT adolescent girls reporting having sex Adolescents (ELA) and reproductive dimensions. Location: Uganda unwillingly (−)**; knowledge about program Description: The program provides (a) vocational skills Follow-up: 2 years after program initiated pregnancy (+)** and HIV (+)***; if sexually to enable adolescent girls to start small-scale, Sample: 4,800 girls living in 100 communities: active, use of condom (+)*** income-generating activities, including hairdressing, 50 communities assigned to control and Behavior related: Gender empowerment tailoring, computing, agriculture, poultry rearing, 50 to treatment index (+)***; early entry into marriage/ and small trade operations; and (b) life skills, including Term used: Life skills cohabitation (−)***; self-reported sexual and reproductive health, menstruation and anxieties about finding a good job in menstrual disorders, pregnancy, sexually transmitted adulthood (−)***; perception of suitable infections, HIV/AIDS awareness, family planning, ages for marriage (+)***; preferred rape, management skills, negotiation and conflict number of children (−)***; perception of resolution, leadership among adolescents, bride suitable age for pregnancy (+)***; girls’ price, child marriage, and violence against women. expectations about their own daughters’ In addition, the clubs also host popular recreational age at marriage (+)*** activities, such as reading, staging dramas, singing, Academic/cognitive related: Plan to start/ dancing, and playing games. return to school for girls who dropped out The intervention is delivered from designated (+)*; number of hours spent on going to ”adolescent development clubs” (a fixed meeting and attending school, homework, or place in each community) rather than in schools. study per week, for enrolled girls (+)* Clubs are typically open five afternoons per week and Economic related: Entrepreneurial timed so that girls enrolled full-time in school can ability (+) ***; engagement in income- attend. Club activities are led by a trained female generating activity (+)***; self-employed mentor from the community. (+)***; monthly consumption Cost: US$85 per participating adolescent girl (2008 expenditure (+)*** dollars) table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results JOBS program Beneficiaries: Individuals who had recently lost a job Authors: Vinokur and others (2000) Health related: Depressive symptoms (−)*; and were unemployed for no longer than 13 weeks Type: RCT likelihood of experiencing a major Objectives: Prevent and reduce negative effects on Location: United States depressive episode in the past year (−)* mental health associated with unemployment and Follow-up: 2 years Behavior related: Role and emotional job-seeking stress, while promoting high-quality Sample: 1,801 individuals who had recently functioning (+)* reemployment. lost a job and were unemployed for no Economic related: Reemployment (+)**; Description: This job-search skill enhancement seminar longer than 13 weeks hours working per week (+)**; months is carried out in five four-hour sessions conducted in Term used: mental health working more than 35 hours (+)*; monthly the morning for one week. The program teaches income (+) ** participants effective strategies for finding and Note: No effect on quality of job as assessed obtaining suitable employment, as well as for by wage rate, stability, or fringe benefits. anticipating and dealing with the inevitable setbacks they will encounter. The program also incorporates elements to increase participants’ self-esteem, sense of control, and job search self-efficacy. Cost: n/a National Guard Beneficiaries: Young people between the ages of 16 and Authors: Bloom, Gardenhire-Crooks, and Health related: Overweight 36 months after Youth ChalleNGe 18 who have dropped out of (or have been expelled Mandsager (2009); Millenky, Bloom, and program completion (−)**; very good/ Program from) school and are unemployed, drug free, and not Dillon (2010); Millenky and others (2011); excellent health 9 months after program heavily involved with the justice system Perez-Arce and others (2012) completion (+)***; obesity 9 months after Objective: Reclaim the lives of at-risk youth who have Type: RCT program completion (−)**; always uses dropped out. Location: United States birth control 36 months after program Description: The 17-month program is divided into Follow-up: 3 years after completion (−)**; ever used other illegal three phases: (a) pre-ChalleNGe, a two-week Sample: 1,173 young people randomly drugs 36 months after program orientation and assessment period (candidates are assigned to intervention (722) or completion (+)* introduced to the program’s rules and expectations; control (451) Behavior related: Collective civic efficacy learn military bearing, discipline, and teamwork; and Term used: life-coping skills (+)**; conventional citizen scale (+)*; begin physical fitness training); (b) a 20-week arrest, conviction, and incarceration rates residential phase structured around leadership/ 9 months after program completion (−)**; followership, responsible citizenship, service to the conviction and violent incidents 21 community, life-coping skills, physical fitness, health months after program completion (−)**; and hygiene, job skills, and academic excellence; and property incidents 21 months after program completion (−)*** 171 table continues next page Table C.1  Out-of-School Program Descriptions (continued) 172 Program Evaluation Name Description Author/description Results (c) a one-year postresidential phase featuring a Academic/cognitive related: Earned GED mentoring program. certificate (+)***; earned college credits During the first two phases, participants live at (+)***; vocational training after 21 and 36 the program site, often on a military base. months(+)**; currently taking college The environment is “quasi-military,” though there courses (+)**; involved in job training after are no requirements for military service. 9 months (+)*; attending high school after Cost: US$11,630 per admitted student 9 and 21 months (+)***; preparing for GED after 9 and 21 months (+)** Economic related: Employment 9 months after (+)***, 21 months after (+)*, 36 months after program completion (+)**; weekly earnings 21 months after (+)*** and 36 months after program completion (+)*; earnings over the past 12 months, 36 months after program completion (+)***; number of months employed 36 months after program completion(+)***; living at parents’ home (−)*; living at own home or apartment (+)** Note: No impacts on self-reported arrests/ convictions, self-reported delinquency 36 months after program completion. Some negative impacts on health outcomes. No impacts on leadership and life-coping skills. Little impact on civic engagements. Year Up Beneficiaries: Low-income young adults, ages 18 to 24 Authors: Roder and Elliot (2011) Economic related: Earnings (+)*; full-time Objective: Prepare vulnerable adults for positions with Type: RCT employment (+)*; working in targeted good wages and career advancement opportunities Location: United States fields (+) in the information technology and investment Follow-up: 24 to 30 months after random Note: No effect on college attendance. operations fields. assignment Description: The program provides (a) six months of Sample: 164 young adults randomly assigned technical skills training that is regularly updated to for treatment (120) or control (44) meet the needs of the program’s corporate partners Term used: Professional skills table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results (includes basic training on operating systems and word processing, spreadsheet, and presentation software, and so forth), (b) classes in business writing and communications, (c) instruction on professional skills through both classroom training and enforcement of a performance contract on the rules of professional behavior (students must maintain high attendance rates, be on time, and complete assignments; additional skills taught include how to present oneself [dress and body language], interact with coworkers, make small talk, engage in social networking, and manage conflict), (d) support and guidance for personal or programmatic issues, and (e) assistance with job search process and/or college enrollment upon completion. Cost: n/a Violence prevention programs Program H Beneficiaries: Young men ages 14 to 25 in three Authors: Pulerwitz and others (2006) Health related: Sexually transmitted low-income communities (favelas) Type: Quasi-experimental infection symptoms (−)*** Objective: Help young men question traditional gender Location: Rio de Janeiro, Brazil Behavior related: Support of inequitable norms and behavior, including violence against women. Follow-up: 12 months after the end of the norms (−)** Description: The program includes educational activities activities Note: The effect was equally great in both and community campaigns. One intervention Sample: 780 men in low-income intervention groups, suggesting group component was six months of interactive group communities were divided into three education was most important. education sessions for young men led by adult male groups: (a) 258 received the education facilitators. The other was a community-wide “lifestyle” intervention only, (b) 250 received social marketing campaign to promote condom use, education and the community lifestyle using gender-equitable messages that also reinforced campaign, and (c) 272 received a delayed those promoted in the group education sessions. intervention after a control period. The education sessions took place weekly for about 2 Term used: Attitudes and behaviors hours each over approximately six months, for a total of about 28 hours. Cost: n/a 173 table continues next page Table C.1  Out-of-School Program Descriptions (continued) 174 Program Evaluation Name Description Author/description Results Involucrando Beneficiaries: Young men between ages 15 and 19 Authors: Obach, Sadler, and Aguayo (2011) Behavior related: Attitudes about gender Hombres Jóvenes Objective: Prevent violence through socioeducational Type: Quasi-experimental design, pre- and equality (+)***; violence toward partner en el Fin de la workshops. posttest with control index (−)***; stalking others index (−)** Violencia de Description: The school- and health-sector-based Location: Chile Note: No impact on the use of contraceptive Género intervention included educational workshops held for Follow-up: At the end of the workshop methods. young men (through the public health sector and in (3 to 5 months) public schools) on the prevention of violence against Sample: The sample was divided into two women, alternatives to violence, and gender equality. groups: (a) intervention, which had 260 School programs lasted from three to five months. individuals in the pretest and 153 in the Cost: n/a posttest; and (b) control, which had 250 pretest and 150 posttest participants. Term used: Social skills Coping with work Team Awareness Beneficiaries: Individuals ages 18 to 55+ Authors: Patterson, Bennett, and Wiitala Health related: Use of tobacco to unwind in Objective: Decrease substance abuse resulting from (2005) women in Choices (+)*; use of over-the- stress at work. Type: RCT counter drugs in women receiving Choices Description: The study examined two prevention Location: United States: southwestern urban (−)**; use of over-the-counter drugs in programs for small business: and suburban communities women receiving Team (+)** 1. Team Awareness-SB (Team), which is a shortened Follow-up: 2 weeks after the training Behavior related: Positive unwinding version (lasting four hours) of Team Awareness, which Sample: 530 employees of small businesses behaviors for Team (+)*** and Choices seeks to decrease substance abuse by addressing (<500) in industries identified as high risk (+)** (especially in men) stress, the social use of alcohol by coworkers, and for alcohol and drug abuse (construction, Note: Training did not significantly change ways to get help for problems. The training was small-aircraft pilot, maintenance, bus substance unwinding. delivered to groups of 6 to 28 individuals and driver, materials moving, hotels, and other included information, games, role playing, and other services) were randomly assigned to three activities on substance abuse prevention, as well as groups: (a) 194 into Team, (b) 124 into individual employee roles in prevention, risks, and Choices, and (c) 212 into control. strengths in the workplace; communication; and peer Term used: Emotional and social health referral skills. 2. Choices in Health Promotion (Choices) is derived from Healthy Workplace, which emphasizes the benefits of decreasing substance use and healthy alternatives to substance use. Choices combines elements from table continues next page Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results Healthy Workplace and Team Awareness and a program in time management and spiritual health into a customized program. Participants set personal goals based on the customized program (it starts with a one-hour face-to-face interview), which helps them create a sense of optimism and future-mindedness. The topics covered include stress management, tobacco, active lifestyle, healthy eating, parenting, time management and spiritual health, safety in the workplace, information on alcohol and moderate drinking, prescription drug use, and team awareness. Then, a four-hour program is developed and carried out with 6 to 39 individuals. Both programs promote social interaction, bonding, and shared positive experiences among training participants. Cost: n/a Returning children to school Kingston YMCA Youth Beneficiaries: At-risk, low-income, inner-city male Authors: Guerra and others (2010) Behavior related: Aggressive behavior (−)**; Development adolescents ages 14 to 17 who are not attending Type: Post hoc analysis aggressive propensity (−)*** Programme school because of academic or social problems, Location: Jamaica typically aggressive and defiant behavior Follow-up: 6 months (RCT) and 5 years with Objective: Provide at-risk, low-income males with matching intensive remedial education, social skills training, Sample: Two samples: (a) 180 males, 125 of and personal development. whom were currently enrolled in the Description: Participants attend the program daily (in program for at least 6 months and 55 of lieu of regular school) until they have attained whom were in a wait-list control group; proficiency on the grade 9 achievement test and are and (b) 116 males, including 56 program returned to regular schools. The typical length of graduates and a matched sample of program participation is four years. Participants 60 community controls receive comprehensive services, including daily Term used: Social/life skills table continues next page 175 176 Table C.1  Out-of-School Program Descriptions (continued) Program Evaluation Name Description Author/description Results supervision (from 9:00 a.m. to 4:00 p.m.), remedial education (20 students per teacher), vocational training, social/life skills instruction, recreation, and positive behavior management. Although no specific social skills curricula are used, the program relies heavily on counseling, guidance, and authoritative discipline (emphasizing rewards for positive behavior), and providing positive male and female role models. Cost: n/a Decrease sexual risk Joven Noble Beneficiaries: Latino male youths ages 10 to 24 Authors: Tello and others (2010) Health related: Sexual activity (+)**; alcohol Objectives: Promote positive youth development, Type: Quasi-experimental or drug consumption before sexual support, and leadership, while preventing a number Location: United States intercourse (−)***; report on having sex of risk-related sexual behaviors within a cultural Follow-up: After the sessions for money, drugs, or other things (−)***; context Sample: 683 adolescent Latino males ages HIV risk knowledge (+)*** Description: This 10-week curriculum aimed at 13 to 17: 64% came from middle school Behavior related: Attitudes toward promoting the character development of young men, and high school, whereas 36% were abstinence (+)***; perceived risk (+)***; while targeting the reduction and prevention of recruited from probation and community- Children and Adolescent Prevention Scale unwanted or unplanned pregnancies, substance based programs. These individuals were (CAPS) talk (+)***; CAPS cool sex (+)***; abuse, and community violence, and increasing the unemployed and looking for work. cultural esteem (−)***. ability of youths to act in a responsible and respectful Term used: Character development Other: Cultural knowledge and beliefs; way in reference to their relationship. cultural esteem Cost: n/a Note: AIDS = acquired immune deficiency syndrome; GED = General Educational Development; HIV = human immunodeficiency virus; RCT = randomized control trial; n/a = not available. Significance levels of results, positive (+) or negative (–), are reported at 1 percent (*), 5 percent (**), and 10 percent (***). Out-of-School Program Descriptions 177 Table C.2 Out-of-School Programs by Component On-the-job Stipend/ Classroom Counseling/ training/internship/ Program subsidy training mentoring apprenticeship Apprenticeship Training for Vulnerable Youth in Malawi Ninaweza Jóvenes en Acción Juventud y Empleo Procajoven entra21 Jordan NOW Galpão Aplauso EPAG program Job Corps National Guard Youth ChalleNGe Program Year Up Kingston YMCA Youth Development Programme Questscope Non-Formal Education program Empowerment and Livelihood for Adolescents (ELA) program JOBS program Program H Involucrando Hombres Jóvenes en el Fin de la Violencia de Género Team Awareness Joven Noble 3 20 8 6 Note: Shading indicates that the program includes that particular component EPAG = Economic Empowerment of Adolescent Girls and Young Women; NOW = New Opportunities for Women; YMCA = Young Men’s Christian Association. References Adoho, Franck, Shubha Chakravarty, Dala T. Korkoyah, Mattias Lundberg, and Afia Tasneem. 2014. “The Impact of an Adolescent Girls Employment Program: The EPAG Project in Liberia.” Policy Research Working Paper 6832, World Bank, Washington, DC. Alvarez de Azevedo, Thomaz, Jeff Davis, and Munene Charles. 2013. “Testing What Works in Youth Employment: Evaluating Kenya’s Ninaweza Program.” Global Partnership for Youth Employment, Washington, DC. Alzúa, María Laura, Guillermo Cruces, and Carolina Lopez Erazo. 2013. “Youth Training Programs beyond Employment: Evidence from a Randomized Controlled Trial.” https://www2.unine.ch/files/content/sites/irene/files/shared/documents/seminaires​ /­Alzua.pdf. Attanasio, Orazio, Adriana Kugler, and Costas Meghir. 2008. “Training Disadvantaged Youth in Latin America: Evidence from a Randomized Trial.” NBER Working Paper13931, National Bureau of Economic Research, Cambridge, MA. ———. 2009. “Subsidizing Vocational Training for Disadvantaged Youth in Developing Countries: Evidence from a Randomized Trial.” IZA Discussion Paper 4251, Institute for the Study of Labor, Bonn. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 178 Out-of-School Program Descriptions ———. 2011. “Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial.” American Economic Journal: Applied Economics 3 (3): 188–220. Bandiera, Oriana, Niklas Buehren, Robin Burgess, Markus Goldstein, Selim Gulesci, Imran Rasul, and Munshi Sulaiman. 2014. “Women’s Empowerment in Action: Evidence from a Randomized Control Trial in Africa.” Suntory and Toyota International Centres for Economics and Related Disciplines, London School of Economics. Bloom, Dan, Alissa Gardenhire-Crooks, and Conrad Mandsager. 2009. “Reengaging High School Dropouts: Early Results of the National Guard Youth ChalleNGe Program Evaluation.” Report, MDRC, New York. Calero, Carla, Carlos Henrique Corseuil, Veronica Gonzales, Jochen Kluve, and Yuri Soares. 2014. “Can Arts-Based Interventions Enhance Labor Market Outcomes among Youth? Evidence from a Randomized Trial in Rio de Janeiro.” IZA Discussion Paper 8210, Institute for the Study of Labor, Bonn. Cho, Yoonyoung, Davie Kalomba, Mushfiq Mobarak, and Victor Orozco. 2013. “Gender Drop-Out Differences in the Effects of Vocational Training: Constraints on Women and ­ Behavior.” Policy Research Working Paper 6545, World Bank, Washington, DC. Groh, Matthew, Nandini Krishnan, David McKenzie, and Tara Vishwanath. 2012. “Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment in Jordan.” Policy Research Working Paper 6141, World Bank, Washington, DC. Guerra, Nancy G., Kirk Williams, Julie Meeks-Gardner, and Ian Walker. 2010. “The Kingston YMCA Youth Development Programme: Impact on Violence among At-Risk Youth in Jamaica.” Caribbean Child Development Centre, University of the West Indies, Kingston, Jamaica. Ibarrarán, Pablo, Laura Ripani, Bibliana Taboada, Juan Miguel Villa, and Brigida Garcia. 2014. “Life Skills, Employability and Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design.” IZA Journal of Labor and Development 3 (1): 1–24. Ibarrarán, Pablo, and David Rosas-Shady. 2006. “Impact Evaluation of the Job Training Component (PROCAJOVEN) of the Assistance Program for the Building of a Training and Employment System in Panama (PN0125).” Report, Inter-American Development Bank, Washington, DC. Millenky, Megan, Dan Bloom, and Colleen Dillon. 2010. “Making the Transition: Interim Results of the National Guard Youth ChalleNGe Evaluation.” Report, MDRC, New York. Millenky, Megan, Dan Bloom, Sara Muller-Ravett, and Joseph Broadus. 2011. “Staying on Course: Three-Year Results of the National Guard Youth ChalleNGe Evaluation.” Report, MDRC, New York. Morton, Matthew H., and Paul Montgomery. 2012. “Empowerment-Based Non-Formal Education for Arab Youth: A Pilot Randomized Trial.” Children and Youth Services Review 34 (2): 417–25. Obach, Alexandra, Michelle Sadler, and Francisco Aguayo. 2011. “Resultados del proyecto involucrando hombres jóvenes en el fin de la Violencia de Género: Intervención multipaís con evaluación de impacto—Caso Chileno.” CulturaSalud/EME, Santiago, ­ Chile. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Out-of-School Program Descriptions 179 Patterson, Camille R., Joel B. Bennett, and Wyndy Wiitala. 2005. “Healthy and Unhealthy Stress Unwinding: Promoting Health in Small Businesses.” Journal of Business and Psychology 20 (2): 221–47. Perez-Arce, Francisco, Louay Constant, David S. Loughran, and Lynn A. Karoly. 2012. “A Cost–Benefit Analysis of the National Guard Youth ChalleNGe Program.” RAND Research Monograph TR1193, RAND Corporation, Santa Monica, CA. Pulerwitz, Julie, Gary Barker, Márcio Segundo, and Marcos Nascimento. 2006. “Promoting More Gender-Equitable Norms and Behaviors among Young Men as an HIV/AIDS Prevention Strategy.” Horizons final report, Population Council, Washington, DC. Roder, Anne, and Mark Elliott. 2011. “A Promising Start: Year Up’s Initial Impacts on Low- Income Young Adults’ Careers.” Economic Mobility Corporation, New York: Schochet, Peter Z., John Burghardt, and Sheena McConnell. 2008. “Does Job Corps Work? Impact Findings from the National Job Corps Study.” American Economic Review 98 (5): 1864–86. Tello, Jerry, Richard C. Cervantes, David Cordova, and Susana M. Santos. 2010. “Joven Noble: Evaluation of a Culturally Focused Youth Development Program.” Journal of Community Psychology 38 (6): 799–811. Vinokur, Amiram D., Yaacov Schul, Jukka Vuori, and Richard H. Price. 2000. “Two Years after a Job Loss: Long-Term Impact of the JOBS Program on Reemployment and Mental Health.” Journal of Occupational Health Psychology 5 (1): 32–47. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Environmental Benefits Statement The World Bank Group is committed to reducing its environmental footprint. In support of this commitment, the Publishing and Knowledge Division leverages electronic publishing options and print-on-demand technology, which is located in regional hubs worldwide. Together, these initiatives enable print runs to be lowered and shipping distances decreased, resulting in reduced paper ­consumption, chemical use, greenhouse gas emissions, and waste. The Publishing and Knowledge Division follows the recommended standards for paper use set by the Green Press Initiative. The majority of our books are printed on Forest Stewardship Council (FSC)–certified paper, with nearly all containing 50–100 percent recycled content. The recycled fiber in our book paper is either unbleached or bleached using totally chlorine-free (TCF), processed chlorine-free (PCF), or enhanced elemental chlorine-free (EECF) processes. More information about the Bank’s environmental philosophy can be found at http://www.worldbank.org/corporateresponsibility. Taking Stock of Programs to Develop Socioemotional Skills http://dx.doi.org/10.1596/978-1-4648-0872-2 Possessing a range of cognitive, socioemotional, and technical skills is important for individuals to maximize their chances of success in many aspects of life. In particular, a growing body of research highlights the effects that socioemotional skills have on a variety of outcomes, from wages and academic performance to health. Programs to help participants develop such skills continue to expand in both developed and developing countries, targeting individuals of almost all ages and life stages. However, the characteristics that make some programs more successful than others—or even what types of outcomes programs use to measure “success”—are less clear. Taking Stock of Programs to Develop Socioemotional Skills: A Systematic Review of Program Evidence aims to fill this knowledge gap through a systematic review of programs that seek to develop socioemotional skills. This analysis rigorously examines a diverse range of more than 80 programs, including programs for toddlers and young children before primary school, programs for students enrolled in formal education, and programs targeted at the out-of-school population. The book develops a conceptual framework that helps to identify the program characteristics and participants’ profiles associated with a range of program outcomes. The book’s findings highlight characteristics and impacts of successful (and less successful) programs. ISBN 978-1-4648-0872-2 SKU 210872