Policy Research Working Paper 8821 Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia Alberto Chong Marco Gonzalez-Navarro Dean Karlan Martín Valdivia Development Economics Vice Presidency Strategy and Operations Team April 2019 Policy Research Working Paper 8821 Abstract This study reports results from a randomized evaluation of data provide no evidence of spillovers to control classrooms a mandatory six-month Internet-based sexual education within treatment schools, and it finds that treatment effects course implemented across public junior high schools in 21 are enhanced when a larger share of a student’s friends also Colombian cities. Six months after finishing the course, the takes the course. The low cost of the online course along study finds a 0.4 standard deviation improvement in knowl- with the effectiveness the study documents suggests this edge, a 0.2 standard deviation improvement in attitudes, technology is a viable alternative for improving sexual edu- and a 55 percent increase in the likelihood of redeeming cation in middle-income countries.. vouchers for condoms as a result of taking the course. The This paper is a product of the Strategy and Operations Team, Development Economics Vice Presidency. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at achong6@gsu.edu. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia Alberto Chong, Marco Gonzalez-Navarro, Dean Karlan, and Martín Valdivia JEL classification: O12, I2, I1 Keywords: information technologies, Internet, sex education, teenagers, field experiment 1. Introduction Providing effective sexual education to teenagers is a pervasive worldwide policy challenge. In many countries, conservative norms lead to restricted sexual education curricula. Deficient sexual education partially explains the high levels of sexually transmitted diseases and teenage pregnancies that are observed in many of the world’s developing countries (WHO 2004). Making matters more consequential for youth, in poor countries there is an acute lack of resources, health system capabilities, and best practices to treat sexually-transmitted diseases (Fortson 2009). Alberto Chong (corresponding author) is a professor at Georgia State University (email: achong6@gsu.edu) and Uni- versidad del Pacifico; Marco Gonzalez-Navarro is an assistant professor at University of California, Berkeley (email: marcog@berkeley.edu); Dean Karlan is a professor at Northwestern University (email: karlan@northwestern.edu); Martín Valdivia is a senior researcher at Group for the Analysis of Development (GRADE–Peru). (email: jvaldivi@grade.org.pe). This research would not have been possible without the sustained support of the Profamilia staff in charge of implement- ing and monitoring the intervention, especially German López and Lyda Díaz. The authors also recognize valuable research assistance from Angela García, César Mora, Juan Pablo Ocampo, Martin Sweeney, and project leadership by Beniamino Savonitto and Rachel Strohm. All errors and omissions are ours. The authors thank the Interamerican Development Bank for funding. All opinions are those of the authors, and not of the participating organizations or donors. The authors retained full intellectual freedom to report the results throughout the study. The study received Institutional Review Board (IRB) approval from Innovations for Poverty Action (IPA) under protocol 117.09June-003. 2 Chong et al. The objective of this research is to test whether, in a predominantly Catholic, middle-income country, information technologies in a school setting can help overcome sexual education–related informational barriers faced by teenagers. Naturally, evaluations of sexual health curricula have been done before. Review papers by Kirby, Laris, and Rolleri (2007); Chin et al. (2012); Fonner et al. (2014), and Goesling et al. (2014) conclude that most comprehensive sexual education programs that have been evaluated rigorously are effective at improving knowledge, attitudes, and self-reported behaviors. Fonner et al. (2014) in particular focus on poor and middle-income country studies and reach the same conclusion. However, this large literature focuses on facilitator-led interventions, which implies that it is difficult to ensure consistent delivery in face-to-face interventions when scaling up. Furthermore, many teachers block this type of education because of dis- comfort discussing sex-related decisions with teens. As a result of these factors, elaborate interventions struggle to translate encouraging results from controlled trials into larger settings (Collins et al. 2002). Information and communication technologies (ICTs) for sexual education hence hold promise to im- prove school-based sexual education along three dimensions: reducing loss of effectiveness when scaling up, reducing costs of implementation, and overcoming educator reluctance to present sexual education material. The study implements a large randomized evaluation of a comprehensive Internet-based sexual health education course geared to adolescents in Colombian public schools. Assignment to treatment is randomized both across 69 schools as well as within schools at the classroom level (in order to measure information spillovers within the school). The course covers topics that range from sexual rights to the use of contraceptives. It was implemented during a full academic semester in close collaboration with public schools as part of the students’ obligatory curricula. The sample consisted of 4,599 students enrolled in 138 ninth-grade classrooms from 69 public schools spread across 21 major Colombian cities. The control group received the status quo: brief biology class coverage or sporadic visits by health personnel.1 To measure differences in outcomes after treatment, the study uses three sources of data: a follow-up survey one week after course completion to measure short-term changes in knowledge and attitudes, a second follow-up survey six months after the course to measure the same outcomes in the longer term; and redemption from local health clinics of a voucher for condoms. The main focus of the study is on knowledge and attitude indicators since these are the main outcomes of interest in the literature studying young adolescents who, for the most part, have not had sex. Furthermore, these two factors have been shown to be the strongest protective factors in preventing sexually transmitted infections (STIs), hu- man immunodeficiency virus (HIV), and pregnancy among teens (Kirby, Lepore, and Ryan 2007). Recent research has also documented the important role that social norms play in responsible sexual behavior (Munshi and Myaux 2005; Ashraf, Field, and Lee 2014). By changing knowledge and attitudes in youth attending school, sexual education can ultimately play a fundamental role in achieving desirable aggregate changes in sexual behavior. The condom voucher makes it possible to avoid obvious problems with self-reported sexual behavior data, and follows O’Donnell et al. (1995) and Thornton (2008). Increased condom demand and distribu- tion is a common policy target and tool as part of efforts to reduce sexually transmitted infections and teenage pregnancy. The researchers view this as a key part of the design of the study, given the concern in this literature regarding the relationship between knowledge and attitude outcomes and actual behav- ior (Ross et al. 2007), and the challenge of gathering accurate responses to sensitive questions on sexual practices. 1 Control schools were asked not to modify their current sexual education because of their participation in the study. Assignment to the course causes a 0.38 standard deviation improvement in an index of sexual knowledge. The effects are practically identical at completion of the course and six months later, suggesting little decay in knowledge. It also generates a 0.24 improvement in an index of sexual attitudes at course completion, and a 0.17 standard deviation improvement after six months. These aggregate improvements in sexual knowledge and attitudes suggest strong effects over time and across almost all of the underlying subindices.2 The sexual knowledge and attitude results are corroborated by the redemption rate of condom vouchers. The data show that 28 percent of treatment students redeem condom vouchers, compared to 18 percent of control students, representing a 55 percent increase in redemption. The data do not provide evidence that that the online course increased sexual activity, which together with the increased condom demand result suggests that the course results in improved sexual practice or at least safe-sex preparedness. The results do not provide any indication of classroom-level spillovers within the same school, which suggests for policy purposes that programs should target entire cohorts, not a subset of classrooms with expectations of spillovers leading indirectly to treatment. The study does, however, find evidence of a reinforcement effect through social networks. Treatment effects are greatest when a large proportion of one’s friends were also treated (as discussed in Manski [2013]).3 From a policy perspective, an Internet-based course has the major advantage of being low cost compared to human led interventions. In this intervention $1,000 is sufficient to provide a course to 68 students for a whole semester.4 It is straightforward to use the estimate of change in demand for condoms due to the course to forecast that $1,000 would generate a reduction of 2.2 STIs among the treated students. Given that averting a sexually transmitted disease is estimated in the literature to be valued at $785 this implies a benefit-to-cost ratio of 1.72. Thus, the results point toward substantial social benefits from this course as a method to improve sexual knowledge, attitudes, and future sexual behavior at a relatively low cost. There is an emerging literature studying the efficacy of computer-based forms of education in developed countries. Noar, Black, and Pierce (2009) review 12 published and unpublished studies using ICTs in sex- ual education among young adults (average age 22) in the United States (and one study in the Netherlands) and find a positive average effect, albeit with more null results than positive in the individual studies. This study builds on this nascent literature by implementing a large-scale program through the public high school system in Colombia, including analysis of knowledge and attitude spillovers through social net- works, and administrative data (through redemption of condom vouchers) rather than self-reported data on condom use. This research also relates to recent evidence suggesting that computers on their own do not change academic outcomes in a discernible manner (Angrist and Lavy 2002; Krueger and Rouse 2004; Barrera- Osorio, and Linden 2009; Fairlie and Robinson 2013). It complements this literature by showing that, once computers have been installed in schools, structured Internet courses—at least for sexual education— can have significant effects. 2 The sexual knowledge index is composed of the following subindices: symptoms and causes of STIs, sexual violence, prevention of STIs, pregnancy prevention, and condom use. The sexual attitudes index is composed of the following subindices: condom use, sexual conservativeness and sexual abuse reporting. 3 These results complement recent literature such as Fletcher (2007), Richards-Shubick (2015), and Card and Giuliano (2013), who find that peer group norms have a first-order effect in explaining sexual health outcomes. 4 All figures in 2012 U.S. dollars. 4 Chong et al. 2. Profamilia’s Internet-Based Sex Education Course The online sexual education course was designed by the local NGO Profamilia.5 Profamilia is Colombia’s largest organization focused on sexual health and reproductive health. With more than 40 years of pres- ence and over 1,800 employees nationwide, Profamilia is well known and used by the local population for sexual health products and services such as contraceptives, HIV testing, and pregnancy tests.6 Motivated by the stubbornly high level of some important adolescent sexual health indicators nation- wide, such as teenage pregnancy rates (Demographic and Health Survey 2005), as well as legal changes, which mandated the introduction of a sexual health curriculum in Colombian public schools, Profamilia embarked on the design of a comprehensive online sexual education course designed for adolescents. The curriculum aims to shape adolescents’ understanding and perceptions of sexuality, risks, repro- ductive health, sexual rights, and dating violence. The overarching theme is a human rights approach to pregnancy and teen sexuality. The course focuses on helping the students recognize themselves as endowed with rights, such as the right to say no to sex, to access basic health services, to access family planning services, and to live without sexual violence.7 Profamilia’s course takes full advantage of Internet con- nectivity to provide an interactive experience and responsive, anonymous counseling. The modules can be potentially accessed any time of day using a password-protected account, and there is a remote tutor available to answer questions via messages and support the learning process. These features aim to create a safe social environment for adolescents to discuss sensitive topics. Treatment consisted of five modules. Students worked on the course for a total of 11 weeks. Each group of treated students was initially given three weeks to become acquainted with the platform and complete activities in the first module. After the first three weeks, each group was given two weeks per module to complete activities in the remaining four modules. Each school dedicated one session of 1.5 hours per week to allow the students to complete the course in the school’s computer labs. In school, each group taking the course worked with the presence of a teacher, who was tasked with helping the students resolve questions about use of and access to the platform, but not questions related to the content of the course. Students were assisted and monitored by an online tutor, who was a trained Profamilia counselor who dedicated part of his or her day to overseeing students during their completion of the course. The tutors had two main roles: answering students’ questions about the course contents and monitoring the students’ performance.8 At the end of every module, the tutor provided the teacher responsible for the group with a grade for each student, based on the results of a test. To incentivize course completion, each school participating in the course included these grades as a component of the grade of one school subject, typically computer education. Each student had to complete module evaluations individually, which were the basis for his or her individual performance report. Participation in the course was mandatory for students. 3. Experimental Design and Estimation Strategies Sample Selection In Colombia, 13.5 percent of adolescents become sexually active by age 15, and 60 percent have sex before age 18.9 Profamilia’s course targets 14–15 year olds precisely because they are close to 5 The organization’s website can be accessed at www.profamilia.org.co. 6 See Miller (2010) for a study of long-term effects of Profamilia family planning services in Colombia. 7 Examples from the course modules can be accessed at www.profamiliaeduca.com/profamilia/index.php. 8 Records of the interactions between tutors and students are not preserved and hence were not available for analysis. 9 Sexual education courses must ideally be targeted at children of the appropriate age to benefit from them. Very young children may not yet be interested in sexuality issues, which points towards the benefits of targeting an older age range. becoming sexually active. The sample frame for the study consists of ninth-grade students in Colom- bian urban public secondary schools. Given the study’s interest in cross-classroom spillovers, it required enrolled schools to have at least two ninth-grade classes. Schools were also required to have at least one computer room with Internet access.10 All participating administrators of the schools had to consent for their school to participate in the field experiment before knowing the results of the randomization. Schools agreed to facilitate data collection and coordination, to make a computer lab available for the prescribed time every week (if selected to implement the course), and to not substantially modify their sexual and reproductive health education for ninth-graders during the study. A short questionnaire for school principals at baseline revealed that sexual education in the sample was either nonexistent, a topic covered in biology class, or consisted of one or two visits per year by a health professional. Schools in the control group received a sports equipment package as compensation at the end of the study. The sample consists of 69 public secondary schools recruited in 21 cities across Colombia.11 From each school, two classrooms of ninth-graders were selected to participate in the study. If the school had more than two classrooms of ninth-graders, a pair was randomly selected by the researchers to partake in the study. Data Collection Strategy Data collection consisted of three rounds of a self-administered in-school pencil and paper survey. These surveys were administered unannounced during class time, so as to minimize data collection costs and attrition. The pencil and paper survey strategy was chosen instead of a computer survey because treatment students would have more familiarity with computers at endline, hence unnecessarily biasing results. The baseline survey was fielded at the beginning of the academic year; the second one at the end of the semester (after taking the course), and the last survey was taken at the end of the academic year, that is, six months after the end of the course. The academic year bracketed the timeline for the final survey data collection since beyond that point there is substantial student attrition due to students switching schools, and more fun- damentally, groups get reshuffled going into the 10th grade. This meant that to interview the participants during the 10th grade the researchers would have had to either locate the students in their new groups for re-interviews or survey the whole cohort to locate the participants’ responses ex- post. The first was deemed unfeasible by school administrators while the latter option was beyond the budget for the study. Six months after the end of the course, students were offered a voucher for six condoms with a total market value of about US$5. The offer was made via an email for all students and additionally via an SMS message for those who provided a cell phone number (86 percent of the sample). Data on which students redeemed their vouchers at the local health clinic were o bta in ed by matching the voucher number to the student to whom the voucher was sent. A timeline of the intervention and data collection strategy is presented in fig. 1. On the other hand, sexual education should in principle be provided before sexual initiation to convey its full benefits. In the United States 15 percent of adolescents have sex before age 15 (Flanigan et al. 2006). 10 The study selected schools with a functioning computer lab connected to the Internet with at least one computer for every three students. On average schools had 38 computers with a ratio of approximately 1 participating student per computer. 11 The sample excludes rural public schools. In urban settings, it is common for schools in Colombia to have two shifts per day (morning and afternoon). A student is offered a place at a certain shift before the beginning of the school year, and once a school is selected, he/she cannot take classes in other shifts or switch shifts. Given the lack of interaction among children of different shifts, the study treats different shifts in the sample as different schools. Both shifts are used for 13 schools in the sample. 6 Chong et al. Figure 1. Timeline Randomization Procedure Because the sexual education course was part of the curriculum of a computer education (or similar) course, treatment was at the classroom level. Hence the randomization unit is the classroom (interchange- ably referred to as group here). There are three types of classrooms: treatment, spillover, and control. The randomization is done in two stages. First, schools are randomly assigned to either treatment or control. Then, within treatment schools, classrooms are randomly assigned to either the treatment condition or the spillover. A spillover classroom does not receive the treatment, but is in the same school as one that does. Table 1 shows the partition of schools and groups in the study. There are 138 groups spread over 69 schools. The total sample size is 4,599 students, with an average of 33 students per group: 46 groups were assigned to control (across 23 schools), 46 groups (across 46 schools) were assigned to treatment, and 46 groups (across the same 46 schools) were assigned to the spillover condition. Randomization of treatment was performed before the baseline survey. The researchers obtained some basic information about participating school characteristics, reported in panel A of table 2. After randomly assigning groups to different conditions, the researchers verified that assignment to treatment was not correlated with any of the available variables.12 Table 1. Experimental Design Schools Classrooms Students Treatment schools Treatment classrooms 46 46 1522 Spillover classrooms 46 1600 Control schools Control classrooms 23 46 1477 Total 69 138 4599 Source: Authors. Note: First, schools were randomly assigned to treatment and control, then two classrooms from each school were randomly selected to participate in the study. In treatment schools one of the classrooms was assigned to treatment and the other one to no treatment (referred to as a spillover classrooms). In control schools both (untreated) classrooms are referred to as control classrooms. 12 Specifically, the study drew randomizations with different starting seed values, testing each one for orthogonality on the set of covariates listed in panel A, and then stopping when a randomization yielded no t-stat larger than 2.0. As discussed in Bruhn and McKenzie (2009), a better approach than what this study did defines a set number of randomizations (e.g., 10,000) and then chooses the one with the most orthogonal assignment. Table 2. Baseline Summary Statistics and Balance Treatment Spillover Control students students students Difference Difference (1) (2) (3) (1–3) (2–3) Panel A: Variables available at random assignment School year begins in January (=1) 0.720 0.731 0.699 0.020 0.032 (0.01) (0.01) (0.01) (0.12) (0.12) Single shift school (=1) 0.606 0.623 0.577 0.028 0.046 (0.01) (0.01) (0.01) (0.13) (0.13) Morning shift (=1) 0.637 0.658 0.652 −0.016 0.006 (0.01) (0.01) (0.01) (0.12) (0.12) City with more than 600,000 people (=1) 0.260 0.239 0.251 0.009 −0.011 (0.01) (0.01) (0.01) (0.11) (0.11) Number of 9th grade classrooms in school 3.226 3.258 3.081 0.145 0.177 (0.03) (0.03) (0.03) (0.32) (0.32) Average number of students in each classroom 37.257 37.330 38.296 −1.039 −0.965 (0.28) (0.29) (0.22) (2.42) (2.47) Number of computers in school 37.669 38.246 35.909 1.761 2.337 (0.44) (0.45) (0.52) (5.17) (5.19) School does not teach sexual education (=1) 0.168 0.167 0.135 0.033 0.032 (0.01) (0.01) (0.01) (0.09) (0.09) p-value from F-test of joint significance on all above variables 0.94 0.89 Panel B: Baseline variables not available at random assignment Male (=1) 0.414 0.402 0.490 −0.076 −0.088* (0.01) (0.01) (0.01) (0.05) (0.05) Not sexually active (=1) 0.617 0.587 0.590 0.026 −0.003 (0.01) (0.01) (0.01) (0.04) (0.04) Age 14.935 15.020 14.977 −0.042 0.043 (0.03) (0.03) (0.03) (0.11) (0.12) Mother’s years of education 12.706 12.641 12.584 0.121 0.056 (0.07) (0.07) (0.07) (0.11) (0.10) Father’s years of education 12.672 12.579 12.503 0.169 0.076 (0.08) (0.08) (0.08) (0.13) (0.13) Socioeconomic level [1, 6] 2.175 2.170 2.162 0.013 0.008 (0.03) (0.03) (0.03) (0.13) (0.13) PC at home (=1) 0.323 0.305 0.326 −0.003 −0.021 (0.01) (0.01) (0.01) (0.04) (0.04) Cellphone (=1) 0.742 0.737 0.716 0.026 0.022 (0.01) (0.01) (0.01) (0.03) (0.03) Does not use Internet in school (=1) 0.447 0.512 0.482 −0.035 0.031 (0.01) (0.01) (0.01) (0.09) (0.09) Does not use Internet (=1) 0.238 0.252 0.252 −0.014 0.000 (0.01) (0.01) (0.01) (0.03) (0.03) Religion is important (=1) 0.619 0.601 0.618 0.001 −0.017 (0.01) (0.01) (0.01) (0.03) (0.03) p-value from F-test of joint significance on all above variables 0.79 0.62 Source: Authors’ analysis from student survey. Note: Columns (1)–(3) report means, with standard errors in parentheses. For columns (4) and (5), each row is one regression of the characteristic on treatment and spillover indicator variables, with the coefficient (standard error, clustered at the school level) on treatment and spillover reported. *** p < 0.01, ** p < 0.05, * p < 0.1. Panel A variables were provided by the schools before the baseline survey took place. The study randomized treatment assignment repeatedly until no t-test comparing treatment to control for any covariate was larger than 2.0. Variables in Panel B became available only after assignment to treatment. The last rows in panels A and B report the p-value on an F-test of joint significance for all variables in the panel from a regression where the dependent variable is a treatment dummy (Column 4) or spillover dummy (column 5). Column 4 excludes the spillover group from the analysis, while column 5 excludes the treatment group from the analysis. 8 Chong et al. Implementation The sexual education course was implemented from August through November 2009 in schools that be- gan their school year in January13 and from November 2009 through March 2010 in schools that began their school year in September. As expected for a middle-income country, it was not difficult to recruit schools with computer labs. However, it proved more difficult to recruit schools with workable Internet connections. In fact, in 3 of the 46 groups assigned to treatment, lack of Internet access prevented imple- mentation of the Internet-based course.14 In some treatment groups, students were unable to complete all five modules due to unforeseen events such as teacher strikes. Grades on the tests at the end of each module were on average 4 out of 10, with a large mass at zero (48 percent). Excluding those students with a score of zero, the average was 8 out of 10, suggesting an acceptable degree of understanding for those actually taking the course and the tests. The high proportion of scores equal to zero highlights the challenges of student compliance associated with Internet-based education. Panel B in table 2 shows sum- mary statistics by treatment condition. The average age is 15 years, 43 percent of the sample is male, and 32 percent have a computer at home.15 Baseline Balance Panels A and B of table 2 show there are no statistically significant differences across treatment, spillover and control groups except for gender: the control group has 7.6 percentage points and 8.8 percentage points more males than the treatment and spillover groups, respectively. Furthermore, an F-test from a re- gression of treatment assignment on a full set of baseline characteristics does not reject the null hypothesis that all baseline coefficients are jointly equal to zero (reported in the final row of each panel). Attrition Attrition was 13 percent between baseline and first follow-up, and 10 percent between baseline and second follow-up. Supplementary appendix table S1.2 shows there is no differential attrition between control and treatment, and control and spillover students. The study also analyzes attrition for the condom voucher offer. Because students had to provide a valid cell phone number and/or email in order to be offered the condom voucher, the offer could not be made to every student in the study: 31 percent of students were missing both pieces of information due to nonresponse, misspelled email addresses, or invalid phone numbers, and could not be offered the vouchers. The table shows there was no difference in condom voucher offers between control and treatment groups or between control and spillover groups. In addition, the table also shows that there was no differential attrition by socioeconomic status of the family16 or parental education. Econometric Specification Randomization allows for identification of reduced form intent-to-treat effects. Let Yijt denote an out- come of interest at follow-up (t = 1 or 2) for individual i in classroom j. Treatment and spillover class- room assignment dummies are denoted by Tj and S j respectively. Treatment classrooms were selected for Internet-based sexual health training whereas spillover classrooms were not selected for the training but are in a school that has a treated classroom. Whenever available, the analysis includes the baseline dependent variable as control for precision. 13 The school year in some regions of Colombia begins in January (Calendario A), whereas in other regions it begins in September (Calendario B). 14 For the statistical analysis, these classrooms are still in the intent-to-treat group. 15 Summary statistics for every question used in the survey are reported in Supplementary appendix table S1.1. 16 A score from 1 to 6 assigned by the Colombian government to households for targeting all social programs in which 1 is poorest and 6 is richest. This score is well known and self-reported by the students in the baseline survey. The following regression model is estimated via ordinary least squares as the main specification: Yijt = α1 + β1Tj + β2 Sj + β3Yij0 + εijt , (1) where the error term ε ijt is assumed to be uncorrelated across schools but not necessarily within them. Hence, standard errors are clustered at the school level. Because Tj and S j were randomly assigned, the estimated coefficients are unbiased estimators of the intent-to-treat effects of the course, which this paper argues are the policy coefficients of interest. The study has multiple measures of sexual health knowledge and attitudes in the survey. However, testing multiple outcomes using (1) for each measure independently increases the probability of rejecting a true null hypothesis for at least one outcome above the significance level used for each test (Duflo, Glennerster, and Kremer 2008). Hence, the study follows Kling, Liebman, and Katz (2007) and defines a summary measure Y ∗ as the unweighted average of all standardized outcomes in a family as follows: ∗ k Yk Yk − μk Y∗= , where Y ∗ k = . k σk Where the mean (μ) and variance (σ 2) at baseline are used in the standardization of each variable Yk. This allows the estimates β1and β2 to be interpreted as the effects of the course in terms of standard deviations of the outcome at baseline. 4. Results The main results appear in tables 3–5, reporting effects on knowledge (table 3), attitudes (table 4), and individual indicators of sexual behavior and condom redemption (table 5). For each indicator, whenever available the results are included from both follow-ups, the first taken one week after the end of the intervention and the second one taken six months after the end of the intervention. While the study focuses more on the results of the second follow-up, the comparisons of effects between the short- and medium-run provide an indication of the durability of the effects. Knowledge Table 3 presents the impacts on five different standardized knowledge indices measuring: (a) STI symptoms and causes, (b) recognition of instances of sexual violence, (c) STI prevention, (d) pregnancy prevention methods, and (e) proper condom use. Columns (11) and (12) in the table show results for an overall index using all the variables used in the partial indices.17 The notes in the table report the definition of the individual variables used in the construction of each index. The aggregate knowledge index shows a 0.37 standard deviation increase in overall knowledge one week after the intervention and a 0.38 standard deviation increase in overall knowledge six months after the intervention compared to students not assigned to the course: Both coefficients are statistically sig- nificant at the 1 percent level. Furthermore, the study finds a robust pattern of positive coefficients in all components of the general index as well as statistical significance at least at the 5 percent level six months after the intervention. The table also shows there is no clear pattern of decay in knowledge outcomes, since some improve while others decrease over time. The second row in the table also shows that there is no clear evidence for cross-classroom spillover effects in terms of sexual knowledge. 17 For space reasons, this article does not report results on every individual outcome, but they are available upon request. The general indices calculate the average for all nonmissing outcomes at the individual level. For the partial subindices, which are composed of few variables, the study drops observations, which have missing values for any component of the subindex. This gives the reader information about item nonresponse. 10 Table 3. Knowledge Indicators Knowledge of symptoms and Sexual violence Prevention of STIs Pregnancy prevention Condom use knowledge General knowledge causes of STIs subindex knowledge subindex knowledge subindex knowledge subindex subindex index One week Six months One week Six months One week Six months One week Six months One week Six months One week Six months post post post inter- post inter- post inter- post inter- post inter- post inter- post inter- post inter- post inter- post inter- intervention intervention vention vention vention vention vention vention vention vention vention vention (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Treatment students 0.282*** 0.202*** 0.254*** 0.109** 0.067 0.519*** 0.299*** 0.335*** 0.262*** 0.166** 0.372*** 0.378*** (0.048) (0.056) (0.057) (0.054) (0.041) (0.139) (0.049) (0.078) (0.046) (0.064) (0.049) (0.080) Spillover students 0.022 0.064 0.034 −0.025 0.024 0.139 0.043 0.061 0.051 0.031 0.015 0.011 (0.044) (0.053) (0.054) (0.059) (0.044) (0.147) (0.050) (0.082) (0.054) (0.064) (0.050) (0.085) Control for baseline Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes value of dep. var. Observations 4,373 3,867 4,354 3,859 4,353 3,836 4,388 3,874 4,384 3,867 4,388 3,903 Source: Authors’ analysis from student surveys. Note: Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1. Knowledge of symptoms and causes of STI subindex: Respondent knows STI symptoms include: (a) Abnormal discharges from the penis/vagina; (b) lesions/sores in genitals; and (c) painful urination; respondent knows (d) vomiting and headache are not STI symptoms; (e) HIV can be transmitted by having sexual intercourse without a condom; (f) HIV can be transmitted by a contaminated blood transfusion; (g) HIV transmission does not depend on hygiene; (h) HIV cannot be transmitted via food sharing; (i) clothes sharing; or (j) being in a pool with an HIV-positive person. Respondent knows that (k) HIV is not transmitted if a condom is used while having sexual intercourse with an HIV-positive individual. Sexual violence knowledge subindex: Respondent identifies (a) nonconsensual touching of genitalia, buttocks, breasts, inner thigh as abusive sexual contact; (b) forcible sex by husband on his wife as a form of sexual abuse; (c) having sex with a person who is impaired due to alcohol as a form of rape; (d) if an individual changes his/her mind about sex even at the last minute, sex is nonconsensual and hence a form of sexual abuse; (e) the use of threats to obtain sex is a form of sexual abuse; respondent knows: (f) sexual abuse is more often than not perpetrated by a known person not a stranger. Prevention of STI knowledge subindex: Respondent knows one of the safest methods to prevent an STI is the use of condom whereas the calendar-based methods, hormone injections, and penis withdrawal are not. Pregnancy prevention knowledge subindex: Respondent disagrees with (a) penis withdrawal is a safe method to avoid pregnancy; respondent knows: (b) women can become pregnant in their first sexual relationship; (c) safe methods to prevent a pregnancy include injections and condom; (d) unsafe methods to prevent a pregnancy include calendar-based methods and penis withdrawal; respondent knows that (e) emergency post-coital contraception pills have secondary effects. Condom use knowledge subindex: Respondent knows (a) one of the safest methods to prevent an STI is the use of a condom; (b) condoms can be used only one time; (c) HIV can be transmitted by having sex without a condom; (d) HIV is not transmitted if a condom is used even if the person is HIV positive; (e) one of the safest methods to prevent a pregnancy is by using a condom. General knowledge index: an index of all the variables used in the subindices of the table. Chong et al. The World Bank Economic Review Table 4. Attitude Indicators Sexually conservative attitudes Sexual abuse reporting attitudes Condom use attitudes subindex subindex subindex General attitudes index One week post Six months post One week post Six months post One week post Six months post One week post Six months post intervention intervention intervention intervention intervention intervention intervention intervention (1) (2) (3) (4) (5) (6) (7) (8) Treatment students 0.170*** 0.100* 0.072 0.133** 0.260*** 0.112** 0.240*** 0.172*** (0.051) (0.051) (0.046) (0.058) (0.048) (0.054) (0.053) (0.056) Spillover students 0.028 −0.024 0.003 0.075 0.035 0.015 0.026 0.022 (0.052) (0.051) (0.044) (0.058) (0.048) (0.051) (0.052) (0.052) Control for baseline Yes Yes Yes Yes Yes Yes Yes Yes value of dep. var. Observations 4,390 3,864 4,389 3,896 4,344 3,854 4,391 3,906 Source: Authors’ analysis from student surveys. Note: Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1. Condom use attitudes subindex: Respondent disagrees with statements (a) “It’s not right to carry a condom because people may think that I planned to have sex”; (b) “If a woman wants to have sex without a condom, the man must not refuse”; (c) “Only women are responsible for unwanted pregnancies”; respondent is (d) confident if requesting that a condom be used; (e) willing to delay sex if condoms are unavailable; respondent thinks (f) he/she will use a condom in his/her next sexual relationship. Sexually conservative attitude subindex: Respondent thinks that (a) it is not right when people of their age have sex with several partners in the same month; (b) people of their age should wait to have sex; respondent’s answer to (c) age at which men and women should start having sex. Respondent is (d) confident he/she will have sex only when emotionally ready. Sexual abuse reporting attitudes subindex: Respondent thinks that when a teenager is suffering from sexual violence (a) he/she must tell his/her family; (b) he/she must tell the authorities; (c) in case of rape, the afflicted individual must seek medical help; respondent disagrees with the idea that in case of rape the person (d) must not tell anyone. General attitudes index: contains all variables used in the other columns of the table. 11 12 Table 5. Sexual Activity and Condom Demand Sexually active last Frequency of sex Number of partners STI presence Pregnancy Redeemed Voucher six months last six months last six months for Free Condoms§ Six months post Six months post Six months post Six months post Six months post Six months post intervention intervention intervention intervention intervention intervention (1) (2) (3) (4) (5) (6) Treatment students −0.003 0.212 −0.009 −0.005 0.000 0.099* (0.029) (0.259) (0.031) (0.004) (0.005) (0.055) Spillover students 0.023 0.278 0.043 −0.001 0.007 0.048 (0.031) (0.270) (0.031) (0.004) (0.006) (0.048) Control for baseline value of dep. var. No No Yes Yes Yes No Mean of dep. var. control group 0.26 1.57 0.37 0.01 0.02 0.18 Observations 4,364 3,857 3,881 3,774 4,252 3,358 Source: Authors’ analysis from student surveys. Note: Dependent variables not standardized. All outcome variables are assessed six months after treatment. Standard errors clustered at the school level in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1. Column (1) includes students attrited for written survey but later tracked over the phone. Table does not control for baseline value of the dependent variable, because outcome was not measured at baseline (except for column (3)). § 3,358 students of the full sample agreed to be contacted for this part of the study. Specification controls for whether individual had a cellphone. Chong et al. The World Bank Economic Review 13 Attitudes Table 4 presents results on attitude indicators. Columns (1) and (2) present results on attitudes towards the use of condoms, (3) and (4) on conservatism with respect to age of initiation of sexual activities, (5) and (6) on attitudes toward denouncing and seeking help in the event of sexual abuse, and columns (7) and (8) show results for an overall index of attitudes, containing all variables used in columns (1−6). For the general attitudes index the study finds significant effects of 0.24 standard deviations in terms of attitudes one week after the intervention and 0.17 standard deviations six months after. Significant effects were again found for each subindex six months after the intervention, and the pattern of positive effects on all partial indices is again apparent. The training was successful in generating more positive attitudes towards the use of condoms at the first follow-up (0.17 standard deviation) and at the second follow-up (0.10 standard deviation). In the sexually conservative attitudes subindex composed of the following variables: (a) indicates that individuals their age should not have multiple sexual partners in the same month; (b) thinks it is too early for individuals of their age to engage in sexual activities; and (c) feels confident he/she will be able to wait to have sex until emotionally prepared to do so, teens scored 0.13 standard deviations higher six months after the intervention. Column (6) shows that treated teens are 0.11 standard deviations more likely to agree with the need to report cases of sexual abuse to the authorities and the need to seek medical attention in such situations. For attitude indicators, there was no consistent evidence of spillovers across classrooms.18 Sexual Activity and Condom Demand Table 5 reports results on sexual activity and condom voucher redemptions. Specifically, columns 1–3 report effects of the course on self-reported sexual activity six months after the end of the course.19 Columns 1–3 uniformly show that the course did not increase or decrease self-reported sexual activity among adolescents. Column 1 shows that students were not more likely to be sexually active in the past six months compared to the control group. Similarly, the frequency of sex did not change in the previous six months compared to the control group (column 2) nor in terms of number of partners (column 3). The finding that sexual education does not result in increases in sexual activity is a robust finding in this literature (cf. Kirby, Laris, and Rolleri 2007). Given that the objective population is teens at high risk of sexual initiation, the setting is not designed to capture other sexual behavior outcomes such as STIs and pregnancy, given their extremely low prevalence (the control group means for these self-reported outcomes are 1 percent and 2 percent; thus there is little room for improvement). The study nevertheless estimated treatment effects for these outcomes and found no statistically significant changes (columns 4 and 5 of table 5). Of course, the validity of studies using self-reported sexual behavior among adolescents has long been a criticism in this literature (Brener, Billy, and Grady 2003). This challenge was addressed by measuring the percentage of students who redeem vouchers for condoms. This strategy makes it possible to address the possible lack of reliability in self-reported outcomes via an objective safe-sex behavior metric. Condom availability is important for adolescent health given the sporadic nature of adolescent sexual activity. Column 6 in table 5 reports the results of the voucher experiment. The administrative data from voucher redemption shows statistically significant and important effects. The table shows that 28 percent of treatment students redeem them, compared to 18 percent of control 18 Classroom level spillover effects were also statistically undetectable using a dose-response model. Specifically, this was implemented by defining the spillover variable as one over the number of classrooms in grade 9 in the school. Spillover coefficients again followed the same pattern of no significant effects. 19 Note that for this table the dependent variables are not an index but a single variable, so the study does not standardize them. This has the benefit of allowing for comparability with other studies. Furthermore, in this table there is no control for baseline value of the dependent variable because it was not available at baseline (except for columns 3-5). 14 Chong et al. Table 6. Friendship Networks Summary Statistics Cases Percent Treatment students with: No friends treated 366 21.2% 1 friend treated 277 16.0% 2 friends treated 266 15.4% 3 friends treated 227 13.1% 4 friends treated 286 16.6% 5 friends treated 183 10.6% 6 friends treated 123 7.1% Spillover students with: No friends treated 1482 88.8% 1 friend treated 133 8.0% 2 friends treated 11 0.7% 3 friends treated 7 0.4% 4 friends treated 7 0.4% 5 friends treated 13 0.8% 6 friends treated 15 0.9% Source: Authors’ analysis from student surveys. Note: Friendship link treatment status is established by matching self-reported list of friends with list of names of students answering the survey at (either) follow-up survey. The number of friends treated for students in control schools is equal to zero. students, a 55 percent increase in redemption (p = 0.07). The study puts substantial weight on this result as it provides objective evidence of an increase in condom demand. In unreported results, the estimated coefficient is basically unchanged when controlling for distance to the health clinic. As a robustness check, bounding exercises were performed with differing assumptions on attrition, as in Karlan and Valdivia (2011), and found that the positive effect on condom voucher in table 5 (0.099 percentage points; standard error = 0.055 percentage points) still holds after imputing the mean minus 0.10 standard deviations of the observed treatment distribution to the nonrespondents in the treatment group, and after imputing the mean plus 0.10 standard deviations of the observed control distribution to nonrespondents in the control group.20 Friendship Network Spillovers and Reinforcing Interactions Next the study takes advantage of the fact that in the surveys students were asked to identify their closest friends by name, and indicate if they were in the same school and/or classroom. This information is used to match each student’s social network to the list of students in the treatment and spillover groups. This makes it possible to analyze treatment and spillover effects differentiating between students for whom a small or a large percentage of friends was also treated. Table 6 presents summary statistics about the network treatment distribution. For 21 percent of stu- dents in a treated classroom their list of friends does not contain anyone from their own classroom. The analysis will consider these students as having a network of friends that is not treated at all. In contrast, 79 percent of students in a treated classroom have at least one friend in their own classroom (who was hence also treated). The table shows substantial variation in the number of friends that are located in the same classroom as a treatment student. The lower panel in the table shows that for students in a spillover classroom, there are few links to students taking the course (in the treatment classroom). Indeed, 89 per- cent of spillover students have no friends in the treatment classroom—this will affect the precision of the spillover estimates. 20 Results of these simulations are not presented here but are available upon request. The World Bank Economic Review 15 With this information, a measure of network treatment intensity is naturally defined as the proportion of the student’s network of closest friends who were treated (friends in a treatment classroom/total listed friends).21 If a student and his or her entire network of close friends were all in the same treatment classroom, then the proportion is equal to one, but if the network of friends includes students from other classrooms or from outside the school, then the proportion is lower. The analysis uses variation in the proportion of close friends that are in the student’s classroom to estimate a heterogeneous treatment effects regression in which the main effects are now interacted with the proportion of friends in the network who were treated (Fij ). The specification becomes: ( ) 11 Yi jt = α3 + β16 Tj + β17 (Fi j × Tj ) + β18 S j + β19 Fi j × S j + β20 Ni j + β21Yi j0 + ε , ijt (3) where in order to control for popularity of a particular student, the regression includes Nij defined as the number of people who mentioned individual i as a friend. Throughout the analysis, standard errors are clustered at the school level. In tables 7, 8, and 9 the interpretation of the main effect (Tj) now becomes the effect of assignment to treatment for someone who has zero friends also treated, whereas the coefficient on (Fij × Tj) is the additional effect of the course for someone whose full set of friends are also treated (analogously for S j). In interpreting these results, the reader should be cognizant that the distribution of a student’s network of friends is not a randomly assigned variable. The identifying variation is coming from whether the student’s friends are in his or her classroom or are rather in another classroom in the school or from outside of the school. This may lead to bias if, for example, more extroverted students have a larger proportion of their best friends in the classroom and this extroversion is related to the outcome beyond the effect stemming from social reinforcement. For this reason, the study conditions on Nij, the number of individuals that mention another student as a best friend.22 The necessary assumption hence becomes that the proportion of friends in the classroom is related to the treatment response only through the network effects (conditional on the number of friends). With this assumption in mind, results for network interactions are presented in tables 7, 8, and 9, which report effects six months after finishing the course. Table 7 provides clear evidence of a significant reinforcing interaction effect for students in the treatment group in terms of the overall knowledge index (column 6). The study is able to identify an effect of 0.46 standard deviation in knowledge for wholly treated networks, as opposed to a 0.28 standard deviation effect if the student’s network is not treated. In contrast, there are no statistically significant effects for spillover students, even if their network was fully treated. As noted before, large standard errors are obtained for the spillover estimates due to the small number of spillover students with treated networks. At the bottom of each column, the table reports the p-value from a test of equality of the friendship interaction effects for treatment and spillover students. The reinforcing interaction effect is positive, large, and significant for all five subindices except for those reporting knowledge concerning sexual violence and knowledge about condom use (columns 2 and 5). Table 8, on attitude indicators, shows an even starker reinforcing interaction effect. In this case, the effects are significant only if the student’s friendship network also took the course. For example, if a student’s full network was treated, the student is predicted to have a 0.24 standard deviation higher attitude index score, whereas the estimated effect is only 0.04 standard deviation if no one in his or her friendship network was treated. Similar outcomes are observed in each of the subcomponents of the index. As in table 7, there is no significant network spillover effect for a student who did not take the course. 21 One shortcoming of this network analysis is that the questionnaire did not clearly differentiate between friendship and romantic relationships. 22 When looking at determinants of being more popular in school, the data shows that kids who are more popular are less likely to smoke, drink, and consume drugs at baseline. 16 Table 7. Knowledge: Network Spillover and Reinforcing Interaction Effects Knowledge of Sexual violence Prevention of STI Pregnancy Condom use symptoms and causes knowledge knowledge prevention knowledge General of STIs subindex subindex subindex knowledge subindex subindex knowledge index (1) (2) (3) (4) (5) (6) Treatment student 0.132* 0.083 0.377** 0.201** 0.135* 0.278*** (0.067) (0.062) (0.153) (0.090) (0.075) (0.081) Spillover student 0.082 −0.023 0.137 0.065 0.035 0.022 (0.055) (0.058) (0.147) (0.082) (0.062) (0.082) Treatment student * % of friends treated 0.136* 0.038 0.258* 0.248** 0.056 0.179* (0.081) (0.080) (0.155) (0.106) (0.079) (0.100) Spillover student * % of friends treated −0.280* 0.170 0.185 −0.154 −0.034 −0.050 (0.159) (0.177) (0.380) (0.213) (0.196) (0.217) Number of friends 0.034* 0.074*** 0.151*** 0.080*** 0.036* 0.104*** (0.018) (0.017) (0.043) (0.019) (0.019) (0.020) Control for baseline value of dep. var. Yes Yes Yes Yes Yes Yes P-value treatment * (% of friends) 0.0387 0.492 0.859 0.0788 0.669 0.334 = spillover * (% of friends) Observations 3,853 3,845 3,828 3,866 3,853 3,888 Source: Authors’ analysis from student surveys. Note: Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1. Knowledge of symptoms and causes of STI subindex: Respondent knows STI symptoms include (a) abnormal discharges from the penis/vagina; (b) lesions/sores in genitals; and (c) painful urination; respondent knows: (d) vomiting and headache are not STI symptoms; (e) HIV can be transmitted by having sexual intercourse without a condom; (f) HIV can be transmitted by a contaminated blood transfusion; (g) HIV transmission does not depend on hygiene; (h) HIV cannot be transmitted via food sharing; (i) clothes sharing; or (j) being in a pool with an HIV-positive person. Respondent knows that (k) HIV is not transmitted if a condom is used while having sexual intercourse with an HIV-positive individual. Sexual violence knowledge subindex: Respondent identifies (a) nonconsensual touching of genitalia, buttocks, breasts, inner thigh as abusive sexual contact; (b) forcible sex by husband on his wife as a form of sexual abuse; (c) having sex with a person who is impaired due to alcohol as a form of rape; (d) if an individual changes his/her mind about sex even at the last minute, sex is nonconsensual and hence a form of sexual abuse; (e) the use of threats to obtain sex is a form of sexual abuse; respondent knows: (f) sexual abuse is more often than not perpetrated by a known person not a stranger. Prevention of STI knowledge subindex: Respondent knows one of the safest methods to prevent an STI is the use of condom whereas the calendar-based methods, hormone injections, and penis withdrawal are not. Pregnancy prevention knowledge subindex: Respondent disagrees with (a) penis withdrawal is a safe method to avoid pregnancy; respondent knows: (b) women can become pregnant in their first sexual relationship; (c) safe methods to prevent a pregnancy include injections and condom; (d) unsafe methods to prevent a pregnancy include calendar-based methods and penis withdrawal; respondent knows that (e) emergency postcoital contraception pills have secondary effects. Condom use knowledge subindex: Respondent knows (a) one of the safest methods to prevent an STI is the use of a condom; (b) condoms can be used only one time; (c) HIV can be transmitted by having sex without a condom; (d) HIV is not transmitted if a condom is used even if the person is HIV positive; (e) one of the safest methods to prevent a pregnancy is by using a condom. General knowledge index: an index of all the variables used in the subindices of the table. Chong et al. The World Bank Economic Review 17 Table 8. Attitudes: Network Spillover and Reinforcing Interaction Effects Condom use Sexually Sexual abuse attitudes conservative reporting attitudes General subindex attitudes subindex subindex attitudes index (1) (2) (3) (4) Treatment student −0.020 0.072 0.024 0.043 (0.064) (0.074) (0.070) (0.073) Spillover student −0.021 0.082 0.021 0.032 (0.049) (0.058) (0.051) (0.051) Treatment student * % of friends treated 0.213*** 0.114 0.166* 0.236*** (0.075) (0.071) (0.093) (0.078) Spillover student * % of friends treated −0.098 −0.059 −0.003 −0.072 (0.170) (0.143) (0.131) (0.147) Number of friends 0.068*** 0.006 0.044*** 0.052*** (0.017) (0.015) (0.016) (0.016) Control for baseline value of dep. var. Yes Yes Yes Yes P-value: treatment * (% of friends) 0.129 0.279 0.270 0.061 = spillover * (% of friends) Observations 3,856 3,882 3,840 3,891 Source: Authors’ analysis from student surveys. Note: Dependent variable is an index of related questions. All components of the indices are standardized to mean 0 and standard deviation 1, based on the sample frame at baseline. Standard errors clustered at the school level in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1. Condom use attitudes subindex: Respondent disagrees with statements (a) “It’s not right to carry a condom because people may think that I planned to have sex”; (b) “If a woman wants to have sex without a condom, the man must not refuse”; (c) “Only women are responsible for unwanted pregnancies”; respondent is (d) confident if requesting that a condom be used; (e) willing to delay sex if condoms are unavailable; respondent thinks (f) he/she will use a condom in his/her next sexual relationship. Sexually conservative attitude subindex: Respondent thinks that (a) it is not right when people of their age have sex with several partners in the same month; (b) people of their age should wait to have sex; respondent’s answer to (c) age at which men and women should start having sex. Respondent is (d) confident he/she will have sex only when emotionally ready. Sexual abuse reporting attitudes subindex: Respondent thinks that when a teenager is suffering from sexual violence (a) he/she must tell his/her family; (b) he/she must tell the authorities; (c) in case of rape, the afflicted individual must seek medical help; respondent disagrees with the idea that in case of rape the person (d) must not tell anyone. General attitudes index: contains all variables used in the other columns of the table. This provides evidence that the relevant group for a reinforcement effect is the network of friends, as suggested in Sacerdote (2011). Table 9 presents evidence of reinforcing interaction effects for self-reported sexual behavior: Students with more friends taking the course report significantly lower sexual activity than those with fewer friends taking the course for two out of three indicators. Column 4 reports results for condom redemption. In contrast to the network interaction results in knowledge and attitudes, the condom redemption results are not significant, although the signs of the coefficients are consistent with tables 8 and 9 in the sense that they point to an improvement the larger the share of friends that is treated. Indeed, treatment students are 8 percentage points more likely to redeem their vouchers than the control group, even if none of their close friends were treated (not statis- tically significant). The coefficient estimate on the reinforcing interaction suggests a 4.3 percentage point higher condom demand for those with all friends being treated, but again the coefficient is not statistically different from zero. Column 4 also shows a puzzling result in which spillover students whose entire friend networks were treated are 14 percentage points less likely to redeem their condom vouchers than the con- trol group. Note there is a small number of observations in the spillover group with friendship links to treated students, and the results in all the other indicators do not show a pattern of signifi- cant spillovers from classroom to classroom, so the authors do not attach much importance to this coefficient. 18 Chong et al. Table 9. Sexual Activity and Condom Demand: Network Spillover and Reinforcing Interaction Effects Sexually active last Frequency of sex Number of partners Redeemed voucher six months last six months last six months for free condoms§ (1) (2) (3) (4) Treatment student 0.042 0.340 0.049 0.076 (0.036) (0.313) (0.039) (0.058) Spillover student 0.016 0.174 0.032 0.056 (0.032) (0.264) (0.031) (0.046) Treatment student * % of friends treated −0.097** −0.257 −0.115*** 0.043 (0.038) (0.440) (0.042) (0.040) Spillover student * % of friends treated 0.110 1.850* 0.186 −0.138* (0.085) (0.986) (0.114) (0.069) Number of friends −0.005 −0.018 0.008 0.009 (0.008) (0.081) (0.010) (0.011) Control for baseline value of dep. var. No No Yes No Mean of dep. var. control group 0.26 1.58 0.37 0.18 P-value: treatment * (% of friends) 0.0195 0.0662 0.0170 0.0335 = spillover * (% of friends) Observations 4,246 3,843 3,868 3,334 Source: Authors’ analysis from student surveys. Note: Dependent variables not standardized. All outcome variables are assessed six months after treatment. Standard errors clustered at the school level in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1. Column 1 includes students attrited for written survey but later tracked over the phone. Table does not control for baseline value of the dependent variable, because outcome was not measured at baseline (except for column 3). § 3,358 students of the full sample agreed to be contacted for this part of the study. Specification controls for whether individual had a cellphone. Overall, while the anlysis reveals that network effects are clearly identified among treated students, it is not surprising that the study had little power to detect network effects among spillover students given the very few friendship network links that occur across classrooms in the data. Cost-Effectiveness and Cost-Benefit Analysis The marginal cost of the Profamilia course is approximately $14.60 per student. The bulk of this cost ($10) is accounted for by the remote tutor, and the remainder comes from Internet platform costs and computer depreciation. The calculations do not include opportunity costs of the time of the students (such as some alternative educational activity, leisure, or work outside of school).23 Compared to non- computer-based sexual health interventions in the United States, which range from $69 to more than $10,000 per student,24 the Profamilia course is extremely low cost. It is also low cost compared to instructor-led programs in developing countries. Kivela, Ketting, and Baltussen (2013) report costs per student of teacher-led school based sexual education programs of $27 in Kenya and $85 in Indonesia.25 Table 10 presents the cost effectiveness and cost-benefit calculations. These estimates obviously rest on many assumptions, but they provide much value for policy makers by allowing for comparisons across different interventions. The key result is that students who take the course are 9.9 percentage points more likely to redeem the condom voucher. For the purposes of table 10, the assumption is made that this indicates a consistent condom user. This makes it possible to link the coefficient to the literature documenting the effect of 23 The calculations also exclude the wage cost of the person supervising students in the computer lab because it is unlikely that a school would hire personnel exclusively for the course. This is in line with guidelines by Dhaliwal et al. (2013), who argue that cost-effectiveness should use marginal costs of adding the program, assuming fixed costs are incurred with or without the program. 24 Chin et al. (2012), pp. 280, with inflated estimates to 2012 dollars. 25 Costs refer to one semester of the course and are expressed in 2012 U.S. dollars. The World Bank Economic Review 19 Table 10. Cost-Effectiveness and Cost-Benefit Analysis Cost effectiveness Marginal cost of course per studenta $14.60 Averted STIs per $1,000 spentb 2.20 90% confidence interval [0.19, 4.20] Cost benefit Cost per averted STIc $455 Benefit per averted STId $785 Source: Authors’ analysis from student surveys. Note: a All figures in 2012 U.S. dollars. Marginal costs correspond to remote tutor wage per student ($10), Internet platform costs ($2.10), and depreciation cost of computers ($2.50). b Averted STIs per $1,000 = (Estimated STI reduction per student*1000/MgCost per student). c Cost per averted STI = (MgCost per stu- dent/Estimated STI reduction per student). d Benefit obtained from STI distribution and DALYs per incident in Ebrahim et al. (2005), and value of DALY from Brent (2011). Estimate assumes the increase in condom demand from table 5 reflects consistent condom use by the adolescent, and a reduction in STIs from condom use from Gallo et al. (2007) of 60 percent, along with the objectively measured STI prevalence from Gallo’s data of 54 percent. consistent condom use on STIs (such as Gallo et al. 2007). In support of making this assumption, Shaffi, Stovel, and Holmes (2007) show that adolescents who use a condom at sexual debut are significantly more likely to have used a condom in their most recent intercourse (on average 6.8 years after sexual debut) and are 50 percent less likely to test positive for chlamydia or gonorrhea. Gallo et al. (2007) estimate that consistent condom use leads to a 60 percent reduction in likelihood of having an STI. Using their baseline STI rate and multiplying by this study’s 9.9 percentage point increase in condom user result, the estimate would imply a 3.2 percentage point reduction in STI prevalence. This in turn means that $1,000 spent on the course generates 2.2 averted STIs (90 percent confidence interval from 0.19 to 4.20). To link the reduction in STIs to Disability Adjusted Life Years (DALYs), it is possible to use the gender- specific distribution of STIs and the implied DALYs lost per STI incident from Ebrahim, McKenna, and Marks (2005). In particular, the latter finds that for every STI episode, 0.11 DALYs are lost.26 Using the estimate of value per DALY of $7,142 in Brent (2011)27 suggests that the benefit of averting an STI is $785. One obtains a similar estimate ($634), using the lifetime costs of an STI presented in Ruger et al. (2014). The lower panel of table 10 summarizes the cost-benefit calculation. The headline result is that the course averts one STI at a cost of $455, indicating a benefit-to-cost ratio of 1.72, well above one. This implies that the course is socially desirable, even with typical deadweight loss factors due to taxation (Auriol and Wartlers 2012). 5. Conclusions This study provides evidence that information technologies can be a powerful tool to provide effective sex education in contexts in which informational barriers may pose a challenge to policy makers. In contexts in which teachers may be unwilling or unable to provide sexual education, Internet-based courses may prove a useful substitute for in-person instruction, and are also more scalable due to the lower marginal cost of delivering the curricula to students. The results presented here show that a six-month web-based sexual education course in Colombian public schools was effective at improving broad measures of knowledge and attitudes among teenagers, and that the course also led to a substantial increase in the rate of condom voucher redemption. This last measure provides plausible evidence that the course was effective in changing safe sex practices, where 26 27 Implied by his estimate of $6,300 (2005 dollars) and an inflation rate of 13.3 percent between 2005 and 2011. 20 Chong et al. the novelty of this approach is that it provides strong evidence that anonymity and confidentiality of information technologies may be of great use in segments of the society in which keeping such anonymity is difficult to overcome.28 The results on knowledge and attitudes are important because these two factors have been shown to be the strongest protective factors in preventing STIs, HIV, and pregnancy among teens (Kirby, Lepore, and Ryan 2007). Furthermore, recent research has documented the important role that social norms play in responsible sexual behavior (Munshi and Myaux 2005; Ashraf, Field, and Lee 2014). By changing knowledge and attitudes in youth attending school, sexual education can ultimately play a fundamental role in achieving desirable aggregate changes in sexual behavior. A second contribution to the sexual health education literature is the focus on spillovers, through a two- stage experimental design. The results indicate that spillovers from treated to untreated classrooms in the same school are negligible. Lastly, the analysis provides strong indications that effects of the course were reinforced when treated individuals had larger percentages of their friend networks in treatment classrooms. The evidence is robust across a large set of sexual health attitude and knowledge indicators. In particular, the analysis found that students whose networks were more intensely treated had significant improvements in knowledge and attitudes, which the study interprets as social reinforcement effects or complementarities. These results demonstrate the positive externalities of the public provision of sex education: When an individual takes a sex education course, this decision has positive effects on sexual health outcomes among his or her close friends. This suggests that without collective action, there is an underprovision of sex education. These results provide an optimistic assessment of the use of information technologies to generate im- proved sexual health outcomes among the youth. The cost-benefit analysis suggests that because Internet- based sexual health education programs are extremely low cost, their benefits in terms of STI reductions actually justify the costs. References AIDS Control and Prevention Project (AIDSCAP). 1997. Making Prevention Work: Global Lessons Learned from the AIDS Control and Preventional Project 1991–1997. Arlington, VA: Family Health International. Angrist, J., and V. Lavy. 2002. “New Evidence on Classroom Computers and Pupil Learning.” Economic Journal 112 (482): 735–65. Ashraf, N., E. Field, and J. Lee. 2014. “Household Bargaining and Excess Fertility: An Experimental Study in Zambia.” American Economic Review 104 (7): 2210–37. Auriol, E., and M. Wartlers. 2012. “The Marginal Cost of Public Funds and Tax Reform in Africa.” Journal of Devel- opment Economics 97 (1): 58–72. Barrera-Osorio, F., and L. L. Linden. 2009. “The Use and Misuse of Computers in Education: Evidence from a Ran- domized Controlled Trial of a Language Arts Program.” Policy Research Working Paper No. 4836, World Bank, Washington, DC. Brener, N. D., J. O. G. Billy, and W. R. Grady. 2003. “Assessment of Factors Affecting the Validity of Self-Reported Health-Risk Behavior Among Adolescents: Evidence from the Scientific Literature.” Journal of Adolescent Health 33 (6): 436–57. Brent, R. J. 2011. “An Implicit Price of a DALY for Use in Cost Benefit Analysis of ARVs.” Applied Economics 43 (11): 1413–21. Bruhn, M., and D. McKenzie. 2009. “In Pursuit of Balance: Randomization in Practice in Development Field Experi- ments.” American Economic Journal: Applied Economics 1 (4): 200–32. 28 This approach provides an alternative to social marketing campaigns that attempt to increase the use of condoms in developing countries, and may be better suited than the latter for the segments of the population described in this paper. Among others, in Tanzania, a campaign to promote condom use among women was entitled “Talk to Him” and included posters and depicting a variety of confident, empowered, young women (AIDS Control and Prevention Project 1997). The World Bank Economic Review 21 Card, D., and L. Giuliano. 2013. “Peer Effects and Multiple Equilibria in the Risky Behavior of Friends.” Review of Economics and Statistics 95 (4): 1130–49. Chin, H. B., T. A. Sipe, R. Elder, S. L. Mercer, S. K. Chattopadhyay, V. Jacob, and H. R. Wethington et al. 2012. “The Effectiveness of Group-Based Comprehensive Risk-Reduction and Abstinence Education Interventions to Prevent or Reduce the Risk of Adolescent Pregnancy, Human Immunodeficiency Virus, and Sexually Transmitted Infections.” American Journal of Preventive Medicine 42 (3): 272–94. Collins, J., L. Robin, S. Wooley, D. Fenley, P. Hunt, J. Taylor, D. Harber, and L. Kolbe. 2002. “Programs-That-Word: CDC’s Guide to Effective Programs That Reduce Health-Risk Behavior of Youth.” Journal of School Health 72 (3): 93–9. Demographic and Health Survey (DHS). 2005. Encuesta Nacional de Demografía y Salud - ENDS Colombia 2005. Washington, DC: World Bank. Dhaliwal, I., E. Duflo, R. Glennerster, and C. Tulloch. 2013. “Comparative Cost-Effectiveness to Inform Policy in Developing Countries: A General Framework with Applications for Education.” In Education Policy in Developing Countries, edited by P. Glewwe, 285–338. Chicago: University of Chicago Press. Duflo, E., R. Glennerster, and M. Kremer. 2008. “Using Randomization in Development Economics Research: A Toolkit.” Handbook of Development Economics 4 (5): 3895–3962. Ebrahim, S. H., M. T. McKenna, and J. S. Marks. 2005. “Sexual Behavior: Related Adverse Health Burden in the United States.” Sexually Transmitted Infections 81 (1): 38–40. Fairlie, R., and J. Robinson. 2013. “Experimental Evidence on the Effects of Home Computers on Academic Achieve- ment among Schoolchildren.” American Economic Journal: Applied Economics 5 (3): 211–40. Flanigan, C. M., K. Suellentrop, M. M. Whitehead, and J. Smith. 2006. “Teens’ Sexual Experience 1995–2002.” Science Says 22: 1–5. Fletcher, J. M. 2007. “Social Multipliers in Sexual Initiation Decisions Among U.S. High School Students.” Demography 44 (2): 373–88. Fonner, V. A., K. S. Armstrong, C. E. Kennedy, K. R. O’Reilly, and M. D. Sweat. 2014. “School Based Sex Education and HIV Prevention in Low and Middle-Income Countries: A Systematic Review and Meta-Analysis.” PlosOne 9 (3): 1–18. Fortson, J. 2009. “HIV/AIDS and Fertility.” American Economic Journal: Applied Economics 1 (3): 170–94. Gallo, M. F., M. J. Steiner, L. Warner, T. Hylton-Kong, P. Figueroa, M. M. Hobbs, and F. M. Behets. 2007. “Self- Reported Condom Use is Associated with Reduced Risk of Chlamydia, Gonorrhea, and Trichomoniasis.” Sexually Transmitted Diseases 34 (10): 829–33. Goesling, B., S. Colman, C. Trenholm, M. A. Terzian, and K. A. Moore. 2014. “Programs to Reduce Teen Pregnancy, Sexually Transmitted Infections, and Associated Sexual Risk Behaviors: A Systematic Review.” Journal of Adoles- cent Health 54(5): 499–507. Karlan, D., and M. Valdivia. 2011. “Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions.” Review of Economics and Statistics 93 (2): 510–27. Kirby, D., A. B. Laris, and L. A. Rolleri. 2007. “Sex and HIV Education Programs for Youth: Their Impact and Important Characteristics.” Journal of Adolescent Health 40 (3): 206–17. Kirby, D., G. Lepore, and J. Ryan. 2007. “Sexual Risk and Protective Factors: Factors Affecting Teen Sexual Behavior, Pregnancy, Childbearing and Sexually Transmitted Disease: Which Are Important? Which Can You Change?” ETR Associates Working Paper, 1–106. Kivela, J., E. Ketting, and R. Baltussen. 2013. “Cost Analysis of School-Based Sexuality Education Programs in Six Countries.” Cost Effectiveness and Resource Allocation 11 (1): 17. Kling, J. R., J. B. Liebman, and L. F. Katz. 2007. “Experimental Analysis of Neighborhood Effects.” Econometrica 75 (1): 83–119. Krueger, A., and C. Rouse. 2004. “Putting Computerized Instruction to the Test: A Randomized Evaluation of a Scientifically Based Reading Program.” Economics of Education Review 23 (4): 323–38. Manski, C. 2013. “Identification of Treatment Response with Social Interactions.” Econometrics Journal 16 (1): 1–23. Miller, G. 2010. “Contraception as Development? New Evidence from Family Planning in Colombia.” Economic Journal 120 (545): 709–36. Munshi, K., and J. Myaux. 2005. “Social Norms and the Fertility Transition.” Journal of Development Economics 70 (1): 1–38. 22 Chong et al. Noar, S. M., H. G. Black, and L. B. Pierce. 2009. “Efficacy of Computer Technology-Based HIV Prevention Interven- tions: A Meta-Analysis.” AIDS 23 (1): 107–15. O’Donnell, L. N., A. S. Doval, R. Duran, and C. O’Donnell. 1995. “Video-Based Sexually Transmitted Disease Patient Education: Its Impact on Condom Acquisition.” American Journal of Public Health 85 (6): 817–22. Richards-Shubik, S. 2015. “Peer Effects in Sexual Initiation: Separating Demand and Supply Mechanisms.” Quanti- tative Economics 6 (3): 663–702. Ross, D. A., J. Changalucha, A. I. Obasi, J. Todd, M. L. Plummer, B. Cleophas-Mazige, A. Anemona, D. Everett, H. Weiss, D. C. Mabey, H. Grosskurth, and R. J. Hayes. 2007. “Biological and Behavioural Impact of an Adolescent Sexual Health Intervention in Tanzania: A Community Randomized Trial.” AIDS 21(14): 1943–55. Ruger, J. P.,A. B. Abdallah, N. Y.Ng, C. Luekens, and L. Cottler. 2014. “Cost-Effectiveness of Interventions to Prevent HIV and STDs Among Injection Drug-Using Women: A Randomized Controlled Trial.” AIDS and Behavior 18 (10): 1913–23. Sacerdote, B. 2011. “Peer Effects in Education: How Might They Work, How Big Are They and How Much Do We Know Thus Far?” In Handbook of Economics of Education, Vol. 3, edited by E. Hanushek, S. Machin, and L. Woessmann, 249–77. Amsterdam: North-Holland (Elsevier). Shaffi, T., K. Stovel, and K. Holmes. 2007. “Association Between Condom Use at Sexual Debut and Subsequent Sexual Trajectories: A Longitudinal Study Using Biomarkers.” American Journal of Public Health 97 (6): 1090–95. Thornton, R. L. 2008. “The Demand For, and Impact Of, Learning HIV Status.” American Economic Review 98 (5): 1829–63. World Health Organization (WHO). 2004. The World Health Report 2004. Geneva: World Health Organization. Supplementary Appendix Do Information Technologies Improve Teenagers’ Sexual Education? Evidence from a Randomized Evaluation in Colombia Alberto Chong, Marco Gonzalez-Navarro, Dean Karlan, and Martín Valdivia Supplementary Appendix S1: Summary Statistics for Individual Variables Table S1.1. Summary Statistics at Baseline INDEX INDIVIDUAL variables MEAN SD MIN MAX Q25 Q75 N Knowledge of Respondent knows: Vomiting is not an STI 0.101 0.301 0 1 0 0 4305 symptoms and symptom causes of STI Headache is not an STI 0.105 0.307 0 1 0 0 4211 index variables symptom Respondent knows STI Abnormal discharges from 0.307 0.461 0 1 0 1 4331 symptoms include: the penis/vagina Lesions/sores in genitals 0.185 0.388 0 1 0 0 4221 Painful urination 0.320 0.467 0 1 0 1 4334 HIV transmission does not Hygiene 0.665 0.472 0 1 0 1 4512 depend on: Food sharing 0.907 0.291 0 1 1 1 4512 Being in a pool with an 0.924 0.265 0 1 1 1 4512 HIV-positive person HIV cannot be transmitted: If a condom is used while 0.628 0.483 0 1 0 1 4512 having sexual intercourse with an HIV-positive individual HIV can be transmitted by: Having sexual intercourse 0.791 0.407 0 1 1 1 4512 without a condom A contaminated blood 0.684 0.465 0 1 0 1 4512 transfusion Sexual violence Respondent identifies as Nonconsensual touching of 0.845 0.362 0 1 1 1 4490 knowledge abusive sexual contact or genitalia, buttocks, breasts, index variables abuse: and inner thigh Forcible sex by husband on 0.758 0.429 0 1 1 1 4490 his wife Having sex with a person 0.759 0.427 0 1 1 1 4490 who is impaired due to alcohol If an individual changes 0.569 0.495 0 1 0 1 4490 his/her mind about sex even at the last minute The use of threats to obtain 0.670 0.470 0 1 0 1 4490 sex Respondent knows sexual abuse is more often than not per- 0.181 0.385 0 1 0 0 4343 petrated by a known person, not a stranger Prevention of Respondent knows one of Calendar-based methods 0.929 0.256 0 1 1 1 4504 STI knowledge the safest methods to Hormone injections 0.795 0.404 0 1 1 1 4504 index variables prevent an STI is not: Penis withdrawal 0.905 0.293 0 1 1 1 4504 Respondent knows one of the safest methods to prevent an 0.737 0.440 0 1 0 1 4504 STI is the use of condoms Table S1.1. Continued. INDEX INDIVIDUAL variables MEAN SD MIN MAX Q25 Q75 N Pregnancy Respondent disagrees that penis withdrawal is a safe method 0.562 0.496 0 1 0 1 4477 prevention to avoid pregnancy knowledge index variables Respondent knows women can become pregnant in their first 0.723 0.448 0 1 0 1 4506 sexual relationship Respondent knows unsafe Calendar-based methods 0.875 0.330 0 1 1 1 4516 methods to prevent a pregnancy include: Penis withdrawal 0.791 0.407 0 1 1 1 4516 Respondent knows safe Injections 0.471 0.499 0 1 0 1 4516 methods to prevent a Condoms 0.759 0.428 0 1 1 1 4516 pregnancy include: Respondent knows that emergency post-coital contraception 0.143 0.351 0 1 0 0 4477 pills have secondary effects Condom use Respondent knows condoms can be used only one time 0.608 0.488 0 1 0 1 4485 knowledge index variables Respondent knows one of the safest methods to prevent an 0.737 0.440 0 1 0 1 4504 STI is the use of a condom Respondent knows HIV can be transmitted by having sex 0.791 0.407 0 1 1 1 4512 without a condom Respondent knows HIV is not transmitted if a condom is 0.628 0.483 0 1 0 1 4512 used even if the person is HIV positive Condom use Respondent knows one of the safest methods to prevent a 0.759 0.428 0 1 1 1 4516 attitudes index pregnancy is by using a condom variables Respondent disagrees with It’s not right to carry a 2.894 1.168 1 4 2 4 4500 statements: condom because people may think that I planned to have sex If a woman wants to have sex 2.835 1.176 1 4 2 4 4525 without a condom, the man must not refuse Only women are responsible 3.516 0.931 1 4 3 4 4514 for unwanted pregnancies Respondent is confident of requesting that a condom be used 1.552 0.716 0 2 1 2 4533 Respondent is willing to delay sex if condoms are unavailable 0.678 0.467 0 1 0 1 4518 Respondent thinks he/she will use a condom in his/her next 0.805 0.396 0 1 1 1 4438 sexual relationship. Sexually Respondent thinks that: It is not right when people of 3.683 0.729 1 4 4 4 4520 conservative their age have sex with attitudes index several partners in the same variables month People of their age should 3.395 0.904 1 4 3 4 4544 wait to have sex Table S1.1. Continued. INDEX INDIVIDUAL variables MEAN SD MIN MAX Q25 Q75 N Respondent’s answer to: Age at which women should 19.577 3.296 10 30 18 20 4501 start having sex Age at which men should 18.449 3.248 10 30 16 20 4509 start having sex Respondent is confident he/she will have sex only when emo- 1.411 0.776 0 2 1 2 4525 tionally ready Sexual abuse Respondent disagrees with the idea that in case of sexual vi- 0.983 0.131 0 1 1 1 4481 reporting olence the person must not tell anyone. attitudes index variables Respondent thinks that when Must tell his/her family 0.713 0.452 0 1 0 1 4502 a teenager is suffering from Must tell the authorities 0.741 0.438 0 1 0 1 4502 sexual violence he/she: In case of rape, must seek 0.596 0.491 0 1 0 1 4502 medical help Must tell someone such as 0.021 0.144 0 1 0 0 4502 teachers, friends, etc. Source: Authors’ analysis from student surveys. Note: Variables with range [1–4] are coded as: totally agree, almost agree, sort of disagree, and totally disagree. Variables with range [0–2] are coded as: 0 – not so sure, 1 – almost sure, 2 – totally sure. Table S1.2. Attrition One week Six months One week Six months One week Six months One week Six months post inter- post inter- Condom post inter- post inter- Condom post inter- post inter- Condom post inter- post inter- Condom Dep. Var.: Attrited = 1 vention vention voucher vention vention Voucher vention vention Voucher vention vention voucher (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Treatment students 0.009 0.012 −0.008 (0.020) (0.019) (0.040) Spillover students 0.013 0.024 0.045 (0.024) (0.018) (0.046) Treatment students * 0.004 0.005 −0.001 socioeconomic level (0.007) (0.008) (0.016) Spillover students * 0.008 0.008 0.015 socioeconomic level (0.010) (0.008) (0.019) Treatment students * father’s 0.001 0.001 −0.000 education (0.002) (0.001) (0.003) Spillover students * father’s 0.001 0.001 0.003 education (0.002) (0.001) (0.003) Treatment students * 0.001 0.001 −0.001 mother’s education (0.001) (0.001) (0.003) Spillover students * mother’s 0.001 0.001 0.003 education (0.002) (0.001) (0.003) Constant 0.126*** 0.100*** 0.313*** 0.115*** 0.098*** 0.304*** 0.117*** 0.150*** 0.333*** 0.134*** 0.143*** 0.393*** (0.016) (0.014) (0.030) (0.017) (0.019) (0.035) (0.019) (0.029) (0.040) (0.023) (0.024) (0.047) Source: Authors’ analysis from student surveys. Note: *** p < 0.01, ** p < 0.05, * p < 0.1. Attrition = 1 for students observed at baseline but not at first follow-up (column 1), second follow-up (column 2), or without working cellphone or email for voucher offer 6 months after intervention (column 3). Columns 4–6 include socioeconomic status variable, columns 7–9 include father’s education variable, and columns 10–12 include mother’s education variable as controls. Standard errors clustered at the school level in parentheses.