DIME Financial and i2i PSD Program in Brief TRANSFORM DEVELOPMENT DIME HOW EARLY ONE SHOULD START FINANCIAL EDUCATION? EVIDENCE FROM A LARGE SCALE EXPERIMENT Introduction country. The pilot program was conducted in Ghana for a full school year but had no effects on learning and attitudinal and Economists disagree on many things, but there is an universal behavioral outcomes (Berry et al. 2015). consensus in the field that education is key for long-term growth and development. Evidence strongly suggests Brazil’s National Strategy of Financial Education (ENEF) piloted that what really matters for development is the quality of a program to strengthen basic financial-literacy skills in young education (Hanushek and Woessmann 2010). Unfortunately, elementary-school students. As with a similar successful there doesn’t seem to be an easy way to improve school program with high-school students, this new initiative tried to quality. How can we ensure that students develop their balance hard and soft (behavioral) skills. The pilot was drafted reading, writing, and math skills from their very first days in in 2015 and was divided into four modules: module 1 was school? This is a challenge that has led many countries to test structured for first to fourth graders, module 2 to fifth and sixth interventions aimed at improving school quality. Hundreds of graders, module 3 to seventh and eighth graders, and module interventions have been tried and rigorously tested, but we 4 to ninth graders. The modules had distinct content and are still far from having a good understanding of what policies approaches. They did not compete with regular content as new are effective in enhancing school quality (Evans and Popova, subjects, but were integrated into math or reading classes. 2015). While the development of reading, writing, and math Though different, the modules complemented each other as skills are important, they also have implications on other fields they focused on the same set of skills that must be built and such as financial literacy (Lusardi 2012). deepened as students progress through the school cycle. A recent meta-analysis compiled 71 experimental evaluations The starting assumption of this initiative is that financial from developing countries. The evaluations tested the impact aspects are already present in the daily lives of students, which of programs aimed at improving the quality of elementary is why financial education should be introduced early in life. schools (proxied by standardized test scores). Overall, the The introduction of financial education should consider two programs included in the meta-analysis showed very limited dimensions: spatial and temporal. The spatial dimension takes effects on learning outcomes (McEwan 2015). into account how individual decisions can have social impacts, while the temporal dimension introduces students to the The evidence suggests that improving school quality is intertemporal dimension embedded in various decision making. difficult. The limited effects found by short-term interventions with completely new subjects—such as financial education— Design of the Evaluation and Data Collection on learning outcomes should not be a surprise. To our knowledge, in addition to our study, there is only one The curriculum was tested in a randomly-selected set of experimental evaluation of a financial-education program schools to evaluate the pilot. The pilot was implemented in that targeted elementary school students in a developing 101 municipal schools in Joinvile, a municipality located in the south, and Manaus, a municipality located in the north. Only the third, fifth, seventh, and ninth grades were included for the evaluation. The idea was to test the impact of each module separately. We selected 72 schools in Joinville and 129 in Manaus for the evaluation, and randomly assigned 36 and 75 schools respectively to receive the pilot. The other 100 schools randomized out were used as the control group. The selection of schools was based on a two-level stratification: city and type of school. Within each city, schools were divided according to the cycles they offered (only primary school, only middle school, and both). This division resulted in six strata. Given the budget constraints, we skipped the baseline and allocated resources to a large-scale follow up at the end of Department for International i2i DIME Development from the British people TRANSFORM DEVELOPMENT the 2015 school year. Administrative data from school census classroom. Also, more than 50 percent of fifth to ninth graders and a large-scale assessment test were used to select the reported they received the material in the middle of the sample of schools and check balance after randomization. school year; and about 18 percent of ninth graders said they Primary data was collected by CAEd (Centro de Políticas received the material at the end of the year. There were some Públicas e Avaliação da Educação), a firm specialized cases where materials were not given at all to the students. in conducting educational surveys. Four different tools This percentage was up to about 10 percent among seventh were used for data collection: a standardized exam was graders. conducted in December 2015 to measure students’ financial proficiency, an attitudinal and behavioral questionnaire, and a Results questionnaire to capture socio-demographic characteristics of students. Teachers responsible for the class (third and fifth The literature on financial education suggests a simple grades) and for the financial-literacy curriculum answered a theory of change where an increase in knowledge precedes questionnaire as well. attitudinal and behavioral changes. The results section is structured around the same logic. Also, we shed light on The standardized exam included a block of questions aimed three potential channels influencing the main results. First, we to measure skills built due to the training. In fact, the exam used the mothers’ level of instruction to see if the children’s used the Item Response Theory (IRT) to measure the level of socio-economic background affects knowledge and attitudinal skills at different grades in a unique scale comparable among outcomes. Second, we checked whether the poor had more grades. difficulty in learning. We did so by using information on whether the child is a beneficiary of the Brazilian conditional Implementation cash transfer, Bolsa Família. Third, we looked at the role played by the quality of program execution on the main outcomes. All The pilot gave schools total freedom to use the modules to results discussed here refer to the intention-to-treat (or ITT). suit their school schedules. A pedagogical coordinator for each This is the impact on those eligible to receive training rather school was trained at the beginning of the school year as part than on those who actually received it. Results should be of the pilot. These trained coordinators were responsible for considered as the lower-bound effect of the program. passing on that knowledge to teachers in charge of integrating the financial-literacy curriculum into their regular classes. Financial proficiency Monitoring questionnaires were sent to coordinators during We used the standardized test score to measure the students’ the school year so that they could keep records of the financial proficiency. The estimates are measured in terms implementation. The questionnaires helped supervisors of standard deviation (SD) so they can be compared across to report any identified issue that could hamper good grades, subjects, and to other studies. execution of the pilot. Two extras questionnaires were sent to supervisors during the first semester of the school year The aggregate effect pooling all grades was 0.07 SD (or 2 so they could report on the intensity and regularity with percent over the control mean). We split the sample between which the materials were used in the classrooms. These primary (third and fifth grades) and middle (seventh and questionnaires were useful in identifying and correcting to ninth grades) school to check whether results were larger some extent ​ implementation issues during the year. The questionnaires answered by teachers during data Figure 1: ITT and QITT Estimates on Financial collection were useful to contextualize the results as they Proficiency of Third and Fifth Graders—95 showed, for instance, that more than half of the teachers percent CI couldn’t cover more than 60 percent of the training content in 0.25 0.20 0.15 Standard-Deviation 0.10 ITT 0.05 0.00 –0.05 –0.10 –0.15 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 2 ITT and QITT Estimates on Financial Figure 2:  ITT and QITT Estimates on Attitudes Figure 3:  Proficiency of Seventh and Ninth Toward Consumption of Fifth Graders—95 Graders—95 percent CI percent CI 0.30 0.25 0.25 0.20 0.20 Standard-Deviation Standard-Deviation 0.15 0.15 0.10 ITT 0.10 ITT 0.05 0.00 0.05 –0.05 0.00 –0.10 –0.15 –0.05 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 for older students. Figure 1 shows the results for third and Heterogeneous Effects fifth graders. The figure shows a positive, though small and statistically insignificant, ITT effect on financial proficiency of The empirical literature suggests that socio-economic younger students. This means the average pool effect was background correlates with children’s capacity to learn. driven by the impact on middle-school students. Interestingly, Experimental evidence shows that children from the QITT estimates show that the program had positive disadvantaged households are less likely to develop socio- effects and statistically significant effects above the median of emotional skills, and that poverty has implications for decision the financial proficiency distribution. making via behavioral biases such as limited attention and lack of self-control. We use the mother’s education as a proxy for Figure 2 presents the estimates for seventh and ninth household socio-economic backgrounds and participation in graders. The ITT effect on the older students is larger (0.09 the Brazilian CCT as a proxy for poverty status. SD) and statistically significant. The QITT estimates suggest that the program had larger effects at the lower tail of financial We proxy quality of pilot implementation using information proficiency distribution. This is an interesting result per se and on when the materials were delivered in the school and the contrasts sharply with the results for younger students. percentage of content covered in classrooms during the school year. Attitudes Toward Consumption and Saving Figure 4 shows the influence of the mothers’ education on We used the first score of a principal component analysis children’s learning achievements. The results indicate that to compute both the consumption and saving indexes. The children with more educated mothers (with at least high- indexes take into account the questions on financial attitudes school degrees) seem to learn more. We believe that this and habits. The indexes don’t include third graders because, might be either due to some type of demonstration effect in their case, the parents or others responsible answered the (assuming that children assimilate some of parents’ behavior questionnaire. and habits) or stimulation effect (more-educated mothers might value education and stimulate their children to study The results suggest that the pilot had positive effects on and learn). Interestingly, the results for attitudes indicate that the consumption index of 0.09 SD. Figure 3 shows that children of less-educated mothers are more likely to have the the ITT estimate for younger students was larger (0.12 SD) better attitudes towards consumption and saving. overall but mainly around the median of the consumption index distribution. This is an interesting finding as it seems These results are not easy to reconcile, but it could be that to suggest that the link between knowledge and attitudes families with less-educated mothers pay more attention outlined in the theory of change is not straightforward. to those issues, given their liquidity constraints. Because Interestingly, QITT estimates are relatively similar to the ITT the attitudinal indexes capture answers under hypothetical (0.08 SD) for older students. The effect on the saving index scenarios, it could be that older children do not take those was smaller (0.05 SD) and statistically significant only for hypothetical questions seriously. It could also be that older primary-school students. Overall, the QITT estimates suggest children are either overconfident and hence less likely to take that the program had distributional effects only for younger precautionary decisions regarding consumption and savings students. or even that they have a good understand of the institutional 3 ITT Estimates on Financial Proficiency by Figure 4:  ITT Estimates on Financial Proficiency by Figure 5:  Mothers’ Education—95 percent CI Poverty Status—95 percent CI 0.4 0.25 0.3 0.20 0.2 0.15 Standard-Deviation Standard-Deviation 0.10 0.1 0.05 0 0 –0.1 –0.05 –0.2 –0.10 –0.3 –0.15 –0.4 –0.20 Less than elementary Elementary High school Non poor Poor school school or more Pooled 3rd and 5th grade 7th and 9th grade Pooled 3rd and 5th grade 7th and 9th grade setting and thus don’t see much value in financial planning. different form zero only for primary school students where the Our data doesn’t allow us to rule out any of those competing program was well implemented. explanations. To check whether the poverty status of children affected their Main Conclusions ability to learn, we compared the impacts of the program on the samples of beneficiaries and non-beneficiaries of The preliminary findings of the impact of a pilot program on Bolsa Família. Interestingly, the point estimates for these financial education to elementary-school students suggest two groups are almost identical, but they are statistically that the program did not have large effects on knowledge significant only for the poor (see figure 5). and attitudes. However, the average effects hide considerable heterogeneity and distributional effects. ITT estimates show Finally, we check the role played by the quality of the that the program affected knowledge of middle-school program’s execution on the main results. Figure 6 shows that students only. For primary-school students, the effect was the training had positive and larger results when students positive and statistically significant only for the upper half of were exposed to at least 60 percent of the curriculum. knowledge distribution. Interestingly, we did not find a strong Again, the pooled effect is driven by middle-school students. correlation between knowledge and attitudinal outcomes as Considering the consumption index, the result was not these showed up stronger among fifth graders. different across implementation quality, it was positive and statistically different from zero regardless of the use of the We hypothesize two possible explanations for that result. First, material. For the savings index, the result is statistically it could be that older students have a clearer understanding of the institutional setting in Brazil and their hesitation in changing attitudes would be a rational decision given the incentives. The ITT Estimates on Financial Proficiency by Figure 6:  incentives in Brazil tend to privilege payments in instalments Quality of Program’s Implementation—95 and encourage use of credit cards, practices that financial percent CI education tries to minimize to avoid over-indebtedness. If 0.25 individuals are not credit constrained, then over-indebtedness shouldn’t be a big problem. An alternative interpretation would 0.20 suggest that formal knowledge doesn’t easily translate into 0.15 attitudinal and behavioral changes. Standard-Deviation 0.10 0.05 We look at heterogeneous effects to test three channels outlined by the empirical literature. First, we look at the 0 influence of the mothers’ education on children’s outcomes. –0.05 Second, we check whether the poor have more difficulty –0.10 learning and assimilating new subjects. Finally, we look at how –0.15 the program’s execution may have influenced the results. –0.20 Up to 60% 60% or more Our results suggest that knowledge is important but doesn’t seem to be sufficient to affect attitudes and behavior. The Pooled 3rd and 5th grade 7th and 9th grade challenge on how to translate knowledge into action remains open. 4