WPS6745 Policy Research Working Paper 6745 Background Paper to the 2014 Global Financial Development Report Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature Margaret Miller Julia Reichelstein Christian Salas Bilal Zia The World Bank Development Research Group Finance and Private Sector Development Team & Financial and Private Sector Development Financial Inclusion and Infrastructure Practice January 2014 Policy Research Working Paper 6745 Abstract This paper presents a systematic and comprehensive and duration of the intervention, delivery channel meta-analysis of the literature on financial education used, and type of population targeted. However, there interventions. The analysis focuses on financial education are a few key outcome indicators where a subset of studies designed to strengthen the financial knowledge papers are comparable, including those that address and behaviors of consumers. The analysis identifies savings behavior, defaults on loans, and financial skills, 188 papers and articles that present impact results of such as record keeping. The results from the meta interventions designed to increase consumers’ financial analysis indicate that financial literacy and capability knowledge (financial literacy) or skills, attitudes, and interventions can have a positive impact in some areas behaviors (financial capability). These papers are diverse (increasing savings and promoting financial skills such as across a number of dimensions, including objectives of record keeping) but not in others (credit default). the program intervention, expected outcomes, intensity This paper is a product of the Finance and Private Sector Development Team, Development Research Group; and the Financial Inclusion and Infrastructure Practice, Financial and Private Sector Development. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The corresponding authors may be contacted at mmiller5@worldbank.org and bzia@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature 1 Margaret Miller Julia Reichelstein Christian Salas Bilal Zia 2 JEL Codes: D12, D14, I22, O12 Sector Board: FSE 1 This paper is part of the background research conducted for the 2014 World Bank Global Financial Development Report on Financial Inclusion. The authors would like to acknowledge comments received on this document from Martin Cihak, from peer reviewers Annamaria Lusardi, David McKenzie, Florentina Mulaj, Valeria Perotti, Oya Pinar and Siegfried Zottel and from Caio Piza and other participants at a May 15 GFDR seminar on this topic. Research assistance was also provided by Hendrick Chan, Meghan Conway, and Anisha Mudaliar. 2 All authors are from the World Bank. Corresponding authors: Margaret Miller (mmiller5@worldbank.org) and Bilal Zia (bzia@worldbank.org) I. Introduction A decade ago there was limited interest in the topic of financial literacy. Now this issue is at the top of the policy agenda for national regulators, international organizations, researchers, and private financial institutions. An important reason for the increased attention to financial literacy is the global financial crisis which highlighted the importance of financial knowledge and skills for consumers. Anecdotal evidence from the crisis immediately suggested that people had taken on financial products – and risks – that they did not fully understand. Later empirical studies confirmed this relationship, including Klapper et al. (2012) with data from Russia and Gerardi et al. (2010) who link outcomes in the subprime housing market in the U.S. with consumers’ financial knowledge and skills. Furthermore, Lusardi and Mitchell (2012) present evidence from around the world suggesting that individuals even in developed countries have a difficult time understanding basic financial concepts. While financial literacy can clearly be a factor in avoiding financial risks, it can also be important for taking advantage of financial opportunities. Studies have shown that financial knowledge is linked to higher levels of retirement planning and savings (Behrman et al., 2012; Alessie, et al., 2011; Bucher- Koenen and Lusardi, 2011; Lusardi and Mitchell, 2011, 2007); investment decisions such as diversification (Abreu and Mendes, 2010) and investments in equities (Van Rooij, et al., 2011; Christelis, et al., 2010); credit management and satisfaction (Akin et al., 2012); and mortgage performance (Gerardi, et al., 2010; Quercia and Spader, 2008; Ding, et al., 2008). At the lower end of the financial market where consumers are seeking their first access to financial products and services such as opening basic accounts or borrowing small sums of money, there is also evidence that financial literacy may be important. Cole et al. (2011) find that use of insurance in India and bank accounts in Indonesia are both linked to higher levels of financial literacy. Using data from Finscope in Africa, Honohan and King (2009) likewise find a positive relationship between financial knowledge and use of financial products and services. There is also ample evidence that financial literacy levels are relatively low across a wide range of countries and negatively correlated with per capita income, 3 hence suggesting that while financial literacy is important for engaging in financial markets, it remains at low levels for many consumers. With this context, it is not surprising that a recent informal global poll of country officials and financial sector experts – the Financial Development Barometer 4 – undertaken for the 2014 World Bank Global Financial 3 The 2014 World Bank Global Financial Development Report (GFDR) compares national survey data on responses to three common questions used to evaluate financial knowledge – calculation of compound interest, understanding inflation, and diversification of risk. Based on data from more than 30 countries, average response rates were 56% correct for the question on compound interest, 63% for the question on inflation, and 48% for the question on risk diversification. Lower income countries scored significantly lower than high income countries - when higher income OECD countries are evaluated separately from the rest of the sample, the difference between the percent correct on compound interest is more than 20 percentage points (OECD at 65.5%, lower income countries at 44.5%). 4 The Financial Development Barometer is an informal poll available on-line through the website for the Global Financial Development Report. In total, officials from 21 developed and 54 developing economies participated in the survey. From 265 polled, 161 responded, a 61 percent response rate. 2 Development Report (GFDR) identifies financial education as the leading response to the question “What is the most effective policy to improve access to finance among low-income borrowers?” (See Figure 1.) What still remains to be proven is whether this faith in financial education is substantiated by evidence of impact. Are there approaches to teaching financial skills or modifying financial behaviors through educational programs, training, or other outreach activities which have reliable, positive results? The objective of this paper is to analyze the evidence of impact for financial literacy and capability interventions through a systematic review of the evidence. The review includes the use of meta-analysis, a statistical technique that pools data from different studies to test for significance in the enlarged sample of observations this creates. This paper is also different from most previous narrative reviews in focusing exclusively on research that analyzes the impact of financial education interventions. Key characteristics of 188 papers are coded to create a rich data set with the characteristics of the interventions as well as statistical information on the impact of programs on outcome variables such as general savings, retirement savings, and credit performance. This data set is then used for a descriptive analysis of the literature and for empirical tests using meta-analysis. More than 140 of the 188 studies identified through this review indicate that financial education can be helpful in improving financial outcomes, although it is important to note that most of these employ non- rigorous empirical methods and may suffer from selection bias or other econometric concerns. Some of the more recent studies employ rigorous analytical tools such as randomized control trials (RCTs), and the impacts reported across these papers are more reliable. Examples of positive impacts come from Cai et al. (2013) who find that financial education sessions for rural farmers increase take-up rates for insurance in China. In South Africa, financial messages delivered through a popular soap opera are shown to improve desirable financial behaviors such as borrowing from formal financial institutions rather than from higher cost options such as retailers (Berg and Zia, 2013). In India, Sarr et al. (2012) find that financial education increases the use of a no-frills savings account even months after the intervention ended. In the U.S. a non-RCT study of the Money Smart financial education curriculum by the FDIC (2007) finds that participants are more likely to open deposit accounts, save money, and adhere to a budget. However, our review of the literature also finds numerous papers (approximately 40 in our sample) citing either no impact or only a modest impact from the intervention which, may not justify the cost of financial education. Cole et al. (2011) for example, find that a financial education intervention among unbanked consumers in Indonesia is less effective at stimulating savings accounts than a small monetary incentive. Likewise, results from a media intervention in Kenya that included comics with financial literacy messages find no significant impact on key variables including savings rates (Eissa et al., 2013). In the U.S., Cole et al. (2013) evaluate mandated personal finance courses in high schools and find that they have no effect on financial outcomes, while training in mathematics is shown to benefit students through greater levels of financial market participation, more investment income, and better management of debt. Similarly, Hung and Yoong (2010) study retirement savings behaviors of adult populations in the U.S. and find that unsolicited financial advice has no impact on savings and investment decisions. 3 Pooling and systematically studying these varied impacts through meta-analysis has the potential to provide valuable policy insight on what works in financial education, as well as help identify where the research gaps lie. Yet, the diversity of the research in this field thus far makes such comparison – and drawing conclusions on effectiveness – difficult even if it is a logical response to the varied and constantly evolving needs in this area. 5 Variation in context, purpose and duration of training, target populations, and outcome measures is evident in Appendix 1, which presents brief information on each study and in the descriptive statistics for the 188 papers, which are presented in Section III. Nevertheless, our meta-analysis presents some key insights after controlling for observable differences across studies. Importantly, we find that financial education can consistently improve outcomes such as savings and record keeping, but does less well in preventing outcomes such as loan default. These results suggest a role for financial education in improving behaviors where individuals have the ability or slack to exert greater control. Arguably, loan default is imposed by external agencies (banks or other financial providers) and hence can only be avoided secondarily or over the long term if financial education leads to more prudent borrowing decisions. Savings and record-keeping, in contrast, are immediate and primary decisions that can be acted upon by targeted consumers. The only other paper which uses meta-analysis to evaluate the literature on financial literacy and capability (Fernandes et al., 2013) discusses another potential source of variance in the results for financial education interventions – omitted variables related to psychological traits such as impulse control, delayed gratification, and self-efficacy. They perform meta-analysis on both financial literacy and capability interventions, which they term “manipulated financial literacy” and on observational literature that links levels of financial knowledge or literacy to outcomes (termed “measured financial literacy” by the authors). Their findings indicate that measured financial literacy has a greater impact on financial outcomes than does manipulated financial literacy. They posit that omitted variables are a source of the different results because people with certain psychometric profiles are more likely to engage in activities that increase their financial literacy levels and improve their financial outcomes, but these behaviors (self-control for example) are not the typical focus of financial education interventions which focus on imparting financial knowledge. Yet, while psychometric measures may indeed explain selection into desirable financial behaviors and choices, they are unlikely to be the only omitted category – moreover, it is difficult to identify empirically whether such measures are the only (or even primary) omitted driver of financial choices. Our paper does not take a stand on the source of statistical bias in observational studies. Instead, we carefully classify and separate observational studies from impact studies, and more than double the number of financial education impact studies which are available for meta-analysis from slightly more than 80 in the case of Fernandes et al. to 188 -- all papers cited in Fernandes et al. are also included in this paper. Another key difference lies in the choice of variables and statistical rigor in the meta-analysis. As indicated previously, there is great diversity in the sample of studies which makes it impossible to 5 Over the life cycle there are a number of different financial skills and behaviors that are needed, innovation in financial markets can quickly create demand for new skills or make others irrelevant (such as writing checks) and new technologies are creating new delivery channels for training. Skills required by employed workers in high income countries, such as investment abilities and pension planning, are irrelevant for low income consumers in developing countries, further adding to the diversity of interventions present in a global review. 4 calculate an effect size that is meaningful and comparable across the entire literature, yet Fernandes et al. estimate such an effect size which includes interventions that are fundamentally different across many characteristics such as how outcomes are measured (binary or continuous), targeted populations, and mode of delivery. We take a more conservative approach and carefully screen and compare studies with similar outcome measures and intervention characteristics. This greater precision comes at a cost, however, as we are only able to compare binary outcomes for a subset of studies that pass our comparability screening. However, this careful analysis allows us to classify interventions among key topics such as savings, record keeping, and debt management, yielding more nuanced results in terms of the evidence of impact. Investigating the details of interventions and outcome measures in this way additionally allows us to present important stylized facts about financial literacy programs. This paper proceeds as follows. Section II describes the systematic review process including a brief discussion of meta-analysis and the rationale for using this tool for such a diverse body of literature. In this context, we look at previous narrative reviews and the literature they cover to help understand why their findings on the impact of financial education have not been consistent. Section II also discusses the search approach used to identify relevant studies in journals, working papers, and other publications, and the inclusion and exclusion criteria. Section III provides information about the data set including both the program descriptions and outcome variables that were identified. Section IV presents the results of the meta-analysis, its potential unique contribution, and limitations. Section V concludes. Appendices 1 and 2 provide supplemental materials on the meta-analysis, and Appendix 3 summarizes ways to strengthen the research protocols of financial literacy studies going forward. II. The Systematic Review Process Including Meta-Analysis This paper uses a systematic review process, including meta-analysis, to compare and contrast the findings of a large body of literature on the impact of financial literacy and capability interventions. The systematic review includes five steps: i) Hypothesis ii) Search approach and inclusion / exclusion criteria iii) Collection and coding of data iv) Statistical analysis (meta-analysis) v) Conclusions A systematic review of a diverse body of literature helps to identify patterns in the data and develop insights on the nature and quality of divergent evidence. The descriptive statistics that are presented in Section III based on the coded data from the 188 studies of financial education interventions provide valuable insights on what has been studied thus far, the research methods which have been used, and initial insights on the evidence. However, there are limits to what can be gleaned from these types of rough comparisons. Meta-analysis was developed to facilitate a statistically rigorous comparison of data across independent studies. By pooling data and statistical information across studies, papers which may individually point 5 to inconclusive or contradictory results may together yield a statistically significant finding. If there are adequate data available on means and standard deviations in the individual research papers, it may also be possible to identify an effect size which indicates the magnitude of change that may be expected. There are two general types of meta-analysis models: fixed effects and random effects. Fixed effects models assume that the studies are all roughly equivalent in terms of the intervention studied and thus are estimating the same outcomes. This means that data can be pooled and that larger studies with more observations are more highly weighted than smaller ones. Medical drug trials are common forms of fixed effects meta-analysis since the interventions are identical (provision of a drug or pill) and the outcomes are similar, observable, and uniformly measurable. Random effects models estimate the mean of a distribution of true effects but assume that each study is measuring a different effect size as the interventions and / or populations are not equivalent. Weights in the random effects models are more balanced across studies and those with large sample sizes do not dominate the results as they would in a fixed effects model. There is only one source of error in a fixed effects model – random error – and as the sample size grows this tends toward zero. In the random effects model, however, there are two sources of error – random error within populations and error in estimation of the true effect size across studies. This means that a large number of observations in a study address only the first kind of error – within population – and not the estimation error across studies, which requires a robust number of studies to increase the precision of the estimate. The diverse nature of the underlying studies on financial literacy and capability clearly leads to the use of a random effects model. When looking at the available studies and data for this exercise, it is reasonable to assume that there are still substantial sources of error from both limited population sizes for specific interventions and inadequate data on the scope of interventions and studies which are available for analysis. The effect sizes that are estimated in the regression analysis in Section IV, therefore, should not be taken as a true measure of effect size for financial literacy and capability interventions. This is not to say that the measures of effect size contain no information. They provide an aggregate measure of impact and are indicative of the state of evidence, but at the same time should not be seen as definitive proof for or against the null hypothesis (defined below) on financial education activities. More technical details on the estimation model used for this study are available in Appendix 2. Hypothesis – The null hypothesis for this study assumes that financial literacy and capability interventions do not affect the financial knowledge and/or financial outcomes of people who are subject to the treatment. Search approach and inclusion / exclusion criteria – For this paper we undertook a comprehensive search of a particular segment of the literature on financial literacy and capability – papers which evaluate the impact of interventions designed to strengthen financial knowledge and behaviors. A broad definition of “financial literacy and capability interventions” was used for this review, which included any kind of intervention (intentional or not) which would impact financial knowledge, attitudes and/or behaviors for individuals. 6 We identified papers meeting the criteria through several sources: 1) Search of peer-reviewed papers in Econlit under the broad terms financial awareness, financial capability, financial competence, financial education, financial knowledge and financial literacy between January 2000 and September 2013. 2) Search of papers included in previous literature reviews (starting with literature reviews published in 2007 – see table below). 3) Recent studies completed within the World Bank, many of which are listed in the website finlitedu.org. 4) Websites that are likely to include relevant studies, including the OECD, World Bank, Global Partnership for Financial Inclusion, and Alliance for Financial Inclusion (AFI). The process of evaluating studies for inclusion or exclusion was performed initially through a review of paper abstracts in the case of both papers identified through Econlit and those cited in previous literature reviews. In the case of World Bank studies and research found on websites the papers themselves were typically available and reviewed directly. All papers that reported on an intervention are included in Appendix 1, even those which lacked sufficient rigor or statistical results for use in the meta-analysis portion of this paper. Descriptive information from these papers is included, where possible, in the descriptive statistics presented in the following section of the paper. In order to reduce the number of studies to review to a manageable size and to focus the evidence on research results which had been screened for quality, only articles from peer reviewed journals were included from Econlit, for the period January 2000 to September 2013. The reasons for exclusion for papers located through Econlit searches are presented in Table 1. Table 1 Reasons for Exclusion of Research Papers Identified Through Econlit n) cy y / or n] ev on ac cy or tio ra m ly pr nti t er ra en it e lu cy r us te pe of ve l li io ) n co rv l l ra io Li sio Pa a et er ia ev ite us er i les ty al d g in nc bs n t nc lu Pr icy de in lL ap ili ci no n a a tic xc in ab an clu lP n ol e ed ia e i i Ar ra rE er t rP e( gf sa t( gf nc ap Fin icl h In a st tic rt lis th fo om in s i n in L a i eo rs n es O su o TA A ub lC f tc n k hi me k re ia t o n i s pe sin F tiv [T ire lin on eo ey P ou li TO n c en y pa ut Pa rip Bu Th rv on lit a a as Ca bo na m an at re at bi Su u ati sc Fi sure Re to ey d to y d tA De e al No t ea rv rv Ev M Su Su Exclusions for Econlit Searches (January 2000 to September 2013) Financial Attitudes 6 1 5 0 0 0 0 0 49 2 63 Financial Awareness 0 0 5 0 0 8 0 0 16 0 29 Financial Capability 2 3 6 1 0 1 1 1 62 0 77 Financial Competence 0 0 2 0 0 3 0 0 15 0 20 Financial Education 34 11 17 2 7 14 14 8 229 5 341 Financial Knowledge 17 14 22 9 4 5 4 1 161 1 238 Financial Literacy 14 13 47 17 2 26 6 7 5 12 149 Many of the excluded papers analyzed the importance of financial literacy or knowledge on various financial and non-financial outcomes. Using the terminology in Fernandes et al. (2013), these are 7 “measured financial literacy” papers. They were excluded because they did not test an intervention but they do provide valuable insights on the importance of financial literacy and knowledge on outcomes. Studies were included when they discussed any program, educational outreach, media intervention or other type of communication or training for consumers which could either strengthen knowledge (financial literacy) or modify attitudes and behaviors (financial capability). One of the insights from the Econlit search on these key terms is that a relatively small percentage of the literature involves an evaluation of some kind of financial literacy or capability intervention or program. Even for the terms “financial education” and “financial literacy” only about 10% of the studies in these categories evaluated the effectiveness of an intervention. Searches of “financial awareness” and “financial competence” yielded no impact evaluations and even the widely used term “financial capability” produced only two papers for the database out of a total number of 77 citations. With the exception of the search for “financial literacy,” a vast majority of the papers listed under the searches conducted on Econlit are not about personal financial literacy or capability. The review of nine previously published narrative reviews identified more than 500 unique references which included more than half of the 188 papers that are listed in Appendix 1. The references cited by these nine narrative literature surveys, however, have relatively little overlap. This can be seen with greater precision in Table 2 below which presents the Pearson correlation coefficients and associated R2 terms for reference lists across the literature survey papers. 6 In only two instances do the survey papers in Table 2 have a correlation coefficient above 0.2 (Martin 2007 with Hathaway and Khatiwada 2008 and Collins and O’Rourke 2010 with Agarwal et.al. 2010), which is already a relatively low level of correlation. The R2 terms in parentheses (which is simply the squared term of the correlation coefficient) are sometimes called the coefficient of determination and provide another indication of the strength of the relationship among the variables. Approximately 55% of the R2 figures are below .01 indicating a very weak relationship and relatively little overlap across studies. 6 A matrix was created based upon the references in each of these papers, for a total of 536 unique citations. For each of the nine literature reviews, a “1” or “0” was entered to indicate whether the reference was cited in the 2 review. This matrix was then used to calculate the Pearson correlation coefficients and r terms. 8 Table 2 Evaluating similarities between recent literature surveys on financial education (Pearson Correlation Table) 12 08 20 20 n 0 13 or 01 a 10 20 yh ad e2 0 20 12 09 01 m ll t iw rk he 20 s 20 im 07 l2 ne ce ou ha itc 20 Sk vi ta k a en ’R K Zi M ic Le e tin m n, O d er d al nd an nd ria or ar an nd f w Re ia cC M ea y ar ad sa Xu wa rd Ag M al ,M lin sa ha G l Lu gs at Co tin H as H 1.000 Martin 2007 (1.000) Hathaway and 0.416 1.000 Khatiwada 2008 (0.173) (1.000) -0.067 0.001 1.000 McCormick 2009 (0.004) (0.000) (1.000) 0.118 0.097 -0.017 1.000 Agarwal et al 2010 (0.014) (0.009) (0.000) (1.000) Collins and 0.161 0.104 0.026 0.425 1.000 O’Rourke 2010 (0.026) (0.011) (0.001) (0.181) (1.000) Gale and Levine 0.171 0.190 -0.088 0.174 0.047 1.000 2010 (0.029) (0.036) (0.008) (0.030) (0.002) (1.000) Hastings, Madrian, -0.057 0.009 -0.060 0.075 0.036 0.084 1.000 Skimmyhorn 2012 (0.003) (0.000) (0.004) (0.006) (0.001) (0.007) (1.000) -0.077 -0.099 -0.065 -0.088 -0.100 -0.083 -0.175 1.000 Xu and Zia 2012 (0.006) (0.010) (0.004) (0.008) (0.010) (0.007) (0.031) (1.000) Lusardi and -0.104 -0.038 -0.155 -0.105 -0.167 0.039 -0.002 -0.217 1.000 Mitchell 2013 (0.011) (0.001) (0.024) (0.011) (0.028) (0.002) (0.000) (0.047) (1.000) Total Number of References 44 35 45 82 57 61 100 132 187 The limited overlap in terms of the literature they are reviewing helps to explain the variety of findings in these narrative literature reviews. Martin (2007), at the Federal Reserve Bank of Richmond, provides one of the earliest reviews and has a limited number of references (44). He assesses the evidence positively, writing “Generally, we can conclude from this literature review that there is a need for financial education and that many existing approaches are effective. More specific conclusions might be best described as tentative given the current scarcity of research in some areas.” 7 Just one year later another literature review published by the Federal Reserve Bank of Cleveland (Hathaway and Khatiwada 2008) came to a much less positive conclusion using many of the same research papers. Hathaway and Khatiwada (2008) state, “Unfortunately, we do not find conclusive evidence that, in general, financial education programs do lead to greater financial knowledge, and ultimately, to better financial behavior. However, this is not the same as saying that they do not or could not – it is just that current studies, while at times illustrating some success, leave us with an unclear feeling about whether we can grant a blanket application of these results specifically, to financial education programs more generally.” 8 7 Martin (2007), page 22. 8 Hathaway and Khatiwada (2008), page 19. 9 Several years later, Gale and Levine 2010 also find little evidence of impact, stating, “None of the four traditional approaches to financial literacy – employer-based, school-based, credit counseling, or community-based – has generated strong evidence that financial literacy efforts have had positive and substantial impacts.” 9 Still, Gale and Levine 2010 recommend further work in this area, since they find financial planning to be beneficial as well as simplification initiatives to facilitate decision making which public policy can support. Lusardi and Mitchell 2013 provide the most comprehensive and recent of the narrative literature reviews with nearly 200 references. They offer a cautious endorsement of financial education activities but also refer to the issues of endogeneity and omitted variables, “Research on efforts to enhance financial literacy suggest that some interventions work well, but additional experimental work is needed to explore endogeneity and establish causality.” 10 While the search on Econlit was limited to papers that had appeared in peer reviewed journals, this exclusion criteria was not used for the many articles identified through the survey papers or from other key sources (websites, World Bank research) in order to increase the coverage of available research on this topic. Many of the papers included in the analysis have been written in the past few years and thus many (about one-third) are still in working paper format (see Figure 2). Apart from universities, the NBER and the World Bank are the main sources of working papers cited here. Figure 2 Type of Publication (n=188) 50% 38% 40% 34% 30% 28% 20% 10% 0% Peer-Reviewed Journal Academic Working Paper Non-Academic Publications & Books There are also numerous papers, briefs, and reports which are released by the many public, private, and non-profit organizations involved in this topic but which are not published through the academic press and subject to peer review. Known as “gray literature” these documents include research reports written by academics and researchers in think-tanks evaluating specific curricula using statistical methods and are thus also included in this literature search. However, since the financial education 9 Gale and Levine (2010), page 2. 10 Lusardi and Mitchell (2013), page 49. 10 provider may often also be the funder for the evaluation, these papers are identified separately from working papers published through academic institutions where such conflicts of interest are less likely to occur. Publication Bias – The first step in any systematic review involves the identification of relevant research so that conclusions are based on a comprehensive body of evidence which is not biased in some important way. If researchers or journal editors have a tendency to publish studies with a certain type of result (for example, in this case studies which show that financial education programs are effective in changing behavior, rather than studies showing no impact) this can result in one form of publication bias. The fact that many studies in the financial education literature (and in social science research more generally) involve little-known research projects with small sample sizes further exacerbates this problem since the broader community of practice may not be aware of instances where research results are not published or widely disseminated. By contrast, medical research trials which involve large public grants and large sample sizes are under more scrutiny and pressure to share results. Including studies from the “gray literature” which have not been published in journals can help to reduce the impact of this type of publication bias. In addition to including gray literature in the systematic review, this paper reviews all of the impact evaluations cited in previous narrative literature surveys as well as all interventions in the recently released meta-analysis by Fernandes et al. (2013). Further, papers were identified through searches of broad terms in Econlit, and on websites and through World Bank research. As such, this paper encompasses a broader, and more comprehensive, analysis of the literature than previous reviews, which reduces the likelihood of this type of publication bias. Appendix 2 also presents evidence from an econometric test (metafunnel in Stata) for publication bias to help show the extent to which the review may be affected by missing studies. Failing to publish the results from research is perhaps the most well-known type of publication bias, but it is not the only one. Other types of publication bias (or dissemination bias) which can negatively impact a systematic review include language bias (including only research in a certain language); availability bias (limiting consideration to research which is readily available to the researcher); cost bias (including only research papers that are available for free or at a low cost); familiarity bias (limiting the research to papers in one’s own discipline); and outcome bias (reporting selectively on the empirical data in papers to highlight certain types of results) (Rothstein, Sutton and Borenstein 2005). These various types of publication bias are not considered significant issues for this review. In terms of language bias, the vast majority of research on this topic internationally has been done in the United States (perhaps due to the greater level of consumer lending and less heavily regulated financial markets) and in the English language. Further, the search process did not use language as a criterion for inclusion and where non- English language research was identified, efforts were made to include these studies. In terms of availability, studies on financial literacy and capability are generally publicly available through online databases available to academic researchers or on websites of relevant NGOs, public agencies or multilateral organizations. No study was eliminated due to a cost consideration and in some cases authors were contacted directly to request copies of articles not readily available on-line (in a few instances of older research articles). Also, this review extends beyond typical economics journals to 11 include research published by academics in consumer and family sciences, education, sociology and psychology. All empirical findings are reported where a financial intervention was an explanatory variable in a regression model explaining a financial outcome or behavior, including financial knowledge gains. III. A Systematic Review of Financial Literacy and Capability Interventions Using Meta-Analysis The extensive literature search described in the previous section resulted in 188 journal articles, reports, and other publications that analyze the effectiveness of financial education interventions. The rigor required for meta-analysis of this literature resulted in a unique database that enables us to more precisely describe the types of financial education programs or interventions which have been evaluated as well as their reported impacts. The main relationship that is tested through meta-analysis in this paper is whether financial education interventions have an impact on outcome variables of interest such as general savings levels, retirement savings, record keeping, and credit performance. If data were available outcome variables could extend to other economic indicators of well-being such as total assets, consumption, or income; or on the other hand, to measures of vulnerability such as declaring bankruptcy or losing major assets such as a home or land. The key characteristics of financial education interventions are also recorded. In the context of meta- analysis these data are referred to as moderator variables because they help to explain the strength of the relationship between the predictor variable (in this case financial education) and the outcome. Moderator variables which are collected for this paper include information on the financial topic which is addressed (i.e. savings, credit, etc.), the type of delivery channel used (classroom instruction, mass media, etc.), the country where the intervention occurred, the location of the intervention (school, workplace, etc.), the number of hours of instruction, whether teacher training was involved, and whether the intervention focused narrowly on the individual or extended to the family or broader social setting. Another important data point which was collected through this exercise concerned the evaluation methods used. In particular, the dataset distinguishes whether studies used a randomized controlled trial (RCT) design. The Dataset – Moderator Variables Until 2008 the vast majority (98%) of studies on financial education interventions which were identified through this search focused on the United States. This started to change after 2008, when the financial crisis focused attention on the importance of this topic not only in the US but around the world (See Figure 3). The World Bank also contributed to the growth of literature on the impact of financial education interventions in developing country markets with the institution’s researchers contributing to more than 20 papers on this topic since 2008. As shown in Figure 4, Africa, Asia, and Latin America are the regions outside the United States with the largest shares of studies in our sample. 12 Figure 3 Share of Studies in USA (Pre- and Post-2008) 98% 100% 90% 80% 70% 60% 54% 50% 88 of 90 40% 53 of 98 30% studies 20% 10% 0% Up to and Including 2008 Post 2008 Figure 4 Region in which Intervention Conducted (n=188) Africa 2% 6% Asia 9% 2% Europe 6% Latin America USA Other 75% There is a wide range of financial topics covered by financial education interventions in the literature which were grouped into six categories in this paper (Figure 5). Interventions which provide information on a variety of financial issues, coded as “mixed” interventions in the data, are the most common type and represent more than 40% of the total sample. Financial literacy programs in the U.S. are most likely to involve either multiple messages (mixed) or focus on savings and retirement. In developing countries 13 courses that include business management are among the most common in our sample together with those involving multiple messages with each comprising about one-quarter of the sample, with savings and retirement messages a close third. Figure 5 Topic of Financial Education (n=188) 50% 43% 40% 30% 30% 20% 9% 7% 10% 6% 5% 0% Mortgage Credit Counseling Business Savings/Retirement Mixed Other Management As is evident in Figure 6, the majority of interventions in this sample involve direct contact with an instructor / advisor typically in a classroom or seminar setting, with a significantly smaller share (9%) involving individualized counseling. In terms of the cost per person reached, these kinds of interventions are likely to be the most expensive. Most interventions studied in the literature also focus on programs using only one delivery channel – classroom, video, phone advice, etc. – although there is widespread awareness in the communications field that a more comprehensive approach using multiple types of media (often referred to as a transmedia or 360o strategy) is more effective. A relatively small number of interventions in the evaluation literature look at use of mass media. These include use of TV soap operas in Africa and in the U.S., both of which show positive results in increasing awareness and shifting attitudes. 14 Figure 6 Intervention Channel (n=188) 60% 54% 50% 40% 30% 18% 20% 9% 6% 10% 5% 3% 3% 3% 0% Course Individual Mass Media Phone Print Online Mixed Other taught in Counseling person While financial education interventions are often associated with youth and programs taught in schools, these are only a relatively small share of the interventions which have been rigorously studied (18% including both high school and university programs). See Figure 7 for locations where financial education programs are provided. Figure 7 Location of Intervention (n=188) 30% 24% 25% 19% 20% 14% 14% 15% 11% 10% 8% 4% 5% 5% 0% 15 Most programs which have been evaluated are provided through community organizations or in the workplace. Workplace programs almost always relate to a “teachable moment” when employees have an opportunity to immediately use the information they are gaining, such as signing up for benefits or to save part of their salary. Many community based interventions also focus on teachable moments, sometimes related to helping people through challenging financial moments (such as credit counseling for people finding themselves overwhelmed by debt or mortgage counseling for prospective home buyers). Community interventions are almost exclusively focused on low income populations or other groups that are seen as financially vulnerable (battered women for example). Most of the financial education interventions which have been evaluated are relatively short in terms both of the number of hours of exposure (see Figure 8) and in terms of the period of time during which the exposure occurred (see Figure 9). As can be seen in Figure 8, slightly more than one-third of the interventions (38%) last 10 hours or less and 16% last just 2 hours or less. These shares are probably understated, as many of the interventions labeled as “varied” in terms of hours of exposure likely fit within the 10 hour or less categories. Figure 8 Intensity of Intervention (hours) 50% (n=148) 43% 40% 30% 20% 16% 11% 11% 11% 10% 6% 2% 0% ≤2 3-6 7-10 11-20 21-50 >50 Varied Figure 9 presents information on the duration of interventions and shows that more than one-third are delivered within one week or less and more than one-half of interventions are delivered within 6 months or less. Just as with the classification of intensity, these shares are likely to be underestimates as many of the interventions listed as “varied” are likely to be of relatively short duration (6 months or less). Since repetition is a key element of learning, the limited amount of time typically devoted to interventions may be a factor working against stronger impact results. 16 Figure 9 Duration of Intervention (n=156) 40% 33% 32% 30% 20% 17% 10% 5% 5% 7% 0% Overall, there has been a noticeable evolution in the literature on financial education interventions over the past five years. The increase in research beyond the United States, mentioned previously, has been accompanied by an increase in more rigorous research methods such as randomized controlled trials (RCTs). As can be noted in Figure 10, RCTs have gone from being relatively rare to frequently used, with more than 40% of papers since 2008 using these rigorous methods. In order to be as comprehensive as possible in reviewing the literature, studies were included which used a variety of research methodologies, not only RCTs. Figure 10 Share of Studies that were RCTs (n=188) 45% 43% 40% 35% 42 of 98 studies 30% 25% 20% 14% 15% 10% 13 of 90 studies 5% 0% Up to and Including 2008 Post 2008 17 IV. Results from Meta-Analysis This section presents the results from the application of meta-analysis statistical techniques on the data assembled on impact evaluations of financial education interventions. Before discussing the specific results it is important to place them in context. The diversity of the literature on financial education interventions means that even where there were common outcome variables (such as likelihood to save) the underlying interventions were each unique, making a direct comparison impossible. The limited number of studies available for evaluation in each of the categories (no more than ten, often closer to five) also reduces the strength of the statistical results. Despite these limitations, it is still useful to systematically and quantitatively evaluate the diverse body of evidence on financial education interventions to provide policy makers with an indication of what is working in financial education and to provide researchers insights on where knowledge gaps lie. Identifying the Issues That Could Be Tested Using Meta-analysis The majority of the 188 papers identified through the search as presenting results of financial literacy or capability interventions include potentially useful statistical data that could be used for meta-analysis. Many papers report on multiple regressions which are of interest and have multiple outcome variables or results from empirical tests on population subsets based on criteria such as educational attainment, income level, or score on a financial literacy test. In some cases, there are several types of explanatory variables related to the intervention. There are a total of 839 observations from the 188 papers; however, not all contain usable statistical information. For the meta-analysis of this paper, outcome (dependent) variables were coded among several major categories based on a review of the variable descriptions (savings, borrowing, financial literacy test scores, account opening, and record keeping). Within these categories there was further disaggregation to ensure comparability across studies for the meta-analysis. For example category “S1” corresponds to the savings category and is a binary variable with a value of “1” if the individual reported having saved, while “S4” is a continuous variable indicating total reported savings during a particular period of time. No effort was made to compare these types of disparate data through meta-analysis. Rather, we used the coding of outcome variables to identify those instances where existing studies had similar impact measurements and thus could be reasonably combined in a meta-analysis. The list of coded outcome variables is provided in Appendix 1. Once the outcome variables were categorized, the data were analyzed to determine the number of studies using this variable and to further identify those which reported adequate statistical information to be able to be included in the meta-analysis. We selected five studies as a minimum number for the meta-analysis.11 Further, each study had to provide the coefficient on the financial literacy / capability intervention and the standard error (or other statistical data from which to construct the standard error) and studies had to have comparable measurement criteria for the outcome variables. For example, studies that reported on a change in a financial literacy score were not used because the studies were based on potentially very different tests and scales for measuring financial literacy and thus did not 11 In the case of one of the outcome variables – loan defaults – the number of eligible papers dropped to just four after further review due to the exclusion of a study (Quercia and Spader 2008) which used a logit model instead of one employing linear probability. 18 represent a true comparison across similar results. This rigorous screening exercise produced just four outcome variables which were used in five or more separate studies. The four qualifying binary variables (listed below) were analyzed using meta-analysis. S1 = savings reported in past period (1 if positive) – binary S11 = contributes to retirement savings (1 if positive) -- binary B6 = defaulted on a loan (1 if positive) – binary R1 = keeps financial records / budgets (1 if positive) – binary In addition to the development of forest plots with the 4 outcome variables mentioned above, meta- regression analysis was used to determine if specific characteristics of the interventions (the explanatory variables) made significant contributions to results. Due to the small sample sizes these explanatory variables were tested individually on each outcome variable. The intensity of the intervention, measured in terms of number of hours of instruction or exposure, was the only continuous explanatory variable which also was widely reported on (with only one or two exceptions) among the studies used in the meta-analysis. The results from the forest plots and regressions looking at the intensity of the intervention are presented in the next section. Both meta-analysis and meta-regressions are conducted on study-level summary data because individual observations from all studies are not available. The meta-analysis performed is a two-stage process involving the estimation of a relevant summary statistic for each of a set of studies followed by the calculation of a weighted average of these statistics across the studies (Deeks, Altman, and Bradburn, 2001). In this case, pre-calculated effect estimates and their standard errors from each study are pooled. The meta-analysis includes a forest plot, in which results from each study are displayed as a square and a horizontal line, representing the intervention effect estimate together with its confidence interval. The area of the square reflects the weight the study contributes to the meta-analysis. The combined-effect estimate and its confidence interval are represented by a diamond. Meta-Analysis Results The most common single-issue topic for financial education interventions was savings and retirement. The greatest number of comparable papers also addresses this issue. The forest plot from the research papers that address the impact of interventions on savings are shown below in Figure 11. A forest plot graphically displays the treatment effects across multiple studies with a solid line representing the null hypothesis of no effect from the intervention – in this case from exposure to financial education outreach / training. Intensity refers to the number of hours of exposure. Channel refers to the type of intervention / content delivery channel and is coded as follows (1= course taught in person; 2= individual consulting; 3= mass media; 4=phone; 5=print; 6=online; 7=mixed; 8=other). 19 Figure 11 Papers testing savings behavior after financial education intervention % Authors Date Country Intensity Channel ES (95% CI) Weight Cole Sampson Zia 2011 Indonesia 2 1 0.01 (-0.07, 0.09) 11.54 Drexler Fischer Schoar 2011 Dominican Rep 15 1 0.14 (-0.02, 0.30) 3.73 Bruhn Ibarra McKenzie 2012 Mexico 4 7 0.03 (-0.01, 0.06) 24.33 Doi McKenzie Zia 2012 Indonesia 13 1 0.10 (0.00, 0.20) 8.29 Berg & Zia 2013 South Africa 26 3 -0.03 (-0.09, 0.03) 15.98 Bruhn Leão et al 2013 Brazil 36 1 0.05 (0.04, 0.06) 36.13 Overall (I-squared = 54.5%, p = 0.051) 0.03 (0.00, 0.06) 100.00 NOTE: Weights are from random effects analysis -.297 0 .297 The results are presented with the statistical output from the meta-analysis as well as with data on important characteristics of the interventions. The weight assigned to each of the six studies is provided in the far-right column and is based on the reported strength of the statistical result in the various studies. The size of the shaded squares for each study reflects their weight in the meta-analysis. The horizontal line through the shaded square is the confidence interval for the reported results. When this line intersects the vertical line which represents the null hypothesis (no effect from financial education) then the null cannot be rejected. In this particular meta-analysis, only two of the six papers reject the null hypothesis and indicate a positive impact on savings from financial education while four cannot reject the null hypothesis. Taken together, however, the meta-analysis finds that these papers do provide evidence of impact for financial education at the 95% confidence interval. The confidence interval for the pooled data across the six papers is represented by the diamond at the bottom of the graph and is on the right hand side of the null hypothesis, rejecting a finding that financial education has no impact on savings behavior. We lack the data on control group means to determine the true effect size for this finding. The meta-analysis in Figure 11 was based only on data from RCT studies to increase the quality of the data and thus to strengthen the cross-study analysis. The I-squared statistic, which is also presented in the forest plots, describes the percentage of variation across studies that is due to heterogeneity rather than chance. Thus, the I-squared is useful for determining the consistency of results between studies in a meta-analysis. A lower I-squared would indicate more consistency underlying the results of the sample studies while a higher I-squared would 20 indicate more heterogeneity in this regard. The p-value also provides a way to test for heterogeneity by examining the null hypothesis that all studies in the meta-analysis are evaluating the same effect. A p- value lower than 0.05 would reject this hypothesis at the 5% significance level and indicate that the results of the studies are more heterogeneous. As seen in Figure 11 (savings behavior), the I-squared was 54.5%, which is not generally considered a low number (indicating consistency) but which is lower than the I-squared terms for the other outcome variables, indicating relatively more consistency among the results for savings. The other meta-analysis results (Figure 12 - retirement savings behavior, Figure 13 - loan default behavior, and Figure 14 - record-keeping behavior) reveal higher I-squared statistics than Figure 11, indicating more heterogeneity between the studies involved in their respective samples. Meta-analysis to evaluate the impact of financial education interventions on retirement savings produced weaker results and was unable to reject the null hypothesis of no effect on retirement savings at the 95% confidence interval, but it could reject the null when the confidence interval was relaxed to 90% (p=0.077). Five studies were included in this analysis, all based on programs in the U.S., none using randomized controlled trials and only two using exclusively classroom methods. Results are shown in Figure 12. Figure 12 Papers testing retirement savings behavior after intervention % Authors Date Country Intensity Channel ES (90% CI) Weight Bernheim & Garret 2003 USA Varied 7 0.12 (0.07, 0.18) 22.23 Muller 2003 USA Varied 1 0.09 (-0.06, 0.24) 12.98 Lusardi and Mitchell - ALP article 2007 USA Seminars NA 0.23 (0.13, 0.33) 17.60 Hung Yoong 2010 USA Varied 2 0.06 (0.01, 0.11) 22.98 Beshears, Choi, Laibson, Madrian, Milkman 2011 USA One letter sent 5 -0.04 (-0.07, -0.01) 24.21 Overall (I-squared = 87.7%, p = 0.000) 0.08 (0.01, 0.16) 100.00 NOTE: Weights are from random effects analysis -.327 0 .327 Care should be taken with the interpretation of these results – the sample diversity and small sample size in both the meta-analysis of general savings and retirement savings require that the results are seen as indicative rather than definitive in terms of the impact of financial education on savings. Still, they 21 provide another piece of information to policy makers interested in the use of financial education for increasing saving, whether for general purposes or for retirement / old age. In addition to the topic of savings, numerous studies focused on credit and borrowing and used information on loan defaults after interventions as an outcome variable. The meta-analysis of these studies found the least evidence of impact from financial education interventions. Figure 13 presents this analysis in a forest plot. Even at the 90% confidence interval, the null hypothesis of no impact from the intervention cannot be discarded. Figure 13 Papers testing loan defaults after intervention % Authors Date Country Intensity Channel ES (90% CI) Weight Agarwal et al 2010 USA 35 1 -0.09 (-0.12, -0.06) 27.54 Gine Mansuri 2011 Pakistan 46 1,2 -0.02 (-0.04, -0.00) 28.82 Bruhn Ibarra McKenzie 2012 Mexico 4 7 0.02 (-0.01, 0.05) 26.81 Berg & Zia 2013 South Africa 26 3 0.04 (-0.04, 0.11) 16.83 Overall (I-squared = 87.6%, p = 0.000) -0.02 (-0.06, 0.02) 100.00 NOTE: Weights are from random effects analysis -.115 0 .115 While the results from the meta-analysis in figure 13 are significantly heterogeneous and do not show a significant overall effect size, they nevertheless provide interesting insights. Two substantive points of difference between the studies that did and did not show impact lie in whether the education intervention was targeted around a specific topic and whether there was an associated immediate opportunity to use the newly-acquired information. The two studies in particular that showed positive impacts on loan behavior were Agarwal et al (2010) and Gine and Mansuri (2011). The study population in Agarwal et al (2010) consisted of low-income households with lower credit scores who voluntarily participated in a counseling program provided by a non-profit organization. In this sense, the participants in the financial education intervention gained targeted information (specific to home ownership) and had the immediate opportunity to apply their 22 learning in a relevant context (purchase of a home). In this same light, the study participants in Gine and Mansuri (2011) were microfinance clients who could choose to participate in a business training course and loan lottery and then apply their learning directly to their microenterprises and loan behavior. On the other hand, Bruhn, Ibarra and McKenzie (2012) studied the effects of financial education programs that covered a broad range of financial topics in which the population did not experience an immediate circumstance like the purchase of a home or the financing of a business in which they could apply their learning. Bruhn, Ibarra and McKenzie (2012) evaluated a broad-based financial education program in Mexico, and among other factors, the diverse variety of topics covered and lack of a “teachable moment” to apply the learning may have contributed to the lower reported effectiveness of their program on loan default behavior. Similarly, while loan default was an outcome measured in Berg and Zia (2013), the topic was not central to their intervention which focused on source of borrowing (formal vs. retail credit) and gambling behavior. The final meta-analysis relates to the impact of financial education on record-keeping. Record keeping and tracking expenditures are often cited as critical elements of gaining control of one’s finances, much the way that many fitness and diet programs focus on recording eating and exercise habits to control weight and improve health. This is a behavior that is fully under the control of the individual as compared to decisions to default or save money which may be influenced by factors outside one’s control such as unexpected illness (and medical fees), loss of a job or other problems that leads to financial distress. The forest plot and regression results (Figure 14) combine data for record keeping related to both personal finance and to microenterprise or business records. This is necessary due to the small sample size but there are also reasons to believe that effects may be similar across these two types of interventions. Microenterprises frequently co-mingle finances between the business and individual owner so record keeping for a business is likely to include some common elements with personal financial management facilitating comparisons between these two types of interventions. The meta-analysis presented in Figure 14 indicates that financial education may positively encourage record keeping behaviors although the meta-analysis results are not quite significant at the 90% confidence interval (p=0.134). The effects are weighted fairly evenly across the studies which involve an unusually diverse set of country cases (Ghana/Kenya, Vietnam, Australia / New Zealand, Indonesia and Mexico) but which use, with one exception, classroom methods. 23 Figure 14 Papers testing record keeping behaviors after intervention % Authors Date Country Intensity Channel ES (90% CI) Weight Mano Akoten Otsuka Sonobe 2010 Ghana / Kenya 45 1 0.18 (0.05, 0.31) 9.16 Sonobe Suzuki Otsuka Nam 2011 Vietnam NA 1 0.06 (-0.00, 0.12) 21.11 Gibson McKenzie Zia 2012 Australia/NZ 2 1 -0.05 (-0.11, 0.02) 19.07 Doi Mckenzie Zia 2012 Indonesia 13 1 0.09 (0.04, 0.14) 23.49 Bruhn Ibarra McKenzie 2012 Mexico 4 7 0.01 (-0.03, 0.04) 27.17 Overall (I-squared = 69.1%, p = 0.012) 0.04 (-0.00, 0.09) 100.00 NOTE: Weights are from random effects analysis -.314 0 .314 Utilizing the meta regression framework, we were also able to analyze the relationship between the intensity of the treatment (in hours of instruction / exposure to the intervention) and the outcome of financial behaviors for savings (Figure 15), loan default (Figure 16), and record- keeping (Figure 17). 12 While the results were not statistically significant in part due to the small number of observations, they nevertheless provide a sense of the directional relationship between each outcome variable and the intensity of the associated treatment intervention. In Figure 15, the overall trend line suggests no direct relationship between intensity of treatment exposure and impact on savings behavior. However, on closer inspection of the individual data points, we can see that there is a positive relationship between hours of treatment exposure and impact on savings up to about 15 hours, whereafter impact declines. This result suggests that more instructional time may be important for increased positive impact, but there may also be a point at which diminishing returns set in. Exploring this relationship would be a good candidate for deeper investigation and further research. 12 Sufficient data on intensity of intervention was not available for the retirement savings behavior variable. 24 Figure 15: Relationship Between Intensity of Exposure and Savings Behavior .15 .1 beta .05 0 -.05 0 10 20 30 40 Intensity Figure 16 suggests a negative relationship between intensity of exposure and the loan default outcome. Increased exposure to the treatment intervention may reduce the likelihood of loan default, again providing an indication that more instructional time can lead to greater treatment impacts. Figure 16: Relationship between Intensity of Figure 17: Relationship between Intensity Exposure and Loan Default Behavior of Exposure and Record Keeping .05 .3 .2 0 beta beta .1 -.05 0 -.1 -.1 0 10 20 30 40 50 Intensity 0 10 20 30 40 50 Intensity Figure 17 shows that the intensity of exposure to a financial capability intervention is associated with improved record-keeping behavior. While this finding does not rise to the level of causality, the data here seem to be the most supportive of a relationship between these two variables when compared with the previous regression results (shown in Figures 15 and 16). It may be the case that additional instructional time is especially valuable for learning record keeping skills. Another possibility is that regardless of one’s financial situation, record keeping 25 can be implemented whereas savings and loan repayment are more dependent on financial standing. Further research to understand these initial findings would help to focus efforts in financial literacy. V. Conclusion The 188 papers that were identified for this meta-analysis represent an incredible level of diversity across numerous study characteristics including, importantly, outcome measures. The diversity in both general outcomes (savings, credit performance, record keeping, financial knowledge, etc.) and in how they are measured severely reduces the number of comparable studies for the meta-analysis and meta- regressions. Further, effect sizes are not comparable as typically data on control group means are not available for this estimation from the published works. Where randomized controlled trials have been done, the results of impact appear limited at best, indicating perhaps omitted variables or publication bias may be present for studies not employing these rigorous methods. With these caveats, some insights do emerge. Our meta-analysis suggests that financial education can impact some financial behaviors, including savings and record keeping. These are both considered fundamental to good personal financial management and are potentially behaviors where individuals can exert greater control than in the case of other outcomes such as loan default. Meta-analysis was unable to provide insights regarding the importance of program characteristics on impact due to the nature of the sample and lack of direct comparability. While the intensity (number of hours of exposure to the intervention) was weakly significant in the case of record keeping it was not significant in the other specifications of the model. Other characteristics which could reasonably be expected to influence the effectiveness of interventions (delivery mechanism, duration or period of time during which subjects were exposed to the treatment, location of the intervention such as school, community, workplace) were not shown to be significant but the small sample size limits the power of the statistical analysis. The difficulty in evaluating this literature for meta-analysis highlights the importance of strengthening and expanding the use of rigorous evaluation methods. Researchers should take care to report more complete statistical information so that others can better understand the strength of the findings and the limitations of the results as well as possibly use the findings in future applications of meta-analysis to this topic. There is also a potential value from defining outcome variables in common terms, including measurement and reporting of dependent variables, to facilitate comparisons across studies. To increase the likelihood that these variables could be used in a large number of studies they should employ simple and replicable definitions and measurement criteria. The use of common questions and/or survey instruments, such as those developed at the World Bank, DfiD, and OECD in the past few years, to measure financial literacy and capability in a target population is a step in the right direction and will help increase the availability of comparable data on what is working in financial education. Finally, new and ongoing research should make use of social science impact evaluation registries undertaken by the American Economic Association and 3ie, to provide a common repository of research in this area for future meta-analyses, as well as avoid issues related to publication bias. Appendix 3 26 details some key steps that could strengthen research protocols in financial literacy studies going forward. Finally, a surprising and yet common omission among virtually all studies is the analysis of costs and benefits of financial literacy interventions. Likewise, we did not find any discussion of potential alternative methods to achieve the desired outcomes across the 188 articles in this literature, which could help benchmark both costs and benefits. Such analyses could be simple to perform in individual studies, and certainly worthwhile to assess returns to development investments. 27 References Agarwal, Sumit, Gene Amromin, Itzhak Ben-David, Souphala Chomsisengphet, and Douglas D. Evanoff. 2009. “Do Financial Counseling Mandates Improve Mortgage Choice and Performance? Evidence from a Natural Experiment.” Federal Reserve Bank of Chicago Working Paper 2007-2009. Agarwal, Sumit, Gene Amromin, Itzhak Ben-David, Souphala Chomsisengphet, and Douglas D. Evanoff. 2010. “Learning to Cope: Voluntary Financial Education and Loan Performance during a Housing Crisis.” American Economic Review: Papers & Proceedings 100: 495-500. Akin, G. Gulsun, Ahmet Faruk Aysan, Serap Ozcelik, and Levent Yildiran. 2013. “Credit Card Satisfaction and Financial Literacy: Evidence from an Emerging Market Economy.” Emerging Markets Finance & Trade 48(5): 103-115. Allgood, Sam, and William B. Walstad. 2012. “The Effects of Perceived and Actual Financial Literacy on Financial Behaviors”. Available on SSRN. Amromin, Gene, Itzhak Ben-David, Sumit Agarwal, Souphala Chomsisengphet, and Douglas D. Evanoff. 2010. “Financial Literacy and The Effectiveness of Financial Education and Counseling: A Review of the Literature.” The Federal Reserve Bank of Richmond, Working Paper Series WP 07-03. Anderson, Joan Gray, Clarissa Uttley, and Claudia Kerbel. 2006. “Outcomes of a Workplace Financial Education Program.” The Journal of Consumer Education 23: 37-49. Anginer, Coville, Di Maro, Kanz, Legovini, Piza, and Zwager. 2013. “The Impact of Financial Education and Learning by Doing on Household Investment Behavior: Evidence from Brazil.” Bali Swain, Ranjula, and Adel Varghese. 2010. “Microfinance ‘Plus’: The Impact of Business Training on Indian Self Help Groups.” Uppsala University, Department of Economics Working Paper Series No 2010: 24. Barcellos, Silvia Helena, James P. Smith, Joanne K. Yoong and Leandro Carvalho. 2012. “Barriers to Immigrant Use of Financial Services.” Financial Literacy Center Working Paper, Wr-923-SSA. Baron-Donovan, Corinne, Richard L. Wiener, Karen Gross, and Susan Block-Lieb. 2005. “Financial Literacy Teacher Training: A Multiple-Measure Evaluation.” Association for Financial Counseling and Planning Education. Barron, John M. and Michael E. Staten. 2011. “Is Technology-Enhanced Credit Counseling As Effective as In-person Delivery?” Federal Reserve Bank of Philadelphia, Research Department Working Paper No. 11-11. Barua, Rashmi, Gauri Kartini Shastry and Dean Yang. 2012. “Evaluating the Effect of Peer-Based Financial Education on Savings and Remittances for Foreign Domestic Workers in Singapore.” Working Paper. Bauer, et al. 2011. “Dollar Works 2: Impact Evaluation.” University of Minnesota, Extension. Minneapolis, MN. Bayer, Patrick, B. Douglas Bernheim, and John Karl Scholz. 2008. “The Effects of Financial Education in the Workplace: Evidence from a Survey of Employers.” Economic Inquiry, Western Economic Association International 47 (4): 605-624. Becchetti, Leonardo and Fabio Pisani. 2012. “Financial education on secondary school students: the randomized experiment revisited.” AICCON Working Papers 98. Becchetti, Leonardo, Stefano Caiazza and Decio Coviello. 2011. “Financial education and investment in attitudes in high schools: evidence from a randomized experiment.” Centre for Economic and Internation Studeis Tor Vergata 9(10): 210. 28 Behrman, Jere R., Olivia S. Mitchell, Cindy K. Soo, and David Bravo. 2012. “The Effects of Financial Education and Financial Literacy: How Financial Literacy Affects Household Wealth Accumulation.” American Economic Review: Papers & Proceedings 102(3): 300-304. Bell, Catherine, Daniel Gorin, and Jeanne M. Hogarth. 2009. “Does Financial Education Affect Soldiers’ Financial Behaviors?” Available on SSRN. Berg, Gunhild, and Bilal Zia. 2013. “Financial Literacy through Mainstream Media: Evaluating the Impact of Financial Messages in a South African Soap Opera.” World Bank Working Paper, Washington, DC. Bernheim, B. Douglas, and Daniel M. Garrett. 2003. “The effects of financial education in the workplace: evidence from a survey of households.” Journal of Public Economics 87: 1487-1519. Bernheim, B. Douglas, Daniel M. Garret, and Dean M. Maki. 2001. “Education and saving: The long-term effects of high school financial curriculum mandates.” Journal of Public Economics 80: 435-465. Berry, Jim, Dean Karlan, and Menno Pradhan. 2013. “Social or Financial: What to focus on in youth financial literacy.” Working Paper. Beshears, John, James J. Choi, David Laibson, and Brigitte C. Madrian. 2006. “Simplification and Saving.” NBER Working Paper 12659, National Bureau of Economic Research, Cambridge, MA. Beshears, John, James J. Choi, David Laibson, Brigitte C. Madrian and Katherine L. Milkman. 2012. “The Effect of Providing Peer Information on Retirement Savings Decisions.” Financial Literacy Center Working Paper: WR-800-SSA. Birkenmaier, Julie, and Sabrina W. Tyuse. 2005. “Does Homeownership Education and Counseling (HEC) Help Credit Scores?” Journal of Social Service Resarch 32(2): 81-103. Bjorvatn, Kjetil, and Bertil Tungodden. 2010. “Teaching Business in Tanzania: Evaluating Participation and Performance.” CMI Working Paper 2009:9. Bolton, Lisa E., Paul N. Bloom, and Joel B. Cohen. 2011. “Using Loan Plus Lender Literacy Information to Combat One-Sided Marketing of Debt Consolidation Loans.” Journal of Marketing Research XLVIII (Special Issue 2011): 51-59. Borden, Lynne M., Sun-A Lee, Joyce Serido, and Dawn Collins. 2008. “Changing College Students’ Financial Knowledge, Attitudes, and Behavior through Seminar Participation.” J Fam Econ Iss 29: 23-40. Bowen, Cathy Faulcon and Heather M. Jones. 2006. “Empowering Young Adults to Control Their Financial Future.” Journal of Family and Consumer Sciences 98(1): 33-39. Boyce, Laurie, and Sharon M. Danes. 1998. “Evaluation of the NEFE High School Financial Planning Program® 1997-1998.” NEFE High School Financial Planning Program, Denver, CO. Braucher, Jean. 2001. “Report on a Study of Debtor Education in Bankruptcy.” American Bankruptcy Institute, Alexandria, VA. Bruhn, Miriam, and Bilal Zia. 2013. “Stimulating Managerial Capital in Emerging Markets: The Impact of Business Training for Young Entrepreneurs.” The World Bank Working Paper, Washington, DC. Bruhn, Miriam, Gabriel Lara Ibarra, and David McKenzie. 2012. “Does Financial Literacy Training Promote Saving and Responsible Credit Card Use?” Financial Literacy & Education Russia Trust Fund, Washington, DC. Bruhn, Miriam, Dean Karlan, and Antoinette Schoar. 2012. “The Impact of Consulting Services on Small and Medium Enterprises: Evidence from a Randomized Trial in Mexico.” Yale Economics Department Working Paper 100, New Haven, CT. 29 Bruhn, Miriam, Luciana de Souza Leao, Arianna Legovini, Rogelio Marchetti, and Bilal Zia. 2013. “Financial Education and Behavior Formation: Large-Scale Experimental Evidence from Brazil.” World Bank Working Paper, Washington, DC. Bucher-Koenen, Tabea, and Annamaria Lusardi. 2011. “Financial literacy and retirement planning in Germany.” Journal of Pension Economics and Finance 10(4): 565-584. Burhouse, Susan E., Angelisa M. Harris, and Luke W. Reynolds. 2007. “Banking on Financial Education.” FDIC Quarterly: Federal Deposit Insurance Corporation. Butt, Nicole, Stephen J. Haessler, Mark C. Schug. 2008. “Incentives-Based Approach to Implementing Financial Fitness for Life in the Milwaukee Public Schools.” Journal of Private Enterprise 24(1): 165-173. Cai, Jing, Alain De Janvry, and Elisabeth Sadoulet. 2013. “Social Networks and the Decision to Insure.” Available on SSRN. Calderon, Gabriela, Jesse M. Cunha, and Giacomo De Giorgi. 2013. “Business Literacy and Development: Evidence from a Randomized Trial in Rural Mexico.” Available on SSRN. Calderone, Margherita, Florentina Mulaj, Satadarshan Sadhu, and Leopold Sarr. 2013. “Does Financial Education Affect Savings Behavior? Evidence from a Randomized Experiment among Low Income Clients of Branchless Banking in India.” Working paper.+ Carlin, Bruice Ian, and David T. Robinson. 2010. “What Does Financial Literacy Training Teach Us?” NBER Working Paper 16271, National Bureau of Economic Research, Cambridge, MA. Carpena, Fenella, Shawn Cole, Jeremy Shapiro, and Bilal Zia. 2011. “Unpacking the Causal Chain of Financial Literacy.” World Bank Working Paper 5798, Washington, DC. Carswell, A.T. 2009. “Does Housing Counseling Change Consumer Financial Behaviors? Evidence from Philadelphia. Journal of Family and Economic Issues 30: 339-356. Chang, Yunhee and Angela Lyons. 2007. “Are Financial Education Programs Meeting the Needs of Financially Disadvantaged Consumers?” Networks Financial Institute Working Paper, Indiana State University. Chatterjee, Swarn, Leslie Green-Pimentel, and Pamela Turner. 2010. “Financial Education and Consumer’s Willingness to Change Behavior.” Investment Management and Financial Innovations 7(3): 73-81. Choi, James J., David Laibson, Brigitte C. Madrian. 2005. “Are Empowerment and Education Enough? Underdiversification in 401(k) Plans.” Brookings Papers on Economic Activity, 2, 151-214. Choi, James J., David Laibson, Brigitte C. Madrian. 2010. “Why Does the Law of One Price Fail? An Experiment on Index Mutual Funds.” The Review of Financial Studies 23(4): 1405-1432. Choi, James J., David Laibson, Brigitte C. Madrian, and Andrew Metrick. 2001. “Saving for Retirement on the Path of Least Resistance.” Prepared for Tax Policy and the Economy 2001. Clancy, Margaret, Michal Grinstein-Weiss, and Mark Schreiner. 2001. “Financial Education and Savings Outcomes in Individual Development Accounts.” Washington University in St. Louis Center for Social Development, St. Louis, MO. Clark, Robert L. 2004, “Sex Differences, Financial Education, and Retirement Goals.” Pension Design and Structure: New Lessons from Behavioral Finance. Clark, Robert L. and Sylvester J. Schieber. 1998. “Factors Affecting Participation Rates and Contribution Levels in 401(k) Plans.” Living with Defined Contribution Pensions: Remaking Responsibility for 30 Retirement, Ed. Olivia S. Mitchell and Sylvester J. Schieber. Philadelphia: University of Pennsylvania Press, 1998. 69-96. Clark, Robert L., Madeleine B. d’ Ambrosio, Ann A. McDermed, and Kshama Sawant. 2003. Presented at Sustainable Community Development: What Works, What Doesn’t, and Why, Washington, DC. Clark, Robert L., Melinda Sandler Morrill, and Steven G. Allen. 2010. “Evaluating Workplace Education for New Hires.” RAND Working Paper WR-791-SSA. Clark, Robert L., Melinda Sandler Morrill, and Steven G. Allen. 2011. “Evaluating Employer-Provided Financial Education Programs for Pre-Retirees.” Report prepared for the FINRA Investor Education Foundation. Clark, Robert L., Melinda Sandler Morrill, and Steven G. Allen. 2011. “Pension Plan Distributions: The Importance of Financial Literacy.” In Financial Literacy: Implication for Retirement Security and the Financial Marketplace, Ed. Olivia S. Mitchell and Annamaria Lusardi, New York: Oxford University Press, 40-58. Clark, Robert, Melinda Sandler Morrill, and Jennifer Maki. 2011. “Encouraging New Hires to Save for Retirement.” Financial Literacy Center Working Paper: WR-892-SSA. Cole, Shawn, and Gauri Kartini Shastry. 2009. “Smart Money: The Effect of Education, Cognitive Ability, and Financial Literacy on Financial Market Participation.” Harvard Business School Working Paper, Cambridge, MA. Cole, Shawn, and Gauri Kartini Shastry. 2012. “Smart Money: The Effect of Education on Financial Behavior.” Harvard Business School Working Paper, Cambridge, MA. Cole, Shawn and Gauri Kartini Shastry. 2010. “Is High School The Right Time To Teach Self-Control? The Effect of Financial Education and Mathematics Courses on Savings Behavior.” Cole, Shawn and Bilal Zia. 2013. “The Impact of Financial Education on Financial Knowledge, Behavior, and Outcomes: Evidence from a Randomized Experiment in South Africa.” Financial Literacy and Education Russia Trust Fund. Cole, Shawn, Jeremy Shapiro, and Bilal Zia. Forthcoming. “Video-Based Financial Literacy: Experimental Evidence on Savings, Credit, Insurance, and Budgeting from India.” Working Paper. Cole, Shawn, Xavier Gine, Jeremy Tobacman, Petia Topalova, Robert Townsend, and James Vickery. 2013. “Barriers to Household Risk Management: Evidence from India.” American Economic Journal 5(1): 104-135. Cole, Shawn, Anna Paulson, and Gauru Kartini Shastry. 2013. “High School and Financial Outcomes: The Impact of Mandated Personal Finance and Mathematics Courses.” Harvard Business School Working Paper 13-064. Cole, Shawn, Thomas Sampson, and Bilal Zia. 2011. “Prices or Knowledge? What Drives Demand for Financial Services in Emerging Markets?” The Journal of Finance LXVI (6): 1933-1967. Collins, J. Michael. 2007. “Exploring the Design of Financial Counseling for Mortgage Borrowers in Default.” Journal of Family and Economic Issues 28: 207-226. Collins, J. Michael. 2012. “The Impacts of Mandatory Financial Education: Evidence from a Randomized Field Study.” Journal of Economic Behavior & Organization. Collins, J. Michael, and Collin M. O’Rourke. 2010. “Financial Education and Counseling - Still Holding Promise.” Journal of Consumer Affairs 44(3): 483-498. Danes, Sharon M. 2004. “Evaluation of the NEFE High School Financial Planning Curriculum: 2003-2004.” National Endowment for Financial Education. 31 Danes, Sharon M. 2010. “Evaluation of the NEFE High School Financial Planning Curriculum: 2009-2010.” National Endowment for Financial Education. Danes, Sharon M. and Heather R. Haberman. 2007. “Teen Financial Knowledge, Self-Efficacy, and Behavior: A Gendered View.” The Journal of the Association for Financial Counseling and Planning Education 18(2): 48-60. Danes, Sharon M, Catherine Huddleston-Casas, and Laurie Boyce. 1999. “Financial Planning Curriculum for Teens: Impact Evaluation.” Association for Financial Counseling and Planning Education. De Mel, Suresh, David McKenzie, and Christopher Woodruff. 2011. “Getting Credit to High Return Microentrepreneurs: The Results of an Information Intervention.” The World Bank Economic Review 25(3): 456-485. Deb, Anamitra, and Mike Kubzansky. 2012. “Bridging the Gap: The Business Case for Financial Literacy.” Citi Foundation. DeVaney, Sharon, Elizabeth Gorham, Janet Bechman, and Virginia Haldeman. 1996. “Cash Flow Management And Credit Use: Effect Of A Financial Information Program.” Association for Financial Counseling and Planning Education. Di Maro, Vincenzo, Aidan Coville, Siegfried Zottel and Felipe Alexander Dunsch. 2013. “The Impact of Financial Literacy through Feature Films: Evidence from a randomized experiment in Nigeria.” Financial Literacy & Education, Russia Trust Fund. Ding, Lei, Roberto G. Quercia, and Janneke Ratcliffe. 2008. “Post-purchase Counseling and Default Resolutions among Low- and Moderate- Income Borrowers.” Journal of Race, Ethnicity, and Religion 30(3): 315-344. Doi, Yoko, David McKenzie and Bilal Zia. 2012. “Who You Train Matters: Identifying Complementary Effects of Financial Education on Migrant Households.” World Bank Working Paper 6157, Washington, DC. Dolvin, Steven D. and William K. Templeton. 2006. “Financial Education and Asset Allocation.” Financial Services Review 15(1): 133-149. Duflo, Esther, and Emmanuel Saez. 2003. “The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment.” The Quarterly Journal of Economics 188(3): 815-842. Duflo, Esther, William Gale, Jeffrey Liebman, Peter Orszag, and Emmanuel Saez. 2006. “Saving Incentives for Low- and Middle-Income Families: Evidence from a Field Experiment with H&R Block.” The Quarterly Journal of Economics, November 2006. Eades, Kenneth M., Jonathan Fox, Arthur Keown, and Michael E. Staten. 2012. “The Role of Professors in Improving Financial Literacy: Roundtable Session Highlights from the 2012 FMA Annual Meeting.” Journal of Applied Finance 23(1): 138-144. Eissa, Nada, James P Habyarimana, and William G Jack. 2013. “Impact of Cartoons on the Effectiveness of Financial Education: Evidence from a Randomized Experiment in Kenya.” Financial Literacy & Education Russia Trust Fund, Washington, DC. Elliehausen, Gregory, E. Christopher Lundquist, and Michael E. Staten. 2007. “The Impact of Credit Counseling on Subsequent Borrower Behavior.” The Journal of Consumer Affairs 41(1): 1-28. Employee Benefit Research Institute (EBRI). 1995. “Are Workers Kidding Themselves? Results of the 1995 Retirement Confidence Survey.” EBRI Issue Brief Number 168. 32 Federal Deposit Insurance Corporation. 2007. “A Longitudinal Evaluation of the Intermediate-Term Impact of the Money Smart Financial Education Curriculum Upon Consumer’s Behavior and Confidence.” Federal Deposit Insurance Corporation. Fernandes, Daniel, John G. Lynch, Jr., and Richard G. Netemeyer. 2013. “Financial Literacy, Financial Education, and Downstream Financial Behaviors.” Forthcoming, Management Science. Field, Erica, Seema Jayachandran, and Rohini Pande. 2010. “Do Traditional Institutions Constrain Female Entrepreneurship? A Field Experiment on Business Training in India.” American Economic Review 100(2): 125-129. Finke, Michael. 2012. “Financial Advice: Does it Make a Difference?” Pension Research Council Working Paper, Philadelphia, PA. Fischer, Greg, Alejandro Drexler, and Antoinette Schoar. 2010. “Keeping it simple: Financial Literacy and Rules of Thumb.” CEPR Development Economics Workshop, Barcelona, Spain. Fox, Jonathan J, and Suzanne Bartholomae. 2006. “Considerations in Financial Education Programming for Women.” The Journal of Consumer Education 23: 77-88. Fry, Tim R. L., Sandra Mihajilo, Roslyn Russell, and Robert Brooks. 2008. “The Factors Influencing Saving in a Matched Savings Program: Goals, Knowledge of Payment Instruments, and Other Behavior.” Journal of Family and Economic Issues 29: 234-250. Gale, William G., and Ruth Levine. 2010. “Financial Literacy: What Works? How Could It Be More Effective?” RAND Corporation Working Paper, Arlington, VA. Garman, E. Thomas, Jinhee Kim, Constance Y. Kratzer, Bruce H. Brunson, and So-hyun Joo. 1999. “Workplace Financial Education Improves Personal Financial Wellness.” Financial Counseling and Planning 10(1): 79-88. Gartner, Kimberly and Richard M. Todd. 2005. “Effectiveness of Online ‘Early Intervention’ Financial Education for Credit Cardholders.” Federal Reserve Bank of Chicago, “Promises and Pitfalls: As Consumer Finance Options Multiply, Who is Being Served and at What Cost?” Conference 2005. Gaurav, Sarthak, Shawn A. Cole, and Jeremy Tobacman. 2011. “Marketing Complex Financial Products in Emerging Markets: Evidence from Rainfall Insurance in India.” Supplement. Journal of Marketing Research 48: S150-S162. Gerardi, Kristopher, Lorenz Goette, and Stephan Meier. 2010. “Financial Literacy and Subprime Mortgage Delinquency: Evidence from a Survey Matched to Administrative Data.” Federal Reserve Bank of Atlanta Working Paper Series 2010-10. Gibson, John, David McKenzie, and Bilal Zia. 2012. “The Impact of Financial Literacy Training for Migrants.” World Bank Working Paper 6073, Impact Evaluation Series No. 57, Washington, DC. Gine, Xavier, and Ghazala Mansuri. 2011. “Money or Ideas? A Field Experiment on Constraints to Entrepreneurship in Rural Pakistan.” The World Bank Group Working Paper, Washington, DC. Gine, Xavier, Dean Karlan, and Muthoni Ngatia. 2013. “Social Networks, Financial Literacy and Index Insurance.” Working Paper. Gine, Xavier, Christina Martinez de Cuellar and Rafael Keenan Mazer. 2013. “Evaluating the effectiveness of loan disclosure reforms on consumer understanding and financial decision making: Evidence from Mexico.” Financial Literacy & Education, Russia Trust Fund. Goda, Gopi Shah, Colleen Flaherty Manchester and Aaron Sojourner. 2012. “What Will My Account Really Be Worth? An Experiment on Exponential Growth Bias and Retirement Saving.” National Bureau of Economic Research Working Paper No. 17927. 33 Goldstein, Markus, Francisco Campos, Obert Pimhizai, Antoinette Schoar, Mattea Stein, and Bilal Zia. 2013 draft. “The Impact and Network Effects of Financial Management and Vocational Training in Informal Industrial Clusters in Uganda.” Financial Literacy & Education Russia Trust Fund. Gray, Bobbi, Jennefer Sebstad, Monique Cohen and Kathleen Stack. 2010. “Can Financial Education Change Behavior? Lessons from Bolivia and Sri Lanka.” Global Financial Education Program Brief. Grimes, Paul W., Kevin E. Rogers, and Revecca Campbell Smith. 2010. “High School Economic Education and Access to Financial Services.” The Journal of Consumer Affairs 44(2): 317-335. Grinstein-Weiss, Michal, Michael Sherraden, William Gale, William M. Rohe, Mark Schreiner, Clinton Key. 2011. “The Ten-Year Impacts of Individual Development Accounts on Homeownership: Evidence from a Randomized Experiment.” Available on SSRN. Han, Chang-Keun, Michal Grinstein-Weiss, and Michael Sherraden. 2007. “Assets beyond Saving in Individual Development Accounts.” Center for Social Development Working Paper No. 07-25. Hartarska, Valentina and Claudio Gonzalez-Vega. 2005. “Credit Counseling and Mortgage Termination by Low-Income Households.” Journal of Real Estate Finance and Economics 30(3): 227-243. Hartarska, Valentina and Claudio Gonzalez-Vega. 2006. “Evidence on the effect of credit counseling on mortgage loan default by low-income households.” Journal of Housing Economics 15: 63-79. Hastings, Justine S, Ali Hortacsu, and Chad Syverson. 2013. “Advertising and Competition in Privatized Social Security: The Case of Mexico.” NBER Working Paper No. 18881. Hastings, Justine S., Brigitte C. Madrian, and William L. Skimmyhorn. 2012. “Financial Literacy, Financial Education and Economic Outcomes.” National Bureau of Economic Research, NBER Working Paper 18412. Hathaway, Ian, and Sameer Khatiwada. 2008. “Do Financial Education Programs Work?” Federal Reserve Bank of Cleveland, Working Paper 08-03. Haynes, Deborah C., George W. Haynes, and Clarann Weinert. 2011. “Outcomes of On-line Financial Education for Chronically Ill Rural Women.” Association for Financial Counseling and Planning Education. Haynes-Bordas, Rebecca, D.E. Kiss, and Tansel Yilmazer. 2008. “Effectiveness of Financial Education on Financial Management Behavior and Account Usage: Evidence from a ‘Second Chance’ Program.” Journal of Family and Economic Issues 29: 362-390. Heinberg, Aileen, Angela A. Hung, Arie Kapteyn, Annamaria Lusardi and Joanne K. Yoong. 2010. “Five Steps to Planning Success.” Financial Literacy Center Working Paper, WR-790-SSA. Hershey, Douglas, John Mowen, and Joy Jacobs-Lawson. 2003. “An Experimental Comparison of Retirement Planning Intervention Seminars.” Educational Gerontology 29: 339-359. Hershey, Douglas A., David A. Walsh, Ruby Brougham, Stephen Carter, and Alicia Farrell. 1998. “Challenges of Training Pre-Retirees to Make Sound Financial Planning Decisions.” Educational Gerontology 24(5): 447-470. Hilgert, Marianne A., Jeanne M. Hogarth, and Sondra G. Beverly. 2003. “Household Financial Management: The Connection between Knowledge and Behavior.” Federal Reserve Bulletin July 2003. Hira, Tahira K., and Cäzilia Loibl. 2005. “Understanding the Impact of Employer-Provided Financial Education on Workplace Satisfaction.” The Journal of Consumer Affairs 39(1): 173-194. 34 Hirad, Abdighani, and Peter M. Zorn. 2001. “A Little Knowledge Is a Good Thing: Empirical Evidence of the Effectiveness of Pre-Purchase Homeownership Counseling.” Low-Income Homeownership Working Paper Series, Joint Center for Housing Studies of Harvard University. Liho-01.4. Holzman, Robert. 2010. “Bringing Financial Literacy and Education to Low and Middle Income Countries: The Ned to Review, Adjust, and Extend Current Wisdom.” IZA Discussion Paper Series, IZA DP No. 5114. Holland, Joseph H., Doug Goodman, and Bethany Stitch. 2008. “Defined Contribution Plans Emerging in the Public Sector: The Manifestation of Defined Contributions and the Effects of Workplace Financial Education.” Review of Public Personnel Administration 28(4): 367-384. Huang, Jin, Yunju Nam, and Margaret S. Sherraden. “Financial Knowledge and Child Development Account Policy: A Test of Financial Capability.” Journal of Consumer Affairs 47(1): 1-26. Hung, Angela A, and Joanne K. Yoong. 2010. “Asking for Help: Survey and Experimental Evidence on Financial Advice and Behavior Change.” RAND Corporation Labor and Population Working Paper, WR-714-1. Joo, So-Hyun, and John E. Grable. 2005. “Employee Education and the Likelihood of Having a Retirement Savings Program.” Association for Financial Counseling and Planning Education: Alexandria, Virginia. Kanz, Martin, Alex Kaufman, Kelly Shue, and Benedikt Wahler. 2013. “Learning by Doing? Using Savings Lotteries to Promote Financial Inclusion.” Working paper.s Karlan, Dean, and Martin Valdivia. 2006. “Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions.” Review of Economics and Statistics MIT Press 93(2): 510- 527. Karlan, Dean, and Jonathan Zinman. 2012. "List Randomization for Sensitive Behavior: An Application for Measuring Use of Loan Proceeds." Journal of Development Economics 98(1): 71-75. Karlan, Dean, Edward Kutsoati, Margaret McConnell, Margaret McMillan and Christopher Udry. 2010. “Savings Account Labeling and Financial Literacy Training in Ghana.” J-PAL, IPA. Karlan, Dean, Margaret McConnell, Sendhil Mullainathan, and Jonathan Zinman. 2010. “Getting to the Top of Mind: How Reminders Increase Saving.” National Bureau of Economic Research, NBER Working Paper No. 16205. Kast, Felipe, Stephan Meier, and Dina Pomeranz. 2012. “Under-Savers Anonymous: Evidence on Self- Help Groups and Peer Pressure as a Savings Commitment Device.” National Bureau of Economic Research, NBER Working Paper No. 18417. Kim, Jinhee. 2007. “Workplace Financial Education Program: Does Impact on Employees’ Personal Finances?” Journal of Family and Consumer Science 99(1): 43-47. Kim, Jinhee, Dorothy C. Bagwell, and E. Thomas Garman. 1998. “Evaluation of Workplace Personal Financial Education.” Proceedings of the Personal Finance Employee Education, 2(1), 187-191, Roanoke, VA. Kim, Jinhee, E. Thomas Garman, and Benoit Sorhaindo. 2003. “Relationships among Credit Counseling Clients' Financial Wellbeing, Financial Behaviors, Financial Stressor Events, and Health.” Association of Financial Counseling and Planning Education: Alexandria, VA. Kim, Jinhee, Jasook Kwon, and Elaine A. Anderson. 2005. “Factors Related to Retirement Confidence: Retirement Preparation and Workplace Financial Education.” Association of Financial Counseling and Planning Education: Alexandria, VA. 35 Kimball, Miles S. and Tyler Shumway. 2006. “Investor Sophistication and the Participation, Home Bias, Diversification, and Employer Stock Puzzles.” Working Paper. Klinger, Bailey and Matthias Schündeln. 2011. “Can Entrepreneurial Activity be Taught? Quasi- Experimental Evidence from Central America.” World Development 39(9): 1592-1610. Koenig, Lori A. 2007. “Financial Literacy Curriculum: The Effect on Offender Money Management Skills.” Journal of Correctional Education 58(1): 43-56. Langrehr, Frederick W. 1979. “Consumer Education: Does It Change Students' Competencies and Attitudes?” Journal of Consumer Affairs 13(1): 41-53. Lawrence, Edward C. 2008. “Student Managed Investment Funds: An International Perspective.” Journal of Applied Finance 18(2): 67-83. Lee, Taejun, Tai Woong Yun, and Eric Haley. 2012. “The Interplay between Advertising Disclosures and Financial Knowledge in Mutual Fund Investment Decisions.” Journal of Consumer Affairs 46(2): 260-287. Loibl, Cäzilia and Tahira K. Hira. 2005. “Self-directed Financial Learning and Financial Satisfaction.” Association of Financial Counseling and Planning Education 16(1): 11-21. Loibl, Cäzilia and Tahira K. Hira. 2006. “A Workplace and Gender-Related Perspective on Financial Planning Information Sources and Knowledge Outcomes.” Financial Services Review 15(1): 21-42. Lusardi, Annamaria. 2002. “Preparing for Retirement: The Importance of Planning Costs.” National Tax Association Proceedings, 95th Annual Conference on Taxation. Lusardi, Annamaria. 2005. “Financial Education and the Saving Behavior of African-American and Hispanic Households.” Working Paper. Lusardi, Annamaria. 2005. “Saving and the Effectiveness of Financial Education.” Journal of Financial Transformation. 15: 159-169. Lusardi, Annamaria. 2008. “Household Saving Behavior: The Role of Financial Literacy, Information and Financial Education Programs.” National Bureau of Economic Research, NBER Working Paper No. 13824. Lusardi, Annamaria, and Olivia S. Mitchell. 2007. “Financial Literacy and Retirement Planning: New Evidence from the Rand American Life Panel.” Michigan Retirement Research Center Research Paper No. WP 2007-157. Lusardi, Annamaria, and Olivia S. Mitchell. 2009. “How Ordinary Consumers Make Complex Economic Decisions: Financial Literacy and Retirement Readiness.” National Bureau of Economic Research, Working Paper No. 15350. Lusardi, Annamaria, and Olivia S. Mitchell. 2011. “Financial Literacy around the World: An Overview.” National Bureau of Economic Research Working Paper 17107. Lusardi, Annamaria, and Olivia S. Mitchell. 2013. “The Economic Importance of Financial Literacy: Theory and Evidence.” National Bureau of Economic Research Working Paper 18952. Lusardi, Annamaria, Punam Anand Keller, and Adam M. Keller. 2009. “Increasing the Effectiveness of Retirement Saving Programs for Females and Low Income Employees: A Marketing Approach.” National Endowment for Financial Education. Lusardi, Annamaria, Punam Anand Keller, and Adam M. Keller. 2009. “New Ways to Make People Save: A Social Marketing Approach.” National Bureau of Economic Research, Working Paper 14715. Lusardi, Annamaria, Daniel Schneider, and Peter Tufano. 2011. “Financially Fragile Households: Evidence and Implications.” Brookings Papers on Economic Activity, Spring 2011: 83-150. 36 Lynch, John G Jr., Daniel Fernandes, and Richard G. Netemeyer. “A Meta-Analytic and Psychometric Investigation of the Effect of Financial Literacy on Downstream Financial Behaviors.” Work in progress, target: Journal of Consumer Research. Lyons, Angela C., and Erik Scherpf. 2004. “Moving from Unbanked to Banked: Evidence from the Money Smart Program.” Financial Services Review 13: 215-231. Lyons, Angela C., Yunhee Chang, and Erik M. Scherpf. 2006. “Translating Financial Education into Behavior Change for Low-Income Populations.” Financial Counseling and Planning 17 (2): 27-45. Lyons, Angela C., Tommye White, and Shawn Howard. 2008. “The Effect of Bankruptcy Counseling and Education on Debtors’ Financial Well-Being: Evidence from the Front Lines.” Money Management International. Maki, Dean M. 2004. “Financial Education and Private Pensions.” In Private Pensions and Public Policies, Eds. William G. Gale, John B. Shoven, and Mark J. Warshawsky. Washington, DC: Brookings Institution Press. Mandell, Lewis. 2005. “Financial Literacy – Does it Matter?” Presented at the ACCI Annual Meeting, Columbus Ohio, April, 2005. Available on Jump$tart. Mandell, Lewis. 2006. “Does Just in Time Instruction Improve Financial Literacy?” Credit Union Magazine, Savingteen Supplement, January 2006. Mandell, Lewis. 2008. “Teaching Young Dogs Old Tricks: The Effectiveness of Intervention in Pre-High School Grades.” in Financial Literacy for Children and Youth, Eds. Thomas A. Lucey and Katherine S. Cooter. DigitalTextbooks.biz, 2008. Mandell, Lewis. 2009. “Starting Younger: Evidence Supporting the Effectiveness of Personal Financial Education for Pre-High School Students.” The National Theater for Children. Mandell, Lewis. 2009. “The Impact of Financial Education in High School and College on Financial Literacy and Subsequent Financial Decision Making.” American Economic Association Meetings, San Francisco, CA. Mandell, Lewis and Linda Schmid Klein. 2007. “Motivation and financial literacy.” Financial Services Review 16(1): 105-116. Mandell, Lewis and Linda Schmid Klein. 2009. “The Impact of Financial Literacy Education on Subsequent Financial Behavior.” Journal of Financial Counseling and Planning 20(1): 15-24. Mano, Yukichi, Alhassan Iddrisu, Yutaka Yoshino, and Tetsushi Sonobe. 2011. “How Can Micro and Small Enterprises in Sub-Saharan Africa Become More Productive? The Impacts of Experimental Basic Managerial Training.” National Graduate Institute for Policy Studies, GRIPS Policy Research Center Discussion Paper: 11-06. Martin, Matthew. 2007. “Literature Review on the Effectiveness of Financial Education.” Federal Reserve Bank of Richmond Working Paper No. 07-03. Martinez A., Claudia, Esteban Puentes, and Jaime Ruiz-Tagle. 2013. “Micro-Entrepreneurship Training and Asset Transfers: Short Term Impacts on the Poor.” Economiay Negocios Universidad De Chile, Departamento De Economia, Santiago, Marzo. Mastrobuoni, Giovanni. 2007. “Do Better-Informed Workers Make Better Retirement Choices? A Test Based on the Social Security Statement.” Collegio Carlo Alberto Working Paper No. 51. Maurer, Trent W. and Sun-A Lee. 2011. “Financial Education with College Students: Comparing Peer-Led and Traditional Classroom Instruction.” Journal of Family and Economic Issues 32(1): 680-689. 37 Mayer, Neil, Peter A. Tatian, Kenneth Temkin, Charles A. Calhoun. 2009. “National Foreclosure Mitigation Counseling Program Evaluation: Preliminary Analysis of Program Effects.” Prepared for NeighborWorks® America, The Urban Institute. Maynard, Nicholas W., Preeti Mehta, Jonas Parker and Jeffrey Steinberg. 2012. “Can Games Build Financial Capability?” Financial Literacy Center Working Paper, WR-963-SSA. McCormick, Martha Henn. 2009. “The Effectiveness of Youth Financial Education: A Review of the Literature.” Journal of Financial Counseling and Planning 20(1):70-83. McKeever, Marybeth, and Education Development Center. 2011. “International Education Week: Somali Soap Opera Teaches Young People to Take Control of Their Finances.” USAID Impact Blog. Mills, Gregory, Rhiannon Patterson, Larry Orr, and Donna DeMarco. 2004. “Evaluation of the American Dream Demonstration: Final Evaluation Report.” Prepared for The Ford Foundation and Charles Stewart Mott Foundation, Abt Associates Inc. Muller, Leslie A., 2003. “Does Retirement Education Teach People to Save Pension Distributions?” Social Security Bulletin 64(4): 48-65. National Endowment for Financial Education. 2009. “Financial Management Practices of College Students from States with Varying Financial Education Mandates.” National Endowment for Financial Education, Denver, CO. Norvilitis, Jill M., Michael G. MacLean. 2010. “The Role of Parents in College Students’ Financial Behaviors and Attitudes.” Journal of Economic Psychology 31(1): 55-63. Osteen, Sissy, Glenn Muske and Joyce Jones. 2007. “Financial Management Education: Its Role in Changing Behavior.” Journal of Extension 45(3): 1077-5315. Peng, Tzu-Chin Martina, Suzanne Bartholomae, Jonathan J. Fox, and Garrett Cravener. 2007. “The Impact of Personal Finance Education in High School and College Courses.” Journal of Family and Economic Issues 28(1): 265-284. Quercia, Roberto and Jonathan Spader. 2008. “Does Homeownership Counseling Affect the Prepayment and Default Behavior of Affordable Mortgage Borrowers?” Journal of Policy Analysis and Management 27(2): 304-325. Robb, Cliff A., Patryk Babiarz, and Ann Woodyard. 2012. “The Demand for Financial Professional’s Advice: The Role of Financial Knowledge, Satisfaction, and Confidence.” Financial Services Review 21(4): 291-305. Romagnoli, Angela, and Maurizio Trifilidis. 2013. “Does Financial Education at School Work? Evidence from Italy.” Bank of Italy Occasional Paper No. 155. Rothstein, Hannah R., Alexander J. Sutton and Michael Borenstein, editors. 2005. Publication Bias in Meta-Analysis – Prevention, Assessment and Adjustments. John Wiley & Sons Ltd. Russell, Roslyn, Sandra Mihajilo and Rob Brooks. 2007. “Saver Plus- encouraging savings and increasing financial capabilities among low-income families.” RMIT University. Ryack, Kenneth. 2011. “The Impact of Family Relationships and Financial Education on Financial Risk Tolerance.” Financial Services Review 20: 181-193. Sanders, Cynthia K., Terri L. Weaver and Meg Schnabel. 2007. “Economic Education for Battered Women: An Evaluation of Outcomes.” Journal of Economic Abuse in the Lives of Survivors 27(1): 411-430. 38 Sarr, Leopold and Santadarshan Sadhu. 2013. “The Role of Financial Access, Knowledge and Service Delivery in Savings Behavior: Evidence from a Randomized Experiment in India.” Financial Literacy and Education Russia Trust Fund. Sarr, Leopald, Santadarshan Sadhu and Nathan Fiala. 2012. Bringing the Bank to the Doorstep: Does Financial Education Influence Savings Behavior among the Poor? Evidence from a Randomized Financial Literacy Program in India.” Institute for Financial Management and Research, Centre for Micro Finance Working Paper. Schreiner, Mark and Michael Sherraden. 2007. Can the Poor Save?: Saving and Asset Building in Individual Development Accounts. New Brunswick, NJ: Transaction, 2007. Schreiner, Mark, Michael Sherraden, Maragret Clancy, Lissa Johnson, Jami Curley, Michal Grinstein- Weiss, Min Zhan and Sondra Beverly. 2001. “Savings and Asset Accumulation in Individual Development Accounts: Downpayments on the American Dream Policy Demonstration.” Center for Social Development, Washington University in St. Louis. Schug, M.C. & Hagedorn, E.A. 2004. “Can charter schools improve financial and economic education: The case of the Milwaukee Urban League Academy of Business and Economics.” Journal of Private Enterprise 20(1) 100-111. Schug, Mark C., M. Scott Niederjohn, and William C. Wood. 2006. “Your Credit Counts Challenge: A Model Program for Financial Education for Low and Moderate Income Adults.” Journal of Private Enterprise 21(2): 196-208. Seligman, Jason S., and Rana Bose. 2012. “Learning by doing: Active employer sponsored retirement saving plan participation and household wealth accumulation.” The Quarterly Review of Economics and Finance 52: 162-172. Servon, Lisa J., and Robert Kaestner. 2008. “Consumer Financial Literacy and the Impact of Online Banking on the Financial Behavior of Low-Income Bank Customers.” The Journal of Consumer Affairs 42(2): 271-305. Seshan, Ganesh, and Dean Yang. 2013. “Transnational Household Finance: A Field Experiment on the Cross-Border Impacts of Financial Education for Migrant Workers.” Working Paper. Shelton, Gladys G. and Octavia Hill. 1995. “First-Time Homebuyers Programs as an Impetus for Change in Budget Behavior.” Financial Counseling and Planning Vol. 6. Sherraden, Margaret, Lissa Johnson, Baorong Guo and William Elliot. 2011. “Financial Capability in Children: Effects of Participation in a School-based Financial Education and Savings Program.” Journal of Family and Economic Issues 32: 385-399. Shim, Soyeon, Bonnie L. Barber, Noel A. Card, Jing Jian Xiao, and Joyce Serido. 2009. “Financial Socialization of First-year College Students: The Role of Parents, Work, and Education.” Journal of Youth Adolescence. 39(12): 1457-1470. Shim, Soyeon, Jing Jian Xiao, Bonnie L. Barber, and Angela C. Lyons. 2009. “Pathway to Life Success: A Conceptual Model of Financial Well-being for Young Adults.” Journal of Applied Developmental Psychology 30: 708-723. Shockey, Susan S. and Sharon B. Seiling. 2004. “Moving Into Action: Application of the Transtheoretical Model of Behavior Change to Financial Education.” Financial Planning and Counseling. Skimmyhorn, William. 2012. “Essays in Behavioral Household Finance.” Faculty of Arts and Sciences Theses and Dissertations, Harvard, Cambridge, MA. 39 Song, Changcheng. 2011. “Financial Illiteracy and Pension Contributions: A Field Experiment on Compound Interest in China.” Working Paper. Sonobe, Tetsushi, Yukichi Mano, John Akoten, and Keijiro Otsuka. 2010. “Assessing the Impacts of Management Skill Training in Ghana and Kenya.” Center for the Study of African Economies, Economic Development in Africa Conference 2010. Sonobe, Tetsushi, Ava Suzuki, Kijiro Otsuka and Vu Hoang Nam. 2011. “KAIZEN for Managerial Skills Improvement in Small and Medium Enterprises: An Impact Evaluation Study in a knitwear cluster in Vietnam.” DEPOCEN Working Paper Series No. 2012/29. Spader, Jonathan, Janneke Ratcliffe, Jorge Montoya, and Peter Skillern. 2009. “The Bold and the Bankable: How the Nuestro Barrio Telenovela Reaches Latino Immigrants with Financial Education.” The Journal of Consumer Affairs 43(1): 56-79. Stuart, Guy. 2012. “Uganda Financial Education Impact Project.” MFO- microfinance opportunities, Washington, DC. Tennyson, Sharon and Chau Nguyen. 2001. “State Curriculum Mandates and Student Knowledge of Personal Finance.” Journal of Consumer Affairs 35(2): 241-262. Tufano, Peter, Timothy Flacke and Nicholas W. Maynard. 2010. “Better Financial Decision Making amoung Low-Income and Minority Groups.” Financial Literacy Center Working Paper, WR-795- SSA. Van Rooij, Maarten, Annamaria Lusardi, and Rob Alessie. 2011. “Financial literacy and stock market participation.” Journal of Financial Economics 101: 449-472. Varcoe, Karen P., Allen Martin, Zana Devitto, and Charles Go. 2005. “Using A Financial Education Curriculum For Teens.” Financial Counseling and Planning 16(1): 63-71. VISA. 2009. “Financial Literacy Education in Sub-Saharan Africa.” VISA, FIFA Worldwide Partner. Walstad, William B., Ken Rebeck, and Richard A. MacDonald. 2010. “The Effects of Financial Education on the Financial Knowledge of High School Students.” The Journal of Consumer Affairs 44(2): 336-357. Wiener, Richard, Corinne Baron-Donovan, Karen Gross and Susan Block-Lieb. 2005. “Debtor education, financial literacy, and pending bankruptcy legislation.” Behavioral Sciences & the Law 23(3): 347- 366. Xiao, Jing Jian, Joyce Serido, and Soyeon Shim. 2012. “Financial Education, Financial Knowledge and Risky Credit Behavior of College Students.” In Consumer Knowledge and Financial Decisions: Lifespan Perspectives. Ed. Douglas J. lamdin. New York: Springer, 113-128. Xiao, Jing Jian, Benoit Sorhaindo, and E. Thomas Garman. 2006. “Financial Behaviors of Consumer in Credit Counseling.” International Journal of Consumer Studies 30(2): 108-121. Xu, Lisa, and Bilal Zia. 2012. “Financial Literacy around the World: An Overview of the Evidence with Practical Suggestions for the Way Forward.” World Bank Working Paper 6107, Washington, DC. 40 Figure 1 “What is the most effective policy to improve access to finance among low-income borrowers?” Financial Other education 33% 32% Promote new Better lending laws techs 18% 17% Responses from the 2012 Survey of the Financial Development Barometer Source: 2014 Global Financial Development Report 41 Appendix 1 No Authors Title Yr Summary Country Overall Subgroup R Intensity TM Subgroup Intervention Impact? Impact? C Type T Agarwal, Amromin, Do Financial Counseling 2009 Evaluates a counseling program (a mandatory third-party USA Yes Yes N 1.5 Y Low-FICO Individual Ben-David, Mandates Improve Mortgage review of mortgage contracts) in Illinois for borrowers (Low Counseling Chomsisengphet and Choice and Performance? considered at high risk. Counseling is seen as a burden, Income) Evanoff lowering both supply and demand of credit, and doesn’t affect default rates overall. However, the counseling is linked to a reduction in defaults among low-FICO-score 1 counseled borrowers. Agarwal, Amromin, Learning to Cope: Voluntary 2009 Evaluates a financial literacy counseling program, run by USA Yes Yes N 35 Y Low-FICO Classroom based Ben-David, Financial Education Programs Indianapolis Neighborhood Housing Partnership, Inc., and Low- seminar Chomsisengphet and and Loan Performance During which targets low- to moderate-income households. income Evanoff a Housing Crisis Analysis shows lower ex-post delinquency rates among counseling program graduates (with strongest effects 2 among low income and low FICO score individuals). Anderson, Uttley and Outcomes of a Workplace 2006 Individuals participated in a workplace financial planning USA Yes N/A N 2 N N/A Classroom based Kerbel Financial Education Program and retirement planning workshop in Rhode Island. For seminar most, participating in a financial education workshop resulted in the adoption of a number of the recommended financial practices. Participants seemed to have a greater inclination to change savings-related behaviors rather than 3 behaviors related to credit. Anginer, Coville, Di The Impact of Financial 2013 This study explores the effectiveness of an online stock Brazil Yes N/A N 10 Y N/A Stock market Maro, Kanz, Education and Learning by market simulator as a tool to overcome investor biases and simulator Legovini, Piza and Doing on Household improve performance. Using data from 40,000 participants, (online) Zwager Investment Behavior: Evidence results show limited impact on behavioral biases but an from Brazil impact on transition to participation in the actual stock 4 market. Bali Swain Varghese Microfinance ‘Plus’: The 2010 Measures impact of MFI training (through Self Help Group India Yes N/A N Varied Y Mostly Classroom based Impact of Business Training on Promoting Institutions) on households in India. The women in seminar Indian Self Help Groups provision of business training with microfinance leads to a Study positive impact on assets for the participating households 5 but no impact on income. Barcellos, Smith, Barriers to Immigrant Use of 2012 Investigated barriers to the use of financial services faced USA No No N Varied N Mostly Mixed Yoong and Carvalho Financial Services by immigrants and designed and evaluated new financial low- education materials targeted at immigrants. Focused on the income, potential barriers of limited English proficiency, lack of minority in U.S. experience and return migration expectations. Used study the RAND American Life Panel materials. Found groups receiving the education intervention had increased financial 6 knowledge but had little effect on overall behavior changes. Baron-Donovan, Financial Literacy Teacher 2005 Evaluates a train the trainer program designed to provide USA Yes N/A N 10-16 N N/A Classroom based Wiener, Gross and Training: A Multiple-Measure instructors with the tools needed to teach financial literacy training sessions Block-Lieb Evaluation to individual debtors. Trained teachers reported satisfaction with their training and felt prepared to teach. Pre- and posttest questionnaires reveal a 9% increase in financial knowledge and positive changes in attitude. 7 42 Barron and Staten Is Technology-Enhanced 2011 Compares outcomes for borrowers who received face-to- USA Yes N/A N Varied N N/A Individual Credit Counseling as Effective face credit counseling with borrowers who got counseling counseling, as In-Person Delivery? via the telephone or Internet. Technology-assisted delivery Internet and was found to generate outcomes no worse- at some margins telephone. 8 better – than face-to-face delivery of counseling services. Barua, Shastry and Evaluating the Effect of Peer- 2012 Evaluates a financial literacy intervention on Filipino Philippine No No Y 27 N Gender Classroom based Yang Based Financial Education on foreign domestic workers in Singapore, looking to effect s seminars Savings and Remittances for financial knowledge and behavior, savings and remittances. Foreign Domestic Workers in Preliminary evidence suggests that financial education has Singapore no effect on financial knowledge, outcomes or behavior. Assignment of a financial education class has a negative 9 effect on saving outcomes among female migrants. Bauer, Son, Hur, Dollar Works 2: Impact 2011 Evaluates the Dollar Works 2 education program, a USA Yes N/A Y 6 N Lower- Classroom based Anderson-Porisch, Evaluation Report University of Minnesota Extension program, targeting a income seminars Heins, Petersen, range of financial topics. The impact evaluation study was Hooper, Marczak, with community participants who are considered high risk Olson, Wiik in the financial education field. Found to be effective in pre- and post-test comparisons in enhancing financial 10 knowledge & behaviors. Bayer, Bernheim and The Effects of Financial 2008 Analysis of retirement planning seminars in workplaces USA Yes Yes N Varied N Lower- Mixed Scholz Education in the Workplace: where the employer sponsored pension plans. Retirement Income Evidence from a Survey of seminars were generally associated with significantly Employers higher rates of participation and contributions (esp. with high seminar frequency. Effect particularly strong for non- 11 highly compensated employees. Becchetti and Pisani Financial education on 2012 Analyzed the effects of financial education on a large Italy Yes Yes Y 16 N Location, Classroom based secondary school students: the sample of secondary school students in Italy. Found that income seminars randomized experiment the course increases significantly financial literacy at both revisited student and class level but the effect is different in different 12 urban environments. Becchetti, Caiazza Financial education and 2011 Evaluated the effects of a financial education training Italy No N/A Y 16 N N/A Classroom based and Coviello investment attitudes in high program for Italian high school students on their seminars schools: evidence from a investment attitudes. Difference-in-difference estimates of randomized experiment the effect of the course are not significant. However, the 13 course in finance did reduce the virtual demand for cash. Bell, Gorin and Does Financial Education 2009 The Amy Emergency Relief and the Federal Reserve Board USA No No N 2 days N N/A Individual Hogarth Affect Soldiers’ Financial jointly offered financial education courses to soldiers at the Counseling Behavior U.S. Army post at Ft. Bliss in El Paso TX. Participants were more likely to both report using an informal spending plan, and know the difference between discretionary and 14 non-discretionary spending. Berg and Zia Financial Literacy Through 2013 Financial education was provided via showing of a soap South Yes N/A Y 26 N N/A Mass Media Soap Operas: A Mass Media opera with financial messages in South Africa. Participants Africa Experiment in South Africa assigned to want the show had higher financial knowledge on issues covered in the soap opera plot, and (while no more likely to change borrowing amounts) were more 15 likely to borrow form formal sources. Bernheim and Garrett The Effects of Financial 2003 Evaluation, through a household survey, of workplace USA Yes N/A N Varied N N/A Mixed Education in the Workplace: financial education among US employers. Findings suggest 16 Evidence from a Survey of that employer-based financial education stimulates saving, 43 Households both in general and for retirement. Bernheim, Garrett Education and Saving: The 2001 Explores effects of state high school financial education USA Yes N/A N Varied N N/A Classroom based and Maki Long Term Effects of High course mandates across the US. Mandates have raised both seminar School Financial Curriculum exposure to financial curricula and subsequent asset 17 Mandates accumulation once exposed students reached adulthood. Berry, Karlan and Social or Financial: What to 2013 Examined the effectiveness of two distinct financial Ghana Yes N/A Y 8 N N/A Classroom based Pradhan Focus on in Youth Financial literacy training programs; Aflatoun, which incorporates seminars Literacy Training social skills into the trainings, and Honest Money Box, which focuses purely on financial skills. Both programs 18 had significant impacts on savings behavior, but little else. Beshears, Choi, Simplification and Saving 2006 Examines the effect of an opportunity to enroll in a USA Yes N/A N N/A Y N/A Sent letter Laibson and Madrian retirement plan with a pre-selected contribution rates and asset allocation. The intervention (a letter offering the plan as a simple, binary choice) increased plan enrollment rates 19 by 10 to 20 percentage points. Beshears, Choi, The Effect of Providing Peer 2013 Measured how receiving information about coworker’s USA No N/A Y N/A Y N/A Sent a simplified Laibson, Madrian Information on Retirement savings behavior affects recipients’ savings choices. Found 401(K) with and Milkman Savings Decisions an oppositional reaction: peer information decreased the information on savings of recipients who were shown the additional saving levels of 20 information. co-workers Birkenmaier and Does Homeownership 2005 Studied effectiveness of credit counseling, by USA No Yes N .75 N Higher Classroom based Tyuse Education and Counseling Homeownership Education and Counseling (HEC), on initial seminar, also (HEC) Help Credit Scores? credit scores. While client age, step-in-service delivery credit using mass systems, and initial credit score were found to be score, age media significant variables, no overarching significant impact was (older) 21 found due to the counseling. Bjorvatin and Teaching Business in 2010 300 small-scale entrepreneurs, clients of the MFI PRIDE Tanzania Yes Yes N 15.75-21 N Lower Classroom based Tungodden Tanzania: Evaluating Tanzania, took a business training course developed and education/tr seminar Participation and Performance implemented by University of Dar es Salaam aining Entrepreneurship Centre. Results found a positive average treatment effect on business knowledge. It also appears that training has a stronger effect on the entrepreneurs with less formal training or more irregular attendance. 22 Bolton, Bloom and Using Loan Plus Lender 2011 Two experiments demonstrate that a financial literacy USA Yes N/A N Varied N N/A Classroom based Cohen Literacy Information to intervention combining information about loans and course Combat One-Sided Marketing lenders can help consumers understand and respond to debt of Debt Consolidation Loans consolidation loan marketing (whereas a basic financial 23 numeracy intervention does not). Borden, Lee, Serido Changing College Students’ 2008 Pilot study examining influence of Credit Wise Cats (a USA Yes Yes N 1.5 N Ethnicity, Classroom based and Collins Financial Knowledge, financial education seminar presented by Students in Free gender seminar Attitudes and Behavior through Enterprise), on the financial knowledge and behaviors of Seminar Participation college students. Seminar effectively increased students’ financial knowledge, increased responsible attitudes toward credit and decreased avoidant attitudes towards credit from pre-test to post-test. A seminar format may be useful in reaching a wider audience of college students and, thus, 24 warrants future longitudinal evaluation. Bowen and Jones Empowering Young Adults to 2006 Evaluated the impact of the Commonwealth Credit Project- USA Yes N/A N 6 N N/A Classroom based 25 Control Their Financial Future a financial education class to college students- on seminar 44 participant’s changes in knowledge, attitudes, and behaviors related to credit cards and other money matters. Results indicate that a narrowly defined program can be a positive step in increasing knowledge, developing skills, and influencing attitudes. Braucher An Empirical Study of Debtor 2001 Study on effects debtor education courses, using data from USA No No N 2-4 Y N/A Classroom based Education in Bankruptcy five Chapter 13 trusteeships, on plan completion. Use of seminar wage orders (having the debtors’ employers pay the trustee directly) is associated with a higher rate of plan completion, while debtor education is not associated with 26 increased plan completion. Bruhn and Zia Stimulating Managerial Capital 2013 Studies the impact of a business and financial literacy Bosnia/ No Yes Y 9 N Women, Classroom based in Emerging Markets: The program on firm outcomes of young entrepreneurs in Herzegovi financial seminar Impact of Business and Bosnia and Herzegovina. The program did not influence na literacy Financial Literacy for Young business survival, but significantly improved business level Entrepreneurs practices, investments, and loan terms for surviving 27 businesses. Bruhn, Ibarra, Does Financial Literacy 2012 Evaluates the effectiveness of a large scale financial Mexico No No N 4 N Gender, Mixed McKenzie Training Promote Savings and literacy program offered by a major bank in Mexico City. Education Responsible Credit Card Use? Attending training results in a 9 percentage point increase level in financial knowledge and a 9 percentage point increase in saving outcomes, but no impact on borrowing behavior. The results suggest people are making optimal choices not 28 to attend financial education courses. Bruhn, Karlan and The Impact of Consulting 2012 RCT provided small and medium enterprises in Puebla Mexico Yes N/A N 208 N N/A Classroom based Schoar Services on Small and Medium Mexico, with the Puebla Institute for Competitive seminar Enterprises: Evidence from a Productivity. The study evaluated weather SMEs with Randomized Trial in Mexico access to management consulting services increased their managerial capital and led to better firm performance. Within one year, firms reported positive effects on return 29 on assets and total factor productivity. Bruhn, Laeo, Zia, Financial Education and 2013 Randomized evaluation of financial education program for Brazil Yes N/A Y 36-108 N N/A Classroom based Legovini and Behavior Formation: Large- Brazilian high school students. Program increased student seminars Marchelli Scale Experimental Evidence financial knowledge by a quarter of a standard deviation- from Brazil which led to a 1.4 percentage point increase in savings. Complementary workshop for parents induces children to 30 save even more. Burhouse, Harris, Banking on Financial 2007 Highlighted results from a Federal Deposit Insurance USA Yes Yes N 11-22 N Students, Classroom based Reynolds Education Corporation study of the effectiveness of the Money Smart Immigrants seminars financial education program. Study found that Money , Low & Smart graduates had positive changes in their knowledge, Moderate confidence, and behaviors related to financial matters. Income 31 Households Butt, Haessler, Schug Incentives-Based Approach to 2008 Evaluated the effectiveness of incentives for teachers to USA Yes N/A N N/A N N/A Classroom based Implementing Financial Fitness implement a financial and economic education curriculum. curriculum for Life in the Milwaukee Utilizing pre- and post-test scores of students to provide Public Schools empirical evidence regarding implementation of the financial education curriculum, analyses found that students’ knowledge gains were statistically significant and 32 that they significantly outperformed students who did not 45 participate in the curriculum. Cai, De Janvry and Social Networks and the 2013 RTC studies the influence of social networks on weather China Yes N/A Y .33-.75 N N/A Classroom based Sadoulet Decision to Insure: Evidence insurance adoption by rural famers in China, by examining seminar from Randomized Experiments influences and effects of financial education. For untreated in China farmers, the effect of having an additional treated friend on take-up is equivalent to offering a 15% reduction in the insurance premium. Positive social network effect is driven 33 by the diffusion of knowledge about insurance. Calderon, Cunha, and Business Literacy and 2013 Examines effectiveness of basic business training, provided Mexico Yes Yes Y 48 N Gender- Classroom based De Giorgi Development: Evidence from by CREA (a Mexican NGO), for female entrepreneurs in only seminar and Randomized Trial in rural Mexico. Found positive effects of basic training on women in Mexico profits, revenues and number of clients served. Also found study a positive effect on the use of “formal” accounting 34 techniques. Calderone, Mulaj, Does Financial Education 2013 Research presents results of a financial education program India Yes N/A Y 6 Y Low Video delivered Sadhu and Sarr Affect Savings Behavior? on savings behavior among low income clients of income in classroom Evidence from a Randomized branchless banking in India, using a RCT design. Two day clients of setting followed Experiment Among Low- intervention increased savings by 29% compared to the branchless by discussion Income Clients of Branchless control group. banking 35 Banking in India Campos, Goldstein, The Impact and Network 2013 The authors find some short--‐term effects from the Uganda Not Not yet Y 24 N Owners of Classroom based Pimhidzai, Stein and Effects of Financial draft financial management training on the core sample of yet avail small firms seminar (part of Zia Management and Vocational members corresponding to financial literacy, technical avail broader technical Training in Informal Industrial knowledge, optimism and adherence to standards and training) Clusters in Uganda procedures. Preliminary results do not show a statistically 36 significant positive impact on profits or revenues. Carlin and Robinson What Does Financial Literacy 2010 Studies how financial education changes savings, USA Yes N/A N 19 Y N/A Went through a Training Teach Us? investment, and consumer behaviors of high school “finance park” students who complete a tour through the Junior making Achievement Finance Park. Treatment effects strong. hypothetical Trained students showed greater up-take of decision financial support that was offered in the park-showing decision decisions support and financial literacy training are complements, not 37 substitutes. Carpena, Cole, Unpacking the Causal Chain of 2011 Studies the experimental impact of financial literacy, for India Yes N/A Y 12.5 N N/A Video based Shapiro and Zia Financial Literacy 1,200 urban households in Ahmadabad, on three classroom dimensions of financial knowledge. Found that financial seminars literacy does not enable individuals to discern costs and rewards that require high numeracy skills, but does 38 improve basic awareness of financial choices and attitudes. Carswell Does Housing Counseling 2009 Examines pre-purchase homeownership counseling for USA No N/A N 1 Y N/A Classroom based Change Consumer Financial low- to moderate-income households, specifically studying seminar Behavior? Evidence from whether counseled borrowers have experienced financial Philadelphia behavior changes within 5 years after having purchased a home. Mixed results. 39 46 Chang and Lyons Are Financial Education 2007 Uses data from a retrospective pre-test to investigate the USA Yes Yes N 8 N Financial Classroom based Programs Meeting the Needs impact that the financial education program for low-income ability seminar of Financially Disadvantaged adults, All My Money, has on financial behaviors. Found (pre), Consumers? that the program benefited all the participants, with the Gender, greatest improvement in behavior coming from those who Education reported lower levels of financial ability prior to the 40 program. Chatterjee, Green- Financial Education and 2010 Uses proprietary data comprising of 4,155 participants who USA No Yes N Varied N Age Classroom based Pimentel and Turner Consumer’s Willingness to attended financial education seminars conducted by a major Income seminar Change Behavior U.S. consumer credit counseling agency to examine Education knowledge gains. Results indicate that those most likely to gain knowledge from attending the financial education seminars were respondents between 18 and 24 years of age, lower income groups and respondents who did not 41 complete college. Choi, Laibson, and Are Empowerment and 2005 Assessed the effectiveness of the “empower and educate” USA No N/A N N/A Y N/A Mixed Madrian Education Enough? regulatory approach and media information at reducing and Underdiversification in 401(k) diversifying away from 401(k) employer stock holdings. Plans Found that educational interventions yielded remarkably 42 small changes in employee behavior. Choi, Laibson, and Why Does the Law of One 2010 Studied how test subjects at Harvard and Wharton allocated USA Modest N/A Y N/A Y N/A Provision of Madrian Price Fail? An Experiment on funds across four S&P index funds. When provided with information Index Mutual Funds transparent and salient fee information, investors shifted towards cheaper funds but did not come close to 43 minimizing index fund fees. Choi, Laibson, Saving for Retirement on the 2001 Assess the impact on savings behavior of several different USA No No N 1 Y N/A Classroom based Madrian and Metrick Path of Least Resistance 401(k) plan features, including financial education. seminar Analysis identifies a key behavioral principle that should partially guide the design of 401(k) plans: employees often follow “the path of least resistance.” Financial education found to not be a powerful mechanism for encouraging 44 401(k) retirement savings overall. Clancy, Grinstein- Financial Education and 2001 Studies effects of financial education provided by the USA Yes N/A N 12 N N/A Classroom based Weiss and Schreiner Savings Outcomes in American Dram Policy Demonstration on IDA savings seminar Individual Development outcomes for the “working poor”. Financial education has Accounts sizeable effects—courses need not be long to take 45 advantage of them. Clark Sex Differences, Financial 2004 Examines sex differences in retirement goals and responses USA Yes Yes N 1 Y Gender Classroom based Education and Retirement to employee financial education seminars. Seminars altered seminar Goals individual retirement objectives, with participants likely to raise income objectives post seminar. Gender differences observed in terms of desired retirement age, and savings 46 and investment choices. Clark and Schieber Factors Affecting Participation 1996 Evaluated factors affecting participation rates and USA Yes N/A N Varied Y N/A Print materials Rates and Contribution Levels contribution levels in 401(k) plans. Found that the quality in 401(k) Plans of communication from the workplace significantly affects participation rates, with greater effects seen from the provision of specifically-tailored information for a company’s own plan rather than generic information. 47 47 Clark, D’Ambrosio, Financial Education and 2003 Measures impact of TIAA-CREF Financial Education USA Yes Yes N 1 N Gender, Classroom based McDermed and Retirement Savings Seminars on employees of education institutions and non- Age seminars Sawant profit organizations’ retirement goals and savings behaviors. A significant proportion of the respondents revised their goals and planned to modify savings and 48 investment. Clark, Morill, and Evaluating Workplace 2010 Examined how employer-provided financial education for USA Yes N/A N Varied Y N/A Classroom based Allen Education for New Hires newly hired workers affects participation in retirement seminar savings plans. Found that auto-enroll programs and match 49 savings do increase retirement savings plan participation. Clark, Morrill and Encouraging New Hires to 2011 Using administrative data from five large employers, USA Yes Yes N Varied Y Age Mixed Maki Save for Retirement examines the impact of employer-provided financial education for newly hired workers on contributions to voluntary retirement savings plans. Average participation rates increased, while the same fraction of workers took 50 advantage of the full employer match once eligible. Clark, Morrill, and Pension Plan Distributions: 2011 Assessed the impact of employer-provided retirement USA Yes N/A N N/A N N/A Classroom based Allen The Importance of Financial planning seminars on workers. Found that participants seminar Literacy change their planned distribution choices after attending seminars that enhance financial knowledge and 51 understanding of retirement plans. Clark, Morrill, and Evaluating Employer-Provided 2011 Evaluated employer-provided financial education USA Yes Yes N Varied Y Low- Classroom based Allen Financial Education Programs programs. Found that workers who participate in the income, seminar for Pre-Retirees retirement planning seminars significantly increase their Less- level of financial literacy and their knowledge of retirement educated, programs. On the basis of this new knowledge, many Women workers alter their retirement plans including their expected age of retirement from their career employer, their plans to work after retirement, their age of claiming Social Security 52 benefits, and the method of accessing pension accounts. Cole and Shastry Is High School The Right Time 2010 Studied variation in state reforms on high school USA No Yes N Varied N Women High school To Teach Self-Control? The graduation requirements in regards to financial education to financial Effect of Financial Education evaluate the impact on asset accumulation. Found that state education classes and Mathematics Courses on mandates on financial literacy courses did not significantly Savings Behavior affect the propensity to save. Also found that reforms increasing the number of required mathematics courses 53 improved financial behavior for women, but not men. Cole and Zia The Impact of Financial 2013 Examined results from a randomized experiment in South South Yes N/A Y 8 N N/A Classroom based Education on Financial Africa to assess how a financial education program offered Africa courses Knowledge, Behavior, and by Old Mutual Bank, the “On the Money” program, affects Outcomes: Evidence from a individuals’ behavior. Found that financial education Randomized Experiment in significantly increases savings, both in terms of the number South Africa of people that report saving and in the monetary value of monthly savings. However, while members of “Women Developing Business” groups that received treatment report an improvement in their financial perceptions and higher welfare, members of burial society groups faced the opposite effects, highlighting the heterogeneous effect of financial education on different populations. 54 48 Cole, Gine, Barriers to Household Risk 2013 Conducted a series of randomized field experiments to test India No No Y 1 N N/A Individual Tobacman, Topalova, Management: Evidence from importance of price and non-price factors (including education sesion Townsend and India financial literacy) on adoption of an innovative rainfall Vickery insurance product. Found lack of trust, liquidity constraints, and limited salience are significant frictions that constrain 55 demand. Cole, Paulson and High School and Financial 2013 Paper reviews impact of state financial education mandates USA N Varied N High Classroom based Shastry Outcomes: The Impact of using plausibly exogenous exposure to these courses and school seminar Mandated Personal Finance administrative data on financial performance in adulthood students and Mathematics Courses such as investment holdings, credit performance. Financial education has no significant impact but mathematics 56 courses do improve financial outcomes. Cole, Sampson and Prices or Knowledge? What 2011 Through a field experiment in India and Indonesia, studied India, Modest Yes Y 2 Y Education, Classroom based Zia Drives Demand for Financial theories of low demand for financial services in emerging Indonesia Financial seminar Services in Emerging Markets? markets. Found strong correlation between financial Literacy literacy and behavior, with modest effects of financial education programs- increasing demand for bank accounts 57 only for those with limited education or financial literacy. Collins Exploring the Design of 2007 Analyzes the effects of the Credit Counseling Resource USA Yes N/A N Varied Y N/A Classroom based Financial Counseling for Center counseling provided to Chicago Neighborhood seminar Mortgage Borrowers in Default Housing Services borrowers in mortgage default. Found that additional counseling results in a reduced probability of foreclosure (greater success with longer durations of 58 counseling). Collins The Impacts of Mandatory 2012 Studied very low-income families in a subsidized housing USA No Yes N 12 N N/A Classroom based Financial Education: Evidence program who received financial education programs. Led seminar from a Randomized Field to improvements in self-reported behaviors but no 59 Study measurable effects on savings or credit. Danes Evaluation of the NEFE High 2004 18 month evaluation of the NEFE High School Financial USA Yes N/A N >10 N N/A Classroom based School Financial Planning Planning Program, a financial education curriculum seminar Curriculum: 2003-2004 provided by the National Endowment for Financial Education. Found that participants reported significant improvement in financial knowledge, behavior, and 60 confidence. Impact increased over time. Danes Evaluation of the National 2010 An impact evaluation of the competency-based NEFE High USA Yes N/A N 6 N N/A Classroom based Endowment for Financial School Financial Planning Program® (HSFPP), a seminars Education (NEFE) High curriculum provided by the National Endowment for School Financial Planning Financial Education® (NEFE®), shows that students who Program 2009-2010 completed the HSFPP reported significant improvement in their financial knowledge, behavior, and confidence 61 immediately after studying the HSFPP. Danes and Haberman Teen Financial Knowledge, 2007 Evaluated gender differences in financial knowledge, USA Yes Yes N >10 N Gender Classroom based Self-Efficacy and Behavior: A confidence, and behavior of teenagers (using NEFE data). seminar Gendered View Found male teens reinforced their existing knowledge, whereas female teens learned significantly more about finances in areas in which they were unfamiliar with prior 62 to the curriculum. Danes, Huddleston- Financial Planning Curriculum 1999 Studied the impact of NEFE financial planning curriculum USA Yes Yes N >10 N Gender Classroom based Casas and Boyce for Teens: Impact Evaluation on the financial knowledge, behavior, and self-efficacy of seminar 63 teenagers. Significant changes in all three found. 49 De Mel, McKenzie Getting Credit to High Return 2011 Examines intervention designed to improve access to credit Sri Lanka No N/A N 4 Y N/A Classroom based and Woodruff Microentrepreneurs: The among high-return microenterprises without subsidizing seminar Results from an Information interest rates or requiring group lending - the intervention Intervention consisted of info sessions providing details of the loan product. 10% of the microenterprises invited to the info meetings received a new loan. Loans seem to go to firms with more household assets (instead of high return firms). Results suggest that info alone is unlikely to be enough for most firms and point to the need for credit bureaus that 64 cover microfinance loans. DeVaney, Gorham, Cash Flow Management and 1996 Examined cash flow and credit use three months after USA Yes Yes N 14 N Gender, Classroom based Mechman and Credit Use: Effect of a participants in the Women’s Financial Information Age seminar Haldeman Financial Information Program Program (WFIP) completed as series of women’s financial information workshops. Highlights importance of feelings and attitudes in changing practices related to cash flow and 65 credit use. Di Maro, Coville, The Impact of Financial 2013 Explores the savings decisions of a group of micro- Nigeria Yes N/A Y 3 N N/A Movie Zottel and Dunsch Literacy Through Feature entrepreneurs and how short-term decisions and longer screenings Films: Evidence from a term behavior are influenced by a motivational film, The Randomized Experiment in Story of Gold. Found that influencing short-term decisions Nigeria is possible, but longer-run behavior is far less malleable 66 and the intervention had no impact on it. Ding, Quercia and Post-Purchase Counseling and 2008 Empirically examines the impact of a proactive post- USA Yes N/A N 1 Y N/A Over the phone Ratcliffe Default Resolutions Among purchase counseling service (provided by the Self-Help counseling Low-Income and Moderate- Ventures Fund’s Community Advantage Program) on Income Borrowers moderately delinquent mortgages. Found that counseling effectively increased the curing probability of delinquent 67 borrowers. Doi, McKenzie and Who You Train Matters: 2012 RCT in Indonesia which provided a financial literacy Indonesia Yes N/A Y 15 Y N/A Classroom based Zia Identifying Complementary program and evaluated its effects on financial knowledge, seminar Effects of Financial Education behaviors, remittance and savings outcomes. Found on Migrant Households training both the migrant and the family had a larger impact than training the family alone. Found large effects when provided at a teachable moment, but large variation of 68 impact. Dolvin & Templeton Financial Education and Asset 2006 Analysis the effects of a financial education seminar USA Yes Yes N 1.5 Y N/A Classroom based Allocation offered to employees in a workplace about to restructure its seminar 401(k) plan. Found seminar attendance is associated with increased portfolio diversification and improved risk 69 management. Duflo and Saez The Role of Information and 2011 Studied the role of information and social interactions in USA Yes N/A Y 9 Y N/A Attendance at a Social Interactions in employees’ decisions to enroll in a Tax Deferred Account benefits Retirement Plan Decisions: retirement plan within University. TDA enrollment was information fair Evidence from a Randomized higher in departments where some individual was treated. at the University Experiment However, effect was almost as large for individuals in treated departments who did not receive training as for 70 those who did. Duflo, Gale, Saving Incentives for Low- and 2006 Analyzed a randomized experiment where 14,000 low- and USA Yes N/A N 1 Y N/A In-person Liebman, Orszag and Moderate-Income Families: middle-income tax filers in H&R Block offices in St. Louis individual 71 Saez Evidence from a Field received varying match percentages. Take up rates counseling 50 Experiment with H&R Block increased accordingly. Tax professionals significantly influenced contribution choices- suggesting that both incentives and information affect behavior. Eades, Fox, Keown, The Role of Professors in 2012 Showed that college students who attend campus financial USA Yes N/A N Varied N N/A Individual Staten Improving Financial Literacy counseling clinics left with reduced levels of financial counseling 72 stress and higher levels of financial self-efficacy. sessions Eissa, Habyarimana The Impact of Cartoons and 2013 Intervention worked with two Kenyan organizations, Well Kenya No N/A Y Varied N N/A Mass media and and Jack Comics on the Effectiveness of Told Story and Junior Achievement Kenya, to compose and print Financial Education: Evidence distribute print and media educational materials that from a Randomized incorporated financial lessons. Overall, found no 73 Experiment in Kenya significant impact of the interventions. Elliehausen, The Impact of Credit 2007 The study examined the impact of individualized credit USA Yes N/A N Varied N N/A Individual Lundquist and Staten Counseling on Subsequent counseling delivered to nearly 8,000 consumer clients. counseling Borrower Behavior Receipt of counseling was associated with a positive sessions 74 change in borrower credit profiles. Employee Benefit Are Workers Kidding 1995 Survey findings indicated that workplace financial USA Yes N/A N Varied N N/A Workplace Research Institute Themselves? Results of the education was effective, as two-fifths of participants financial 1995 Retirement Confidence reported that education materials led them to increase their education Survey contributions to their 401(k) plans. One-half indicated that financial education materials led them to change their asset allocations. Workers with less formal education less likely to utilize the provided education material, but when they do use the material, they are equally or more likely to alter 75 their behavior as a result. FDIC A Longitudinal Evaluation of 2007 Evaluated the Money Smart financial education curriculum USA Yes N/A N Varied N N/A Online course or the Intermediate-Term Impact upon the financial opinions and behaviors of course classroom based of the Money Smart Financial participants during the survey period. Data showed that seminars Education Curriculum Upon training positively affected consumer behaviors (and held Consumers’ Behavior and over time). Participants were more likely to open deposit Confidence accounts, save money in a mainstream deposit product, and 76 use and adhere to a budget. Field, Jayachandran Do Traditional Institutions 2010 Explores how a randomized field experiment training poor India Yes Yes Y 18 N Religion Classroom based and Pande Constrain Female self-employed women in India in basic financial literacy seminar Entrepreneurship? A Field and business skills influenced their financial attitudes and Experiment on Business behaviors. Studies how traditional religious and caste Training in India institutions impose restrictions on women’s’ behavior and influence their business activity. Found training increased borrowing and business income for Hindu women facing 77 more restrictions, however Muslim women do not benefit. Finke Financial Advice: Does it 2012 Provides evidence that financial advisers can improve USA Yes Yes N Varied N Retirement Personal Make a Difference? financial outcomes when the interests of the advisor and minded financial advisor household are aligned- yet professional advice can harm consumers if conflicts of interest create high agency costs. Examines how differences in compensation methods and regulatory frameworks affect incentives – finds essential to 78 improving the breadth and quality of professional advice. Fischer, Drexler and Keeping it Simple: Financial 2010 Test the impact of financial training on firm-level and Dominica Yes Yes Y 36 N Education Classroom based Schoar Literacy and Rules of Thumb individual outcomes for micro entrepreneurs in the n seminar Dominican Republic. No significant effect found from a Republic 79 fundamentals-based accounting training. However, a rule- 51 of-thumb training produced meaningful improvements in business practices. Fox and Bartholomae Considerations in Financial 2006 Studied impacts of a community-based financial education USA Yes N/A N 9 N Only Classroom based Education Programming for program, Ohio Women and Money, on perceived financial women in seminar Women knowledge and learning. Found length of program, and study whether or not information was thought to be personally relevant, was important factors in increasing knowledge. Also, women with prior financial decision making 80 experience made fewer gains. Fry, Mihajilo, The Factors Influencing Saving 2008 Explores the saving behavior of the group of low-income Australia Yes Yes N 10 N N/A Classroom Russell and Brooks in a Matched Savings Program: households that have participated in the Australian Saver based, highly Goals, Knowledge of Payment Plus matched savings program. Found that even after interactive, Instruments, and Other controlling for the unobservable individual response to the seminars Behavior program incentive, the saving goal and education/ financial literacy variables play a positive role in encouraging saving 81 behavior. Garman, Kim, Workplace Financial Education 1999 Studied employee benefits in a workplace financial USA Yes N/A N 5 N N/A Classroom based Kratzner, Brunson Improves Personal Financial education program, administered by the EDSA Group, at a seminar, data and Joo Wellness Southeastern chemical production plant. Found strong collected through evidence that workplace financial education is effective mail survey 82 because it resulted in better financial wellness for workers. Gartner and Todd “Effectiveness of Online 2005 Three credit card issuers conducted randomized tests of USA No N/A Y N/A Y N/A Online “Early Intervention” Financial whether offering online credit card education to credit education Education for Credit cardholders is effective in changing behavior. Found that Cardholders” completion of online credit education correlates with more responsible credit card usage, but the experiments don’t prove that the education causes this behavior as the results 83 were not statistically significant. Gaurav, Cole, and Marketing Complex Financial 2011 Evaluated a field experiment involving the offer of rainfall India Yes N/A Y N/A Y N/A Classroom based Tobacman Products in Emerging Markets: insurance to 600 small-scale farmers in India. Found that a seminar Evidence from Rainfall customized financial literacy and insurance education Insurance in India module communicating the need for personal financial management had a positive and significant impact on 84 rainfall insurance adoption. Gibson, McKenzie The Impact of Financial 2012 Measured the impact of providing financial literacy training Australia No N/A Y 2 N N/A Classroom based and Zia Literacy Training for Migrants to migrants in Australia and New Zealand. Training and New seminar appeared to increase financial knowledge and information Zealand seeking behavior and reduce the risk of switching to costlier remittance products. Did not find an impact on 85 either the frequency or level of remittances. Gine and Mansuri Money or Ideas? A Field 2011 Field experiment in rural Pakistan where microfinance Pakistan Yes Yes Y 46 Y Gender Classroom Experiment on Constraints to clients were provided business training and the opportunity setting and Entrepreneurship in Rural to participate in a lottery to access larger business loans. individual Pakistan Found offering training led to increased business counseling knowledge, better business practices and improvements in household and member outcomes. Concentrated 86 improvements for men. Gine, Karlan and Social Networks, Financial 2013 Field experiment in Kenya measuring the direct impact and Kenya Yes N/A Y N/A Y N/A Shown a comic Ngatia Literacy and Index Insurance social network spillovers of providing financial literacy and on index 87 discount vouchers on farmers’ decision to purchase index- insurance and 52 based drought insurance. Found social network spillovers given discount to the provision of financial literacy materials but no vouchers for the spillovers to the provision of discount vouchers on farmers’ insurance decision to purchase insurance. Goda, Manchester What Will My Account Really 2012 Measures how provision of retirement income projections USA Yes N/A N N/A Y N/A Individual and Sojourner Be Worth? Experimental along with enrollment information affects individuals’ counseling Evidence on How Retirement contributions to employer-sponsored retirement accounts. Income Projections Affect Measure effect on participation and the level of Saving contributions. Those who sent retirement income projections were more likely to change their contribution level and they increased annual contributions more than 88 those without the intervention. Gray, Cohen, Sebstad Can Financial Education 2010 Analyzed effects of financial education targeted at Bolivia Yes N/A N Mixed N N/A Classroom based and Stack Change Behavior? Lessons addressing issues of budgeting, debt management and and Sri seminars and from Bolivia and Sri Lanka savings. Results from quantitative and qualitative data from Lanka group discussion three partners suggest that knowledge and behavior change did occur, including increased knowledge regarding the product characteristics clients should understand when evaluating their loans as well as how to calculate their debt 89 capacity. Grimes, Rogers & High School Economic 2010 Assesses high school economic education by examining an USA Yes Yes N Varied N N/A Classroom Smith Education and Access to individual’s decision to have a bank account. Found courses Financial Services courses in economics and business reduced the probability 90 that an adult was unbanked, ceteris paribus. Grinstein-Weiss, The Ten-Year Impacts of 2012 Evaluated the impact of an Individual Development USA No Yes Y 12 Y Above- Classroom based Sherraden, Gale, Individual Development Account (IDA) program in Oklahoma that provided low- sample courses Rohe, Screiner, Key Accounts on Homeownership: income households with financial education and matching median Evidence from a Randomized funds for qualified savings withdrawals, including a 2:1 income Experiment match for housing down payments. The IDA program had no significant effect on homeownership rates among the full sample and had no effect on the duration of 91 homeownership during the study period. Han, Grinstein- Assets beyond Saving in 2007 Examined whether participation in Individual Development USA Yes N/A Y 12 Y N/A Classroom based Weiss, Sherraden Individual Development Accounts (IDAs) leads to significant growth in assets courses Accounts beyond saving in the IDA accounts. Program participation included 12 hours of financial education classes. Results show that IDA participants have more real assets and total assets than the control group while there are no significant differences in liquid and financial assets between the 92 treatment and control. Hartarska and Evidence on the Effect of 2006 Studies the effects of counseling on default by adopting an USA Yes N/A Y Varied Y N/A Individual Gonzalez-Vega Credit Counseling on Mortgage option-based approach to mortgage termination, with counseling Loan Default by Low-Income sources of data coming from a large Midwest bank, Households Community churches, and a local development company. Found evidence that counseled borrowers defaulted less and that counseling affects the optimal exercise of the 93 default option. Hartarska and Credit Counseling and 2005 Using a competing-risks framework, studies the effects on USA Yes N/A N Varied N N/A Varied Gonzalez-Vega Mortgage Termination by default and prepayment of a counseling program 94 Low-Income Households implemented in several Midwest states. Found weak 53 evidence that the default hazard was lower for graduates of the counseling program but that their default behavior was more optimal. The prepayment hazard was higher for counseled borrowers, but their prepayment behavior was not more optimal. Hastings, Hortacsu Advertising and Competition in 2012 Examined data on pension fund choices in Mexico’s Mexico Yes N/A N N/A N N/A N/A and Syverson Privatized Social Security: The privatized social security system, seeing how advertising Case of Mexico can affect prices, competition, and efficiency in a private pension market. Estimated a flexible model of demand for fund managers, and found that advertising was a key channel used to gain customers by increasing brand value 95 and decreasing price sensitivity. Haynes, Haynes and Outcomes of On-line Financial 2011 Study part of a larger longitudinal study (Women to USA Yes N/A Y Varied N N/A Online Weinert Education for Chronically Ill Women Project) of chronically ill rural women to Rural Women determine if computer tech could be effective in allowing them to take control of their own wellbeing. This study looked at the effects of an on-line personal finance education program on the financial literacy level of the women. Results indicate that intensive and intermediate intervention strategies significantly increase financial 96 knowledge. Haynes-Bordas, Kiss, Effectiveness of Financial 2008 The Get Checking program is a ‘‘second chance’’ program USA Yes Yes N 6 Y Non- Classroom based and Yilmazer Education on Financial that aims to provide financial education to consumers who whites, seminar Management Behavior and were reported to ChexSystems by a previous financial young Account Usage: Evidence institution for account abuse or mismanagement. Findings adults from a ‘Second Chance’ show that the program was successful in positively Program influencing the financial management behavior of participants in terms of recording transactions and 97 communicating with financial institutions. Heinberg, Hung, Five Steps to Planning Success 2010 Investigated the effectiveness of videos and narratives in USA Yes N/A Y Varied N N/A Mass media and Kapteyn, Lusardi and improving people’s understanding of basic financial online course Yoong planning. Administered a quiz in the American Life Panel and discussion to establish a baseline. Found significant improvements in understanding the concepts, as compared to the control 98 group. Hershey, Mowen and An Experimental Comparison 2003 The efficacy of three different retirement seminars USA Yes N/A N 1.5-3 N N/A Combination of Jacobs-Lawson of Retirement Planning (combining goal setting and informational content) was individual Intervention Seminars evaluated on the basis of goal clarity and planning and counseling and savings practices. Found strongest impact on behavior classroom setting came on those participating in both information and goal seminars. setting seminars, and a modest impact of information only 99 seminars. Hershey, Walsh, and Challenges of Training Pre- 1998 Measures the effectiveness of a brief education training USA Modest N/A N NEED NEE N/A NEED INFO Brougham Retirees to Make Sound program to improve decision performance of pre-retirees INFO D FROM PAPER Financial Planning Decisions regarding the affordability of retiring from regular FROM INF employment. Results indicate that although subjects’ PAPER O knowledge of the domain increased significantly as a FRO function of training, the overall quality of their decisions MP did not significantly improve from pretest to posttest. APE 100 R 54 Hilgert, Hogarth, and Household Financial 2003 Analyzed data from the Surveys of Consumers to explore USA Yes N N Varied N N/A Mixed Beverly Management: The Connection some of the connections between knowledge and behavior between Knowledge and and where households obtain their financial knowledge. Behavior Found that households with low knowledge index scores were less likely to report learning from friends and family. Households that reported learning a lot from the media and the Internet had a 50 percent chance of being in the high 101 index group. Hira and Loibl Understanding the Impact of 2005 Examines the linkage between workplace financial USA Yes N/A N Varied N N/A Classroom based Employer-Provided Financial education and workplace satisfaction. Data gathered from a seminars Education on Workplace national sample of employees of an insurance company Satisfaction favor the hypothesis that employees who participate in workplace financial education more fully understand personal finances and recognize how financial literacy 102 impacts their future financial expectations. Hirad and Zorn A Little Knowledge Is a Good 2001 Uses data on mortgages originated under Freddie Mac’s USA Yes N/A N Varied Y N/A Mixed- Thing: Affordable Gold program to assess the claim that pre- classroom, home Empirical Evidence of the purchase homeownership counseling programs lower study, one on Effectiveness of mortgage delinquency rates. Found counseling can increase one, other Pre-Purchase Homeownership the success of affordable lending programs by helping Counseling families keep their homes. However, counseling’s effectiveness is influenced far more by how it is delivered 103 than by the organization that delivers it. Holland, Goodman, “Defined Contribution Plans 2008 Evaluated results from a nonexperimental single group USA Yes N/A N N.A N N/A DVD Sessions and Stich Emerging in the Public Sector: pretest/posttest design to investigate the effects of a The Manifestation of Defined workplace financial literacy program. The findings suggest Contributions and the Effects that participants are less stressed, more satisfied with their of Workplace Financial financial situation, less worried about monthly living Education” expenses, and more confident about overcoming financial emergencies after participating in the financial literacy 104 program. Huang Nam Financial Knowledge and 2012 Evaluated results from the SEED OK program, a statewide USA Yes N/A Y N/A Y N/A Print materials Sherraden Child Development Account randomized policy experiment of child development Policy: A Test of Financial accounts (CDAs) to encourage families to accumulate Capability savings for their children’s future. Found that the treatment group who received additional financial incentives and information on the benefits of the OK 529 plan had a positive impact on participants’ likelihood to open a 529 105 account. Hung and Yoong Asking for Help: Survey and 2010 Using data from current defined-contribution plan holders USA No N/A Y 1 or N N/A Individual Experimental Evidence on in the RAND American Life Panel, found little evidence of Varied counseling or Financial Advice and Behavior improved DC plan behaviors due to advice. Results show online Change that unsolicited advice has no effect on investment behavior, in terms of behavioral outcomes 106 Joo and Grable Employee Education and the 2005 Develops a framework that can be used to examine the USA Yes Yes N Varied N N/A Varied Likelihood of Having a retirement savings decision using Retirement Confidence Retirement Savings Program Survey data determined that respondents with higher education levels, higher income, a smaller household size, 107 and favorable financial attitudes tend to have a retirement 55 savings program. Found those who are exposed to workplace financial education are more likely to have a retirement savings program and having a retirement savings program related positively to retirement confidence. Kanz, Kaufman, Learning by Doing? Using 2013 Examined unique intervention of the “I-Save-I-Win” Nigeria No N/A N N/A N N/A Mass media Shue and Wahler Savings Lotteries and Social program. Several components comprised the intervention- Marketing to Promote giving cash and non-cash prizes for holding saving Financial Inclusion accounts at certain levels, social media, flyers, and text message reminders and advertisements, celebrity endorsements, and a lottery for cash. No evidence incentive 108 program inspired long term savings increases. Karlan and Valdivia Teaching Entrepreneurship: 2006 Measured the marginal impact of adding business training Peru Yes Yes Y 6-104 N Business Classroom based Impact of Business Training on to a Peruvian group lending program for female size, seminar Microfinance microentrepreneurs. Treatment led to improved business education, Clients and Institutions knowledge, practices and revenues. Found treatment led to interest ex- improved business knowledge, practices and revenues. The ante program also improved repayment and client retention rates towards 109 for the microfinance institution. training Karlan, McConnell, Getting to the Top of Mind: 2010 Analyzed three field experiments that randomly assign Bolivia, Yes N/A N 1 text or N N/A Reminder letters Mullainathan and How Reminders Increase reminders to new savings account holders. Tested their Peru, and letter per and texts Zinman Saving model of limited attention in intertemproal choice- positing Philippine month that individuals fully attend to consumption in all periods s but fail to attend some future lumpy expenditure opportunities. Also saw that reminders may increase saving and will be more effective when increase the salience of a 110 specific expenditure. Kast, Meier, and Under-Savers Anonymous: 2012 Using randomized field experiments among Chile Yes N/A Y N/A Y N/A Feedback texts Pomeranz Evidence on microentrepreneurs in Chile, found that holding people Self-Help Groups and Peer accountable through feedback test messages can be a Pressure as a Savings powerful tool to encourage savings, with an effect size 111 Commitment Device similar to that of self-help peer groups. Kim Workplace Financial Education 2007 Examines the impacts of a Cooperative Extension USA Yes N/A N 8 N N/A Classroom based Program: Does It Have An workplace financial education program on university seminars Impact on Employees Personal employees’ financial knowledge, behavior, and perceived Finances? well-being. Found participants made significant 112 improvements in all three categories. Kim, Bagwell, and Evaluation of Workplace 1998 Investigated the evaluation of a financial education USA Yes N/A N N/A N N/A Classroom based Garman Personal Financial Education seminar. The survey results show that participants were seminars satisfied and intended to take actions in personal finances 113 due to the financial education. Kim, Garman, and Relationships among Credit 2003 Evaluated data from a large credit counseling organization USA Yes N/.A N N/A N N/A Individual Sorhaindo Counseling Clients’ Financial to examine the relationships among credit counseling, Counseling Well-Being, Financial financial behaviors, financial stressor events, perceived Behaviors, Financial Stressor financial well-being, and health. Credit counseling and debt Events, and Health management program directly affect financial stressor events in a helpful way and indirectly affect perceived financial well-being and health of the participants after 18 months. Results provide some evidence of the effectiveness of credit counseling in improving financial health variables. 114 56 Kim, Kwon and Factors Related to Retirement 2005 Examined the 2004 Retirement Confidence Survey to asses USA Yes Yes N Varied N N/A Varied Anderson Confidence: Retirement retirement confidence. Results suggest that those who Preparation and Workplace calculated their retirement fund needs, had more savings, Financial Education higher levels of confidence in government programs such as Social Security and Medicare, higher household income, better health, and who received workplace financial education and advice had higher levels of retirement 115 confidence than others. Kimball and Investor Sophistication, and the 2007 Evaluated data from the April 2005 Survey of Consumer USA Yes N/A N Varied N N/A Varied Shumway Participation, Home Bias, Attitudes, finding evidence that financial education might Diversification, and Employer help investors overcome suboptimal behavior. 116 Stock Puzzles Klinger and Can Entrepreneurial Activity 2011 Studied the effects of a business training program for micro El Yes N/A N Varied Y N/A Classroom based Schundeln be Taught? Quasi- entrepreneurs in Central America. Found that business Salvador, seminars Experimental Evidence from training significantly increases the probability that an Guatemal Central America applicant to the workshop starts a business or expands an a, 117 existing business. Nicaragua Koening Financial Literacy Curriculum: 2007 Studied the effects of a basic financial curriculum on USA Yes N/A N 4 N N/A Classroom based The Effect on Offender Money offenders within a medium security correctional facility. seminars Management Skills Offenders gained financial knowledge, as measured by pre 118 and posttests covering financial topics. Langrehr Consumer Education: Does It 1979 Examined research conducted in a state that requires USA Yes N/A N Varied N N/A Classroom based Change Students' consumer education of all students to see if there was a courses Competencies and Attitudes? change in competency level and attitudes toward business. Found that students who took a course specifically designed to present consumer education topics did improve their consumer economics competency and developed a 119 more positive attitude toward business. Lawrence Student Managed Investment 2008 Evaluated the effectiveness of student-managed investment USA Yes N/A/ M N/A N Varied Mixed Funds: An International funds in increasing student knowledge and career success. (classroom-based 120 Perspective Found positive impacts. and hands-on) Lee, Yun, and Haley The Interplay between 2012 Examined the relationship between financial disclosures USA Yes Yes Y Varied N Low Advertising Advertising Disclosures and and investors' financial knowledge on investor decision financial disclosures Financial Knowledge in making within the context of mutual fund advertising. knowledge Mutual Fund Investment Results suggested that mutual fund ads with financial Decisions disclosures are more likely to generate higher levels of recall and positive thoughts regarding advertised information for the mutual fund, more favorable attitudes 121 toward the mutual fund, and greater investment intention. Loibl and Hira A Workplace and Gender- 2006 Examines financial learning in the workplace through USA Yes N/A N Varied N N/A Self-direct Related Perspective on employer provided, self-directed financial learning media, learning from Financial Planning Information such as newsletters, print publications, software, and the education Sources and Knowledge Internet. Results suggest that the social network influences materials Outcomes utilization of employer-provided financial learning media which, in turn, increases actual retirement-specific and self- 122 reported financial knowledge. Loibl and Hira Self-directed Financial 2005 Analyzed the effect of self-directed financial learning from USA Yes N/A N Varied N N/A Self-directed Learning and Financial employer-provided materials on financial management from educational Satisfaction practices, financial satisfaction, and career satisfaction. materials 123 Results demonstrated the effectiveness of self-directed 57 financial learning and the role good financial management practices play in the relationship of financial learning with financial satisfaction and workers’ satisfaction with their career progression. Lusardi Preparing for Retirement: The 2002 Uses data from the Health and Retirement Study (HRS) to USA Yes Yes N Varied N Income, Classroom based Importance of Planning Costs examine workers’ planning activity toward retirement, Education seminars looking at whether employers’ initiatives to reduce planning costs via retirement seminars affect savings. Found seminars do stimulate savings, especially for low 124 income and less educated individuals. Lusardi Financial Education and the 2005 Uses the 1992 HRS’ data set to examine the effect of USA Yes Yes N Varied N Race, age Classroom based Saving Behavior of African financial education seminars on retirement accumulation seminars American and Hispanic for older African American and Hispanic households. Households Found seminars did not affect the portfolio choice of minorities. Only African-Americans were affected by the 125 education- Hispanics were not. Lusardi Saving and the Effectiveness of 2005 Examines whether employers’ initiatives to reduce USA Yes Yes N Varied N Education, Classroom based Financial Education planning costs via retirement seminars have an effect on amount of seminars workers’ saving. Uses data from the Health and Retirement savings Study. Found that seminars foster saving- especially for (low 126 those with low education and those who save little. income) Lusardi Household Saving Behavior: 2008 Largely a survey paper detailing the extent of financial USA Yes Yes N Varied N Minority Varied The Role of Financial Literacy, illiteracy in the US- especially among women, minority status, Information and Financial populations, and low-income people. Briefly mentions and gender, Education Programs provides evidence that financial education may help to income improve savings a financial decision-making. No specific 127 financial literacy intervention for this paper done. Lusardi and Mitchell Financial Literacy and 2007 Using the Rand American Life Panel (ALP) dataset, USA Yes Yes N Varied N Age, Varied Retirement Planning: evaluates financial knowledge of employees. Financial gender, New Evidence from the Rand literacy proved to be a key determinant of retirement education American Life Panel planning. Also found that respondent literacy is higher when participants were exposed to economics in school and 128 to company-based financial education programs. Lusardi and Mitchell How Ordinary Consumers 2009 Assesses the American Life Panel (ALP) on a number of USA Yes Yes N Varied N Age, Varied Make Complex Economic financial literacy measures, trying to evaluate the causal gender, Decisions: relationship between financial literacy and retirement education Financial Literacy and planning. Found financial literacy contributes importantly Retirement Readiness to retirement readiness, after correcting for potential 129 endogeneity biases. Lusardi, Keller and Increasing the Effectiveness of 2009 Studied effects of a financial education intervention on USA Yes Yes N Varied N Gender and Mass media and Keller Retirement Saving Programs participant’s mental wellbeing and financial outlook. income literature for Females and Low Income Measured financial awareness, retirement saving behavior, Employees: A Marketing and overall anxiety. Across the board, found the Approach communication program helped to reduce anxiety levels about future retirement needs, increased financial knowledge, and increased participation and contribution to 130 supplementary retirement accounts. Lusardi, Keller and New Ways to Make People 2009 Used a social marketing approach to develop a planning aid USA Yes Yes N N/A N Gender, Print and online Keller Save: A Social Marketing to help new employees at a not-for-profit institution income 131 Approach contribute to supplementary pensions. Found there was a 58 sharp increase in supplementary retirement accounts after the implementation of the program: the election rate more than tripled in a 30-day period and doubled in a 60-day period. Lusardi, Schneider Financially Fragile 2011 Examined households’ financial fragility by looking at their USA Yes Yes N N/A N Education, N/A and Tufano Households: capacity to come up with $2,000 in 30 days, using data income Evidence And Implications from the 2009 TNS Global Economic Crisis Study. Found approximately one-quarter of U.S. respondents are certain they could not come up with the sum. While financial fragility is more severe among those with low educational attainment and no financial education, families with children, those who suffered large wealth losses, and those who are unemployed, a sizable fraction of seemingly “middle class” Americans also judge themselves to be 132 financially fragile. Lyons and Scherpf Moving from unbanked to 2004 Using data collected from participants of the FDICs Money USA No N/A N 5-10 N N/A Classroom based banked: Evidence from the Smart program, this study investigates the impact that seminar Money Smart program financial education has on an individual’s decision to move from unbanked to banked. The best measure of program “success” may not be the number of accounts opened, but instead whether the program has provided the unbanked with the skills and tools necessary to make sound financial 133 decisions given their financial circumstances. Lyons, Chang and Translating Financial 2006 Evaluated the effectiveness of a financial education USA No N/A N 16-20 N N/A Classroom based Scherpf Education into Behavior program geared towards low-income individuals. Results seminar Change provide some evidence that financial education may result for Low-Income Populations in improved financial behaviors. However, prior level of financial experience may matter more than the number of 134 lessons completed. Lyons, White and The Effect of Bankruptcy 2008 Money Management International, Inc. (MMI) examined USA Yes N/A N 3-3.5 Y N/A Phone Howard Counseling and the education value of the Bankruptcy Abuse and Education on Debtors’ Consumer Protection Act’s educational provisions. The Financial Well-Being: evidence suggests that debtors’ financial literacy improved Evidence from the Front Lines as a result of the counseling. Moreover, even after participating in the counseling, debtors’ financial literacy 135 continued to improve as a result of the education. Maki Financial Education and 2004 Evaluated the impact of workplace and high school USA Yes N/A N Varied N N/A Varied Private Pensions financial education on financial knowledge. Found that both employer and school-based financial education appear to reduce employees’ ignorance of their pension plans, and high school financial education improved respondents’ 136 knowledge of relative asset returns and pension plans. Mandell Does Just in Time Instruction 2006 Four Jumpstart surveys since 1997, encompassing more USA No N/A N Varied N N/A Classroom based Improve Financial Literacy? than 10,000 high-school seniors, showed little overall courses difference in financial literacy test scores between those who had taken an entire semester's course in financial 137 management and those who had not. Mandell Financial Literacy – Does it 2005 Evaluated data from Four Jumpstart Surveys since 1997, USA Yes N/A N Varied N N/A Classroom based Matter? finding that financial education somehow affects courses 138 propensity to save without necessarily improving financial 59 literacy. Students who take a financial education course may emerge from the course with little improvement in financial literacy, as measured by test scores of what they know, but end up more savings oriented than those who have not had such a course. Mandell Teaching Young Dogs Old 2006 Examined the effectiveness of a financial education USA Modest Yes N <1 N Sixth-grade Live play Tricks: The Effectiveness of intervention at the middle-school level in improving students, Intervention in Pre-High saving-related knowledge and behavior on younger boys School Grades students. After seeing a live play on the advantages of saving, students showed a small improvement in knowledge, but this improvement was most pronounced among the youngest students, in the 6th grade. Some positive attitudinal change was also found but it was not possible to find behavioral differences as a result of the 139 intervention Mandell Starting Younger: Evidence 2009 Evaluated the effect of personal finance education in the USA Modest N/A Y <1 Y N/A Live play and Supporting the Effectiveness of form of a live play and supporting classroom instruction classroom Personal Financial Education revolving around the benefits of saving. Pre- and post-test instruction for Pre-High School Students results showed that learning and attitude change were inversely related to age, suggesting that earlier education might be more effective. The treatment group was not significantly more likely to engage in savings behavior as a 140 result of the educational intervention. Mandell The Impact of Financial 2009 Examines the effectiveness of personal finance education USA No N/A N 10 N N/A Mixed Education in High School and on both financial literacy and behavior in a sample of College On Financial Literacy college and high school students. Finds there is little and Subsequent Financial evidence showing that courses increase financial literacy. 141 Decision Making However, do find positive impacts on financial behavior. Mandell and Klein Motivation and Financial 2009 Examined Jump$tart survey of high school seniors to USA No N/A N 10 N N/A Classroom based Literacy assess the hypothesis that low financial literacy scores seminar among adults is related to lack of motivation to learn or retain these skills, regardless of participation in personal finance courses. Found that the motivational variables significantly increased ability to explain differences in 142 financial literacy. Mandell and Klein The Impact of Financial 2009 Examined the differential impact on 79 high school USA No N/A N Varied N N/A Classroom based Literacy Education on students of a personal financial management course courses Subsequent Financial Behavior completed 1 to 4 years earlier. Findings indicated that those who took the course were no more financially literate than 143 those who had not. Mano, Akoten, Assessing the Impacts of 2009 Study carries out a randomized experiment in which Ghana / Yes N/A N 9 or 45 N N/A Classroom based Otsuka and Sonobe Management Skill Training entrepreneurs in study sites in Kumasi, Ghana, and Nairobi, Kenya seminar in Ghana and Kenya Kenya receive managerial training, and the improvements in their behaviors and business results are measured. The econometric analysis finds strong evidence of positive 144 training impacts on business practices and performances. Martinez, Puentes Micro-Entrepreneurship 2013 RCT of a large-scale publicly run micro-entrepreneurship Chile Yes Yes Y 60 Y Income Classroom based and Ruiz-Tagle Training and Asset Transfers: program in Chile assessing the effectiveness of business seminars Short Term Impacts on the training and asset transfers on individual’s employment and 145 Poor income. Found significant increases in individuals’ 60 employment and income and improvement of the business practices of its beneficiaries. Effect on wage earners only positive for low income individuals. Mastrobuoni Do Better-Informed Workers 2007 Analyzed the effect of the provision of information through USA Yes Yes & No N N/A N Low- Make Better Retirement the Social Security Statement on workers’ retirement and educated Choices? A Test Based on the claiming decision making. Found that despite the previous workers Social Security Statement availability of information, the Statement has a significant (Y), black impact on workers’ knowledge about their benefits and workers 146 consequent decisions. (N) Maurer and Lee Financial Education with 2011 Project compares the learning gains from teaching financial USA Yes N/A N 1 or 6.25 N N/A Classroom based College Students: Comparing literacy skills to undergraduate students through multiple seminars or peer Peer- Led and Traditional methods. Students in the course received extended led discussion Classroom Instruction instruction on budgeting, credit, and other topics. Results session found similar gains between methods on shared content and 147 on planned financial behaviors. Mayer, Tatian, National Foreclosure 2009 Studied data from the National Foreclosure Mitigation USA Yes N/A N Varied Y N/A Mixed Temkin and Calhoun Mitigation Counseling Counseling program, administered by NeighborWorks Program Evaluation- America, which provided a foreclosure intervention Preliminary Analysis of counseling program in response to the nationwide Program Effects foreclosure crisis. So far, have found strong evidence to suggest the program is having intended effect of helping homeowners cure existing foreclosures, receive loan modifications, and sustain cures to delinquencies or 148 foreclosures. Maynard, Mehta, Can Games Build Financial 2012 Evaluated Doorways to Creams (“D2D”) Fund’s Financial USA Yes N/A N Varied N N/A Video games Parker and Steinberg Capability? Entertainment innovation, which incorporates financial education into video games. Data suggests that the FE games can be successful at engaging consumers, cultivating financial self-efficacy and financial literacy, and enabling real-world financial action. More rigorous research and 149 testing is needed. Mills, Patterson, Orr, Evaluation of the American 2004 Evaluated the effects of an individual development USA Y 12 DeMarco Dream Demonstration: Final accounts (IDAs) program (which included financial Evaluation Report education classes) on the savings and asset accumulation of low-income individuals. Found that that the Statement significantly improve the decisions of low–educated workers and works with a dependent spouse who usually disregarded their spouse’s benefits in their calculations. The information did not seem to help back workers. and black, made, on average, worse retirement decisions, and that workers with a dependent spouse usually disregarded their own spouse’s benefits in their calculations. The information contained in the Statement appears to have helped both groups, though with the important exception of 150 black workers. Muller Does Retirement Education 2003 Examines effect of retirement education on how employees USA No Yes N Varied N Age, Classroom based Teach People to Save Pension use distributions from their defined contribution pension Gender, seminars Distributions? plans. Found education substantially increases the Education probability that participants age 40 and under will save a 151 distribution, but decreases the probability that college 61 graduates and women will save one. National Endowment Financial Management 2009 Uses three categories of financial outcome indicators USA No Yes N Varied N Program Mixed for Financial Practices of College Students (financial knowledge, dispositions, and behaviors) to assess forms Education from Sates with Varying the effectiveness of high school financial education Financial Education Mandates programs. Find results vary by state policy (even with host of controls), showing that both social learning and formal education are important determinants of financial 152 behaviors. Norvilitis, MacLean The Role of Parents in College 2010 Examined how parents’ teaching and modeling of financial USA Yes N/A N Varied N N/A Parental Students’ Financial Behaviros concepts affects college student credit card debt. Parental instruction and Attitudes hands-on mentoring of financial skills was most strongly 153 related to lower levels of credit card debt Osteen, Muske and Financial Management 2007 The study examined the results of Money 2000TM and its USA Yes N/A N 3 years N N/A Classroom based Jones Education: Its Role in ability to influence behavior and financial preparedness. seminars Changing Behavior Participants made greater use of banks and less use of loan and check cashing services, increased savings, and decreased debt. The data supports financial literacy training 154 as enhancing financial well­being. Peng, Bartholomae, The Impact of Personal 2007 This study investigates the impact of personal finance USA No No N Varied N N/A Varied Fox and Cravener Finance Education Delivered in education delivered in high school and college on High School and College investment knowledge and household savings rates Courses measured years after the financial education was delivered. Participation in a college level personal finance course was associated with higher levels of investment knowledge. No significant relationship between taking a high school course and investment knowledge was found. Financial experiences were found to be positively associated with 155 savings rates. Querica and Spader Does Homeownership 2008 Examines the impact of HEC completion on pre-payment USA No Yes N Varied Y Different Mixed- Counseling Affect the and default among borrowers receiving HEC from a variety counseling classroom based Prepayment and Default of providers across 42 states. Found programs based on channels seminars, phone, Behavior of Affordable classroom instruction and individual counseling improves a or home study Mortgage Borrowers? borrower’s exercise of the mortgage prepayment options, materials 156 though no effect on default propensities. Robb Babiarz The Demand for Financial 2012 Analyzed the effect of using financial advice on personal USA Yes N/A N Varied N N/A Individual Woodyard Professional’s Advice: The financial knowledge and satisfaction. Found that financial counseling Role of Financial Knowledge, knowledge (both objective and subjective) and satisfaction Satisfaction, and Confidence are positively related to using any type of financial advice, and specifically with using advice related to investing and saving, mortgage decisions, insurance, and tax planning. In contrast, knowledge and satisfaction are inversely related to 157 the use of debt counseling. Romagnoli & Does Financial Education at 2013 Examines the effectiveness of a nation-wide initiative by Italy Yes Yes N Varied N N/A Classroom based Trifilidis School Work? Evidence from the Bank of Italy and the Italian Ministry of Education that courses Italy incorporated financial education into school curricula. Knowledge gains are measured via series of tests, results find the education successful in increasing financial 158 knowledge. Russell, Mihajilo and Saver Plus- encouraging 2007 Report provides results from the second follow-up survey USA Yes Yes N Varied N Income, Classroom based 159 Brooks savings and increasing of post-Saver Plus saving behavior. Program includes a age, gender seminars and 62 financial capabilities among matched savings ratio of 1:1, financial literacy education matching low-income families and support from the delivering organizations. Results find program significant increases in saving-oriented financial behaviors. Ryack The Impact of Family 2011 This study explores how family relationships and financial USA No N/A N Varied N N/A Varied – Relationships and Financial education impact financial risk tolerance using a sample of surveyed past Education on Financial Risk college freshmen and their parents. College students who exposure to Tolerance had some financial education in high school are found to be financial more risk tolerant, especially when they played a stock education 160 market game as part of the course. Sanders, Weaver and Economic Education for 2007 Looked at the financial literacy outcomes of an economic USA No No N 12 N N/A Classroom based Schnable Battered Women: An education program which was created specifically for seminars Evaluation of Outcomes battered women. Aim of the program was to increase awareness of financial choices, opportunities, and consequences. Found limited gains in financial knowledge 161 and significant improvements in financial self-efficacy. Sarr and Sadhu The Role of Financial Access, 2013 Details the impacts of a two pronged financial intervention India Yes N/A Y 2 N N/A Classroom based Knowledge and Service in rural India. First, to promote financial inclusion, CSPs seminar Delivery in Savings Behavior: were set up in key rural locations. Financial education Evidence from a Randomized classes were also held periodically. 162 Experiment in India Sarr, Sadhu and Fiala Bringing the Bank to the 2012 Explores the uptake of branchless banking in one of the India Yes Yes Y 5 N Gender Classroom Doorstep: Does Financial largest banking correspondent programs, FINO. Did a setting where Education Influence Savings randomized financial literacy training program to FINO video shown Behavior among the Poor? clients on the transaction activities in their savings account. Evidence from a Randomized Results show a persistent treatment effect on account usage Financial Literacy Program in in the short run, with more pronounced effects for females. 163 India Schreiner and Can the Poor Save? 2007 Examines the effectiveness of Individual Development USA Yes N/A N Varied Y N/A Classroom based Sherraden Accounts in helping the poor to save. Found that savings seminars increase significantly as a result of the first 1 to 10 hours of financial education. Results were not significant for 10 to 164 20 hours or 20 to 30 hours. Schreiner, Sherraden, Savings and Asset 2001 Provides midway results of the American Dream USA Yes N/A N 10.5 N N/A Classroom based Clancy, Johnson, Accumulation in Individual Demonstration’s intervention, which tests if the seminars Curley, Grinstein- Development Accounts: introduction of Individual Development Accounts will help Weiss, Zhan and Downpayments on the the poor save and build financial literacy. Preliminary Beverly American Dream Policy research suggests increased savings. 165 Demonstration Schug & Hagedorn Can Charter Schools Improve 2004 Describes an inner city charter school the Milwaukee USA Yes Yes N Varied N N/A Classroom based Financial and Economic Urban League Academy of Business and Economics that courses Education? The Case of the specializes in a business and economics curriculum. Milwaukee Urban League Describes the special curriculum in one charter school and Academy of Business and present an evaluation of the second year of the effort to 166 Economics implement the curriculum. Schug, Niederjohn Your Credit Counts Challenge: 2006 Reports on the impact on attitudes of a program designed to USA Yes Yes N 4 N Income Classroom based and Wood A Model Program for Financial improve the financial literacy of low- and moderate-income seminar Education for Low and adults. The program involves three national partners- Moderate Income Adults Household International, National Center for Neighborhood Enterprise, and National Council on 167 Economic Education. 63 Seligman and Bose Learning by Doing: Active 2012 Investigated the impact of household exposure to employer USA Yes Yes N Varied N N/A Mixed Employer Sponsored pension plan features using the Health and Retirement Retirement Savings Plan Survey. Does exposure to active management or Participation and Household participation in plan-sponsored financial education Wealth Accumulation seminars impact household portfolio allocations or wealth? Found that both of the plan features improve asset allocations and financial outcomes for recent retirees, 168 especially when used together. Servon and Kaestner Consumer Financial Literacy 2008 Analyzed a demonstration program mounted by a major USA No N/A Y 12 N Only low- Computer based and the Impact of Online bank to understand whether access to information and to classroom Banking on the Financial communications technologies, combined with financial moderate- seminars Behavior of Lower-Income literacy training, can help low- and moderate-income income Bank Customers individuals in inner-city neighborhoods be more effective studied 169 financial actors. No significant effects found. Seshan & Yang Transnational Household 2013 Study examined effectiveness of a financial literacy Qatar Yes Yes N 5 N Income Classroom based Finance: A Field Experiment intervention for migrant Indian works in Qatar on the seminar and on the Cross-Border Impacts of financial decision-making of the migrant workers, discussion dinner Financial Education on migrants’ attempts to influence the financial decision- Migrant Workers making of their wives in the home country, migrant beliefs about their wives’ behaviors, and the wives’ actual behaviors. Treatment led to substantial changes in migrant financial practices, and more joint financial decision- making. Migrants with below-median baseline savings are 170 most responsive to the treatment. Shelton and Hill First-Time Homebuyers 1995 Examined whether participation in a First-Time USA Yes Yes N 12 Y Gender, Classroom based Programs as an Impetus for Homebuyers Educational Program affects budgeting Age, seminars Change in Budget Behavior behavior. Results suggest that the program had a substantial Income, impact on consumers’ knowledge, regardless of gender, Education, 171 race, age, education level or income category. Race Sherraden, Johnson, Financial Capability in 2011 Analyzed an innovative four-year school-based financial USA Yes N/A N 150 N N/A Classroom based Guo and Elliott Children: Effects of education and savings program, “I Can Save”, impacts’ on seminars Participation in a School Based financial knowledge. Results suggest young children Financial Education and increase financial capability when they have access to Savings Program financial education and it is accompanied by participation 172 in financial services. Shim, Barber, Card, Financial Socialization of First- 2010 Structural equation modeling from a cross-sectional study USA Yes N/A N Varied N N/A Varied Xiao, and Serido year College Students: The indicates that parents, work, and high school financial Role of Parents, Work, and education during adolescence predicted young adults' Education current financial learning, attitude and behavior, with the role played by parents substantially greater than the role played by work experience and high school financial 173 education combined. Shim, Xiao, Barber, Pathway to Life Success: A 2009 Tested a conceptual model of the potential antecedents and USA Yes N/A N Varied N N/A Classroom based Lyons Conceptual Model of Financial consequences of financial well-being in young adulthood. courses Well-being for Young Adults Results suggest the importance of formal financial education at school in the ways that young adults acquire knowledge about financial matters and form attitudes and 174 behavioral intentions based on that knowledge. Shockey and Seiling Moving into Action: 2004 Study sued the Transtheoretical Model of Behavior Change USA Yes N/A N 10 N Only low- Classroom based 175 Application of the to assess change in financial behavior among participants income seminars 64 Transtheoretical Behavior enrolled in an IDA financial education program. All six participants Change to Financial Education money management behavior means increased. studied Skimmyhorn Essays in Behavioral 2012 Estimated the effects of the Personal Financial USA Yes N/A N 8 N N/A Classroom based Household Finance: Assessing Management Course (PFMC) on retirement savings and seminars Financial Education: Evidence other financial outcomes using a RTC within the U.S. from a Personal Financial Army. Found attendance doubles retirement savings, and Management Course has moderately sized beneficial effects for other financial decisions (such as reducing cumulative credit account 176 balances and monthly credit payments). Song Financial Illiteracy and Pension 2011 Evaluated results from a randomized control trial testing China Yes N/A Y N/A Y N/A Individual Contributions: A Field the effectiveness of a financial education program on rural counseling Experiment on Compound households in China. Found that the financial education Interest in China treatment had a positive and significant effect on household understanding of compound interest and increased 177 contributions to pension savings. Sonobe, Suzuki, KAIZEN for Managerial Skills 2011 Presents preliminary results of ongoing study of a Vietnam Yes N/A Y N/A N N/A Classroom based Otsuka and Nam Improvement in Small and corrective financial education program in a knitwear cluster seminars Medium Enterprises: An of industrial development in Hatay, Vietnam. Found Impact Evaluation Study in a training program improved the participants’ business Kitwear Cluster in Vietnam practices and their recognition of the importance of 178 management knowledge. Spader, Ratcliffe, The Bold and the Bankable: 2009 Evaluates a Spanish telenovela, Nuestro Barrio, in its USA Yes N/A N 13 N N/A Mass-media Montoya and Skillern How the Nuestro Barrio effectiveness to teach general financial education. The telenovela Telenovela Reaches Latino teaching method relies on an incremental build-up of Immigrants with Financial knowledge to sustain long term behavior changes. Found Education behavioral outcomes are consistent with intended 179 educational effects. Stuart Uganda Financial Education 2012 Evaluated a Habitat for Humanity financial education Uganda No N/A N 7 N N/A Classroom based Impact Project intervention in rural Uganda, using Financial Diaries and seminars interviews to gain data about the knowledge, skills, attitudes, and behaviors of the participants. Did not find 180 significant effects of behavior changes. Tennyson and State Curriculum Mandates and 2001 Analyzed the relationship between high school students’ USA Yes N/A N Varied N N/A Classroom based Nguyen Student Knowledge of Personal scores on a test of personal financial literacy and their courses Finance state’s personal finance curriculum mandate. Results showed that students in states that required specific financial education course work scored significantly higher than those in states with either a general mandate or with 181 no mandate. Tufano, Flacke and Better Financial Decision 2010 Details a Doorways to Dreams Fund (D2D) project which USA N/A N/A N 1.5 N N/A Video game Maynard Making Among Low-Income tests the effectiveness of financial education through video and Minority Groups games. Geared towards low-income and minority adults, Farm Blitz and Bit Club are meant to teach effective money management and saving behaviors. Study quantifying 182 effectiveness to come. Varcoe, Martin, Using a Financial Education 2005 Examines the effectiveness of the “Money Talks: Should I USA Yes N/A N Varied N N/A Classroom based Devitto and Go Curriculum for Teens be Listening?” curriculum on financial knowledge and seminars behavior of teenagers. Findings indicate that using the curriculum improved financial literacy high school 183 students, with positive behavior changes, improved 65 knowledge, and new skills in making their money go farther. VISA Financial Literacy Education in 2009 Details the Visa Financial Literacy Roadshow, a living South Yes N/A N 1 N N/A “Industrial Sub-Saharan Africa tutorial theatre spurred and sponsored by the 2010 FIFA Africa, Theatre” in World Cup South Africa, which teaches financial Botswana, townships knowledge. Found high understanding and retention of key Zambia, 184 messages by surveyed audience members. Kenya Walstad, Rebeck and The Effects of Financial 2010 Investigates the effects of a financial education program, USA Yes N/A N 6 N N/A Media based MacDonald Education on the Financial Financing Your Future, on high school student’s classroom Knowledge of High School knowledge of personal finance. Financial knowledge post- learning Students intervention scores increased regardless of specific course 185 in curriculum. Wiener, Baron- Debtor Education, Financial 2005 Evaluates a financial education training program for New USA Yes N/A N 3 Y N/A Classroom based Donovan, Gross and Literacy and Pending York residents who had filed for bankruptcy. Results seminar Block-Lieb Bankruptcy Legislation revealed that trained debtor’s financial knowledge increased, as did positive financial behaviors such as 186 cautious use of credit cards, paying bills, and budgeting. Xiao, Serido, and Financial Education, Financial 2010 Examined associations among financial education, USA Yes N/A N Varied N N./A Classroom based Shim Knowledge and Risky Credit financial knowledge, and risky credit behavior of college courses Behavior of College Students students. Found that both high school and college personal finance courses are positively associated with subjective financial knowledge, which in turn reduces the chance of 187 engaging in risky credit behavior. Xiao, Sorhaindo and Financial Behaviors of 2006 Positive financial behaviors of consumers were examined USA Yes N/A N Varied N N/A Mixed Garman Consumer in Credit Counseling using a national sample of consumers who use credit counseling services in the US from a behavior economic perspective. Found indicate that consumers in credit counseling may follow a hierarchal pattern- paying off 188 debts and adjusting spending before considering saving. 66 GROUP VARIABLE DEFINITION Authors Authors of the Paper General Study Information Date Date that paper was published or released as a working paper Country Country in which intervention was conducted Years Years over which intervention was conducted Duration Number of years that intervention lasted Intensity Number of hours in total that were effectively “taught” T_after Amount of time in months elapsed between intervention and endline evaluation TT 1=Individuals assigned to intervention treatment were treated; 0=Individuals were offered intervention treatment Channel Channel of Intervention: 1=Course Taught in Person; 2=Individual Consulting; 3=Mass Media; 4=Phone; 5=Print; 6=Online; 7=Mixed; 8=Other Intervention Information Location Location of Intervention: 1=High School; 2= University; 3= Workplace; 4=Community/Third-Party; 5=Home-based; 6=Other; 7=Mixed; 8=Financial Institution Topic Topic Taught through Intervention: 1= Mortgage; 2=Credit Counseling; 3=Business Management; =Savings/Retirement; 5=Other; 6=Mixed Method Methodology Used to Identify Impact of Intervention: 1=Randomized Control Trial; 2=Natural Experiment; 3=Non-experimental regression analysis Journal Publication in which Study of Intervention Appeared: 1=Peer-reviewed Journal; 2=Non-academic Publication or Book; 3=Academic Working Paper DV_id ID assigned to dependent variable (see Dependent Variable ID Legend) DepVar Name of dependent variable tm Teachable Moment Intervention: 1=Intervention directly linked to behavior/action soon to take place; 0=Intervention does not involve teachable moment f Function of the dependent variable: 1=Level; 2=Logarithm; 3=Z-score; 4=Binary; 5=% in 0 to1; 6=% in 0 to 100 Estimation Information SR Self-Reported Data: 1=Data is self-reported; 0=Data is not self-reported cmean Control mean of dependent variable csd Control standard deviation of dependent variable mean Mean of dependent variable SD Standard deviation of dependent variable 67 beta Estimated coefficient SE Standard error of the estimated coefficient pv p-value associated with estimation of the beta coefficient tvalue t-value associated with estimation of the beta coefficient Sample From the total set of individuals that participated in the intervention, which sample was used in the estimation? N Number of observations used in the estimation 68 doff Degrees of freedom of estimation (N minus the number of regressors used in the estimation) DEPENDENT VARIABLE ID THEME DESCRIPTION S01 Savings Binary: 1 = saves at least something (in a given period of time in the past); 0 = if not S02 Savings Total amount of money saved S03 Savings Savings rate: total savings in a given period divided by total income in a given period S04 Savings Total amount of money saved in a given period (e.g. amount saved last month) S05 Savings Change in contribution rate to retirement savings plan S06 Savings Change in participation rate to retirement savings plan S11 Savings Binary: 1 = contributed to retirement savings (in a given period of time in the past); 0 = if not S12 Savings Total amount of money allocated as retirement savings T13 Retirement savings rate: total retirement savings in a given period divided by total income in a Savings given period T14 Total amount of money allocated for retirement in a given period (e.g. amount allocated for Savings retirement savings last month) T16 Savings Participation rate for retirement savings plan B01 Borrowing Binary: 1 = borrows or has borrowed at least something; 0=if not B02 Borrowing Total amount of money borrowed B03 Borrowing Borrowing rate: total borrowing in a given period divided by total income in a given period B04 Borrowing Total amount of money borrowed in a given period (e.g. amount borrowed last month) B05 Borrowing Credit score B06 Borrowing Binary: 1 = has defaulted on a loan; 0 = if not B07 Borrowing Credit score (binary over cutoff) B11 Borrowing Binary: 1 = was bankrupt; 0 = if not T01 Financial Literacy Test Score on a financial literacy test that evaluates content of financial literacy intervention Score T02 Financial Literacy Test Dummy indicating whether above/below a predetermined financial literacy score Score A01 Opening of Accounts Binary: 1 = opened any bank account (in a given period of time in the past); 0 = if not A02 Opening of Accounts Binary: 1 = opened a savings account (in a given period of time in the past); 0 = if not A03 Opening of Accounts Binary: 1 = has any bank account (in a given period of time in the past); 0 = if not A04 Opening of Accounts Binary: 1 = has a credit card (in a given period of time in the past); 0 = if not 69 R01 Recordkeeping / Budgeting Binary: 1 = keep records (in a given period of time in the past); 0 = if not R02 Recordkeeping / Budgeting Binary: 1 = knows how to make a budget (expense + income); 0 = if not R03 Recordkeeping / Budgeting Binary: 1 = set financial goals (in a given period of time in the past); 0 = if not R04 Recordkeeping / Budgeting Binary: 1 = makes budget (in a given period of time in the past); 0 = if not R05 Binary: 1 = separates business and personal accounts (in a given period of time in the past); 0 = if Recordkeeping / Budgeting not DQ01 Delinquencies Varied IA01 Investments / Assets Varied IA02 Investments / Assets Stock Ownership K01 Knowledge - other Know difference: discretionary & non-discretionary K02 Knowledge - other Understands interest rate K03 Knowledge - other Can compare financial options N01 Intention to do something Intention to save P01 Participation Discusses financial matters with parents, peers, etc X01 Behavior - other Uses an informal saving plan X02 Behavior – other Reached saving goal PS01 Psychology / Personality Varied 70 Appendix 2: Additional Meta-Analysis In this paper we chose to use a random effects model using the method of DerSimonian and Laird (1986), which is a variation on the inverse-variance method in order to incorporate the assumption that the different studies are estimating different, yet related, intervention effects. The Stata command used is metan. This produces a random-effects meta-analysis, and the simplest version is known as the DerSimonian and Laird method. To undertake a random-effects meta-analysis, the standard errors of the study-specific estimates are adjusted to incorporate a measure of the extent of variation, or heterogeneity, among the intervention effects observed in different studies. The amount of variation, and hence the adjustment, can be estimated from the intervention effects and standard errors of the studies included in the meta-analysis. On the other hand, a meta-analysis regression routine intends to investigate the extent to which statistical heterogeneity between results of multiple studies can be related to one or more characteristics of the studies. In this case, we perform a random-effects meta-regression with the use of the Stata command metareg, which uses the effect estimates and its standard error for each study in an equivalent way as the metan command does for the meta-analysis described above. In summary, this procedure first estimates the between-study variance, and then estimates the coefficient corresponding to the study characteristic by weighted least squares, using the weights 1/(xi+ yi), where xi is the between-study variance and yi is the standard error of the estimated effect in each study i. The graphs shown below plot each study in circles, where the size of the circle corresponds to its weight, and a line whose slope is the resulting coefficient associated with the study characteristic. 71 Funnel Plots to evaluate publication bias by outcome variable. Record keeping Savings Funnel plot with pseudo 95% confidence limits Funnel plot with pseudo 95% confidence limits 0 0 .02 .02 .04 .04 se se .06 .06 .08 .08 -.1 0 .1 .2 -.1 0 .1 .2 beta beta Retirement Loan Defaults Funnel plot with pseudo 95% confidence limits Funnel plot with pseudo 95% confidence limits 0 0 .01 .02 .04 .02 se se .06 .03 .08 .04 .1 -.1 -.05 0 .05 -.2 -.1 0 .1 .2 beta beta 72 Appendix 3: Ways to Strengthen the Body of Research on Impact Analysis of Financial Literacy and Capability Interventions 1) Report fully on the intervention being tested. Basic data on the intervention(s) being tested should be clearly reported in any impact evaluation. For meta-analysis in this paper, the following data were coded for each study when available. a. Country where the intervention occurred b. Year when the intervention occurred c. Duration of the intervention (period of time over which the intervention occurred) d. Intensity of the intervention 13 (time that students/ subjects were directly engaged in the intervention) e. Main intervention channel (1= course taught in person; 2= individual consulting; 3= mass media; 4=phone; 5=print; 6=online; 7=mixed) f. Main Location of financial literacy (1=High School; 2= University; 3= Workplace; 4= Community/Third Party; 5= Home-based; 6= other; 7=mixed; 8=financial institution) g. Main Topic of financial education (1= Mortgage; 2=Credit Counseling; 3= business management; 4=savings/retirement; 5=other; 6=mixed) h. Main Methodology used to identify impact of intervention (1=RCT; 2=Natural Experiment; 3=Non-experimental regression analysis) i. Indication if people who were assigned to treatment were actually treated j. Level of Peer Review (1 = study published in peer reviewed journal; 2 = appears in non- academic publication; 3 = academic working paper) k. Amount of time elapsed between intervention and evaluation (in months) Based upon insights from literature on this topic, some additional data which should be reported upon in papers includes: l. Linked directly with a “teachable moment” – Is the financial education intervention linked to a financial decision or opportunity immediately available for participants? m. Indication of motivation – Do participants volunteer or self-select into the training / intervention or are they required to participate? n. Leveraging social relationships – Does the intervention leverage or involve social relationships (parents and children; remittance senders and receivers; co-workers; spouses; peers; etc.)? o. Training for instructors – What training or specific skills do any instructors have in financial education? p. Provide links to the actual financial education materials when possible. Multi-country studies which evaluate the same (or a very similar) intervention should also be encouraged to add to insights on what elements of an intervention are most critical for success. 13 To understand the difference between duration and intensity, imagine a 6 week course that meets for two hours each week. The duration is 6 weeks and the intensity is 12 hours. 73 2) Report on the cost of the financial education intervention There is virtually no reporting on the cost of the financial education intervention in papers which evaluate impact. The following issues should be addressed to clarify the value proposition of any particular intervention. a. Total direct cost of providing the financial education intervention. This data should include any development costs, licensing materials, paying instructors, print materials, technology costs, counseling expenses (for one-on-one support), etc. b. Estimated per person cost for the intervention. This could be presented in the context of the specific intervention being reported upon (which may be a pilot in some cases) and estimated for eventual roll out to a larger audience. c. Cost of the impact evaluation indicating cost of data collection (by survey or administrative data) and analysis / research. d. Any imbedded or indirect costs (teacher time dedicated to financial education; work time spent by employees in financial education; etc.) Where possible, more than one intervention or combination of interventions should be evaluated to determine the most cost effective approach. This may include other types of actions such as choice architecture, defaults, generic products, etc. as well as financial education activities. 3) Improve the comparability of research through increased standardization in how outcomes are measured (including the measurement of financial knowledge / capability) Comparing results across studies is facilitated when the interventions are similar (or at least when differences in the interventions can be accounted for – see point #1 in this Appendix) and when common outcome indicators are used in the empirical tests. The literature on financial capability interventions does not currently have standardized metrics to determine whether an intervention was successful. Instead, outcome measures are highly specific to each study which makes comparison difficult. The following suggestions could increase comparability of results: a. Use common measurement tools for financial knowledge, financial capability and psycho-social traits. There are several widely available survey tools which can be used including the core financial knowledge questions used by Lusardi and Mitchell, the World Bank survey for financial capability and consumer protection and the OECD survey for financial literacy. The World Bank survey is formulated with lower income, lower access environments in mind, such as those found in many developing countries and includes psycho-social indicator questions. b. Increase the use of binary dependent variables when evaluating the impact of financial capability interventions. When the dependent variable is too narrowly defined it becomes hard to compare impacts across studies. In addition to study-specific variables, there may often be scope for organizing the results according to more general descriptive variables such as binary variables which indicate whether any impact was noted (i.e. an increase in savings, a reduction in defaults, opening an account, etc.). Some specific recommendations for binary variables include the following: i. Savings (increase in total savings = 1; increase in savings in specific account = 1; increase in savings with provider = 1) 74 ii. Wealth (increase in total assets = 1; increase in financial assets = 1) iii. Credit (defaulted on a loan = 1; late on a credit payment in last 6 months = 1; credit score improved = 1; reduced total debt = 1) iv. Financial inclusion (opened an account = 1; use formal sources of credit = 1; use mobile payments = 1) v. Pension / retirement (enrolled in pension plan = 1; increased contribution to retirement savings / plan = 1; delayed age for taking pension benefits = 1) vi. Record keeping and budgeting (initiated record keeping = 1; uses a budget to plan and track expenditures = 1; follows budget = 1) c. There are also ways to facilitate comparison of continuous data across studies including by providing information on income and assets as well as financial data so that figures can be scaled. In terms of credit, including data on loan maturity, interest rates, monthly payments and whether the loan is secured or unsecured can also facilitate more relevant comparisons. 4) Register studies (prior to results) and report on results afterward – In the medicine and public health literature in particular, there are several registries for research studies and clinical trials. These include the listing of clinical trials administered by the U.S. National Institutes of Health (http://clinicaltrials.gov/) as well as the Cochrane Collaboration(http://www.cochrane.org/) for evidence based health care and the Campbell Collaboration which is an international research network that systematically reviews social research (http://www.campbellcollaboration.org/). These networks promote knowledge sharing and communities of practice as well as increase transparency (and perhaps reduce publication bias) by encouraging researchers to register their projects before results are available. The databases that can be developed through these collaborative endeavors can be used for meta-analysis and other systematic reviews of the literature and findings. In economic research, there are new efforts underway to create similar registries, for instance the American Economic Association Social Science RCT Registry (https://www.socialscienceregistry.org) and the 3ie and RAND collaborative effort for experimental and quasi-experimental evaluations (http://www.3ieimpact.org/en/evaluation/ridie). Ongoing and new research should take advantage of these registries. 75