Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature Margaret Miller, Julia Reichelstein, Christian Salas and Bilal Zia This paper presents a systematic and comprehensive meta-analysis of the literature on financial education interventions that focuses on financial education studies designed to strengthen the financial knowledge and behaviors of consumers. The analysis identifies 188 papers and articles that present impact results of interventions designed to increase consumers’ financial knowledge (financial literacy) or skills, attitudes, and behaviors (financial capability). These papers are diverse across a number of dimensions, including objectives of the program intervention, expected outcomes, intensity and duration of the in- tervention, delivery channel used, and type of population targeted. However, there are a few key outcome indicators where a subset of papers are comparable, including those that address savings behavior, defaults on loans, and financial skills such as record keeping. The results from the meta-analysis indicate that financial literacy and capability interventions can have a positive impact in some areas (e.g., increasing savings) but not in others (e.g., reducing loan defaults). Financial education, financial literacy, meta Analysis. JEL codes: C93, D03, D14, O12, O17 A decade ago there was limited interest in the topic of financial literacy. Now this issue is at the top of the policy agenda for national regulators, international organi- zations, researchers, and private financial institutions. An important reason for the increased attention to financial literacy is the global financial crisis, which high- lighted the importance of financial knowledge and skills for consumers. Anecdotal evidence from the crisis immediately suggested that people had taken on financial products—and risks—that they did not fully understand. Empirical studies con- firmed this relationship, including Klapper et al. (2013), who utilize data from The World Bank Research Observer # The Author 2015. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lkv009 Advance Access publication May 4, 2015 30:220–246 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Russia, and Gerardi et al. (2010), who link outcomes in the subprime housing market in the United States with consumers’ financial knowledge and skills. Furthermore, Lusardi and Mitchell (2011) present evidence from around the world suggesting that individuals (even in developed countries) have a difficult time un- derstanding basic financial concepts. While financial literacy can clearly be a factor in avoiding financial risks, it can also be important for taking advantage of financial opportunities. Studies have shown that financial knowledge is linked to higher levels of retirement planning and savings (Behrman et al. 2012; Alessie et al. 2011; Bucher-Koenen and Lusardi 2011; Lusardi and Mitchell 2012; 2007); investment decisions such as diversifica- tion (Abreu and Mendes 2010) and investments in equities (Van Rooij et al. 2011; Christelis et al. 2010); credit management and satisfaction; and mortgage perfor- mance (Gerardi et al. 2010; Quercia and Spader 2008; Ding et al. 2008). At the lower end of the financial market where consumers are seeking their first access to financial products and services such as opening basic accounts or borrow- ing small sums of money, there is also evidence that financial literacy may be impor- tant. Cole et al. (2011) find that the use of insurance in India and bank accounts in Indonesia are both linked to higher levels of financial literacy. Using data from Finscope 1 in Africa, Honohan and King (2009) likewise find a positive relationship between financial knowledge and the use of financial products and services. There is also ample evidence that financial literacy levels are relatively low across a wide range of countries and negatively correlated with per capita income,2 sug- gesting that while financial literacy is important for engaging in financial markets, it remains low for many consumers. It is thus not surprising that a recent infor- mal global poll of country officials and financial sector experts—the Financial Development Barometer3—undertaken for the 2014 World Bank Global Financial Development Report (GFDR) identifies financial education as the leading response to the question “What is the most effective policy to improve access to finance among low-income borrowers?” What remains to be proven is whether this faith in financial education is substan- tiated by evidence of impact. Are there approaches to teaching financial skills or modifying financial behaviors through educational programs, training, or other outreach activities that have reliable, positive results? The objective of this paper is to analyze the evidence of impact for financial literacy and capability interventions through a systematic review of the evidence. The review includes the use of meta- analysis, a statistical technique that pools data from different studies to test for sig- nificance in the enlarged sample of observations this creates. This paper is different from most previous narrative reviews in that it focuses exclusively on research that analyzes the impact of financial education interventions. Key characteristics of 188 papers are coded to create a rich data set with the characteristics of the interven- tions, as well as statistical information on the impact of programs on outcome Miller et al. 221 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 variables such as general savings, retirement savings, and credit performance. This data set is then used for a descriptive analysis of the literature and for empirical tests using meta-analysis. More than 140 of the 188 studies identified through this review indicate that financial education can be helpful in improving financial outcomes, although it is important to note that most of these employ non-rigorous empirical methods and may suffer from selection bias or other econometric concerns. Some of the more recent studies employ rigorous analytical tools such as randomized control trials (RCTs), and the impacts reported across these papers are more reliable. Examples of positive impacts come from Cai et al. (2013), who find that financial education ses- sions for rural farmers increase take-up rates for insurance in China. In South Africa, financial messages delivered through a popular soap opera are shown to improve desirable financial behaviors such as borrowing from formal financial insti- tutions rather than from higher-cost options such as retailers (Berg and Zia 2013). In India, Sarr et al. (2012) find that financial education increases the use of a no- frills savings account even months after the intervention ended. In the United States a non-RCT study of the Money Smart financial education curriculum by the FDIC (2007) finds that participants are more likely to open deposit accounts, save money, and adhere to a budget. However, our review of the literature also finds numerous papers (approximately 40 in our sample) citing either no impact or only a modest impact from the inter- vention, which may not justify the cost of financial education. Cole et al. (2011), for example, find that a financial education intervention among unbanked consum- ers in Indonesia is less effective at stimulating savings accounts than a small mone- tary incentive. Likewise, results from a media intervention in Kenya that included comics with financial literacy messages find no significant impact on key variables, including savings rates (Eissa et al. 2013). In the United States, Cole et al. (2013) evaluate mandated personal finance courses in high schools and find that they have no effect on financial outcomes, while training in mathematics is shown to benefit students through greater levels of financial market participation, more in- vestment income, and better debt management. Similarly, Hung and Yoong (2010) study retirement savings behaviors of adult populations in the United States and find that unsolicited financial advice has no impact on savings and investment decisions. Pooling and systematically studying these varied impacts through meta-analysis has the potential to provide valuable policy insight on what works in financial education, as well as to help identify where the research gaps lie. Yet the diversity of the research in this field thus far makes such comparison—and drawing con- clusions on effectiveness—difficult, even if it is a logical response to the varied and constantly evolving needs in this area.4 Variation in context, purpose and duration of training, target populations, and outcome measures is evident in the online 222 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 appendix 1, which presents brief information on each study, as well as in the descriptive statistics for the 188 papers, which are presented in main analysis of the paper. Nevertheless, our meta-analysis presents some key insights after controlling for observable differences across studies. Importantly, we find that financial educa- tion can affect financial outcomes such as savings and improved record keeping, but does less well in preventing negative outcomes such as loan defaults. These results suggest a role for financial education in improving behaviors where individ- uals have the ability to exert greater control. Arguably, loan default is imposed by external agencies (banks or other financial providers), and hence can only be avoided secondarily or over the long term if financial education leads to more prudent borrowing decisions. Savings and record-keeping, in contrast, are immedi- ate and primary decisions that can be acted upon by targeted consumers. The only other paper that uses meta-analysis to evaluate the literature on finan- cial literacy and capability (Fernandes et al. 2014) discusses another potential source of variance in the results for financial education interventions—omitted var- iables related to psychological traits such as impulse control, delayed gratification, and self-efficacy. These authors perform meta-analysis on both financial literacy and capability interventions, which they term “manipulated financial literacy” and on observational literature that links levels of financial knowledge or literacy to out- comes (termed “measured financial literacy” by the authors). Their findings indi- cate that measured financial literacy has a greater impact on financial outcomes than does manipulated financial literacy. These authors posit that omitted variables are a source of the different results because people with certain psychometric pro- files are more likely to engage in activities that increase their financial literacy levels and improve their financial outcomes, but these behaviors (self-control for example) are not the typical focus of financial education interventions that focus on impart- ing financial knowledge. Yet, while psychometric measures may indeed explain se- lection into desirable financial behaviors and choices, they are unlikely to be the only omitted category. Moreover, it is difficult to identify empirically whether such measures are the only (or even primary) omitted driver of financial choices. Our paper does not take a stand on the source of statistical bias in observational studies. Instead, we carefully classify and separate observational studies from impact studies, and more than double the number of financial education impact studies that are available for meta-analysis from slightly more than 80 in the case of Fernandes et al. (2014), to 188 (all papers cited in Fernandes et al. (2014) are also included in this paper). Another key difference from Fernandes et al. (2014) lies in the choice of variables and statistical rigor in the meta-analysis. As indicated previ- ously, there is great diversity in the sample of studies, which makes it impossible to calculate an effect size that is meaningful and comparable across the entire litera- ture, yet Fernandes et al. (2014) estimate an effect size that includes interventions that are fundamentally different across many characteristics such as how outcomes Miller et al. 223 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 are measured (binary or continuous), targeted populations, and mode of delivery. We adopt a more conservative approach and carefully screen and compare studies with similar outcome measures and intervention characteristics. This greater preci- sion comes at a cost, however, as we are only able to compare binary outcomes for a subset of studies that pass our comparability screening. Nevertheless, this careful analysis allows us to classify interventions among key topics such as savings, record keeping, and debt management, thus yielding more nuanced results in terms of the evidence of impact. Investigating the details of interventions and outcome measures in this way additionally allows us to present important stylized facts about financial literacy programs. This paper proceeds as follows. Section II describes the systematic review process including a brief discussion of meta-analysis and the rationale for using this tool for such a diverse body of literature. In this context, we examine previous narrative reviews and the literature they cover to help us understand why their findings on the impact of financial education have not been consistent. Section II also discusses the search approach used to identify relevant studies in journals, working papers, and other publications, as well as the inclusion and exclusion criteria. Section III provides information about the data set, including both the program descriptions and outcome variables that were identified. Section IV presents the results of the meta-analysis, its potential unique contribution, and its limitations. Section V con- cludes. Appendices 1 and 2 provide supplemental materials on the meta-analysis, and appendix 3 summarizes ways to strengthen the research protocols of financial literacy studies. The Systematic Review Process, Including Meta-Analysis This paper uses a systematic review process, including meta-analysis, to compare and contrast the findings of a large body of literature on the impact of financial liter- acy and capability interventions. The systematic review includes five steps: 1) Hypothesis 2) Search approach and inclusion/exclusion criteria 3) Collection and coding of data 4) Statistical analysis (meta-analysis) 5) Conclusions. A systematic review of a diverse body of literature helps us to identify patterns in the data and develop insights on the nature and quality of divergent evidence. The de- scriptive statistics presented in the next section are based on the coded data from the 188 studies of financial education interventions, and provide valuable insights on what has been studied thus far, the research methods that have been used, and 224 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 initial insights on the evidence. However, there are limits to what can be gleaned from these types of rough comparisons. Meta-analysis was developed to facilitate a statistically rigorous comparison of data across independent studies. By pooling data and statistical information across studies, papers that may individually point to inconclusive or contradictory results may together yield a statistically significant finding. If there are adequate data avail- able on means and standard deviations in the individual research papers, it may also be possible to identify an effect size which indicates the magnitude of change that may be expected. From a policy standpoint, meta-analysis can help enhance the external validity and general application of policy interventions, such as finan- cial literacy programs, and hence can serve as an important tool in developing and achieving specific development goals. There are two general types of meta-analysis models: fixed effects and random effects. Fixed effects models assume that the studies are all roughly equivalent in terms of the intervention studied, and thus are estimating the same outcomes. This means that data can be pooled and that larger studies with more observations are more heavily weighted than smaller ones. Medical drug trials are common forms of fixed effects meta-analysis since the interventions are identical ( provision of a drug or pill) and the outcomes are similar, observable, and uniformly measurable. Random effects models estimate the mean of a distribution of true effects but assume that each study is measuring a different effect size as the interventions and/ or populations are not equivalent. Weights in the random effects models are more balanced across studies and those with large sample sizes do not dominate the results as they would in a fixed effects model. There is only one source of error in a fixed effects model—random error—and as the sample size grows this tends toward zero. In the random effects model, however, there are two sources of error—random error within populations and error in estimations of the true effect size across studies. This means that a large number of observations in a study address only the first kind of error—within population—and not the estimation error across studies, which requires a robust number of studies to increase the precision of the estimate. The diverse nature of the underlying studies on financial literacy and capability clearly leads to the use of a random effects model.5 When looking at the available studies and data for this exercise, it is reasonable to assume that there are still substan- tial sources of error from both limited population sizes for specific interventions and in- adequate data on the scope of interventions and studies that are available for analysis. The effect sizes that are estimated in the regression analysis presented in this paper, therefore, should not be taken as a true measure of the effect size for financial literacy and capability interventions. This is not to say that the measures of effect size contain no information. They do provide an aggregate measure of impact and are indicative of the state of evidence, but at the same time should not be seen as definitive proof for or against the null hypothesis (defined below) on financial education activities. Miller et al. 225 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Hypothesis The null hypothesis for this study assumes that financial literacy and capability interventions do not affect the financial knowledge and/or financial outcomes of people who are subject to the treatment. Search Approach and Inclusion/Exclusion Criteria For this paper we undertook a comprehensive search of a particular segment of the literature on financial literacy and capability—papers that evaluate the impact of interventions designed to strengthen financial knowledge and behaviors. A broad definition of “financial literacy and capability interventions” was used for this review, which included any kind of intervention (intentional or not) that would impact financial knowledge, attitudes and/or behaviors for individuals. We identified papers meeting the criteria through several sources: 1) Search of peer-reviewed papers in Econlit under the broad terms “financial awareness”, “financial capability”, “financial competence”, “financial educa- tion”, “financial knowledge” and “financial literacy” between January 2000 and September 2013. 2) Search of papers included in previous literature reviews (starting with literature reviews published in 2007 – see table below). 3) Recent studies completed within the World Bank, many of which are listed on the website finlitedu.org. 4) Websites that are likely to include relevant studies, including the OECD, World Bank, Global Partnership for Financial Inclusion, and Alliance for Financial Inclusion (AFI). The process of evaluating studies for inclusion or exclusion was performed initially through a review of paper abstracts in the case of both papers identified through Econlit and those cited in previous literature reviews. For World Bank studies and research found on websites, the papers themselves were typically available and re- viewed directly. All papers that reported on an intervention are included in the online appendix 1, even those which lacked sufficient rigor or statistical results for use in the meta-analysis portion of this paper. Descriptive information from these papers is included, where possible, in the descriptive statistics presented in the fol- lowing section of the paper. In order to reduce the number of studies to review to a manageable size and to focus the evidence on research results that had been screened for quality, only articles from peer reviewed journals were included from Econlit, and only from January 2000 to September 2013. 226 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 One of the insights from the Econlit search on these key terms is that a relatively small percentage of the literature involves an evaluation of some kind of financial literacy or capability intervention or program. Even for the terms “financial educa- tion” and “financial literacy”, only about 10% of the studies in these categories evaluated the effectiveness of an intervention. Searches of “financial awareness” and “financial competence” yielded no impact evaluations, and even the widely used term “financial capability” produced only two papers for the database out of a total number of 77 citations. With the exception of the search for “financial litera- cy,” a vast majority of the papers listed under the searches conducted on Econlit are not about personal financial literacy or capability. Many of the excluded papers analyzed the importance of financial literacy or knowledge on various financial and non-financial outcomes. Using the terminology in Fernandes et al. (2014), these are considered “measured financial literacy” papers; they were excluded because they did not test an intervention but they do provide valuable insights on the importance of financial literacy and knowledge on outcomes. Studies were included when they discussed any program, educational outreach, media intervention or other type of communication or training for con- sumers that could either strengthen knowledge (financial literacy) or modify atti- tudes and behaviors (financial capability). A detailed table presenting the reasons for exclusion for the various search terms in Econlit is provided in a supplementary appendix on the WBRO website, along with full references and a table describing the 188 included papers. The review of nine previously published narrative reviews identified more than 500 unique references that included more than half of the 188 papers listed in the online appendix 1. The references cited by these nine narrative literature surveys, however, have relatively little overlap. This can be seen by calculating the Pearson correlation coefficients and associated R2 terms for reference lists across the litera- ture survey papers.6 In only two instances do the survey papers have enough common papers to result in a correlation coefficient above 0.2 (Martin 2007 with Hathaway and Khatiwada 2008, and Collins and O’Rourke 2010 with Agarwal et al. 2010), which is a relatively low level of correlation. The R2 terms (which is simply the squared term of the correlation coefficient), are sometimes called the co- efficient of determination and provide another indication of the strength of the rela- tionship among the variables, or in this case, the overlap in articles surveyed. Approximately 55% of the R2 figures are below .01, indicating a very weak relation- ship and relatively little overlap across studies. The limited overlap in terms of the literature being reviewed helps to explain the variety of findings in these narrative literature reviews. The Hathaway and Khatiwada (2008) summary of the findings is typical: “Unfortunately, we do not find conclusive evidence that, in general, financial education programs do lead to greater financial knowledge, and ultimately, to better financial behavior. However, Miller et al. 227 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 this is not the same as saying that they do not or could not – it is just that current studies, while at times illustrating some success, leave us with an unclear feeling about whether we can grant a blanket application of these results specifically, to financial education programs more generally.”7 Lusardi and Mitchell (2013) provide the most comprehensive and recent of the narrative literature reviews with nearly 200 references. These authors offer a cautious endorsement of financial edu- cation activities but also refer to the issues of endogeneity and omitted variables: “Research on efforts to enhance financial literacy suggest that some interventions work well, but additional experimental work is needed to explore endogeneity and establish causality.”8 While the search on Econlit was limited to papers that had appeared in peer reviewed journals, this exclusion criteria was not used for the many articles iden- tified through the survey papers or from other key sources (websites, World Bank research) in order to increase the coverage of available research on this topic. Many of the papers included in the analysis have been written in the past few years and thus many (about one-third, i.e., 34%) are still in working paper format, compared with 28% from peer-reviewed journals. Apart from uni- versities, the National Bureau of Economic Research and the World Bank are the main sources of working papers cited here. There are also numerous papers, briefs, and reports that are released by the many public, private, and non-profit organizations involved in this topic but which are not published through the academic press and are subject to peer review (38%). Known as “gray literature” these documents include research reports written by ac- ademics and researchers in think-tanks evaluating specific curricula using statisti- cal methods, and are thus also included in this literature search. However, since the financial education provider may often also be the funder for the evaluation, these papers are coded separately from working papers published through academic insti- tutions where such conflicts of interest are less likely to occur. Publication Bias The first step in any systematic review involves identifying relevant research so that conclusions are based on a comprehensive body of evidence that is not biased in some important way. If researchers or journal editors have a tendency to publish studies with a certain type of result (e.g., studies showing that financial education programs are effective in changing behavior, rather than studies showing no impact), this can result in one form of publication bias. The fact that many studies in the financial education literature (and in social science research more generally) involve little-known research projects with small sample sizes further exacerbates this problem since the broader community of practice may not be aware of instances 228 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 where research results are not published or widely disseminated. By contrast, medical research trials involving large public grants and large sample sizes are under more scrutiny and pressure to share results. Including studies from the “gray literature” that have not been published in journals can help to reduce the impact of this type of publication bias. In addition to including gray literature in the systematic review, this paper reviews all of the impact evaluations cited in previous narrative literature surveys as well as all interventions in the recently released meta-analysis by Fernandes et al. (2014). Further, papers were identified through searches of broad terms in Econlit, on websites, and through World Bank research. As such, this paper encom- passes a broader and more comprehensive analysis of the literature than previous reviews, which reduces the likelihood of this type of publication bias. The online materials for this paper include evidence from an econometric test (metafunnel in Stata) for publication bias to help illustrate the extent to which the review may be affected by missing studies. Failing to publish the results from research is perhaps the most well-known type of publication bias, but it is not the only one. According to Rothstein, Sutton and Borenstein (2005), other types of publication bias (or dissemination bias) that can negatively impact a systematic review include the following: language bias (includ- ing only research in a certain language); availability bias (limiting consideration to research that is readily available to the researcher); cost bias (including only research papers that are available for free or at a low cost); familiarity bias (limiting the research to papers in one’s own discipline); and outcome bias (reporting selec- tively on the empirical data in papers to highlight certain types of results. These various types of publication bias are not considered significant issues for this review. In terms of language bias, the vast majority of research on this topic internationally has been done in the United States ( perhaps due to the greater level of consumer lending and less heavily regulated financial markets), and in the English language. Further, the search process did not use language as a criterion for inclusion and where non-English language research was identified, efforts were made to include these studies. In terms of availability, studies on financial literacy and capability are generally publicly available through online databases available to academic researchers or on websites of relevant NGOs, public agencies, or multilateral organi- zations. No study was eliminated due to cost consideration, and in some cases authors were contacted directly to request copies of articles not readily available online (in the few instances of older research articles). Also, this review extends beyond typical economics journals to include research published by academics in consumer and family sciences, education, sociology, and psychology. All empirical findings are reported where a financial intervention was an explanatory variable in a regression model explaining a financial outcome or behavior, including financial knowledge gains. Miller et al. 229 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 A Systematic Review of Financial Literacy and Capability Interventions Using Meta-Analysis The extensive literature search described in the previous section resulted in 188 journal articles, reports, and other publications that analyze the effectiveness of financial education interventions. The rigorous meta-analysis of this literature re- sulted in a unique database that enables us to more precisely describe the types of fi- nancial education programs or interventions that have been evaluated, as well as their reported impacts. The main relationship that is tested through meta-analysis in this paper is whether financial education interventions have an impact on outcome variables of interest such as general savings levels, retirement savings, record keeping, and credit performance. If data were available, outcome variables could extend to other economic indicators of well-being such as total assets, consumption, or income; or on the other hand, to measures of vulnerability such as declaring bankruptcy or losing major assets such as a home or land. The key characteristics of financial education interventions are also recorded. In the context of meta-analysis these data are referred to as moderator variables because they help to explain the strength of the relationship between the predictor variable (in this case financial education) and the outcome. Moderator variables that are collected for this paper include information on the financial topic that is addressed (i.e., savings, credit, etc.), the type of delivery channel used (classroom instruction, mass media, etc.), the country where the intervention occurred, the lo- cation of the intervention (school, workplace, etc.), the number of hours of instruc- tion, whether teacher training was involved, and whether the intervention focused narrowly on the individual or extended to the family or broader social setting. Another important data point that was collected through this exercise concerned the evaluation methods used. In particular, the dataset distinguishes whether studies used a randomized controlled trial (RCT) design. The Dataset—Moderator Variables The vast majority (98%) of studies on financial education interventions conducted prior to 2008 identified through this search focused on the United States. This started to change after 2008, when the financial crisis focused attention on the im- portance of this topic not only in the United States, but around the world. Indeed, only 54% of later studies (2009 forward) in the dataset focus on the United States. The World Bank also contributed to the growth of literature on the impact of finan- cial education interventions in developing country markets, with the institution’s researchers contributing to more than 20 papers on this topic since 2008. Table 1 230 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Miller et al. Table 1. Descriptive Statistics for 188 Studies in Dataset (percentages of total) Latin Africa Asia Europe America United States Other Location of intervention 6 9 2 6 75 2 Business Credit Counseling Mixed topics Mortgage Savings & Other Management Retirement Financial topic taught 7 9 43 5 30 6 Course taught in Individual counseling Mass media Mixed Online Phone Print Other person methods Delivery channel 54 9 5 18 3 3 6 3 Community / third Financial institution High school Home-based Mixed University Workplace Other party Setting for instruction 24 11 14 8 14 4 19 5 Less than or equal Three to six Seven to ten 11– 20 21 – 50 More than Varied to two fifty Intensity of intervention 16 11 11 11 6 2 43 (hours) Less than or equal Between one week and One to six Six months More than one Varied to one week one month months to one year year Duration of 33 5 17 5 7 32 intervention 231 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 provides data on a number of characteristics of the studies, including region where the intervention took place. Africa, Asia, and Latin America are the regions outside the United States with the largest shares of studies in our sample. A wide range of financial topics are covered by financial education interventions in the literature, which were grouped into six categories in this paper. Interventions that provide information on a variety of financial issues, coded as “mixed” interventions in the data, are the most common type and represent more than 40% of the total sample. Financial literacy programs in the United States are most likely to involve either multiple messages (mixed) or focus on savings and retirement. In developing countries, courses that include business management are among the most common in our sample, as are classes involving multiple messages, with each comprising about one-quarter of the sample; savings and retirement messages are a close third. The majority of interventions in this sample involve direct contact with an in- structor/advisor, typically in a classroom or seminar setting, with a significantly smaller share (9%) involving individualized counseling. In terms of the cost per person reached, these kinds of interventions are likely to be the most expensive. Most interventions studied in the literature also focus on programs using only one delivery channel (classroom, video, phone advice, etc.) although there is widespread awareness in the communications field that a more comprehensive approach using multiple types of media (often referred to as a transmedia or 360o strategy) is more effective. A relatively small number of interventions in the evaluation literature examine the use of mass media. These interventions might include using TV soap operas in Africa and in the United States, both of which show positive results in in- creasing awareness and shifting attitudes. While financial education interventions are often associated with youth and programs taught in schools, these represent only a relatively small share of the interventions that have been rigorously studied (18% including both high school and university programs). Most programs that have been evaluated are provided through community orga- nizations or in the workplace. Workplace programs almost always relate to a “teach- able moment” when employees have an opportunity to immediately use the information they are gaining, such as signing up for benefits or to save part of their salary. Many community-based interventions also focus on teachable moments, sometimes related to helping people through challenging financial times (such as credit counseling for people finding themselves overwhelmed by debt, or mortgage counseling for prospective home buyers). Community interventions are almost ex- clusively focused on low income populations or other groups that are seen as finan- cially vulnerable (e.g., battered women). Most of the financial education interventions that have been evaluated are relatively short in terms of the number of hours of exposure (intensity of the inter- vention) and the period of time during which the exposure occurred (duration of the intervention). Slightly more than one-third of the interventions (38%) last 232 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 10 hours or less, and 16% last two hours or less. These shares are probably under- stated, as many of the interventions labeled as “varied” in terms of hours of expo- sure likely fit within the 10 hour or less categories. More than one-third of financial capability interventions are delivered within one week or less, and more than one-half of interventions are delivered within six months or less. Just as with the classification of intensity, these shares are likely to be underestimates, as many of the interventions listed as “varied” are likely to be of relatively short duration (six months or less). Since repetition is a key element of learning, the limited amount of time typically devoted to interventions may work against stronger impact results. Overall, there has been a noticeable evolution in the literature on financial educa- tion interventions over the past five years. The increase in research beyond the United States mentioned previously has been accompanied by an increase in more rigorous research methods such as randomized controlled trials (RCTs). These RCTs have gone from being relatively rare (through 2008 only 14% of studies in the dataset use RCTs), to frequently used, with 43% of papers from 2009 onward using these rigor- ous methods. In order to be as comprehensive as possible in reviewing the literature, studies were included that used a variety of research methodologies, not only RCTs. Results from Meta-Analysis This section presents the results from applying meta-analysis statistical techniques on the data assembled on impact evaluations of financial education interventions. Before discussing the specific results it is important to place them in context. The diversity of the literature on financial education interventions means that even where there were common outcome variables (such as likelihood to save) the under- lying interventions were each unique, making a direct comparison impossible. The limited number of studies available for evaluation in each of the categories (no more than ten, often closer to five) also reduces the strength of the statistical results. Despite these limitations, it is still useful to systematically and quantitatively evalu- ate the diverse body of evidence on financial education interventions to provide policy makers with an indication of what is working in financial education, and to provide researchers with insights on where knowledge gaps lie. Identifying the Issues That Could Be Tested Using Meta-Analysis The majority of the 188 papers identified through the search as presenting results of financial literacy or capability interventions include potentially useful statistical data that could be used for meta-analysis. Many papers report on multiple regressions that Miller et al. 233 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 are of interest and have multiple outcome variables or results from empirical tests on population subsets based on criteria such as educational attainment, income level, or score on a financial literacy test. In some cases, there are several types of explanatory variables related to the intervention. There are a total of 839 observations from the 188 papers; however, not all contain usable statistical information. For the meta-analysis of this paper, outcome (dependent) variables were coded among several major categories based on a review of the variable descriptions (savings, borrowing, financial literacy test scores, account opening, and record keeping). Within these categories there was further disaggregation to ensure com- parability across studies for the meta-analysis. For example, category “S1” corre- sponds to the savings category and is a binary variable with a value of “1” if the individual reported having saved, while “S4” is a continuous variable indicating total reported savings during a particular period of time. No effort was made to compare these types of disparate data through meta-analysis. Rather, we used the coding of outcome variables to identify those instances where existing studies had similar impact measurements, and thus could be reasonably combined in a meta- analysis. The list of coded outcome variables is provided with other detailed back- ground information in the supplementary online appendix. Once the outcome variables were categorized, the data were analyzed to deter- mine the number of studies using this variable, and to further identify those which reported adequate statistical information to allow them to be included in the meta- analysis. We selected five studies as a minimum number for the meta-analysis.9 Further, each study had to provide the coefficient on the financial literacy/capability intervention and the standard error (or other statistical data from which to construct the standard error), and studies had to have comparable measurement criteria for the outcome variables. For example, studies that reported on a change in a financial liter- acy score were not used because the studies were based on potentially very different tests and scales for measuring financial literacy, and thus did not represent a true comparison across similar results. This rigorous screening exercise produced just four outcome variables that were used in five or more separate studies. The four qualifying binary variables listed below were analyzed using meta- analysis: S1 ¼ savings reported in past period (1 if positive)—binary S11 ¼ contributes to retirement savings (1 if positive)—binary B6 ¼ defaulted on a loan (1 if positive)—binary R1 ¼ keeps financial records/budgets (1 if positive)—binary In addition to the development of forest plots with the 4 outcome variables men- tioned above, meta-regression analysis was used to determine if specific characteris- tics of the interventions (the explanatory variables) made significant contributions to results. Due to the small sample sizes, these explanatory variables were tested 234 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 individually on each outcome variable. The intensity of the intervention, measured in number of hours of instruction or exposure, was the only continuous explanatory variable that was also widely reported on (with only one or two exceptions) among the studies used in the meta-analysis. The results from the forest plots are presented in the next section and were produced using study-level summary data because in- dividual observations from all studies are not available. The meta-analysis performed is a two-stage process involving the estimation of a relevant summary statistic for each of a set of studies, followed by the calcula- tion of a weighted average of these statistics across the studies (Deeks, Altman, and Bradburn 2001). In this case, pre-calculated effect estimates and their stan- dard errors from each study are pooled. The meta-analysis includes a forest plot in which results from each study are displayed as a square and a horizontal line, representing the intervention effect estimate together with its confidence inter- val. The area of the square reflects the weight that the study contributes to the meta-analysis. The combined-effect estimate and its confidence interval are rep- resented by a diamond. Meta-Analysis Results The most common single-issue topic for financial education interventions was savings and retirement. The greatest number of comparable papers also addresses this issue. The forest plot from the research papers that address the impact of inter- ventions on savings are shown below in figure 2. A forest plot graphically displays the treatment effects across multiple studies, with a solid line representing the null hypothesis of no effect from the intervention—in this case from exposure to finan- cial education outreach/training. Intensity refers to the number of hours of expo- sure. Channel refers to the type of intervention/content delivery method. The results are presented with the statistical output from the meta-analysis, as well as with data on important characteristics of the interventions. The weight as- signed to each of the six studies is provided in the far-right column and is based on the reported strength of the statistical result in the various studies. The size of the shaded squares for each study reflects their weight in the meta-analysis. The hori- zontal line passing through the shaded square is the confidence interval for the re- ported results. When this line intersects the vertical line, which represents the null hypothesis (no effect from financial education), then the null cannot be rejected. In this particular meta-analysis, only two of the six papers reject the null hypothesis and indicate a positive impact on savings from financial education, while four cannot reject the null hypothesis. Taken together, however, the meta-analysis finds that these papers do provide evidence of impact for financial education at the 90% confidence interval. The confidence interval for the pooled data across the Miller et al. 235 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Figure 1. “What is the Most Effective Policy to Improve Access to Finance Among Low-Income Borrowers?”. Responses from the 2012 Survey of the Financial Development Barometer. Source: 2014 Global Financial Development Report. Figure 2. Papers Testing Savings Behavior after Intervention. Note: Weights are from random effects analysis. 236 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 six papers is represented by the diamond at the bottom of the graph and is on the right-hand side of the null hypothesis, rejecting a finding that financial education has no impact on savings behavior. We lack the data on control group means to determine the true effect size for this finding. The meta-analysis in figure 2 was based only on data from RCT studies to increase the quality of the data and thus to strengthen the cross-study analysis. The I-squared statistic, which is also presented in the forest plots, describes the percentage of variation across studies, which is due to heterogeneity rather than chance. Thus, the I-squared is useful for determining the consistency of results between studies in a meta-analysis. A lower I-squared would indicate more consis- tency underlying the results of the sample studies, while a higher I-squared would indicate more heterogeneity in this regard. The p-value also provides a way to test for heterogeneity by examining the null hypothesis that all studies in the meta- analysis are evaluating the same effect. A p-value lower than 0.05 would reject this hypothesis at the 5% significance level, and indicate that the results of the studies are more heterogeneous. As seen in figure 2 (savings behavior), the I-squared was 54.5%, which is not generally considered a low number (indicating consistency), but which is lower than the I-squared terms for the other outcome variables, indicating relatively more consistency among the results for savings. The other meta-analysis results (figure 3—retirement savings behavior; figure 4—loan default behavior; and figure 5—record-keeping behavior) reveal higher I-squared statistics than figure 2, indicating more heterogeneity between the studies involved in their respective samples. Meta-analysis evaluating the impact of financial education interventions on re- tirement savings produced similar results at the 90% confidence interval, as shown in figure 3. Five studies were included in this analysis, all based on programs in the United States, none using randomized controlled trials, and only two using exclu- sively classroom methods. Care should be taken with the interpretation of these results—the sample diver- sity and small sample size in both the meta-analysis of general savings and retire- ment savings require that the results are seen as indicative rather than definitive in terms of the impact of financial education on savings. Still, they provide another piece of information to policy makers interested in the use of financial education for increasing saving, whether for general purposes or for retirement/old age. In addition to the topic of savings, numerous studies focused on credit and bor- rowing and used information on loan defaults after interventions as an outcome variable. The meta-analysis of these studies found the least evidence of impact from financial education interventions. Figure 4 presents this analysis in a forest plot. Even at the 90% confidence interval, the null hypothesis of no impact from the intervention cannot be discarded. Miller et al. 237 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Figure 3. Papers Testing Retirement Savings Behavior after Intervention. Note: Weights are from random effects analysis. Figure 4. Papers Testing Loan Defaults after Intervention. Note: Weights are from random effects analysis. While the results from the meta-analysis in figure 4 are significantly heteroge- neous and do not show a significant overall effect size, they nevertheless provide in- teresting insights. Two substantive points of difference between the studies that did and did not show impact lie in whether the education intervention was targeted 238 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Figure 5. Papers Testing Record Keeping Behaviors after Intervention. Note: Weights are from random effects analysis. around a specific topic and whether there was an associated immediate opportunity to use the newly-acquired information. The two studies that particularly showed positive impacts on loan behavior were Agarwal et al. (2010) and Gine and Mansuri (2011). The study population in Agarwal et al. (2010) consisted of low-income households with lower credit scores who voluntarily participated in a counseling program provided by a non-profit orga- nization. In this sense, the participants in the financial education intervention gained targeted information (specific to home ownership) and had the immediate opportunity to apply their learning in a relevant context ( purchase of a home). In this same light, the study participants in Gine and Mansuri (2011) were microfi- nance clients who could choose to participate in a business training course and loan lottery, and then apply their learning directly to their microenterprises and loan behavior. On the other hand, Bruhn, Ibarra, and McKenzie (2012) studied the effects of financial education programs that covered a broad range of financial topics in which the population did not experience an immediate circumstance like the purchase of a home or the financing of a business in which they could apply their learning. Bruhn, Ibarra, and McKenzie (2012) evaluated a broad-based finan- cial education program in Mexico, and among other factors, the diverse variety of topics covered and lack of a “teachable moment” to apply the learning may have contributed to the lower reported effectiveness of their program on loan default behavior. Similarly, while loan default was an outcome measured in Berg and Zia Miller et al. 239 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 (2013), the topic was not central to their intervention, which focused on source of borrowing (formal vs. retail credit) and gambling behavior. The final meta-analysis relates to the impact of financial education on record- keeping. Record keeping and tracking expenditures are often cited as critical ele- ments of gaining control of one’s finances, much the way that many fitness and diet programs focus on recording eating and exercise habits to control weight and improve health. This is a behavior that is fully under the control of the individual as compared to decisions to default or even to save money, which may be influenced by factors outside of one’s control such as unexpected illness (and medical fees), loss of a job or other problems leading to financial distress. The forest plot and regression results (figure 5) combine data for record keeping related to both personal finance and to microenterprise or business records. This is necessary due to the small sample size, but there are also reasons to believe that effects may be similar across these two types of interventions. Microenterprises frequently co- mingle finances between the business and individual owner, so record keeping for a business is likely to include some common elements with personal financial manage- ment facilitating comparisons between these two types of interventions. The meta-analysis presented in figure 5 indicates that financial education may positively encourage record keeping behaviors, although the meta-analysis results are not quite significant at the 90% confidence interval ( p ¼ 0.134). The effects are weighted fairly evenly across the studies, which involve an unusually diverse set of country cases (Ghana/Kenya, Vietnam, Australia/New Zealand, Indonesia, and Mexico) but which use, with one exception, classroom methods. Utilizing the meta regression framework, we were also able to analyze the rela- tionship between the intensity of the treatment (in hours of instruction/exposure to the intervention) and the outcome of financial behaviors for savings, loan default, and record-keeping.10 The corresponding graphs for these exercises can be found online at the WBRO website. While the results were not statistically significant in part due to the small number of observations, they nevertheless provide a sense of the directional relationship between each outcome variable and the intensity of the associated treatment intervention. The overall trend line suggests no direct relation- ship between intensity of treatment exposure and impact on savings behavior. However, upon closer inspection of the individual data points, we can see that there is a positive relationship between hours of treatment exposure and impact on savings up to about 15 hours, whereafter impact declines. This result suggests that more instructional time may be important for increased positive impact, but there may also be a point at which diminishing returns set in. This relationship would be a good candidate for deeper investigation and further research. Findings from the sample suggest a negative association between intensity of exposure and the loan default outcome. Increased exposure to the treatment inter- vention may reduce the likelihood of loan default, again providing an indication 240 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 that more instructional time can lead to greater treatment impacts. Finally, analysis suggests that the intensity of exposure to a financial capability intervention is asso- ciated with improved record-keeping behavior. While this finding does not rise to the level of causality, the data here seem to be the most supportive of a relationship between these two variables when compared with the previous regression results. It may be the case that additional instructional time is especially valuable for learning record keeping skills. Another possibility is that regardless of one’s financial situation, record keeping can be implemented, whereas savings and loan repayment are more dependent on financial standing. Further study into these initial findings would help focus researchers’ efforts to help people achieve financial literacy. Conclusions The 188 papers that were identified for this meta-analysis represent an incredible level of diversity across numerous study characteristics including, importantly, outcome measures. The diversity in both general outcomes (savings, credit per- formance, record keeping, financial knowledge, etc.) and in how they are measured severely reduces the number of comparable studies for the meta-analysis and meta- regressions. Further, effect sizes are not comparable as typically data on control group means are not available for this estimation from the published works. Where randomized controlled trials have been done, the results of impact appear limited at best, indicating that perhaps omitted variables or publication bias may be present for studies not employing these rigorous methods. With these caveats, some policy insights do emerge. Our meta-analysis suggests that financial education can impact some financial behaviors, including savings and record keeping. These are both considered funda- mental to good personal financial management and are potentially behaviors where individuals can exert greater control than in the case of other outcomes such as loan defaults. From a policy perspective, these behaviors are also relatively simple to target since advocating for regular record keeping and household saving do not necessarily require institutional changes or the creation of new financial products, as would be the case for loans or insurance. Meta-analysis was unable to provide insights regarding the importance of program characteristics on impact due to the nature of the sample and lack of direct comparability. For example, the intensity (number of hours of exposure) of the treat- ment was weakly correlated with improved financial outcomes in the case of record keeping, but was not significant in the other specifications of the model. Other char- acteristics that could reasonably be expected to influence the effectiveness of inter- ventions (delivery mechanism, duration, or period of time during which subjects were exposed to the treatment, location of the intervention such as school, Miller et al. 241 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 community, workplace) were not shown to be significant, but the small sample size limits the power of the statistical analysis. These are important factors in creating fi- nancial education policy and more research is needed to increase the body of evi- dence on the types of intervention characteristics and their influence on financial education effectiveness. Moreover, the difficulty in evaluating this literature for meta-analysis highlights the importance of strengthening and expanding the use of rigorous evaluation methods, not only for comparable research but also to understand policy implica- tions. Researchers should take care to report more complete statistical information so that others can better understand the strength of the findings and the limitations of the results, as well as possibly use the findings in future applications of meta- analysis to this topic. There is also a potential value from defining outcome variables in common terms, including measuring and reporting dependent variables, to facili- tate comparisons across studies. To increase the likelihood that these variables could be used in a large number of studies future research should employ simple and replicable definitions and measurement criteria. Using common questions and/ or survey instruments such as those developed at the World Bank, DfiD, and OECD in recent years to measure financial literacy and capability in a target population is a step in the right direction and will help increase the availability of comparable data on what is effective in financial education. Finally, new and ongoing research should make use of social science impact evaluation registries undertaken by the American Economic Association and 3ie, to provide a common repository of re- search for future meta-analyses, as well as avoid issues related to publication bias. The supplementary online appendix 3 details some key steps that could strengthen research protocols in financial literacy studies going forward. Finally, a surprising and yet common omission among virtually all studies is the analysis of costs and benefits of financial literacy interventions. Likewise, we did not find any discussion of potential alternative methods to achieve the desired outcomes across the 188 articles in this literature, which could help benchmark both costs and benefits. Such analyses could be simple to perform in individual studies, and certainly worthwhile to assess returns to development investments. Supplementary Data The supplemental appendix to this article is available at (http://wbro.oxfordjournals. org/). Notes This paper is part of the background research conducted for the 2014 World Bank Global Financial Development Report on Financial Inclusion. The authors would like to acknowledge comments received 242 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 on this document from Martin Cihak, from peer reviewers Annamaria Lusardi, David McKenzie, Florentina Mulaj, Valeria Perotti, Oya Pinar, and Siegfried Zottel, and from Caio Piza and other partici- pants at a May 15, 2014 Global Financial Development Report seminar. Research assistance was also provided by Hendrick Chan, Meghan Conway, and Anisha Mudaliar. All authors are from the World Bank. Corresponding authors are Margaret Miller (mmiller5@worldbank.org) and Bilal Zia (bzia@ worldbank.org). 1. Finscope surveys are nationally representative surveys on financial outcomes and behaviors. The surveys are conducted by Finmark Trust and have thus far been done in 18 countries across Africa and Asia. 2. The 2014 World Bank Global Financial Development Report (GFDR) compares national survey data on responses to three common questions used to evaluate financial knowledge—calculation of compound interest, understanding inflation, and diversification of risk. Based on data from more than 30 countries, average response rates were 56% correct for the question on compound interest, 63% for the question on inflation, and 48% for the question on risk diversification. Lower-income countries scored significantly lower than high-income countries—when higher income OECD countries are eval- uated separately from the rest of the sample, the difference between the percentage correct on com- pound interest is more than 20 points (OECD at 65.5%, lower income countries at 44.5%). 3. The Financial Development Barometer is an informal poll available online through the website for the Global Financial Development Report. In total, officials from 21 developed and 54 developing economies participated in the survey. From 265 polled, 161 responded, that is, a 61% response rate. 4. A number of different financial skills and behaviors are needed over the course of one’s life. Further, innovation in financial markets can quickly create demand for new skills or make others irrel- evant (such as writing checks), and new technologies create new delivery channels for training. Skills required by employed workers in high-income countries, such as investment abilities and pension planning, are irrelevant for low-income consumers in developing countries, further adding to the diversity of interventions present in a global review. 5. In this paper we chose to use a random effects model using the method of DerSimonian and Laird (1986), which is a variation on the inverse-variance method employed in order to incorporate the assumption that the different studies are estimating different yet related intervention effects. The Stata command used is metan. This produces a random-effects meta-analysis, and the simplest version is known as the DerSimonian and Laird method. To undertake a random-effects meta-analysis, the standard errors of the study-specific estimates are adjusted to incorporate a measure of the extent of variation, or heterogeneity, among the intervention effects observed in different studies. The amount of variation, and hence the adjustment, can be estimated from the intervention effects and standard errors of the studies included in the meta-analysis. 6. A matrix was created based upon the references in each of these papers, for a total of 536 unique citations. For each of the nine literature reviews, a “1” or “0” was entered to indicate whether the refer- ence was cited in the review. This matrix was then used to calculate the Pearson correlation coefficients and r2 terms. The survey papers are all listed in the references and indicated with an asterisk (*). 7. Hathaway and Khatiwada (2008), page 19. 8. Lusardi and Mitchell (2013), page 49. 9. In the case of one of the outcome variables—loan defaults—the number of eligible papers dropped to just four after further review due to the exclusion of a study (Quercia and Spader 2008), which used a logit model instead of one employing linear probability. 10. Sufficient data on intensity of intervention was not available for the retirement savings behav- ior variable. References Abreu, M., and V . Mendes. 2010. “Financial Literacy and Portfolio Diversification.” Quantitative Finance 10 (5) : 515 –28. Miller et al. 243 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Agarwal, S., G. Amromin, I. Ben-David, S. Chomsisengphet, and D. D. Evanoff. 2010. “Learning to Cope: Voluntary Financial Education and Loan Performance during a Housing Crisis.” American Economic Review: Papers & Proceedings 100 : 495 –500. Gulsun, A. G., A. F. Aysan, S. Ozcelik, and L. Yildiran. 2013. “Credit Card Satisfaction and Financial Literacy: Evidence from an Emerging Market Economy.” Emerging Markets Finance & Trade 48 (5) : 103 –15. Alessie, R., M. Van Rooij, and A. Lusardi. 2011. “Financial Literacy and Retirement Preparation in the Netherlands.” Journal of Pension Economics and Finance 10 : 527– 45. Amromin, G., I. Ben-David, S. Agarwal, S. Chomsisengphet, and D. D. Evanoff. 2010. “Financial Literacy and The Effectiveness of Financial Education and Counseling: A Review of the Literature.” The Federal Reserve Bank of Richmond, Working Paper Series WP 07 –03. Behrman, J. R., O. S. Mitchell, C. K. Soo, and D. Bravo. 2012. “The Effects of Financial Education and Financial Literacy: How Financial Literacy Affects Household Wealth Accumulation.” American Economic Review: Papers & Proceedings 102 (3) : 300– 04. Berg, G., and B. Zia. 2013. “Financial Literacy through Mainstream Media: Evaluating the Impact of Financial Messages in a South African Soap Opera.” World Bank Working Paper, Washington, DC. Bernheim, B. D., and D. M. Garrett. 2003. “The Effects of Financial Education in the Workplace: Evidence from a Survey of Households.” Journal of Public Economics 87 : 1487–1519. Beshears, J., J. J. Choi, D. Laibson, B. C. Madrian, and K. L. Milkman. 2012. “The Effect of Providing Peer Information on Retirement Savings Decisions.” Financial Literacy Center Working Paper: WR- 800-SSA. Harvard University. Bruhn, M., G. Lara Ibarra, and D. McKenzie. 2012. “Does Financial Literacy Training Promote Saving and Responsible Credit Card Use?” Financial Literacy & Education Russia Trust Fund, Washington, DC. Bruhn, M., L. de Souza Leao, A. Legovini, R. Marchetti, and B. Zia. 2013. “Financial Education and Behavior Formation: Large-Scale Experimental Evidence from Brazil.” World Bank Working Paper, Washington, DC. Bucher-Koenen, T., and A. Lusardi. 2011. “Financial Literacy and Retirement Planning in Germany.” Journal of Pension Economics and Finance 10 (4) : 565 –84. Cai, Jing, Alain De Janvry, and Elisabeth Sadoulet. 2013. “Social Networks and the Decision to Insure.” Available on Social Science Research Network (www.ssrn.com). Christelis, Dimitris, Tullio Jappelli, and Mario Padula. 2010. “Cognitive Abilities and Portfolio Choice.” European Economic Review 54 : 18 –38. Cole, S., A. Paulson, and G. K. Shastry. 2013. “High School and Financial Outcomes: The Impact of Mandated Personal Finance and Mathematics Courses.” Manuscript, Harvard Business School. Cole, Shawn, Thomas Sampson, and Bilal Zia. 2011. “Prices or Knowledge? What Drives Demand for Financial Services in Emerging Markets?” The Journal of Finance 66 (6) : 1933–67. Collins, J. Michael, and Collin M. O’Rourke. 2010. “Financial Education and Counseling - Still Holding Promise.” Journal of Consumer Affairs 44 (3) : 483– 98. Deeks, JJ, DG Altman, and MJ Bradburn. 2001. “Statistical Methods for Examining Heterogeneity and Combining Results from Several Studies in Meta-Analysis.” In M. Egger, D. Smith, and D. G. Altman, eds., Systematic Reviews in Health Care: Meta-Analysis in Context (2nd edition). London, UK: BMJ Publication Group. Ding, Lei, Roberto G. Quercia, and Janneke Ratcliffe. 2008. “Post-Purchase Counseling and Default Resolutions among Low- and Moderate-Income Borrowers.” Journal of Race, Ethnicity, and Religion 30 (3) : 315– 44. 244 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Doi, Yoko, David McKenzie, and Bilal Zia. 2012. “Who You Train Matters: Identifying Complementary Effects of Financial Education on Migrant Households.” World Bank Working Paper 6157, Washington, DC. Drexler, Alejandro, Greg Fischer, and Antoinette Schoar. 2014. “Keeping It Simple: Financial Literacy and Rules of Thumb.” American Economic Journal: Applied Economics 6 (2) : 1–31. Eissa, Nada, James P Habyarimana, and William G Jack. 2013. “Impact of Cartoons on the Effectiveness of Financial Education: Evidence from a Randomized Experiment in Kenya.” Financial Literacy & Education Russia Trust Fund, Washington, DC. Federal Deposit Insurance Corporation. 2007. “A Longitudinal Evaluation of the Intermediate-Term Impact of the Money Smart Financial Education Curriculum Upon Consumer’s Behavior and Confidence.” Federal Deposit Insurance Corporation, Washington, DC. Fernandes, Daniel, John G. Lynch, Jr., and Richard G. Netemeyer. 2014. “Financial Literacy, Financial Education, and Downstream Financial Behaviors.” Management Science 5 (8): 1861–83. Gale, William G., and Ruth Levine. 2010. “Financial Literacy: What Works? How Could It Be More Effective?” RAND Corporation Working Paper, Arlington, VA. Gerardi, Kristopher, Lorenz Goette, and Stephan Meier. 2010. “Financial Literacy and Subprime Mortgage Delinquency: Evidence from a Survey Matched to Administrative Data.” Federal Reserve Bank of Atlanta Working Paper Series 2010–10. Gibson, John, David McKenzie, and Bilal Zia. 2012. “The Impact of Financial Literacy Training for Migrants.” World Bank Working Paper 6073, Impact Evaluation Series No. 57, Washington, DC. Gine, Xavier, and Ghazala Mansuri. 2011. “Money or Ideas? A Field Experiment on Constraints to Entrepreneurship in Rural Pakistan.” World Bank Group Working Paper, Washington, DC. Hastings, Justine S., Brigitte C. Madrian, and William L. Skimmyhorn. 2012. “Financial Literacy, Financial Education and Economic Outcomes.” National Bureau of Economic Research, NBER Working Paper 18412, Cambridge, MA. Hathaway, Ian, and Sameer Khatiwada. 2008. “Do Financial Education Programs Work?” Federal Reserve Bank of Cleveland, Working Paper 08 –03. Honohan, P., and M. King. 2009. “Cause and Effect of Financial Access: Cross-Country Evidence from the Finscope Surveys”. Paper prepared for the World Bank Conference “Measurement promotion and impact of access to financial services”, Washington, DC. Hung, Angela A, and Joanne K. Yoong. 2010. “Asking for Help: Survey and Experimental Evidence on Financial Advice and Behavior Change.” RAND Corporation Labor and Population Working Paper, WR-714-1, Washington, DC. Klapper, Leora, Annamaria Lusardi, and Georgios Panos. 2013. “Financial Literacy and its Consequences: Evidence from Russia during the Financial Crisis.” Journal of Banking and Finance 37 (10) : 3904–23. Lusardi, Annamaria, and Olivia S. Mitchell. 2007. “Financial Literacy and Retirement Planning: New Evidence from the Rand American Life Panel.” Michigan Retirement Research Center Research Paper No. WP 2007–157, Ann Arbor, MI. , 2011. “Financial Literacy Around the World: An Overview,” National Bureau of Economic Research, Working Paper No. 17107, Cambridge, MA. . 2012. “Financial Literacy and Planning: Implications for Retirement Wellbeing.” In A. Lusardi, and O. S. Mitchell, eds., Financial Literacy: Implications for Retirement Security and the Financial Marketplace. Oxford: Oxford University Press. . 2013. “The Economic Importance of Financial Literacy: Theory and Evidence.” National Bureau of Economic Research, Working Paper 18952, Cambridge, MA. Miller et al. 245 Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Mano, Yukichi, Alhassan Iddrisu, Yutaka Yoshino, and Tetsushi Sonobe. 2011. “How Can Micro and Small Enterprises in Sub-Saharan Africa Become More Productive? The Impacts of Experimental Basic Managerial Training.” National Graduate Institute for Policy Studies, GRIPS Policy Research Center Discussion Paper: 11 –06. Martin, Matthew. 2007. “Literature Review on the Effectiveness of Financial Education.” Federal Reserve Bank of Richmond, Working Paper No. 07 – 03. McCormick, Martha Henn. 2009. “The Effectiveness of Youth Financial Education: A Review of the Literature.” Journal of Financial Counseling and Planning 20 (1) : 70 –83. Muller, Leslie A., 2003. “Does Retirement Education Teach People to Save Pension Distributions?” Social Security Bulletin 64 (4) : 48 –65. Quercia, Roberto, and Jonathan Spader. 2008. “Does Homeownership Counseling Affect the Prepayment and Default Behavior of Affordable Mortgage Borrowers?” Journal of Policy Analysis and Management 27 (2) : 304–25. Hannah R. Rothstein, Alexander J. Sutton, and Michael Borenstein, eds. 2005. Publication Bias in Meta-Analysis – Prevention, Assessment and Adjustments. Hoboken, NJ: John Wiley & Sons Ltd. Sarr, Leopald, Santadarshan Sadhu, and Nathan Fiala. 2012. Bringing the Bank to the Doorstep: Does Financial Education Influence Savings Behavior among the Poor? Evidence from a Randomized Financial Literacy Program in India.” Institute for Financial Management and Research, Centre for Micro Finance Working Paper, Chennai, India. Sonobe, Tetsushi, Ava Suzuki, Kijiro Otsuka, and Vu Hoang Nam. 2011. “KAIZEN for Managerial Skills Improvement in Small and Medium Enterprises: An Impact Evaluation Study in a knitwear cluster in Vietnam.” Development and Policies Research Center Working Paper Series No. 2012/ 29, Hanoi, Vietnam. Van Rooij, Maarten, Annamaria Lusardi, and Rob Alessie. 2011. “Financial Literacy and Stock Market Participation.” Journal of Financial Economics 101 : 449–72. World Bank. 2013. “Global Financial Development Report”. Washington, DC. Xu, Lisa, and Bilal Zia. 2012. “Financial Literacy around the World: An Overview of the Evidence with Practical Suggestions for the Way Forward.” World Bank Working Paper 6107, Washington, DC. 246 The World Bank Research Observer, vol. 30, no. 2 (August 2015) Downloaded from https://academic.oup.com/wbro/article-abstract/30/2/220/1655290/Can-You-Help-Someone-Become-Financially-Capable-A by Sectoral Library Rm MC-C3-220 user on 06 October 2017