June 2013 FINANCIAL LITE RACY EDUCATION RUSA TRS FN Pkr EVALUATION studies of programs to in low- and middle-income countries Evaluation studies of programs to enhance financial capability in low- and middle-income countries Summary FINANCIAL LITERACY EDUCATION r9wr WmI @ 2013 International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org The findings, interpretations, and conclusions expressed here do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because the World Bank encourages dissem- ination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: oubrights@worldbank.or. Cover photos: World Bank Cover design/layout: Nita Congress 件一江 一”織 皺’&’鰍一！一’ orldwide interest in empowering consumers to engage effectively with the proliferating number and complexity of financial products and services available has grown dramatically over recent decades, with initiatives quickly spreading from developed to developing countries. Although many countries have introduced national strategies for financial literacy and education, these are often not supported by robust impact and outcome evaluations. Despite growing interest in the topic, research on the results achieved by programs to enhance the financial capability of individuals remains limited. Understanding of the relative effec- tiveness of different approaches remains meager, and there is little concrete guidance for policy makers on how best to design and manage the many types of programs that are now in place. EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES E SUMMARY PAUCITY OF RIGOROUS EVALUATIONS IN THE FIELD Several factors have contributed to the limited base of evidence in this area: " In recent years, the debate around this topic has shifted from an initial focus on financial literacy defined as knowledge and numeracy to a broader defi- nition that includes skills, attitudes, and behavior.' In this broader approach, a financially capable person makes good decisions in the context of his or her individual circumstances and environment. This conceptual development has expanded the range of interventions beyond simply expanding knowl- edge through financial education to a variety of alternative methods, such as education through entertainment (sometimes called "edutainment"), social marketing, consumer protection-all more subtle efforts to alter behavior. Most evaluations, however, have addressed traditional financial education programs. m Good evaluations need to be integrated into program design to collect suffi- cient and appropriate data and to credibly determine causality and attribute outcomes. Such integration is an expensive and challenging process that is typically not uppermost in the minds of those designing and funding programs. Consequently, most financial literacy evaluations that have been under- taken have been retrospective, evaluating programs after they have been completed, which limits the potential scope and value of the analysis. a By their very nature, financial capability enhancement programs involve complex methodological challenges. They seek to address behavior that is the result of the interaction of a number of factors, and the outcomes of interest will typically play out over an extended period of time. This prolonga- tion makes it difficult to establish a frame of reference (or a counterfactual) against which to measure changes. These programs also involve delivery to specific groups with particular characteristics, making it difficult to separate the outcomes from these characteristics; or they entail wide dissemination of information or materials in a way that makes traditional research methods of establishing experimental and control groups very difficult. 1 Defining and measuring financial capability is a major element of the work performed under the Russia Financial Literacy and Education Trust Fund. E. Kempson, V. Perotti, and K. Scott, Measuring Financial Capability: A New Instrument and Results from Low- and Middle-Income Countries and Measuring Financial Capability: Questionnaires and implementation Guidance for Low- and Middle-Income Countries (Washington, DC: World Bank, 2013). (Washington, DC: World Bank, 2013). These publications and their summary booklet are available at www.finlitedu.org. EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY m Most of the evaluation of programs has been done in higher-income country settings, which is where the majority of financial education programs have been implemented in the past. The results achieved in these settings may not be applicable to the very different conditions existing in other countries where these programs are now becoming more prevalent. ADDRESSING THE CHALLENGE: A METHODOLOGICAL TOOLKIT AND A COORDINATED PROGRAM OF EVALUATION STUDIES To address these challenges, the World Bank's Russia Financial Literacy and Educa- tion Trust Fund has undertaken two coordinated initiatives: m Developing methodological and operational guidance in the form of a toolkit specifically targeted to financial capability programs in low- and middle-in- come countries; this toolkit is described in a detailed report and summarized in a companion booklet2 m Providing financial and methodological support for a carefully selected group of field evaluations for a wide range of programs in low- and middle-income countries; these were selected on the basis of their innovative nature and potential to develop and illustrate solutions to methodological challenges. This booklet highlights the latter initiative, briefly summarizing the studies under- taken by the World Bank, including the background for their selection and the lessons learned from their implementation. IDENTIFICATION OF GAPS AND SELECTION OF PROPOSALS To better understand the state of knowledge and to identify gaps in literature to guide the selection of program evaluations, a review of existing work was undertaken to identify underresearched areas. This effort concluded that programs directed toward budgeting and saving had received the most attention. Topics found to be underrepresented included financial capability related to credit, loans, investments, and use of insurance. The majority of existing evaluations were found to focus on education through workshops and seminars, classroom education, and counseling. 2 J. Yoong, K. Mihaly, S. Bauhoff, L. Rabinovich, and A. Hung, A Toolkit for the Evaluation of Finan- cial Capability Programs in Low- and Middle-Income Countries (Washington, DC: World Bank, 2013). This publication and its summary booklet are available at wwwfinlitedu.org. EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY Alternative means of delivery-such as the use of media, marketing, behavioral treat- ments, or mixed interventions-were found to be understudied. Most studies either used process evaluations or randomized controlled trials. Very few employed mixed methods or examined spillover effects. The review revealed a gap in conducting comparison studies that measure the impact of interventions across countries, contexts, and populations. Although a number of efforts to utilize choice architecture to influence behavior were under way-particularly in the field of commitment devices-there was little analysis of their effectiveness. Most studies treated approaches derived from the emerging field of behavioral economics as a distinct alternative to financial capability training. The gaps identified through this review became the priority areas in a competitive selection of 17 programs for which funding was provided to undertake evaluations; figure 1 shows the sites of the selected programs. In addition to financial support, the selected applicants received technical assistance in the design of their evalu- ations. Further, the project managers were given access to evaluation experts to develop their evaluation methodologies and to help plan the research. This arrange- ment offered expertise to applicants who were able to identify programs but did not have the technical capacity to develop research proposals, greatly expanding the scope of accessible programs. A number of experienced researchers provided highly innovative methods in their successful proposals as well. FIGURE 1 COUNTRY LOCATIONS OF PROGRAM EVALUATION STUDIES L DOMMUnc4 S. .. . .. . EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY The evaluations supported by the Trust Fund have produced a range of studies and reports that are being published as World Bank Policy Research Working Papers, in academic journals, and in other World Bank publications. As they are completed, all of this literature will be made available on the Trust Fund website (www.finlitedu.org). The studies can be organized into three groups: 1. Evaluations that address more traditional methods of financial education 2. Studies addressing more innovative methods of enhancing financial capability and influencing behavior 3. Studies that utilize a combination of methods. The studies included in the three groups are summarized in the following boxes; the impacts found across the studies are summarized in table 1. TABLE 1 SUMMARY OF IMPACT FROM THE RUSSIA FINANCIAL LITERACY AND EDUCATION TRUST FUND EVALUATIONS MIXED EVIDENCE EVIDENCE OF STUDY EVIDENCE OF OF LACK OF INCOMPLETE INTERVENTION EFFECTIVENESS EFFECTIVENESS EFFECTIVENESS (AS OF APRIL 2013) School-based/seminars and workshops Brazil (schools) X Australia and New Zealand (seminar) X India (seminar) X Mexico (seminar) X South Africa (seminar) X India (seminar, counseling, SMS reminders) X Entertainment media and marketing South Africa (soap operas) X Nigeria (feature films) X Kenya (comics) X Nigeria (marketing) X Brazil (online stock market simulator) x Other and mixed interventions Kenya (comics) X Uganda (financial diaries) x Mexico (information disclosure) X Dominican Republic (CCTs and seminars) X Malawi (labeled banking accounts) x I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES E SUMMARY BOX 1 EVALUATIONS OF TRADITIONAL FINANCIAL EDUCATION PROGRAMS FINANCIAL EDUCATION AND BEHAVIOR FORMATION: LARGE-SCALE EXPERIMENTAL EVIDENCE FROM BRAZIL * Pathways to change: Classroom financial education * Thematic focus: Budgeting, savings, and general financial management * Target groups: High school students * Evaluation method: Randomized controlled trial design * Data collection: Self-administered questionnaires * Results: Impact on improving knowledge, attitudes, and behavior The study assesses the impact of high school financial education in Brazil. It includes nearly 900 schools and 26,000 students. Administration of the program through schools allowed for a broad coverage of content in the curriculum. To control for quality of content, the educational material was developed by experts. Separate training was provided to a group of parents of the students to examine whether inside-the-household interactions influenced behavior. Results found that the program increased student financial knowledge by a quarter of a standard deviation, which led to a 1.4 percentage point increase in savings-a relatively large and economically relevant effect. A complementary workshop for parents induced children to save even more. Both current attitudes and forward-looking intentions to save improved. THE IMPACT OF FINANCIAL LITERACY TRAINING FOR MIGRANTS: EVIDENCE FROM AUSTRALIA AND NEW ZEALAND * Pathways to change: Group-based financial literacy seminar * Thematic focus: Remittances, credit and financial product selection * Target groups: Migrant workers * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys * Results: Impact on increasing knowledge and information-seeking behavior and reducing the risk of switching to costlier remittance products. No impact on changing frequency or levels of remittances. The study examined the impact of financial education targeting migrant workers and their remitting behavior in Australia and New Zealand. Training consisted of a two-hour session on reasons to remit, strategies for comparing costs, and information about different remittance products. Results show that the training led to increases in financial knowledge; migrants were more likely to know it is cheaper to send one large transfer than individual smaller ones, and more likely to know cheaper methods of remitting. The study also found that migrants changed behavior in response to knowledge gained; however, training was not found to change frequency of remitting, amount remitted, or the take-up of products. (continued) I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY BOX 1 EVALUATIONS OF TRADITIONAL FINANCIAL EDUCATION PROGRAMS (continued) DOES FINANCIAL EDUCATION AFFECT SAVINGS BEHAVIOR? EVIDENCE FROM A RANDOMIZED EXPERIMENT AMONG LOW-INCOME CLIENTS OF BRANCHLESS BANKING IN INDIA * Pathways to change: Classroom training seminar * Thematic focus: Budgeting, savings, and general financial management * Target groups: Low-income households * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys * Results: Impact on savings and attitudes related to financial management. Financial literacy levels did not improve. The study measured the impact of a classroom financial literacy training on the uptake of branchless banking and on savings behavior. The intervention consisted of a two-day training that covered the role of formal banking in people's lives; and responsible borrowing, spending, saving, and cash management. The experiment was conducted on a random sample of 3,000 clients of branchless banking across two adjacent districts in the state of Uttar Pradesh. The results reveal that the intervention had impact on savings and that attitudes related to financial management improved, but overall financial literacy did not. This suggests that a causal chain reaction from knowledge to behavior might not necessarily be required in such order to achieve desired outcomes. MEASURING THE IMPACT OF FINANCIAL LITERACY ON SAVINGS AND RESPONSIBLE CREDIT CARD USE: EVIDENCE FROM A RANDOMIZED EXPERIMENT IN MEXICO * Pathways to change: Classroom training seminar * Thematic focus: Saving, retirement, use of credit * Target groups: Bank credit card customers * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys * Results: Impact on improving knowledge and savings rates, but no impact on credit card usage The study tested the impact of financial literacy training on savings and borrowing behavior and credit card usage patterns of credit card customers in Mexico City. It involved approximately 40,000 bank consumers. The training course lasted for about four hours and consisted of four modules on savings, retirement, credit cards, and responsible use of credit. Results show a 9 percentage point increase in financial knowledge, and a 9 percentage point increase in saving outcomes, but no impact on credit card behavior, retirement savings, or borrowing. Moreover, administrative data suggest that the savings impact is relatively short-lived. The results point to the limits of using general-purpose workshops to improve financial literacy and decision-making patterns for the general population. (continued) EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES E SUMMARY BOX 1 EVALUATIONS OF TRADITIONAL FINANCIAL EDUCATION PROGRAMS (continued) THE IMPACT OF FINANCIAL EDUCATION ON FINANCIAL KNOWLEDGE, BEHAVIOR, AND OUTCOMES: EVIDENCE FROM A RANDOMIZED EXPERIMENT IN SOUTH AFRICA * Pathways to change: Group-based interactive financial literacy seminar * Thematic focus: Remittances, credit, and financial product selection * Target groups: Members of burial societies * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys and self-administered questionnaires * Results: The study is still ongoing. Based on early observations, the intervention was found to improve budgeting and savings, reduce gambling, and decrease risk aversion. The study examines the impact of one-day financial education training on savings, financial planning, budgeting, and debt management. The purpose is to encourage individuals to seek more efficient financial vehicles, as well as to save and use credit wisely. The target population consists of members of burial societies and women's development groups in the Eastern Cape area of South Africa. It uses a randomized experiment involving approximately 1,300 individuals. Outcome measures are obtained from a variety of sources, including individual surveys and administrative data. While the study is still ongoing, preliminary results show that the intervention had impact on budgeting skills and savings behavior, as well as on reducing gambling and decreasing risk aversion. THE ROLE OF FINANCIAL ACCESS, KNOWLEDGE, AND SERVICE DELIVERY IN SAVINGS BEHAVIOR: EVIDENCE FROM A RANDOMIZED EXPERIMENT IN INDIA * Pathways to change: Classroom seminar, counseling, and reminders through phones * Thematic focus: Savings, budgeting, and selection of financial products * Target groups: Low-income households * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys * Results: No results reported at this stage. The intervention is ongoing. The study measures the impact of a mixed and multilayered intervention on savings behavior. The program consists of three treatments: a classroom financial education training, followed by reminders through mobile SMS and voice messages over a period of several months, and personal counseling through physical visits to participants' homes. It aims to measure the overall impact but also to disentangle the separate effects from the individual treatments. The novelty of this study is that it experiments with combining traditional financial education with behavioral treatments and explores a multilayered intervention over a longer period of time as opposed to a single and one-time treatment. I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY BOX 2 EVALUATIONS OF INNOVATIVE PROGRAMS AND METHODS HARNESSING EMOTIONAL CONNECTIONS TO IMPROVE FINANCIAL DECISIONS: EVALUATING THE IMPACT OF FINANCIAL EDUCATION IN MAINSTREAM MEDIA IN SOUTH AFRICA * Pathways to change: Entertainment media (television soap opera) * Thematic focus: Debt management * Target groups: Low-income households with and without existing consumer debt * Evaluation method: Randomized encouragement design * Data collection: Phone and face-to-face surveys and qualitative focus groups * Results: Improvement of knowledge and borrowing behavior The study investigates whether debt management may be improved through TV soap operas. To control for content quality, the project develops a soap opera storyline through focus groups. It involves around 1,000 randomly selected individuals divided into treatment and control groups. The treatment watches a soap opera with financial literacy messages, called Scandal!; the control watches a different show aired at the same time but with no financial literacy messages. Financial incentives are provided to ensure compliance. Results report that individuals assigned to watch Scandal! had higher financial knowledge on issues highlighted in the storyline. Scandal! viewers were more likely to borrow from formal sources, less likely to engage in gambling, and less prone to enter hire purchase agreements. LEARNING BY DOING? USING SAVINGS LOTTERIES AND SOCIAL MARKETING TO PROMOTE FINANCIAL INCLUSION: EVIDENCE FROM AN EXPERIMENT IN NIGERIA * Pathways to change: National marketing campaign and savings lottery * Thematic focus: Savings and use of banking services * Target groups: Existing and new bank users * Evaluation method: Regression; discontinuity design * Data collection: Microdata collected from the banks on daily account balances * Results: Increased savings and use of additional financial products within a week of intervention. However, no evidence on persistent changes after incentives removed. The study measures the impact of a national marketing campaign launched by one of the largest banks in Nigeria to encourage savings. It entails lottery prizes for individuals who open a savings account and maintain a threshold amount in the account for 90 days during a period of three months. The lottery is publicized through advertisements with celebrity endorsements and media releases. The research assesses the extent to which the different components of the campaign affect the take-up rate. It also measures the impact of "learning-by-doing"-the extent to which interactions with banking motivate people to continue to save. Results report that during the intervention there was improvement on savings behavior and on the usage of the bank's other financial products. However, there was no evidence that the incentive program led to persistent changes after explicit incentives were removed. (continued) EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY BOX 2 EVALUATIONS OF INNOVATIVE PROGRAMS AND METHODS (continued) THE IMPACT OF CARTOONS AND COMICS ON THE EFFECTIVENESS OF FINANCIAL EDUCATION: EVIDENCE FROM A RANDOMIZED EXPERIMENT IN KENYA * Pathways to change: Comic books * Thematic focus: Financial management and saving * Target groups: Schoolchildren * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys * Results: No impact on literacy levels and on savings behavior. However, it found impact on the likelihood that students want to start a business in the future. The study tests the absolute and relative impact of different program delivery mechanisms on the financial capability and behavior of Kenyan youth. It compares delivering education though classroom with that through comic books and radio shows. It uses a sample of 220 high schools, randomly assigned to two main treatment groups, a placebo group and a control. One novelty of this study is that in addition to completing both baseline and endline surveys, students are also asked to make financial decisions using real resources. This allows recording how actions differ from stated intentions, and how both stated intentions and actions change over time. Results show little evidence that the interventions improved financial literacy. Similarly they find no effect on stated and actual savings behavior. However, the study finds impacts on the likelihood that students want to start a business in the future. THE IMPACT OF FINANCIAL LITERACY THROUGH FEATURE FILMS: EVIDENCE FROM A RANDOMIZED EXPERIMENT IN NIGERIA * Pathways to change: Entertainment media (feature film) * Thematic focus: Savings and credit * Target groups: Low-income households and small business owners * Evaluation method: Randomized controlled trial design * Data collection: Surveys and self-administered questionnaires * Results: The study is still ongoing. Early observations report on the impact on perceptions and trust in microfinance institutions and increasing the take-up of savings accounts in the short run. Limited evidence of an impact on longer-term behavioral change. The study involves a sample of 3,000 individuals in Nigeria to assess the extent to which a feature film can promote responsible borrowing and improve savings. A basic premise on which the study is developed is that emotions have an influence on actions, and while the emotional state might be transient and short-lived, the decisions reached under the emotional state could potentially provide the basis for future actions. To capture immediate actions, the experiment includes the presence of microfinance institutions at the movie screening venues. Furthermore, to control for spillover effects-that is, the extent to which individuals in the treatment and control groups might talk to each other and share information-the experiment adopts a mix of individual and cluster randomization. (continued) I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY BOX 2 EVALUATIONS OF INNOVATIVE PROGRAMS AND METHODS (continued) THE IMPACT OF FINANCIAL EDUCATION AND LEARNING-BY-DOING ON HOUSEHOLD INVESTMENT BEHAVIOR: EVIDENCE FROM BRAZIL * Pathways to change: Online stock market simulator * Thematic focus: Stock market participation and investments * Target groups: Stock market simulator users and participants * Evaluation method: Randomized controlled trial design and regression methods * Data collection: Stock market simulator data and individual stock market data * Results: No results reported at this stage. The intervention is ongoing. The study measures the impact of financial education on stock market participation and investment in Brazil. It involves an online stock market simulator that serves as a platform through which financial education is passed to participants and as a mechanism through which participants develop practical experience. A combination of education, warning, and reminders about good practices and learning-by-doing is expected to improve decisions over time. The study uses data of approximately 600,000 investors. Understanding why people make the investments they do can help identify interventions to improve consumer protection and support development of capital markets. I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- BOX 3 EVALUATIONS OF COMBINED PROGRAMS AND METHODS SOCIAL NETWORKS, FINANCIAL LITERACY, AND INDEX INSURANCE: EVIDENCE FROM A RANDOMIZED EXPERIMENT IN KENYA * Pathways to change: Comic books * Thematic focus: Long-term planning; index-based weather insurance * Target groups: Rural, small-scale farmers * Evaluation method: Randomized controlled trial design and regression methods * Data collection: Face-to-face and phone surveys * Results: Impact on encouraging the take-up of index-based drought insurance The study examines the role of financial literacy on farmers' decisions to purchase index-based weather insurance in Kenya. It also examines social network spillovers of financial literacy provision. It uses a randomized approach covering around 14 villages. The program uses comic books as a delivery mechanism of financial education; the comic details the index-insurance product and how it can help families protect themselves from the risk of drought. The study also compares the effect of financial literacy materials and discount vouchers in influencing farmers' purchase decisions. Results provide evidence that financial literacy materials are effective in encouraging the take-up rates. UNDERSTANDING FINANCIAL CAPABILITY THROUGH FINANCIAL DIARIES AND IN-DEPTH INTERVIEWS IN UGANDA * Pathways to change: Classroom training sessions * Thematic focus: General financial education topics * Target groups: Low-income households * Evaluation method: Qualitative (financial diaries) * Data collection: Face-to-face surveys * Results: Qualitative findings suggest changes in knowledge, skills, and attitudes; however, they also indicate that they do not always translate directly into behavior change, at least not within a short time frame. The study uses financial diaries in combination with in-depth interviews in Uganda to understand and measure the financial capabilities of low-income households. It compares changes in knowledge, skills, attitudes, and behaviors of respondents in the treatment and comparison groups, highlighting situations where the former underwent a change that might be the result of the impact of the financial education. Results suggest that financial education affects knowledge, skills, and attitudes. Nevertheless, they also indicate that they do not always translate into behavioral change and affect decision making, at least not within a short time frame. However, there is some suggestion of change in savings behavior in terms of saving at home. (continued) EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY BOX 3 EVALUATIONS OF COMBINED PROGRAMS AND METHODS (continued) EVALUATING THE EFFECTIVENESS OF LOAN DISCLOSURE REFORMS ON CONSUMER UNDERSTANDING AND FINANCIAL DECISION MAKING: EVIDENCE FROM MEXICO * Pathways to change: Product disclosure formats, and mobile SMS and telephone counseling * Thematic focus: Savings and credit * Target groups: Low-income consumers of credit * Evaluation method: Randomized controlled trial design * Data collection: Surveys and credit reports * Results: Preliminary results suggest that disclosure and transparency improve the ability of consumers to select the best product out of several options and to identify the least expensive of several credit products offered. The study measured the impact of product-specific information disclosure on financial decisions. It assumes that the more transparent and relevant the information, the better consumer decisions will be with regard to product selection. It involved development and testing of a series of alternate product-specific disclosure formats, which were then used by low-income consumers in Mexico to choose between a series of credit or savings products. The testing of formats was complemented with financial education information delivered prior to the exercise to some participants, either via SMS messages or by phone consultation. Preliminary results suggest that disclosure and transparency improve the ability of consumers to select the best product out of several options and the ability to identify the least expensive of several credit products offered. These findings point to potential benefits of focusing on product-specific information disclosure and consumer education. INCREASING THE IMPACT OF CONDITIONAL CASH TRANSFER PROGRAMS THROUGH FINANCIAL LITERACY IN THE DOMINICAN REPUBLIC * Pathways to change: Professional and peer trainings * Thematic focus: Household and business financial management * Target groups: Conditional cash transfer beneficiaries * Evaluation method: Randomized controlled trial design * Data collection: Surveys * Results: No results reported at this stage. The intervention is ongoing. The study assesses whether conditional cash transfer (CCT) programs can be leveraged to deliver financial education and affect both knowledge and behavior. Evaluations of CCT programs have shown that they can be successful in increasing usage of health care and education services. This project explores the extent to which CCTs can improve financial capabilities. Working with around 60 beneficiaries of the Solidaridad cash transfer program in the Dominican Republic, the study randomly selects one group of beneficiaries to participate in the training and another group that does not to serve as a control. The experiment is further divided into subtreatments to test whether training delivered by professionals versus peers has different effects. It also measures the relative impact of business training versus soft job skills training on decreasing unemployment among beneficiaries. (continued) EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES E SUMMARY BOX 3 EVALUATIONS OF COMBINED PROGRAMS AND METHODS (continued) FINANCIAL DEVELOPMENT AND THE PSYCHOLOGY OF SAVINGS: EVIDENCE FROM A RANDOMIZED STUDY IN MALAWI * Pathways to change: One-on-one training; labeled banking accounts * Thematic focus: Household and business financial management * Target groups: Low-income agricultural wage earners and smallholder farmers * Evaluation method: Randomized controlled trial design * Data collection: Surveys * Results: No results reported at this stage. The intervention is ongoing. The study investigates innovative ways to address low levels of formal savings by leveraging psychological mechanisms. The target population consists of low-income agricultural wage earners and smallholder farmers in Malawi. It examines whether direct deposit of wages as opposed to cash payments can help individuals match desired savings and expenditure patterns with actual behavior. It also tests whether labeling particular bank savings accounts with particular expenditures and labels (for example, college fund, car purchase, etc.) reinforces commitments to save. THE IMPACT AND NETWORK EFFECTS OF FINANCIAL MANAGEMENT AND VOCATIONAL TRAINING IN INFORMAL INDUSTRIAL CLUSTERS: EVIDENCE FROM UGANDA * Pathways to change: Practice-based vocational training and classroom-based business training * Thematic focus: Vocational and business training, and business network effects * Target groups: Small-scale industries and business owners * Evaluation method: Randomized controlled trial design * Data collection: Face-to-face surveys * Results: Very preliminary results suggest some short-term effects of training on financial literacy and technical knowledge, optimism, and adherence to technical standards, but not on core business outcomes. The intervention is ongoing. The study explores the extent to which personal financial choices are affected by peers. It consists of a randomized evaluation in Uganda to identify the impact of a comprehensive financial management and vocational training program for small-scale industries, focusing on network effects. The study identifies business networks and examines whether the enhanced knowledge received through the training program spreads to other businesses and across networks, influencing certain behavior among the untreated population. Potential positive spillovers would constitute efficient ways to scale the impact of trainings and provide a natural source of leverage for these programs. EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY LESSONS Results from the completed and ongoing research projects provide some useful lessons that can guide future work. The following are especially important: 1. Financial capability may be achieved through different interventions; programs that utilize mass media and social marketing tools promise to be especially effective. Results suggest that financial capability programs, whether delivered through schools and workshops or through more innovative methods such as mass media and marketing, can be effective in changing both knowledge and behavior. Alternative interventions, such as using television soap operas, films, and promotion campaigns, prove to be especially effective. For these interventions, two features are presumed to affect the outcome. First, they can be more entertaining and have the capacity to transmit messages through appealing stories that stick in the memory; and second, they can serve as mech- anisms through which messages can be repeated and reinforced over time, keeping the audience engaged in treatment for longer than traditional financial education mechanisms do (e.g., soap operas can last for months or years). 2. Quality of content delivered affects outcomes. Results suggest that finan- cial capability programs work better when the content is relevant, targeted at the right audience, and delivered at teachable moments. While this might come across as an obvious point, most financial capability programs developed by both program providers and researchers do not tend to focus much on devel- oping the financial education content. The stocktaking exercise conducted found that most of the programs in the field use similar material or rely on a generic curriculum that addresses key concepts on financial management without tailoring the subject to the targeted audience or the specific setting. 3. Financial behavior improves even in the absence of improvements in financial literacy. Especially important is the finding that programs can be effective in changing individual decision making even in cases where the finan- cial literacy (i.e., knowledge) of the individuals does not improve. This finding thus underscores the importance of paying more attention to the broad range of social and psychological factors that-along with knowledge-influence financial behavior. If the objective of a program is to improve savings rates, for example, providing financial education might not be necessary-and in certain cases might not even be cost-effective. Interventions that employ media and marketing tools could improve savings rates without following a cognitive route. This does not suggest that financial education should not be promoted, but rather that interventions other than traditional financial education are avail- able to policy makers to help people make better financial choices. I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES a SUMMARY 4. Impacts from one-time interventions fade over time. Repetition of treat- ment is to sustain behavioral change. The results suggest that improvement of financial capability among the poor is extremely difficult, especially when the improvement sought is related to longer-term behavior change and habit formation. The results from these evaluations and other studies suggest that while interventions may be successful in influencing immediate postinterven- tion behavior, the effects tend to fade over time. This observation is in line with research in other areas that focuses on mechanisms to change deci- sion making-for example, in health (promoting a healthy lifestyle) or energy (changing consumption patterns). This does not necessarily indicate that finan- cial capability interventions are not effective, but rather that treatments may need repetition over time to maintain their effect. GOING FORWARD The Russia Financial Literacy and Education Trust Fund supported a wide range of evaluations that yield information of value to policy makers and program managers with respect to the design, content, and effective delivery of financial capability enhancement programs undertaken in low- and middle-income countries. However, this initial effort raises as many questions as it answers. Going forward, it suggests the following areas for the focus of future endeavors: " Explore interventions other than financial education. The limitation of traditional financial education in changing consumer financial behavior is well documented by research and is confirmed by the results from the studies. Even if such efforts work, they typically have high treatment costs per indi- vidual. In contrast to traditional and classroom-based financial education, interventions that utilize mass media (television programs, radio, commercials, etc.), social marketing techniques (promotion campaigns, etc.), and behavioral treatments (reminders, choice framing, peer pressure, etc.) show promise in being effective in influencing consumer choices, have low costs per individual, and can be rapidly scaled up. A main condition is high technology penetration, which is currently met in most environments. Future research needs to further explore the scope and limitations of these interventions to better understand what specific mechanisms work best and in which settings. " Explore the difference between financial literacy and financial capa- bility and conduct comparative research to measure the relative impact on consumer decision making. These two terms, while to an extent used interchangeably in the literature in recent years, mean different things and may have different objectives and outcomes. Achieving financial literacy I EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES I SUMMARY means improving knowledge that may improve decisions for some financial outcomes-for example, budgeting or specific investments. Achieving finan- cial capability means improving financial decisions for the same or, perhaps, other outcomes-for example, increasing the savings rate or seeking financial advice. This conjecture emerges strongly from both the financial measurement work as well as the program evaluations conducted under the Trust Fund, but the differences and overlap with regard to policy interventions (e.g., between classroom-based education and mobile phone reminders) and their effective- ness are little known. " Further validate the results of the evaluation of the Brazil program of financial education in the schools. The Brazil study indicates that a school- based financial education program, if very well done, works both to increase financial literacy as well as financial outcomes (saving) and offers the promise of long-term effects. The results of the study suggest that such an outcome requires (1) a relevant high-quality material/textbook developed by experts; (2) a well-trained and highly motivated staff (through incentives); (3) multiple treatments over a longer period (three consecutive academic semesters); and, importantly, (4) the involvement of parents. Yet even with these elements, it is not yet clear if the observed effects will be sustainable. It is therefore important to replicate and document the approach and results in different countries to gain external validity-or at least to understand under what conditions such results can be expected. " Explore the importance of the intensity or level of service delivered. There is little evidence on how much education and information is optimal and how often it should be reinforced. Studies should be developed to look at optimal dosages for different interventions and examine timing and repetition effects-that is, whether impacts diminish as the time spent in classes or exposure to messages declines and how long effects last. " Focus on maintenance of long-term behavior change. Related to the point above, the results suggest that most interventions that find the desired results usually refer to short-term impact. Individuals generally tend to regress to former behavior (pre-intervention). It is not enough for researchers to identify interventions that succeed in controlled trials. They must also show that the impact detected can continue over time; otherwise an intervention cannot be deemed successful and suggested for replication. Future research should focus on the maintenance stage: ways (or additional treatments needed over time) to help individuals maintain behavior in the longer term. F EVALUATION STUDIES OF PROGRAMS TO ENHANCE FINANCIAL CAPABILITY IN LOW- AND MIDDLE-INCOME COUNTRIES a SUMMARY m Good policy design often requires behavioral treatments to address limitations and biases of individuals. The recent revolution in behavioral economics research has provided a range of empirical evidence that contra- dicts the classical rational optimization model of behavior. This research suggests that people often behave in contrast to their best interest and that they are very sensitive to the way choices are framed. This finding is especially relevant in the field of decision making and personal finance. Future research should explore this area further and examine how behavioral models can inform personal financial decision making. Specifically, it should examine what types of behavioral biases affect choices, in what contexts, and how to collect data around behavioral characteristics. Furthermore, it should examine the interaction between individual behavioral characteristics and external environ- mental factors that might influence behavior. m Integrate cost-effectiveness analysis in research design to inform scale. Research should address both the impact of an intervention on desired outcomes and also the cost of developing and delivering the intervention. Such research is especially important for lower-income settings, because if a program is deemed effective but expensive, countries might not have the means to deliver at scale. Therefore, as the second-generation research in this area advances, it is crucial that studies report on cost-effectiveness of outcomes. This research should be presented in terms of the cost of desired outcome per individual or similar measures. Studies should also focus on cost comparisons between alternative interventions, such as the cost of traditional financial education programs versus alternative methods.     The Russia Financial Literacy and Education Trust Fund was established in 2008 at the World Bank with funding provided by the Ministry of Finance of the Russian Federation. The work supported by the Trust Fund is jointly managed by the World Bank and the Organisation for Economic Co-operation and Development (OECD) and is directed toward improving public pol- icies and programs to enhance financial knowledge and capabilities in low- and middle-income countries. This effort has focused on the review of national strategies for financial education, the development of methods for the measurement of financial knowledge and capabilities, methods for evaluating the impact and outcome of programs, and research applying these methods to programs in developing countries. The products of this program of work can be found at the Trust Fund website at: www.finlitedu.org THE WORLD BANK J."' USAEIEDERAO ECD QP EC