SALMAN ALIBHAI NIKLAS BUEHREN RACHEL COLEMAN MARKUS GOLDSTEIN FRANCESCO STROBBE Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia Innovation has been and will be pivotal for reaching sustained, scalable solutions to the world’s most complex problems.1 Amhara Credit and Savings Institute’s (ACSI) product has a high potential for serving more women but its reach is limited by a lack of appropriate appraisal tools, limited skills and that the women borrowers are constrained by the traditional collateral, such as houses, which ACSI requires.2 All around the world, women-owned businesses have less access to credit than their male counterparts. This unequal access is largely driven by inequalities in ownership of fixed assets such as houses and land, which can serve as collateral to secure loans. Without needed credit, women-owned enterprises are starved of opportunities to grow. In Ethiopia, and in emerging markets around the world, this collateral constraint is amplified by the absence of financial sector infrastructure such as credit information systems that can help lenders identify creditworthy borrowers. Faced with a dearth of information, financial institutions impose unduly large collateral requirements in order to minimize their exposure and risk. Five years ago, it wasn’t uncommon for Ethiopian banks to require collateral worth three times the value of a loan, limiting credit access to only the wealthiest borrowers. Limitations to the reach of financial services are not endemic to Ethiopia, or to emerging markets. The tide of financial tech- nology, or ‘fintech’, is sweeping across the largest banks around the globe, leading them to adapt business models, adopt new tools, demonstrate agility, and ultimately enhance their customer offerings. But beyond making banks more efficient and profitable, fintech also holds the potential to open new horizons for financial inclusion, enabling lenders to appraise and reach borrowers traditionally excluded or underserved by the banking system. In 2014, a World Bank team set out to explore how developments in the fintech industry could be harnessed to unlock the collateral challenge facing Ethiopia’s women entrepreneurs. They looked to a technology at the frontiers of financial inclusion: psychometric credit scoring. A tech start-up in Cambridge, Massachusetts had developed a short interactive assessment 1 “A Call for Innovation in International Development”, pubdocs.worldbank.org/en/851851446842304852/Call-for-Innovation-in-International-Development.pdf. 2 “Ethiopia Women Development Project Microfinance Institutions Assessment Report”, Business & Finance Consulting (BFC), 2011 This work has been funded in part by the government of Canada. 1 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 that could be taken by potential borrowers on a tablet computer and could predict the likelihood that the borrower would repay a loan by drawing on the latest advances in big data and machine learning. If psychometric credit scoring worked, the team surmised, perhaps it could be an antidote to the information asymmetry faced by Ethiopian financial institutions, and could ease collateral requirements for creditworthy women entrepreneurs. The following case study tells the story of the evolution of psychometric credit scoring as an innovative solution in a World Bank operation, from its humble beginnings as a small pilot in Ethiopia, to the current movement to replicate its use for similar challenges in countries across the continent – in Tanzania, Zimbabwe, Madagascar, and beyond. The story is one of both achievements and setbacks, just as the future of fintech holds both promise and limitations. It is shared with a view to better understand how psychometrics and fintech more broadly can be utilized to solve critical development challenges, and help get finance to those who need it most around the world. A Changing Industry – the Rise of Fintech Fintech is commonly defined as an industry composed of companies that use technology to make financial systems more efficient.3 Fintech industry players have taken the financial sector by storm in recent years. They cover a wide range of sub-industries such as peer-to-peer lending, credit scoring, payments, crowdfunding and digital currencies. Investment in financial technology has grown exponentially in the past decade - rising from $1.8 billion in 2010 to $19 billion in 2015.4 De- spite different innovative business models, these companies share a common objective: to create and implement technology that improves the efficiency of financial markets.5 Fintech companies are increasingly supporting the growth of financial inclusion by reimagining traditional banking products and procedures. They are doing this by bringing new and innovative approaches to better serve clients with reliable, relevant and affordable financial solutions that work for underserved communities.6 In this way, financial technology is allowing cus- tomers to overcome traditional barriers to accessing finance. EXHIBIT 1 Get a Full Picture and Better Serve Clients by Leveraging Big Data utility payments point of sale mobile transaction devices web email browser CREDIT SCORE 842 social credit networks bureau 3 Williams, Christophe. “What Is Fintech?”, Wharton Fintech, 16 Feb. 2016, www.whartonfintech.org/blog-archive/2016/2/16/what-is-fintech. 4 “Digital Disruption - How FinTech Is Forcing Banking to a Tipping Point”, CITI GPS: Global Perspectives and Solutions, 2016, www.nist.gov/sites/default/files/ documents/2016/09/15/citi_rfi_response.pdf. 5 Williams, Christophe. “What Is Fintech?” 6 Chaia, Alberto, et al. “Counting the World’s Unbanked”, McKinsey Quarterly, Mar. 2010, www.mckinsey.com/industries/financial-services/our-insights/counting-the-worlds-unbanked. 2 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 Credit assessment is an area where fintech is poised to make particularly large gains for financial inclusion. Using fintech, financial institutions can access previously untapped data that reveals insights into customers and markets.7 This alternative data is derived from sources such as mobile phones, social media, web browsers, utility payments, and point-of-sale trans- action devices (See Exhibit 1). Using this data, fintechs can better understand a borrower’s cashflows, character traits, and networks. Lenders can use this information to better calculate risk for current clients and to expand their reach to new and previously unbanked borrowers. Integrating technology into financial institutions heralds the opportunity to dramatically advance financial inclusion. Two billion people around the world remain unbanked,8 and fintech promises the potential of more convenient, less expensive, and high- er quality financial services. But technology in itself is not a panacea for financial inclusion, and implementing technological change in a financial institution, as we will see from the following case, can be a complex and oftentimes non-linear process. The Idea In early 2014, a multidisciplinary World Bank team working on banking, entrepreneurship, and gender in Ethiopia be- gan its first forays into the world of fintech. The team’s objective was to identify a technology that could address the collateral constraint facing women entrepreneurs in Ethiopia. The team understood that information scarcity was the key issue preventing banks from lending to women, and that addressing this scarcity would be critical to finding a sustainable solution. At the time, a myriad of new technologies were emerging to help banks address information asymmetries and scarcities. Harnessing alternative data, for example, could enable lenders to make reliable predictions about the creditworthiness of potential borrowers. Despite their potential, however, many of these technologies were better suited to developed markets EXHIBIT 2 Predictive Power and Availability Vary By Data Source Psychometric is the only data source with 100% availability - everyone has a personality More Available 100% PSYCHOMETRIC DATA 90% (Hit rate on sample of low-income applicants in emerging markets) 80% MOBILE DATA 70% 60% Availability 50% BUREAU DATA 40% 30% Less Available ONLINE DATA 20% 10% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Less Predictive More Predictive Predictive Power (Out of sample gini coefficient with comparable model & sample) Source: https://www.eflglobal.com/alternative-credit-scoring-emerging-markets/; https://data.worldbank.org/indicator/IC.CRD.PRVT.ZA; EFL Analysis 7 “Innovations in Payments: The Future Is Fintech”, BNY Mellon, 2015, www.bnymellon.com/_global-assets/pdf/our-thinking/innovation-in-payments-the-future-is-fintech.pdf. 8 Demirguc-Kunt, Asli, et al. “The Global Findex Database 2014 - Measuring Financial Inclusion around the World ”, World Bank Policy Research Working Paper 7255, documents. worldbank.org/curated/en/187761468179367706/pdf/WPS7255.pdf#page=3 3 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 in which the majority of citizens had a digital footprint. These tech- EXHIBIT 3 nologies were less viable in a market like Ethiopia, where only 16% of the population uses the internet and 51 out of 100 people have mobile phone subscriptions.9 For this reason, psychometrics – lit- erally ‘measurement of the mind’ - emerged as a promising option WEDP: Women Entrepreneurship Development Project for creating a better picture of Ethiopian borrowers. Unlike other fintech data solutions, psychometrics could actually create new What We Do Provide technical assistance data on borrowers, where it didn’t exist before. to MFIs to enable them to $45.9M ABOUT US disburse a credit line of: Our Goal Psychometric data held great potential (See Exhibit 2). While it is Where We Work6 Cities in four regions of Ethiopia Increase the earnings and employment of widely known that a borrowers’ character relates to their likelihood 12 women-owned MFIs, Pre-selected by Our Partners enterprises in Ethiopia, the World Bank. through access to to repay a loan, financial institutions had found it challenging to finance and business 6647 development services quantify these traits. Years of research and development have Results as of July, 2017 Clients. In the third year of lending… proven that someone’s personality could be broken down into 10,573 Entrepreneurs Trained: 80% of RESULTS measurable traits such as locus of control, fluid intelligence, im- participants rated the training as excel- lent, 19% as good, and 1% as average. pulsiveness, confidence, delayed gratification and conscientious- 3 e Average jobs projected nc Jobs projected ness, and that these traits could be used to predict credit risk.10 by WEDP borrowers : ina F 61% ss First-time borrowers as ce a portion of the portfolio: Through a connection with a researcher at Harvard Business Ac 237,000 ETB School, the World Bank team was introduced to the Entrepreneur- Average WEDP loan size. Pre-WEDP average loan size was 27,000 Bir ial Finance Lab (EFL)11 in early 2014 (now Lenddo after a corporate An increase of 870% merger in October 2017)12. EFL had worked with over 50 finan- Loans > ETB 1M Sustainability MFIs are using their own funds to disburse WEDP loans cial institutions in 20 countries as a provider of alternative credit Targeted by banks 915 Million ETB scores. Their value proposition was a universal credit score that Missing Middle 469 was calculated using psychometric and behavioral data. They had Million ETB Targeted by MFIs built predictive psychometric credit models around the world in- Loans < 100k WEDP loans (WB line of credit) WEDP loans (MFs own funds) cluding in Peru, Mexico, Guatemala, Ecuador, India, Indonesia, 46% of WEDP loans were disbursed using non-traditional forms of collateral, such as vehicles, personal guarantees, post-dated cheques, and psychometric scores Kenya, and Ghana and across different population segments. WEDP Coordinates With From their first conversations with EFL, the World Bank team was Italy (AICS) UK (DFID) Japan (JICA) World Bank Canada (GAC) impressed. EFL’s track record was strong and their work in African MoFEC countries was unique, as many fintech companies were only just DBE FUJCFSA starting to dip their toes in African markets. They had designed a DAI/PEPE/ WEDP MFIs psychometric credit assessment which could be taken on a tablet For more information, contact or a mobile phone: a powerful solution that made sense in a data Francesco Strobbe: fstrobbe@worldbank.org, Salman Alibhai: aalibhai@worldbank.org scarce environment where the level of digitalization of loan files, mo- bile phone usage and internet penetration is low. Generating new data on an entrepreneur’s character that could predict their creditworthiness could be a breakthrough for Ethiopian lending. While the team saw promise in EFL’s tool, the challenge ahead was a bold one. The objective was not simply to introduce psychometric screening, but to use it as a tool that could replace traditional asset collateral for high-potential women entre- preneurs. And the low-tech context in Ethiopia was one unlike any other that EFL had worked in before. If this worked and a locally calibrated statistical model could be developed for predicting loan repayment, the Ethiopian financial sector would be equipped with a powerful tool for expanding access to credit for hundreds of thousands of collateral constrained borrowers. Though the challenges of introducing a new technology into a low-tech environment were real, it soon became clear the opportunities of piloting the psychometric credit assessment to address the collateral constraint far outweighed the risks. In late 2014, a pilot project was launched. 9 International Telecommunication Union, World Telecommunication/ICT Development Report database, data as of 2016 10 These characteristics have been statistically correlated to loan performance in a global database of over 75,000 applications to create a robust credit risk evaluation. 11 EFL [or “The psychometric credit scoring company used”] provides credit scores for SMEs in emerging markets based on psychometric questions. Their core business has produced compelling results – see Enterprising Psychometrics and Poverty Reduction by Bailey Klinger, Asim Ijaz Khwaja, Carlos del Carpio – and this project was an R&D attempt organized by the World Bank to apply the same techniques in the micro-credit space, where there are considerable differences in income, literacy, and bank processes among other potentially contributing factors. 12 EFL merged with Lenddo in October 2017. Learn more about the merger and the combined company at Include1Billion.com 4 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 The Situation in Ethiopia EXHIBIT 4 Association of Ethiopian Over the past decade, Ethiopia has achieved high economic Microfinance Institutions (AEMFI) growth, establishing the country among the fastest growing econ- Data on ACSI MFI omies both in Africa and the developing world. However, Ethiopia 1st Quarter - March 31, 2016 is falling behind its peers in the area of provision of credit to the ETB = .046 USD private sector. According to the World Bank’s Enterprise Surveys access to finance is perceived as the main business environment Name ACSI constraint by micro (41%), small (36%) and medium (29%) enter- prises in Ethiopia, compared to a Sub-Saharan average of 24%, No. of Active Borrowers 1,056,390 20% and 16% respectively.13 No. of Women Borrowers 621,245 Access to finance is a challenge for men and women alike but Loans Outstanding 8,568,663,201 difficulties are amplified for women, who are less likely to own as- Voluntary Saving 7,431,441,029 sets, and opportunities for women entrepreneurs in Ethiopia lag Compulsory Saving 787,699,181 far behind those of men. In the Economist’s Women’s Economic Opportunity index, Ethiopia occupies the 123rd rank out of 128 Total Saving 8,219,140,210 countries.14 Most growth-oriented women entrepreneurs fall into Total Asset 13,898,277,412 a ‘missing middle’ trap, in which they are served neither by com- mercial banks nor by microfinance situations. High minimum loan Total Liabilities 10,668,396,923 sizes and excessive collateral constraints restrict women’s ac- Total Capital 3,229,880,489 cess to loans from commercial banks. Microfinance institutions (MFIs) often lend without asset collateral, but they primarily cater to micro-firms with group lending schemes that provide very small loans, which are not sufficient in size to support business growth. In 2012, a $50 million World Bank investment lending operation was designed to address this missing middle challenge. The Women Entrepreneurship Development Project (WEDP) aimed to increase the earnings and employment of female-owned micro and small enterprises in Ethiopia, by improving the capacity of existing microfinance institutions to serve female en- trepreneurs. To do this, MFIs were provided with tools, technical assistance, and liquidity to allow them to ‘upscale’ and provider larger, individual loans to growth-oriented women entrepreneurs who needed them. In addition to introducing MFIs to existing best practices in individual lending, the World Bank team also sought to introduce innovations that could enable MFIs to reach a greater number of women entrepreneurs with individual loans (See Exhibit 3). The Lender and the Fintech After deciding on EFL’s psychometric credit assessment tool, the next step was to decide which microfinance institution in Ethiopia would be the best partner for a pilot. At the time, the WEDP project involved 7 of the country’s leading MFIs. After introducing the technology to each of them and conducting a careful assessment, one MFI - the Amhara Credit and Savings Institute (ACSI) - came out as the leading candidate. Located in the Amhara region of Ethiopia, ACSI was the largest MFI in the country, with over 1 million active borrowers and 440 branches (See Exhibit 4). ACSI had a well-established business in the group lending market but individual lending was still a new product line. Individual loans made up just over 10% of ACSI’s lending portfolio and screening for individual loans was still based on a traditional appraisal mechanism that relied heavily on the existence of fixed asset collateral, such as houses or buildings. ACSI saw the pilot project as an opportunity to improve their ability to lend on an individual basis. They were optimistic that the psychometric score could be used to help graduate group borrowers who lacked collateral and therefore did not qual- ify for individual loans under the current credit assessment processes. This would allow former group borrowers to access “SME Finance in Ethiopia: Addressing the Missing Middle Challenge”,World Bank Group, 2015, documents.worldbank.org/curated/en/805371468247458154/ 13  pdf/943650WP0Box380nt0Feb01002015040Web.pdf. “Women’s Economic Opportunity Index 2012”, The Economist, www.eiu.com/Handlers/WhitepaperHandler.ashx?fi=WEO_full_report_final. 14  pdf&mode=wp&campaignid=weoindex2012. 5 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 larger value loans and create an individual credit history. They were eager to learn if psychometric screening could be used to predict credit risk for borrowers in their portfolio. A tripartite pilot agreement was formed and in January 2015, the World Bank, EFL and ACSI set out to pilot psychometric testing as an alternative to collateral for ACSI borrowers. A common objective was established: to measure both the feasibil- ity of the screening tool in predicting credit risk, and the impact of the loans on female borrowers. The agreement was that the World Bank would fund the start-up phase of this partnership, enabling ACSI to pilot EFL’s technology before deciding whether to make a longer-term investment. The agreement also stipulated that the technology would be provided at afford- able rates to other financial institutions in Ethiopia if the pilot proved successful. First Years and Early Success PREPARATION EFL’s technology, an interactive test on a tablet computer, was the epicenter of the pilot and the key to predicting the likelihood the entrepreneur would repay their loan. The rules of the game: if a client scored above a certain score cut-off on the test they could use their test score as a form of collateral and receive a loan without providing a fixed asset as collateral (See Exhibit 5). Clients who already had collateral could also take the test to qualify for a larger loan size. EXHIBIT 5 EFL Questionnaire Administration and Scoring 1 2 3 4 ### Client takes EFL Lender uploads responses Lender receives Borrower receives loan Test on tablet to EFL test when internet 3-digit EFL score if their EFL score is in 45 minutes is available and EFL that indicates client’s above the threshold processes against relative risk 250,000 records Although the psychometric test had been used in many other contexts, Ethiopia’s context was unique and required significant adaptation. In addition to translating the test into Amharic, EFL worked to include more visuals and interactive exercises, since the ACSI microfinance client population had low literacy levels. Other adaptations were also needed to ensure that the test was as concise and easy-to-follow as possible, since the population had limited familiarity with digital technology (See Exhibit 6). Adjustments were guided by the over-arching principle that the test should be quick, data-rich and text-light. The test needed to have a maximum time associated with each exercise and each exercise had to capture multiple traits in order to obtain as much information in as little time as possible.15 These adaptations involved some significant shifts in the content of EFL’s test, but ultimately helped EFL to engage in other low-tech, low-literacy contexts globally over the following years. 15 Meade, Kyle. “From SME to Micro: Developing New Content for the BOP”, EFL Global, 2 Mar. 2015, www.eflglobal.com/efl-microfinance/. 6 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 EXHIBIT 6 EFL Psychometric Test Question Examples How are you How do people Imagine that one feeling today? behave? day, you find 10 gold coins... How would you spend them? PRE-PILOT The pre-pilot phase began in March 2015 in two ACSI branches in the town of Bahir Dar, close to ACSI’s headquarters. The purpose of this phase was to help the team understand the challenges staff and clients could face in implementing the tool and how to appropriately set the loan terms before scaling up to additional branches. During the pre-pilot phase, 420 tests were administered and 133 clients received loans. To administer the test, EFL pur- chased tablets, pre-paid SIM cards and airtime for each of the credit officers. An agreement was made at this time that only clients serviced in urban branches would be eligible to take the test. The pre-pilot stage enabled ACSI to iron out early implementation issues and determine how the product should be marketed, how to integrate it into existing credit screening processes, and how to ensure ownership of the credit scoring tool by branch-level staff. It was an important phase in en- abling ACSI to get familiar and comfortable with the new technology. VOTE OF CONFIDENCE – THE PILOT STAGE After a year, the pre-pilot stage expanded to a full pilot phase in a total of 12 branches. ACSI chose to expand to branches that were networked in the core banking software (not all branches were at the time of the pilot) to ease the credit score de- livery process. During this stage, 2,496 borrowers took the psychometric credit assessment and 1,132 loans were disbursed using ACSI’s standard loan decision-making criteria (See Exhibit 7). Initially, ACSI was hesitant about relying too much on the psychometric test scores. ACSI’s senior management was excited about the potential of the psychometric testing, but wanted more evidence that it could actually predict credit risk in their population. During the pilot phase, therefore, the focus was on testing clients without using the score as the basis for the credit decision. In this way, ACSI could observe how accurate the test was, without taking on any undue credit risk. EFL sponsored a trip for three ACSI deputy managers to visit Janalakshmi Financial Services (JFS), a bank in India that had suc- cessfully implemented psychometrics to graduate group borrowers, and develop an individual loan portfolio. This learning experience positively impacted the commitment of the managers to the pilot. 7 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 The results of the pilot took time to emerge, partially due to the unique context at ACSI. Across its broader portfolio, ACSI’s PAR30 EXHIBIT 7 ACSI Clients Writing the rate (customers with payments due for more than 30 days) was Psychometric Test with below 2%, with very few borrowers in arrears and almost no de- Loan Officers faults. ACSI’s strong book was coupled with long loan tenures. In the early months of the pilot, hundreds of loans were disbursed, usually with a 24-month tenor, and almost none of the borrowers went into arrears. However, ACSI knew that customers were more likely to miss payments as the loan term progressed, with more de- faults emerging after six months, and the peak number of defaults emerging after 18 months. As months passed and loans matured, data began to come in on the clients that had taken the psychometric test. Customers who scored higher on the test were seven times more likely to repay their loans compared to lower performing customers (See Exhibit 8). Across customer segments, higher test scores meant better repayment. There was a clear trend between psychometric profiles and loan per- formance, and the test had proven its value as a tool for predicting credit risk amongst ACSI’s borrowers. Through the pilot phase, ACSI was able to develop a proof of concept, showing that psychometrics could accurately predict which of their borrowers were likely to go into arrears or default. The next step would be to move towards using the psycho- metrics as a substitute to traditional collateral, and to shift to a model where they psychometric score became the basis of the loan decision. Is Newer Shinier? Some Reflections on Alternative and Traditional Data One shortcoming in looking at correlations of psychometric TABLE 1 scores and loan performance is the potential of an attribution Correlations of Variables bias, where the scores themselves are masking the effect of with Default other factors which are influencing loan performance. This could lead to overly optimistic conclusions about the value of Default Business Bad the psychometric scores in predicting credit risk. Age Score Variables Years Count A wide body of evidence shows that, across countries and At 1 day -4.2% 14.4% -9.6% 116 contexts, age and business experience have been powerful At 7 days -7.3% 15.3% -13.2% 63 predictors of both business performance and loan repayment. At 14 days -7.8% 12.2% -12.6% 49 We analyzed the age and years of business experience of the borrowers who took psychometric loans in Ethiopia, in order At 30 days -6.9% 13.2% -4.6% 21 to compare the predictive value of these variables against the psychometric score of the given sample of borrowers. When we examine the correlation of the psychometric score with age and business years, we find weak relation- ships, suggesting that an attribution bias is not at play. However, we still want to understand the value of the psy- chometric score in relation to other potential predictors of loan performance. In Table 1, we run correlations of our three variables (Age, Business Years, Score) and default. Higher numbers in either direction indicate better predic- tive power of our variables. We use a ‘default at 7 days’ variable, as it is the most severe definition at which we have more than 50 defaults.16 16 LenddoEFL typically relies on a minimum of 50 default outcomes to assess a model’s predictive power 8 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 The most interesting finding is that business years have a TABLE 2 positive correlation with default. This means that the older the Regressions on Probability business, the riskier it is. This is an unexpected result, that of Default at 7 Days could suggest that older businesses take out loans if they are in some sort of trouble, which leads to higher rates of default. (1) (2) (3) While psychometric scores, age, and business experience all   Default Default Default correlate with default, the latter two seem less useful from a Age -0.0478*** -0.0312* -0.0421*** policy-standpoint, since it would likely be both inequitable and impractical for financial institutions to exclude or differentiate (0.0157) (0.0161) (0.0158) borrowers on the basis of demographic characteristics such Business 0.107*** 0.116*** 0.113*** as age or years of experience. Years (0.0227) (0.0234) (0.0231) While age and business experience do not seem to hold as EFL Score -0.0149*** useful determinants of lending policy on their own, the remain- (0.00426) ing question is whether the psychometric score adds any val- EFL Score -0.996*** ue beyond the predictive power of these variables. Table 2 Above 350 shows statistical regressions on loan performance with and (0.341) without EFL scores. The regressions use both the EFL score Constant -1.095* 3.953** -0.511 as well as a cutoff of EFL>350 to construct a dummy variable, (0.587) (1.541) (0.607) and test that as a potential predictor of default. These regres- sions provide a basis to believe that EFL scores are a signif- Observations 818 818 818 icant predictor of default probability in addition to individual age and business years. Pseudo 0.058 0.086 0.075 R-squared Overall, our findings suggest that when controlling for age and Standard errors in parentheses years of experience, the psychometric score still has value in * p<0.1 ** p<0.05 *** p<0.01 predicting default. The psychometric score, then, does pro- vide insights into predicted loan performance beyond basic demographic characteristics. However, it’s important to rec- ognize that psychometric credit scoring is not a silver bullet that can replace other forms of loan appraisal or substitute tried-and-tested sources of data traditionally relied on by financial institutions. Alternative data shouldn’t replace traditional data, but tools such as psychometrics can complement and enhance the ability to predict credit risk in information-scarce environments. Scaling Up and Its Complications The move from pilot to scale was laden with complexities, and holds valuable lessons for organizations interested in scaling fintech initiatives. In theory, the tripartite agreement stipulated that ACSI would use the psychometric tool to offer large, un- secured loans to a subset of women entrepreneurs. In practice, although the technology had proven to be predictive, there were several key challenges and trade-offs that ACSI faced in using the psychometric screening tool in this way. ACSI’s evolution into the largest MFI in the country, and one of the largest on the continent, had been premised on a low- cost distribution model, homogeneous product offering, and a high-performing portfolio, with almost no default. While AC- SI’s leadership was very progressive and had a strong interest in financial inclusion, the organization was also committed to maintaining high repayment rates and absorbing minimal risk. While ACSI slowly began to gain trust in the psychometric screening process, they remained reticent to offer large, unsecured loans against the scores. 9 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 A TUMULTUOUS POLITICAL LANDSCAPE The challenges around the risk of scaling the new psychometric technology were amplified in mid-2016, when civil and political con- flict broke out in the Amhara region, where ACSI operates. The conflict broke out just as the pilot phase began to get underway. As a result of the prolonged conflict, late repayments temporarily spiked, as mass closures of businesses, violence, and protests brought the local economy to a grinding halt. As ACSI contended with significant portfolio risk stemming from the conflict, it became more reluctant to roll out the unsecured psychometric loans. Ultimately, the dramatic shift from secured to unsecured lending envisioned by the World Bank team seemed to be a larger leap than ACSI was willing to make in the midst of a portfolio-wide shock. LEGACY SYSTEMS Beyond the concerns about risk, one major operational challenge for ACSI in taking the new technology to scale related to the ability to integrate a new, digital product into an aging and rudimentary IT infrastructure. ACSI had few dedicated IT staff, a dated core banking system, and little connectivity between branches and headquarters. Simple processes, such as reload- ing sim cards and charging tablets overnight, seemed unusually challenging. More complex processes such as matching loan performance data from ACSI’s core banking system with the psychometric score of a given client, required months of effort and extensive trial-and-error. Ultimately, the transition from analog to digital was a slow, belabored process, and is in many ways still a work in progress. This challenge is not one unique to ACSI. Fintechs and banks around the world are struggling with the integration of tech- nology and the limitations of legacy systems. A survey by Accenture of senior financial services industry executives revealed that 72% feel their bank has only a fragmented or opportunistic strategy to dealing within digital innovation. All of the re- spondents felt that legacy technology presented an issue to their organization, but only just over half said their bank had a strategic approach to decommissioning this old technology. The overwhelming majority felt that the big challenge for leading banks is their organizational ability to adopt a collaborative approach with new innovators and start-ups.17 Simply put, for ACSI and for banks around the world, the promise of new technology alone isn’t sufficient: organizations need to be com- mitted and connected to technological change. EXHIBIT 8 Model Performance at ACSI Bad30 ever at 17 months on book 1200 8% 7% 7% 1000 Number of Loans 6% 800 Bad30 Rate 5% 600 4% 4% 3% 400 2% “Bad” borrowers 200 1% 1% “Good” borrowers 0% 310 - 378 378 - 412 412 - 475 Bad 30 Rate EFL Score Buckets 17 Skan, Julian. “The Future of Fintech and Banking: Digitally Disrupted or Reimagined?”, Accenture, 2015, bankingblog.accenture.com/wp-content/uploads/2015/04/accenture- future-fintech-banking.pdf. 1 0 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 ANALOG OPERATIONS AND HUMAN CENTRIC LENDING EXHIBIT 9 ACSI Group Clients ACSI had traditionally relied heavily on human-centric processes. Marketing Poster Loan officers visit clients at their homes and businesses, managers oversee branch processes through daily visits, and communica- tion of priorities is done via written letters and faxes. The introduc- tion of the psychometric technology required a number of orga- nization-wide changes to the credit process to enable unsecured lending. Organization-wide communications around policy were a key chal- lenge. Since there are no staff email accounts at ACSI, new pro- cesses or procedures are communicated through written circulars hand-delivered or faxed to each of ACSI’s 440 branches. Messag- es pass through several layers of the institution before reaching credit officers and clients, and the process is very dependent on word-of-mouth communication. As a result, memorandums about the new psychometric technology were understood in very differ- ent ways by different branches and staff. While some of the larg- er, more visited branches (referred to by ACSI as ‘micro-banks’) seemed to integrate the technology according to policy, the small- er and more distant branches often had challenges around compli- ance with the testing process and use of the new scores. Compliance challenges due to human-centric communications were compounded by a highly hierarchical management structure, typical of large Ethiopian para-statals. The branches which were not complying with the psychometric testing policy often would not alter their practice until receiving multiple, official memorandums. The organizational culture, in which risk of reprimand significantly outweighed rewards for innovation led branch-level staff to be cau- tious and risk-averse in adopting policy changes. In this way, new processes became complex and time-consuming to realize. INCENTIVES AND DETERRENTS The commitment of ACSI’s senior leadership to the use of psychometric testing as an alternative to traditional collateral was strong from the outset. ACSI’s General Manager and its Board were enthusiastic about being a first-mover in piloting and introducing this new technology, which they felt could address a critical challenge they faced around graduating bor- rowers from group to individual loans. However, it soon became clear that ACSI’s rank-and-file - from branch managers to loan officers themselves - didn’t always share this excitement. For loan officers in particular, the new technology involved an additional workload, as it required them to administer the 45 minute psychometric test and put together extra paperwork for eligible clients. The new psychometric product also posed a direct risk to these loan officers, whose performance was judged on the health of the loans they approved. Getting branch staff on board required oversight by the World Bank and EFL, extensive training, and careful attention to the needs of loan officers. For example, loan officers were initially having difficulty explaining the value of the test and administering it in their already busy work program, so a package of marketing materials was developed for them to com- municate more easily with clients about the graduating and lower collateralized products offered (See Exhibit 9). 1 1 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 Vision and Reality EXHIBIT 10 ACSI Client Profile As issues were addressed and time passed, ACSI became more comfortable and confident with the psychometric technology. They Abeba produces and wholesales Ethiopian bread were willing to incrementally adjust how they were using the tech- (injera). She operates 3 stoves in her compound nology to make loans. These changes included adjusting the score and sells up to 150 injera each day. After more cutoff rate to allow more clients to pass the test and increasing the than 10 years as a group loan borrower at ACSI, loan size available with a passing EFL score. While at the start of she was finally able to take her first individual the pilot many branch level staff were skeptical of the technology, loan after passing the psychometric test. As an over time they began to trust the tool and see the benefits it offered individual borrower, she is no longer constrained for their clients. One branch level staff stated, “This product helps by the group loan size limit of approximately and is appropriate to help customers graduate from group to indi- $1000 USD, and can borrow higher value loans. vidual loans and bridge the collateral gap.” The larger loans are fueling business growth. With the first individual loan she was able to build With ongoing commitment from ACSI, EFL, and the World Bank another room in her compound that she rents out team, the technology began to take root. But the utilization of to diversify her income. Despite the transition the technology was not as the World Bank team had initially en- to individual lending, the social cohesion from visioned. Instead of using the psychometric technology as a full the group endures. Many of the former group replacement for asset collateral, ACSI began to see it as an addi- members are now individual borrowers also, tional data point on potential borrowers in the credit decision. but they work together and take turns to go to the MFI to make monthly repayments on each Today, though the support from EFL and the World Bank has sub- other’s behalf. sided, ACSI’s branches are continuing to use the tool as a valuable addition to their credit assessment process. In some branches, collateral requirements are being relaxed or reduced for high-scor- ing borrowers, or loan sizes were increased. In other branches, the test is used to screen trusted group borrowers for the potential to graduate to individual loans (See Exhibit 10). The test became a way for ACSI to improve its appraisal process and provide better products to more borrowers, even if it fell short of becoming a tool to transform ACSI’s underwriting process. Reactions on the Markets Financial institutions in Ethiopia had initially responded to the idea of psychometric testing with tepid optimism. Most banks and MFIs were curious about psychometrics, but wanted to observe the results at ACSI before committing to the technology themselves. Three years later the spirit has shifted decidedly. Psychometrics have demonstrated the ability to predict credit risk in the Ethiopian market, and have succeeded with one of the country’s largest financial institutions. The technology created a buzz, with multiple MFIs and banks requesting their own pilots. To respond to the demand, the World Bank team has launched a second psychometric intervention with one of the coun- try’s leading private MFIs, Wasasa. With knowledge built from the ACSI experience the tool is being operationalized bet- ter this time around. The MFI underwent a readiness assessment to understand their current capacity to adopt financial technology. This exercise created a detailed roadmap for the financial institution to follow that captures what they need to act on from the start to allow for increased efficiency in their operations and greater insight into credit risk from financial technology. At the same time, the pilot has sparked a broader interest in the fintech credit screening space in Ethiopia, with global fintechs also beginning to look to the Ethiopian market for the first time. A New York-based fintech firm, First Access, recently designed and launched a first-ever data driven credit screening tool for the Ethiopian market, which will enable MFIs to tap into alternative data from digitized loan files. 1 2 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019 The reaction from Washington has ranged from a spectrum of exuberance to aversion. The team has received dozens of requests from country teams across the World Bank Group to learn more about psychometrics. World Bank operations in Tanzania, Madagascar, and Zimbabwe have explored integrating psychometrics into the design of credit lines and finan- cial sector operations. Others have expressed resistance to psychometric technology and other fintech credit screening tools, due to the infancy of the technology and the portfolio risk it could introduce. The team’s response has been to moderate expectations. The psychometric credit assessment won’t work for every finan- cial institution, and can’t be implemented overnight. Roll-out of the technology is invariably slow to take root when it must be implemented through loan officers, and is dependent on a number of conditions being in place in a financial institution. Changing the way a bank makes loans takes time, and most lenders are cautious when reforming these fundamentals. Still, as this case study demonstrates, psychometrics can differentiate risk for borrowers with no information and for many financial institutions it could be a valuable tool for reaching more customers with better-fit services. The technology seems to hold particular value in contexts where the coverage of credit bureaus is low, and where other forms of data are unavailable. Conclusions and Options for the Future The small pilot of psychometrics in Ethiopia points to broader questions about how the World Bank Group should engage with and support disruptive technologies. The team hasn’t gotten as far as originally imagined, and lending in Ethiopia still remains largely asset-based. The pilot, however, has made notable impacts. Financial institutions in Ethiopia have gained exposure to technologies that improve the appraisal process and expand access to credit. Psychometric assessment has been proven as a viable way to predict and manage credit risk in female Ethiopian small business borrowers. The fintech industry is starting to see markets like Ethiopia as viable destinations, despite the dearth of digital and financial sector infrastructure. And the World Bank team itself has learned a great deal about how to ensure success in the implementation of fintech innovations. Fintech can be disruptive, and can push the frontiers of inclusion. But like all technology and all change, its introduction must be premised on strong commitment and readiness of the implementing institution, and on a clear vision of how the technol- ogy will be used to expand access. The lesson for the World Bank Group is clear: by introducing the right fintech to the right partners, and providing flexible, responsive, long-term technical assistance, we can help transform the credit landscape and further improve access to finance for credit worthy entrepreneurs. This is collaborative work of the World Bank Africa Gender Innovation Lab and the Finance, Competitiveness and Innovation Global Practice. This work has also been funded in part by The Umbrella Facility for Gender Equality (UFGE), a World Bank Group multi-donor trust fund expanding evidence, knowledge and data needed to identify and address key gaps between men and women to deliver better development solutions that boost prosperity and increase opportunity for all. The UFGE has received generous contributions from Australia, Canada, Denmark, Finland, Germany, Iceland, Netherlands, Nor- way, Spain, Sweden, Switzerland, United Kingdom, and the United States. 1 3 Disruptive Finance: Using Psychometrics to Overcome Collateral Constraints in Ethiopia February 2019