A study shows policy professionals, like everyone else, are subject to cognitive biases. The Project Working together with the Department for International Development (DFID), we conducted a study designed to identify decision-making biases Policy professionals can’t within a sample of our own colleagues- 4,724 World Bank and 1,148 DFID staff. We used a series of defeat their own biases experiments adapted to the development context to test for several decision-making areas and biases that loom in development policy making. Most individuals like to think that they are objective, First, we studied confirmation bias, our tendency to logical, and capable of evaluating all relevant, opt for information that confirms our existing beliefs available information in order to make good or ideas. To do so, we showed respondents identical decisions. But despite our best intentions, we’re all sets of data using two frames: one asking whether a susceptible to cognitive biases and heuristics – those skin cream was effective at treating a rash, and the mental shortcuts that enable us to make quick, often other asking whether or not minimum wage laws necessary, decisions in our daily lives, but which can reduce poverty. also inhibit thoughtful, equitable decision-making and policy design. Second, we tested sunk cost bias, our natural impulse to continue an ineffective endeavor once an investment Even those well-informed of the research on cognitive in money, effort, or time has been made. A major biases aren’t immune to its effects. Policy professionals challenge in government agencies involves inertia; play an essential role in the design and implementation in particular, bureaucracies sometimes continue of policies, programs, and projects across the world. initiatives even when they have been shown not to Cognitive biases of policy professionals – which may work. For our experiment, we asked respondents how be influenced by social environment, mental models, or likely they were to continue investing in a US$500 limited cognitive bandwidth – can thus have significant million project, which, due to changes in policy, was impact on key policies and decisions. Biases can unlikely to achieve any results. compromise work effectiveness, and subsequently, efforts towards poverty reduction. Last, we analyzed the effects of framing on risk aversion. How information is framed affects risk It would be unfair to say that organizations are perception, as well as decisions to take risky decisions altogether unaware of these challenges. To promote on behalf of others. We examined whether loss or gain impartial and proper use of evidence and decision- framing affects willingness to take on risk in a health making, they recruit and rely on judgments of well- context relevant for policy professionals during the qualified content experts, and implement procedural Ebola virus pandemic. We made respondents decide safeguards like peer review and deliberation, cost- between two alternative medical treatments, but benefit analysis, and other kinds of evaluations that randomly assigned them to a frame either emphasizing in theory should counteract cognitive biases. gains (“will be saved”) or losses (“will die”). The Results Confirmation bias changes policy People take more risks when they professionals’ decisions. believe there is nothing to lose. Even though the data were identical, respondents Respondents were 45% more likely to select the risky were significantly less accurate in the minimum wage option when the decision problem was framed as a treatments relative to the skin cream treatments (45% loss. 22% of the respondents assigned to the gains vs. 65%). In addition, respondents were also asked frame selected the risky policy option while 65% of the about their views regarding wage inequality and were respondents assigned to the losses frame did the same. more likely to answer incorrectly when the right answer conflicted with their prior belief. This suggests a bias in interpreting data on ideologically charged interventions. Debating % and exchanging ideas with others improves accuracy. 70 80 90 100 In a follow-up experiment with a small sample of DFID The more time and money spent on a project, staff, we tested whether deliberation can improve the more difficult it becomes to admit failure. decision-making. Respondents were asked to answer We found, on average, a 40% likelihood of disbursing the same questions – first individually, and later in pairs, 60 the remaining funds when 30% of costs were already following a brief period of deliberation. Deliberation 20 30 40 50 spent; this increased to 49% when 70% of the costs improved accuracy by 12% for the confirmation bias were already invested. Interestingly, people also experiment, and the sunk cost bias was completely 65% reported significantly higher likelihood that others in mitigated. However, no real changes were observed in 45% their organization would disburse the remaining funds, the results of the framing experiment, probably because 10 suggesting that individuals may be influenced by what there isn’t a right answer in this case, unlike the first they believe is the norm in their organizations. two experiments. 0 Skin Cream Minimum Wage CONFIRMATION BIAS FRAMING BIAS Correct Responses Percentage of respondents % % choosing the risky policy option 70 80 90 100 70 80 90 100 60 60 20 30 40 50 20 30 40 50 65% 65% 45% 22% 10 10 0 0 Skin Cream Minimum Wage Gains Frame Losses Frame Policy Implications As our findings show, policy professionals, like everyone strengths and weakness of different courses of action else, are subject to cognitive biases. But because policy or views), “dogfooding” products and services (e.g., makers’ decisions often have large sampling the products and services effects on the lives of citizens, it But because policy makers’ that consumers or citizens use before is especially important to create decisions often have large effects rollout), prediction tournaments, and mechanisms that check and correct group deliberation. on the lives of citizens, it is for these biases and blind spots. especially important to create As behavioral economists, a lot of Some procedures can help mechanisms that check and correct our time and efforts are dedicated to mitigating biases, such as “red for these biases and blind spots. understanding underlying cognitive teaming” major decisions (e.g., biases, schemas, and heuristics that implementing mock adversarial arguments, as in playing prevent beneficiaries from engaging in a desired behavior. the “devil’s advocate,” or war games to identify the It’s time we start doing the same with ourselves. About eMBeD The Mind, Behavior, and Development Unit (eMBeD), the World Bank’s behavioral science team in the Poverty and Equity Global Practice, works closely with project teams, governments, and other partners to diagnose, design, and evaluate behaviorally informed interventions. By collaborating with a worldwide network of scientists and practitioners,  the eMBeD team provides answers to important economic and social questions, and contributes to the global effort to eliminate poverty and enhance equity. Stay Connected eMBeD@worldbank.org #embed_wb worldbank.org/embed bit.ly/eMBeDNews April 2, 2018