An experiment in Nigeria showed that psychological tools, like priming, can have powerful impacts on survey respondents. The Study In this context, we decided to test how priming - a The Right Way to Ask tool in the field of psychology in which memories or responses are influenced by a reminder, picture, or other stimulus - could impact measurements. There is no doubt that data is the new gold. Governments rely on data to determine policies and In this brief, we focus on two measures: to measure the growth of the economy. But the truth is that data collection is not simple. On the contrary, 1. an indicator of depression developed from a everyone working on this arena knows that results version of the Center for Epidemiological Studies often vary for many reasons. Depression scale (CESD) - a widely used, validated, and reliable population-based measure, and In 2016, the Nigerian National Bureau of Statistics, in 2. an indicator of discounting and patience related collaboration with the World Bank, conducted a survey to immediate versus late rewards developed from experiment in the latest round of the General Household a set of time preference questions. Survey Panel to examine how talking about sensitive matters affects responses. If people are reminded about To measure priming, respondents were randomly their recent history of adverse events such as shocks and assigned to a treatment or control group. In the priming conflicts, will they change their later answers about topics treatment, respondents answered questions about such as depressive symptoms and level of patience? conflicts, shocks, and deaths (which we will group into one term, “adverse events”) affecting their household in It is not often that a country decides to nationally the past two to six years before answering the module collect data on psychological constructs, yet doing so is on behaviors, preferences, and depression, while in important to get a pulse on these issues and preferences the control group, they answered these questions on in order to inform the development of policies and adverse events after. government services. As one might imagine, however, measuring depression is not an easy task. This is The hypothesis was that being primed on conflicts, particularly the case in Nigeria, which has suffered in deaths, and/or shocks cause people to reflect on recent years from fighting and displacement, and their families’ recent histories of these events, and where surveying people about sensitive topics is a consequently influence the way in which they answered challenging task. the modules on behaviors, preferences, and depression. The Results People who have been affected by adverse Priming is particularly impactful on depression events are more likely to show depressive and time preferences for those who have been symptoms when primed. impacted by personal shocks. For those who have experienced these events, We found that those who have experienced a answering the conflicts, shocks, and deaths module personal shock, such as the loss of a job, and first actually increased the chance that they were who were primed on all adverse events had an classified as as having depressive symptoms by 3.3 increased chance of having depressive symptoms percentage points in comparison with those who of 8.8 percentage points, and were 16 percentage answered that module later in the survey. points less present-biased. Those who have not experienced adverse events appear better off when primed. Priming does not impact measures for Being primed actually made people who had not been those who have been affected by conflicts. affected both less depressed and less present-biased Being primed about conflicts did not impact (the tendency to over-value immediate rewards at the depression or preferences in a significant way expense of our long-term intentions). One explanation? for these respondents. This suggests that the Being primed served as a reminder to respondents who experience of conflict is salient enough to lead have not experienced these events that things in their to an increased probability of depression and lives could be worse, so their self-reported mental decreased probability of present bias without health and present-day patience measures improved. the help of reminders. Policy Implications We need to rethink how we structure surveys to minimize measures are affected by the order of survey modules. measurement error. Amidst the need To this end, we should review the reliability to fill in data gaps, countries across and validity of certain measurements, and the globe collect household-level data We need to rethink under which circumstances they are “true.” on a regular basis, while researchers Measuring depression and time preferences and corporations continuously survey how we structure well and in a standardized and cost- their target groups. However, this research shows that there is a need surveys to minimize effective way will allow policymakers to take the right decisions for their countries’ for better understanding how to measurement error. needs. It will also help researchers answer design surveys. If priming can impact important questions on the correlation answers, especially in the context of sensitive topics, between investments, education, and conflicts, shocks, then we need to understand exactly how our and deaths. About eMBeD The Mind, Behavior, and Development Unit (eMBeD), the World Bank’s behavioral science team in the Poverty and Equity Global Practice, works closely with project teams, governments, and other partners to diagnose, design, and evaluate behaviorally informed interventions. By collaborating with a worldwide network of scientists and practitioners,  the eMBeD team provides answers to important economic and social questions, and contributes to the global effort to eliminate poverty and enhance equity. Stay Connected eMBeD@worldbank.org #embed_wb worldbank.org/embed bit.ly/eMBeDNews Last Update: April 5, 2018