WPS4904 Policy Research Working Paper 4904 Frame-of-Reference Bias in Subjective Welfare Regressions Kathleen Beegle Kristen Himelein Martin Ravallion The World Bank Development Research Group Poverty Team April 2009 Policy Research Working Paper 4904 Abstract Past research has found that subjective questions about economic status of the theoretical vignette households, an individuals' economic status do not correspond as well as their own. The vignette rankings are used to closely to measures of economic welfare based on reveal the respondent's own scale. The findings indicate household income or consumption. Survey respondents that respondents hold diverse scales in assessing their undoubtedly hold diverse ideas about what it means to welfare, but that there is little bias in either the economic be "poor" or "rich." Further, this heterogeneity may be gradient of subjective welfare or most other coefficients correlated with other characteristics, including welfare, on covariates of interest. These results provide a firmer leading to frame-of-reference bias. To test for this bias, foundation for standard survey methods and regression vignettes were added to a nationally representative survey specifications for subjective welfare data. of Tajikistan, in which survey respondents rank the This paper--a product of the Poverty Team, Development Research Group--is part of a larger effort in the department to test new methods of measuring well-being. Policy Research Working Papers are also posted on the Web at http://econ. worldbank.org. The authors may be contacted at kbeegle@worldbank.org, khimelein@worldbank.org and mravallion@ worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Frame-of-Reference Bias in Subjective Welfare Regressions Kathleen Beegle, Kristen Himelein and Martin Ravallion1 Development Research Group, World Bank 1818 H Street NW, Washington DC, 20433, USA 1 This paper would not have been possible without the work of the Tajikistan Living Standards Survey Team, including Diane Steele, Sasun Tsirunyan, Vladimir Kolchin, Farhod Khamidov, Oleksiy Ivaschenko and staff of the National Statistics Committee of Tajikistan (Goskomstat). We thank Gero Carletto and seminar participants at IFPRI and the AEA annual meetings for very useful comments. All views are those of the authors and do not reflect the views of the World Bank or its member countries. 1. Introduction Subjective measures of welfare are widely used in psychological and social sciences, including economics.2 Typically a survey respondent is asked to rate their "economic welfare," "satisfaction with life" or "happiness" on an ordinal scale, sometimes referred to as a Cantril ladder following Cantril (1965). 3 A large literature has studied the covariates of answers to such subjective welfare questions.4 The most common method is a regression (typically an ordered probit) of the survey responses on individual and household characteristics, including age, gender, income, education, employment status and household demographics. Such regressions have been used to assess the welfare effects of, inter alia, own income ("does money buy you happiness?"), employment ("does unemployment lower welfare at given income?") and relative position ("do people care about relative deprivation?"). Measurement problems clearly confound interpersonal comparisons of welfare using subjective data (as they do with objective data). A long standing concern about subjective welfare (and health) questions is that different people may use different criteria for scaling their welfare--that they have different ideas about what it means to be "rich" or "poor," or what it means to be "satisfied" or not with one's life. 5 Such latent heterogeneity in scales has often been seen as invalidating the use of subjective welfare questions for inferring utility.6 This recognizes explicitly that latent heterogeneity in factors that are essentially irrelevant to welfare but influence responses to subjective welfare questions casts doubt on the implied interpersonal comparisons of welfare from a subjective measure. In these circumstances, heterogeneity in 2 A cross-country compendium of the questions asked and a summary of the answers can be found in Veenhoven et al. (1993). The literature in economics is reviewed by Di Tella and MacCulloch (2006). The psychological literature on subjective welfare is reviewed in Diener et al. (1999) and Furnham and Argyle (1998). Since 2000, a scholarly journal has been devoted to the scientific study of subjective welfare, namely the Journal of Happiness Studies. 3 An alternative approach is to ask what level of income is needed to attain a given position on a Cantril ladder, such as not being "poor." This is the "Leyden method" devised by van Praag (1968). While we do not use this type of data here, the same concerns about bias arise in the Leyden method. 4 Examples include van Praag (1968), van de Stadt et al. (1985), Clark and Oswald (1994, 1996), Kapteyn et al. (1998), Easterlin (1995), Oswald (1997), Winkelmann and Winkelmann (1998), Pradhan and Ravallion (2000), McBride (2001), Ravallion and Lokshin (2001, 2002, 2009) Senik (2004), Luttmer (2005), Ferrer-i-Carbonell (2005) and Fafchamps and Shilpi (2009). 5 While this paper focuses on heterogeneity in scales, there are other concerns. For example, Conti and Pudney (2008) find that minor re-designs in questions on satisfaction of life/work led to large changes in answers, particularly for women. Moreover, they conclude that these distortions in survey responses influence research findings with respect to understanding women's job satisfaction. 6 For a recent discussion, see Di Tella and MacCulloch (2006). 2 scales will translate into corresponding differences in subjective welfare at any given level of objective welfare or other relevant covariates. This will, of course, reduce the explanatory power of the regression models for subjective welfare. However, if such heterogeneity was purely random then it would not invalidate inferences from such regressions (at least for linear models). Thus it has been argued that, while inter-personal welfare comparisons are invalidated by heterogeneous scales, the regressions are likely to be robust to such heterogeneity.7 However, that claim is questionable. People will answer subjective questions in surveys relative to their personal frame-of-reference, which depends on the respondent's own knowledge and experience, and therefore is likely to vary systematically with the characteristics of that person, including objective measures of economic welfare. For example, it can be conjectured that people living in poor areas of a developing country tend to have a more limited knowledge of the full range of levels of living found in the society as a whole. Someone living in a poor, remote area who has only infrequently left the village and gone no further than the district town, may rate her welfare higher than someone with the same real income who lives in a city and sees far greater affluence around her. Similarly, it can be conjectured that relatively well-off people are often unaware of how poor some people are, and may thus rate their own welfare lower on a Cantril scale. When this effect is present and it impacts on the coefficients of interest, we shall say that there is a frame-of-reference bias (FORB). The potential for FORB raises concerns about the (enumerable) regression models found in past literature. Consider, for example, the many papers that have used subjective welfare regressions to test for reference-group effects, such as whether higher neighbors' income makes one feel worse off through perceptions of relative deprivation. It seems likely that the same reference group also influences the respondents' interpretation of scales used in subjective questions. The reference group acts as both the comparator in assessing relative position and a key element of the information set used by respondents when interpreting the Cantril scales. To give a sharp illustration of the problem in the present context, suppose that three people are asked to rate their own welfare on a scale of 1,2 and 3, with "1" the poorest. For the sake of the argument, let us also suppose that "wealth" is the only parameter for defining 7 For example, Frey and Stutzer (2002, p.406) note the possibility of heterogeneity in the scales used in self-reported welfare questions but claim that this does not invalidate regression models for such data. 3 "welfare." Wealth is normalized to be in the [0, 1] interval. The first person is relatively poor, the second has the overall modal wealth, denoted M, and the third is relatively rich. The frame-of- reference effect implies that the poor person is only aware of levels of wealth in the interval [0, M] while the rich person is only aware of those in the interval [M, 1]. Furthermore, the poor person has a wealth somewhere near the middle of the [0, M] interval while the rich person has wealth near the middle of [M, 1]. This suggests a potentially large downward bias in the regression coefficient of subjective welfare on wealth. Indeed, all three responses to the subjective welfare question may well be "2". This type of bias may also be present when using other concepts of well-being such as life satisfaction or happiness. There are antecedents to the idea of frame-of-reference effects in the literature. It is a well-established idea that people assess their welfare relative to some "comparison group" such as neighbors or co-workers.8 This argument has emphasized relativist welfare comparisons. It has also been argued that reference groups play an important role in expectations formation.9 It is a small step from these ideas to the proposition that survey respondents answer questions with reference to their immediate experiences and that this may well be highly localized in some relevant social or geographic dimensions. In the specific context of subjective welfare measurement, Seidl (2004) argues that van Praag's (1968) method of calibrating a utility function to subjective data confounds the underlying utility function with a "welfare evaluation function" whereby (for example) "respondents belonging to the middle income strata can evaluate the welfare of the middle income range more precisely" (Seidl, 2004, p.1653).10 The frame-of-reference effect can also be interpreted as a special case of what is termed "differential item functioning" (DIF) in the literature on educational testing. In this literature, DIF exists if students with equal latent ability have different probabilities of giving a correct answer.11 This paper explores the role of the frame-of-reference effect in influencing self-reported economic status and offers various tests for FORB, to see whether the standard regressions for subjective welfare found in the literature are contaminated by this effect and to assess the consequences of corrections for this bias to the types of results reported in the literature. Toward 8 See, for example, Frank (1997). Frey and Stutzer (2002) provide a useful overview of evidence related to comparison-group effects. 9 This was argued by Hirshman (1973). For more recent discussions and evidence, see Ravallion and Lokshin (2000) and Senik (2004). 10 See the response by Van Praag and Kapteyn (1994) to this critique. 11 For an overview of the history and methods of addressing DIF, see Angoff (1993). 4 these aims, we adapt the vignette methodology that has been used in a number of recent studies of subjective data on (inter alia) health status, political efficacy and job satisfaction. King et al. (2004) and King and Wand (2007) designed vignettes to establish common points on the heterogeneous reference scales regarding political efficacy in China and Mexico. Kristensen and Johansson (2008) used vignettes in anchoring subjective scales for job satisfaction. Bago d'Uva et al. (2008) used them for correcting self-assessed health data for reporting bias. Vignettes have also been used in testing the competence of doctors (Das et al., 2008). To our knowledge, this is the first work to use vignettes for anchoring self-reported economic status, although Kapteyn et al. (2008) use vignettes to compare life satisfaction between respondents in the U.S. and the Netherlands. The paper provides three tests for FORB where our subjective welfare measure is self- reported economic status. In the first we simply test whether vignette responses are correlated with the regressors typically used in subjective welfare analysis, including objective welfare measures. If everyone has essentially the same idea of what it means to be "poor" or "rich" then we would not expect to find significant correlations between the vignette responses and the covariates used to explain subjective welfare. Under certain forms of FORB, we may find a negative economic gradient in the vignette responses. Consider again the three people in the example above and now suppose that these three people are the vignettes. A plausible set of responses to the vignette questions is given in Table 1, indicating that the wealthier the respondent, the lower (or at least not higher) the rating of each vignette. So Test 1 provides a very direct test for FORB in subjective welfare regressions, under which we would look for a negative wealth gradient in each set of vignette responses as well as correlation with other household characteristics. We then provide two further tests that can help quantify the extent of any bias due to heterogeneity in scales by providing a method of correcting subjective welfare regressions for the presence of individual-specific scales or standards. In Test 2, the vignette responses enter as control variables in standard regressions, to purge the error term of the heterogeneity in scales used, under the assumption that differences in vignette responses are solely attributable to differences in the personal scales used. In the third test, an alternative to Test 2, we use the re- scaling method proposed by King et al. (2004) in which subjective welfare measures are re- calibrated for consistently across respondents based on the vignette responses. 5 We use data from Tajikistan. For the purpose of this paper, we specially designed and included a set of vignettes in the 2007 national household survey for Tajikistan. Respondents were asked to place themselves on a subjective welfare ladder with six rungs. Later in the questionnaire they were asked to place four vignette households on this ladder, and finally to (again) place their own household on the same ladder after ranking the vignettes. Unlike past research using vignettes, we have assured that the subjective welfare questions were asked both before and after the vignettes. The second (post-vignette) subjective reporting enables the re- scaling of subjective responses, since we need respondents to place their household in reference to the vignettes themselves. We begin with a description of our data in section 2. Section 3 presents our results on FORB while section 4 concludes. 2. Data Tajikistan is one of the poorest and most isolated of the countries in the former Soviet Union, with a per capita income of 430 USD in 2007. Its mountainous location and deteriorating physical infrastructure make transportation difficult and leave certain parts of the country completely isolated during winter months. About one-third of households are located in the capital Dushanbe or other urban areas. Overall, 47 percent of the population lived below the country's poverty line,12 and 21 percent in 2004 lived below the World Bank's international poverty line of $1.25 a day at 2005 purchasing power parity (estimated using PovcalNet at iresearch.worldbank.org/PovcalNet/jsp/index.jsp). Survey data and instrument design The 2007 Tajikistan Living Standards Measurement Survey (TLSMS) surveyed a random sample of 4,860 households in September and November 2007. The sample is designed to be representative at the national level, urban and rural levels, and at the oblast (administrative region) level. Data were collected in two visits, with the subjective welfare modules being asked in the first visit. Summary statistics for the variables used can be found in Appendix 1. 12 See World Bank (2008). The poverty line is based on the cost of 2,250 calories per day and includes an additional non-food component calculated from the share of total consumption going to non- food purchases with the reference population being households with food spending just above the food poverty line. All values are regionally deflated. 6 In addition to the standard questions on household characteristics and expenditures (including imputed values of self-produced food) common to multi-topic household questionnaires, subjective welfare data were collected at two different points in the questionnaire. In the Subjective Poverty and Food Security Module, respondents were asked: "Imagine a 6-step ladder where on the bottom, the first step, stand the poorest people, and the highest step, the sixth, stand the rich. On which step are you today?" In a later section of the questionnaire, respondents were asked to place four vignettes of hypothetical households on a six-step ladder and then to place themselves on the same scale. The latter is the same question asked in the earlier portion of the questionnaire. The actual vignettes from the questionnaire (translated into English) are given in Appendix 2. We developed the vignettes for this experiment in consultation with local counterparts. The vignettes were designed to capture representative snapshots of various levels of welfare in Tajikistan. Characteristics incorporated in the vignettes include land holdings, education, diet, clothing, and the ability to heat the home during the winter. The vignettes were developed in a clear, expected hierarchy with respect to welfare, with all aspects of socio-economic status increasing monotonically. This structure was used to minimize the effects of multi- dimensionality. Multi-dimensionality can lead to the perverse sequencing of the vignettes with respect to the intended ordering if respondents place different values on various characteristics contained in the vignettes with respect to overall welfare (King et al., 2004). This does not seem to be a major concern for our vignettes: of the 4,860 households in the sample, only 89 have any instances of an "incorrect" ordering. The most common characteristic of respondents who perversely order the vignettes is a low level of education of the household head; see Appendix 3 for results from a probit estimation of the correlates of perverse ordering. These 89 households are excluded from our analysis. Pre and post vignettes If responses are influenced by heterogeneous scales, it might be expected that subjective welfare responses will be affected by familiarity with the vignettes. By asking the vignette questions, the survey may focus the respondent to think about, and possibly revise, their own scale used to report their self-assessment welfare. In Table 2 we compare the pre- and post-vignette responses to the household's self- assessment welfare. On average, respondents place themselves between steps 2 and 3 in both the 7 pre-vignette and post-vignette question, though the mean was slightly higher post-vignette (2.75 vs. 2.80). Most respondents place themselves in the same position in the pre- and post-vignette self-reporting, although fully 25 percent change their position. Of those who change their position, 57 percent adjust upwards and 43 percent downwards. Generally respondents give similar responses; 82 percent of those changing their position move only one step up or down. Among the 4,771 households, only 39 respondents switch dramatically from the "rich" category (steps 4-6) when asked before the vignettes to the "poor" (steps 1-2) category when asked after the vignettes. In the opposite direction, 72 report themselves in the "poor" categories pre- vignette and in the "rich" category post-vignette. We explore whether specific types of households are more inclined to change pre- and post-vignette reports, but we do not observe any striking differences in such changes across different categories of households (Table 3). For example, comparing those classified as poor using objective measures (per capita households expenditure below the poverty line), the poor report a lower subjective welfare score than the non-poor. Both groups adjust their scores up slightly following the vignettes, but neither experiences a marked transformation. We observe similar patterns for other groups, namely urban/rural populations, households in which the head is employed or unemployed, male and female headed households, and households headed by persons older or younger than 65. Subjective vs. objective economic welfare To compare the two subjective measures against our objective measure based on expenditure per person, we define the sample sizes of six categories of expenditure per capita, based on the distribution of households in the subjective categories. If there are N respondents who place themselves on the lowest subjective step, the lowest N households in terms of expenditure per capita will make up the lowest category in the objective measure. Table 4 presents the results for the pre-vignette subjective rankings. If the subjective measures are perfectly explained by the objective measure, all observations in the matrix would be along the shaded diagonal. Though the subjective measures are highly correlated with the objective measure, the matching is imperfect. Nearly half (43 percent) of those in the lowest objective classification place themselves on steps 3 or higher in the subjective measure. Of the richest decile of the population according to objective measures, 19 percent place themselves on the lowest two rungs of the subjective ladders, and less than half 8 position themselves on the top three rungs. Of those households classified as extreme poor, living below the food poverty line (roughly 15 percent of the population), only 55 percent place themselves on the lowest two subjective rungs. Similarly, among poor households, only 45 percent place themselves on the lowest two subjective rungs. The post-vignette rankings show a similar relationship to objective measures as for the pre-vignette rankings (Table 5). Twenty percent of those in the richest decile position themselves on the lowest two rungs. Among extreme poor households and poor households, 53 and 44 percent respectively place themselves on the bottom two rungs. It is also interesting to note that the pre-vignette rankings seem more consistent with our objective measure than the post-vignette rankings. The Cramer's V statistic is higher in the pre-vignette question, indicating a stronger association between the rows and columns, and a better overall fit to the objective measure. Comparing the correlation between the pre- and post-vignette rankings and the objective per capita expenditure measure, the correlations are higher for pre-vignette reports (Table 6, row 1). This is also true universally across various populations of interest. It is interesting to note that within some household categories we find lower average subjective welfare reports among the sub-group with higher per capita expenditure. This is true, for example, among female and male headed households (where female-headed households are on average richer by objective measures). Across all household categories the post-vignette scores are higher than those reported prior to the vignettes. Placement of vignettes The vignettes were designed such that the first vignette presented a scene of extreme poverty, the second vignette of improved conditions though still poor, the third of middle class and the fourth of relative affluence. If the vignettes are an effective method of imposing a uniform scale, we would expect the placement of particularly the first vignette to be a good indicator of objective welfare. As the first vignette presents a picture of extreme poverty, most respondents would be expected to place this vignette on step 1. Households which position the first vignette on step 2 might be expected, on average, to be poorer than those who placed it on step 1, as they are able to conceive of a household situation that was even poorer than the first vignette. Similarly households which place the first vignette on step 3 we might assume them to be poorer than those who place the vignette on steps 1 or 2 as they could imagine even worse living conditions compared to that 9 depicted in the first vignette. Following a similar logic, we would also expect the average subjective welfare score to be higher for those that locate the first vignette on step 1 as opposed to those placing it on steps 2 or 3. The data, however, did not bear out this expectation (Table 7). Those households that place the first vignette on step 1 are, on average, poorer than those who place the first vignette on steps 2 or higher. Though there is no statistical significance in terms of objective measures between households that place the first vignette on step 1 and those who place it on steps 2 or higher, there is a statistically significant difference in terms of subjective welfare measures. This would indicate that those households who position the first vignette on steps 2 or higher perceive themselves as better off than their objective circumstances would indicate. 3. Results Test 1: We begin by asking whether vignette responses are correlated with widely-used covariates from the literature, including objective measures of economic welfare. We assume an ordered probit specification, which has become standard in the literature. The specification for the underlying continuous variable (generating the ordinal categorical responses) is as follows: VWik k ln PCEi k X i ki (k=1,4; i=1,..,N) (1) where VWik is a latent continuous variable for respondent i's assessment of vignette k's welfare, which generates a discrete response on the scale from 1-6, PCE is per capita expenditure, X is a vector of other household-level variables. Table 8 summarizes the results while Appendix 4 presents the complete ordered probit regressions estimating the parameters of (1) for the four vignettes. The pseudo R2's are low, at approximately 0.02. In general, vignette rankings are not consistently or significantly correlated with household characteristics. Geographic characteristics are more likely to be significant for the vignettes higher on the consumption scale (vignettes 3 and 4). For vignettes 3 and 4 (but not 1 and 2), we find a positive and statistically significant relationship between log PCE and the vignette rankings. Richer households are more likely to give a high welfare ranking to the better-off households described by vignettes 3 and 4. This is not what one would expect with the frame-of-reference effect described in the introduction, which would suggest that poor people would tend to rank the relatively rich vignette higher than 10 rich people. More suggestive of this FORB is our finding that smallholders (in terms of land) tend to rate the poorest vignette higher than do other households. So the results of Test 1 show that there are only a few significant correlates of vignette responses amongst the types of regressors commonly used in subjective welfare regressions. But the evidence is mixed on FORB: How much do these effects bias the standard subjective welfare regressions found in the literature? Test 2: In our second test we examine a standard subjective welfare regression, employing widely-used covariates from the literature, with the difference that we also estimate specifications augmented with the vignettes. The augmented specification is as follows: SWi ln PCEi X i Vi i (i=1,..,N) (2) where SWi is a latent continuous variable for the subjective welfare of respondent i, which also generates a discrete response on the scale from 1-6 and V is a vector representing the vignette responses. The vector V translates the vignette responses into a series of dummy variables. This eliminates FORB under the assumption that inter-personal differences in vignette responses stem solely from differences in how the scales are interpreted. We refer to the estimated as the economic gradient in subjective welfare. With six possible steps for each of the four vignettes, there would theoretically be twenty dummy variables with one step omitted for each vignette. In practice, however, some steps are omitted due to an insufficient number of responses, leaving a total of fourteen vignette dummy variables to capture the complete set of responses observed in the data. We examine the two measures of SW: SW reported before the vignettes and the post-vignette SW. Table 9 presents the results of the ordered probit based on equation (2) using the pre- vignette self-reported welfare. We estimate four alternative specifications. In column 1 we find that logged per capita expenditure is significantly positively associated with SW. When vignettes are introduced in the second specification (column 2), the coefficient on PCE is basically unchanged. We do, however, find that the set of vignette dummy variables are jointly significant. Column 2 suggests that there is a frame-of-reference effect on SW, although comparing columns 1 and 2 the vignette effects are not sufficiently correlated with the household's PCE to generate more than negligible bias in the unconditional economic gradient in subjective welfare. That is, there is negligible FORB. 11 In column 3 of Table 9 we include non-income household characteristics and omit the vignettes. A number of these characteristics have significant effects on subjective welfare controlling for PCE. Female-headed households have lower SW. Households where the head has completed higher education, those where the head has a professional job (such as sales, service and public administration) and larger households generally have higher SW. We do not find consistent urban/rural patterns across areas. Households in urban Gbao report higher SW compared to the reference group in Dushanbe. In rural Khatlon and RRP as well as urban Khatlon, households have lower SW compared to households in Dushanbe. One might expect that households with migrants would have lower subjective welfare scores as they have a wider scope of knowledge and are therefore less likely to overstate their position. The coefficient is positive though not statistically significant. After adding controls for other household characteristics, we find an increase in the economic gradient (the increase in the coefficient on PCE in columns 1 and 3 of Table 9). This is a statistically significant difference at the 10 percent level. The household characteristics in column 3 may capture several things, including frame-of-reference effects or other effects such as perceived vulnerability, permanent income, or security. Introducing both the vignettes and the set of non-income household characteristics in column 4, we find that the economic gradient is basically unchanged from column 3. The set of vignette dummy variables remain statistically significant. Thus, while it appears that there is a frame-of-reference effect being picked up by the vignettes, it is still not influencing the economic gradient in subjective welfare. In addition, including the vignettes does not alter the coefficients on other household characteristics. Table 10 gives our results using the post-vignette subjective welfare measures. The magnitude of the coefficients tends to be lower. In columns 1 and 2, there is still no statistical significant difference between the economic gradient in subjective wellbeing. As before, we observe an increase in the economic gradient when we control for non-income household characteristics; the coefficient on PCE is statistically different at the 10% level between columns 1 and 3. And, again, there is no significant change in this coefficient when we further include the vignettes (columns 3 and 4). Likewise, the coefficients on other covariates remain largely unchanged. Our finding that FORB is negligible remains valid. 12 Test 3: A concern with Test 2 is that the vignette responses are strictly endogenous, given that they come from the same respondent at the same time and so could be jointly influenced by some latent characteristic. One response to this concern is to instead use the vignette responses to re-calibrate the subjective welfare responses. Our rescaling method follows that developed by King et al. (2004) and King and Wand (2007) in which only the relative position of the self- reported score in relation to the vignette rankings matters to the analysis. For example, all respondents who ranked themselves below vignette 1 would have a score of 1 in the re-scaled rankings, regardless of the actual values given to either the self-reported score or the ranking of the first vignette. Similarly, all those respondents who placed themselves at the same level as the first vignette would have a re-scaled ranking of 2, those between vignettes 1 and 2 would have a 3, and so on. Rescaling therefore gives nine possible values to the dependent variable. Table 11 repeats the analysis presented in Tables 9 and 10 with a rescaled post-vignette ranking as the dependent variable. (For completeness we also estimate regressions for the re- scaled responses with controls for the vignettes.) The rescaled regressions show similar relationships with the household characteristics, including significant correlations with PCE, higher education, professional jobs, larger households and the geographic variables. The vignettes, in general, lose their significance with the notable exception of vignette 3. Tables 9-11 assess the impact of the vignettes on the economic gradient in subjective welfare under the assumption that this impact is constant across the income distribution. To allow for more flexibility in the specification we estimate non-parametric regression functions with linear controls ("partial linear regressions").13 By comparing the non-parametric results with and without vignettes (as linear controls), we can test for FORB across the income distribution. We define two alternative binary outcome variables: SW being poor (steps 1-2) and SW being rich (steps 4-6). Since we have both pre- and post-vignette SW, we have 4 outcome variables (pre/post, poor/rich). Further, we assess the FORB from estimates with and without the non- income traits. Thus, we have a total of eight pairs of partial linear regressions (Figures 1-4). Figure 1 shows the results for reporting oneself as poor and as rich, with and without vignettes as controls. The "poor" curve is downward sloping: households are less likely to place themselves on the lowest two steps as PCE increases. The opposite is true for those who place themselves on the upper rungs ("rich"). The economic gradient in subjective wellbeing is 13 We used the PLREG program for STATA written by Lokshin (2006). 13 unchanged if we control for the household's scoring of vignettes. That is, the FORB appears to be minimal across the whole income distribution. If there was FORB, we would expect the curves with the vignettes to be steeper than those without, assuming that the poor would overstate and the rich understate their subjective welfare, with attenuation towards the mean. We find the same results when we include non-income controls (Figure 2) and when we define the dependent variables using the post-vignette SW (Figures 3 and 4). As noted above, the economic gradient was lower for the post-vignette SW compared to the pre-vignette score (Tables 9 and 10, column 2). This difference in the economic gradient may also vary across the income distribution. Partial linear regressions of poor SW and rich SW are presented in Figure 5, which compares the results based on pre- and post-vignette SW. Figure 5 shows that households at all income levels are more likely to report being rich after scoring the vignettes compared to their prior response; that is, the "rich" curve is shifted up after the hearing the vignettes. There is little change in the probability of reporting being poor for the pre- versus post-vignette SW. Non-income household characteristics have a modest impact the estimates of the economic gradient of subjective welfare. The economic gradient with respect to self-reporting as poor is slightly steeper when additional control variables are included (Figure 6 "poor"). There is only a slight shift in the gradient with respect to self-reporting as rich after adding controls (Figure 6 "rich"). These results are the same for the post-vignette SW (Figure 7). 4. Conclusions A cloud of doubt has hung over subjective welfare regressions, arising from concerns about likely heterogeneity in the interpretation of the Cantril scales widely used to measure subjective welfare. This heterogeneity undoubtedly reduces the power of standard covariates in explaining perceived welfare. More worrying, however, is the possibility that the heterogeneity in scales is leading to biased inferences from studies of subjective welfare, including biases in identifying its economic gradient, as well as the effects of other variables such as education, employment status and relative economic position. Bias arises if these variables are correlated with the latent heterogeneity in scales. It can be conjectured that poorer people tend to have more limited horizons in life, due to more limited experiences with the extent of the disparities in levels of living in society as a whole; a poor person's idea of what it means to be "rich" may then 14 be very different to that of a middle or upper income person with a very different frame of reference. This may be correlated with certain attributes of the household, such as if rural or more isolated households may overstate their welfare given that they have a limited experience or exposure to higher living standards with which to judge their own economic standing. We have provided various tests for bias due to such heterogeneity in individual scales. The tests entailed adding vignettes of hypothetical households to a national household survey for Tajikistan in 2007. Respondents placed these vignettes on the same six-step ladder used to report their own subjective economic welfare. In our first test for this bias, we do find some significant covariates for vignette responses amongst the regressors commonly used to explain subjective welfare. However, the effects are neither very strong nor consistent across different vignettes. In the second and third tests, we explore the extent of bias due to the frame of reference effect by embedding vignette rankings by households into standard regressions for subjective welfare. We do this in two ways: we include vignettes among the covariates and, to address endogeneity concerns with this approach, we also recalibrate the self-assessments of welfare to accord with the heterogeneity in scales revealed by the vignette responses. The striking finding of these further tests is that the bias is negligible with respect to the "income effect" on subjective welfare as well as other covariates. Based on this study, the concerns that past uses of subjective economic welfare data are compromised by systematic differences in the meaning given to the scales used appear to be unwarranted. 15 Table 1: Plausible responses to vignette questions when there is a frame-of-reference bias Respondent's wealth Poor Middle Rich Poor 2 1 1 Vignette Middle 3 2 1 Rich 3 3 2 Table 2: Pre-vignette and post-vignette subjective welfare rankings Post-vignette 1 2 3 4 5 6 Pre-vignette poorest richest Total 1 poorest 247 64 23 3 4 0 341 2 72 933 232 53 7 5 1,302 3 34 242 1,735 223 37 4 2,275 4 4 33 112 592 34 1 776 5 2 0 2 11 51 1 67 6 richest 0 0 1 1 4 4 10 Total 359 1,272 2,105 883 137 15 4,771 Table 3: Summary statistics pre and post-vignettes Pre-vignette Post-vignette Overall mean 2.75 2.8 (0.87) (0.92) Household characteristics Poor 2.53 2.59 Non-poor 2.94 2.99 Rural 2.67 2.73 Urban 2.91 2.93 Unemployed head 2.61 2.66 Employed head 2.82 2.88 Female head 2.58 2.62 Male head 2.79 2.84 Pensioner head 2.78 2.63 Non-pensioner 2.59 2.83 Note: Standard deviation in parentheses. 16 Table 4: Comparison of pre-vignette subjective welfare with objective measure Subjective Expenditure per capita rank welfare 2 3 4 5 6 rank 1 poorest richest Total 1 poorest 84 120 108 26 3 0 341 2 109 461 586 137 8 1 1,302 3 130 582 1,184 345 29 5 2,275 4 18 135 369 227 23 4 776 5 0 3 25 36 3 0 67 6 richest 0 1 3 5 1 0 10 Total 341 1,302 2,275 776 67 10 4,771 Notes: Cramer's V = 0.1484; Chi-square=525 (significant at 1%) Table 5: Comparison of post-vignette subjective welfare with objective measure Subjective Expenditure per capita rank welfare rank 1 poorest 2 3 4 5 6 richest Total 1 poorest 78 128 98 47 7 1 359 2 126 422 549 154 19 2 1,272 3 126 560 998 362 53 6 2,105 4 23 150 402 260 43 5 883 5 6 10 53 54 13 1 137 6 richest 0 2 5 6 2 0 15 Total 359 1,272 2,105 883 137 15 4,771 Notes: Cramer's V = 0.1355; Chi-square= 438 (significant at p <0.0005) 17 Table 6: Correlations with objective measures Expenditure per Correlation with Correlation with post- capita pre-vignette vignette ranking (Tajik somoni) ranking Overall mean 176 0.203 0.178 (162) Household characteristics Poor 100 0.188 0.176 Non-poor 245 0.135 0.106 Rural 157 0.206 0.187 Urban 218 0.193 0.169 Unemployed head 161 0.176 0.176 Employed head 185 0.209 0.173 Female head 192 0.167 0.135 Male head 173 0.225 0.202 Pensioner head 157 0.151 0.157 Non-pensioner 180 0.209 0.179 Note: Standard deviation in parentheses. Table 7: Placement of vignettes First Vignette ranked Expenditure per Subjective on Step: capita position N 1 173 2.64 3,276 2 181 2.95 1,352 3 195 3.12 140 Note: Three households which ranked the first vignette above step 4 are excluded. 18 Table 8: Significant predictors of how households rank the four vignettes (See Appendix 4 for complete regression results) Vignette 1 (poorest) 2 3 4 (richest) Significant Household size (+) Special secondary PCE (+) PCE (+) covariates at the Basic education (-) schooling (+) Uzbek (+) Uzbeck (+) 10% level or Services sector Number of Primary schooling Number of better occupation (+) employed (+) (+) employed (+) Small holding (+) Small holding (-) General secondary Agriculture sector Khatlon urban (-) Sogd (+) (+) (-) Gbao rural (-) Khatlon (+) Number of Small and medium Gbao urban (-) employed (+) holding (-) Gbao rural (+) Small holding (-) Sogd (-) Sogd rural (-) Khatlon urban (+) Khatlon urban (+) Khatlon rural (-) RRP rural (-) RRP (-) Gbao (+) Gbao rural (+) Pseudo R2 0.022 0.018 0.016 0.019 19 Table 9: Pre-vignette self-assessed subjective welfare positions (1) (2) (3) (4) PCE + PCE + PCE + household PCE household vignettes controls + controls vignettes Coef. se Coef. se Coef. se Coef. se Log expenditure per capita 0.630*** (0.037) 0.631*** (0.037) 0.727*** (0.043) 0.736*** (0.043) Household Demographics Female headed household -0.151*** (0.053) -0.148*** (0.053) Age of household head -0.014 (0.010) -0.017* (0.010) Age of household head squared 0.000 (0.000) 0.000* (0.000) Household size 0.076*** (0.013) 0.082*** (0.013) Number of children -0.007 (0.018) -0.011 (0.018) Number of older adults -0.030 (0.047) -0.037 (0.048) Number of migrants 0.049 (0.043) 0.034 (0.044) Ethnicity (Reference: Tajik) Uzbek 0.020 (0.046) 0.019 (0.046) Russian -0.337** (0.157) -0.350** (0.165) Other -0.693*** (0.159) -0.718*** (0.150) Education (Reference: No Education) Primary 0.066 (0.116) 0.001 (0.118) Basic -0.118 (0.109) -0.129 (0.111) General Secondary 0.028 (0.107) 0.007 (0.109) Special Secondary 0.143 (0.114) 0.111 (0.115) Technical Secondary 0.103 (0.117) 0.068 (0.119) Higher Education 0.304*** (0.115) 0.285** (0.117) Graduate School 0.545 (0.368) 0.617** (0.305) Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed 0.035* (0.018) 0.032* (0.018) Agriculture / Fishing / Forestry 0.009 (0.064) -0.003 (0.065) Manufacture / Mining -0.023 (0.094) -0.009 (0.096) Services 0.363*** (0.132) 0.307** (0.131) Construction -0.063 (0.080) -0.093 (0.080) Public Administration / Education / Health 0.186*** (0.070) 0.190*** (0.071) Sales and Services 0.310*** (0.064) 0.314*** (0.065) Other -0.158 (0.107) -0.156 (0.105) Agriculture (Reference for holdings: No Land) Small Holding -0.063 (0.066) -0.037 (0.067) Medium Holding -0.103 (0.079) -0.087 (0.080) Large Holding 0.020 (0.081) 0.051 (0.082) Any livestock, poultry, etc -0.007 (0.058) -0.008 (0.058) Geography (Reference: Dushanbe) Sogd Urban -0.011 (0.075) -0.067 (0.076) Sogd Rural -0.106 (0.079) -0.159** (0.080) Khatlon Urban -0.184* (0.095) -0.344*** (0.097) Khatlon Rural -0.509*** (0.074) -0.570*** (0.075) RRP Urban -0.158 (0.108) -0.200* (0.110) RRP Rural -0.263*** (0.075) -0.319*** (0.077) 20 (1) (2) (3) (4) PCE + PCE + PCE + household PCE household vignettes controls + controls vignettes Coef. se Coef. se Coef. se Coef. se Gbao Urban 0.343*** (0.107) 0.409*** (0.113) Gbao Rural -0.062 (0.080) -0.076 (0.081) Vignette 1 (positions 3 to 6 omitted) vign1==1 -0.258* (0.140) -0.281* (0.144) vign1==2 -0.022 (0.133) -0.067 (0.138) Vignette 2 (position 1 omitted) vign2==2 1.106*** (0.223) 1.168*** (0.227) vign2==3 1.337*** (0.230) 1.439*** (0.233) vign2==4 1.515*** (0.261) 1.668*** (0.265) vign2==5 1.724*** (0.574) 1.670*** (0.468) Vignette 3 (positions 1 and 2 omitted) vign3==3 -0.777*** (0.257) -0.735** (0.290) vign3==4 -0.852*** (0.263) -0.754** (0.295) vign3==5 -0.844*** (0.270) -0.742** (0.301) vign3==6 -1.505*** (0.401) -1.631*** (0.457) Vignette 4 (positions 1 to 3 omitted) vign4==4 0.735*** (0.245) 0.630** (0.281) vign4==5 0.794*** (0.247) 0.594** (0.282) vign4==6 0.793*** (0.251) 0.582** (0.285) Number of observations 4,771 4,771 4,771 4,771 Pseudo R2 0.037 0.055 0.080 0.099 Note: *** p<0.01, ** p<0.05, * p<0.1 21 Table 10: Post-vignette self-assessed subjective welfare positions (1) (2) (3) (4) PCE + PCE + PCE + household PCE household vignettes controls + controls vignettes Coef. se Coef. se Coef. se Coef. se Log expenditure per capita 0.560*** (0.037) 0.573*** (0.037) 0.652*** (0.041) 0.673*** 0.042 Household Demographics Female headed household -0.178*** (0.052) -0.173*** (0.053) Age of household head -0.005 (0.009) -0.008 (0.009) Age of household head squared 0.000 (0.000) 0.000 (0.000) Household size 0.060*** (0.013) 0.068*** (0.013) Number of children 0.008 (0.017) 0.002 (0.018) Number of older adults -0.039 (0.046) -0.053 (0.046) Number of migrants 0.070 (0.043) 0.054 (0.043) Ethnicity (Reference: Tajik) Uzbek 0.064 (0.046) 0.070 (0.046) Russian -0.412** (0.161) -0.435*** (0.166) Other -0.650*** (0.159) -0.664*** (0.149) Education (Reference: No Education) Primary -0.013 (0.113) -0.112 (0.115) Basic -0.184* (0.109) -0.220** (0.111) General Secondary -0.086 (0.105) -0.129 (0.107) Special Secondary -0.021 (0.113) -0.072 (0.114) Technical Secondary -0.067 (0.115) -0.123 (0.116) Higher Education 0.189* (0.112) 0.157 (0.114) Graduate School 0.511 (0.399) 0.612* (0.368) Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed 0.013 (0.018) 0.007 (0.019) Agriculture / Fishing / Forestry 0.063 (0.063) 0.056 (0.064) Manufacture / Mining 0.054 (0.092) 0.082 (0.095) Services 0.447*** (0.120) 0.391*** (0.118) Construction -0.032 (0.077) -0.071 (0.077) Public Administration / Education / Health 0.094 (0.067) 0.100 (0.067) Sales and Services 0.236*** (0.063) 0.241*** (0.063) Other -0.142 (0.108) -0.141 (0.107) Agriculture (Reference for holdings: No Land) Small Holding -0.039 (0.064) 0.006 (0.066) Medium Holding 0.055 (0.078) 0.089 (0.080) Large Holding 0.088 (0.077 0.138* (0.078) Any livestock, poultry, etc -0.017 (0.056) -0.021 (0.057) Geography (Reference: Dushanbe) Sogd Urban -0.016 (0.072) -0.087 (0.072) Sogd Rural -0.090 (0.078) -0.170** (0.079) Khatlon Urban -0.092 (0.089) -0.334*** (0.088) Khatlon Rural -0.471*** (0.073) -0.560*** (0.074) RRP Urban -0.286*** (0.107) -0.343*** (0.108) RRP Rural -0.243*** (0.076) -0.306*** (0.077) Gbao Urban 0.571*** (0.115) 0.657*** (0.120) 22 (1) (2) (3) (4) PCE + PCE + PCE + household PCE household vignettes controls + controls vignettes Coef. se Coef. se Coef. se Coef. se Gbao Rural 0.036 (0.079) 0.006 (0.080) Vignette 1 (positions 3 to 6 omitted) vign1==1 -0.385*** (0.126) -0.404*** (0.135) vign1==2 -0.184 (0.120) -0.231* (0.129) Vignette 2 (position 1 omitted) vign2==2 0.940*** (0.193) 0.991*** (0.203) vign2==3 1.345*** (0.199) 1.449*** (0.209) vign2==4 1.525*** (0.230) 1.693*** (0.240) vign2==5 2.842*** (0.363) 2.882*** (0.314) Vignette 3 (positions 1 and 2 omitted) vign3==3 -0.115 (0.243) -0.045 (0.281) vign3==4 -0.265 (0.249) -0.157 (0.287) vign3==5 -0.160 (0.257) -0.056 (0.294) vign3==6 -0.636 (0.431) -0.698* (0.423) Vignette 4 (positions 1 to 3 omitted) vign4==4 0.666** (0.265) 0.584** (0.287) vign4==5 0.667** (0.268) 0.510* (0.290) vign4==6 0.652** (0.271) 0.492* (0.293) Number of observations 4,771 4,771 4,771 4,771 Pseudo R2 0.029 0.057 0.061 0.092 note: *** p<0.01, ** p<0.05, * p<0.1 23 Table 11: Rescaled SW responses using Post Vignettes (1) (2) (3) (4) PCE + PCE + PCE + household PCE household vignettes controls + controls vignettes Coef. se Coef. se Coef. se Coef. se Log expenditure per capita 0.466*** 0.034 0.559*** 0.035 0.553*** 0.039 0.646*** 0.040 Household Demographics Female headed household -0.157*** 0.052 -0.163*** 0.052 Age of household head -0.011 0.009 -0.009 0.009 Age of household head squared 0.000 0.000 0.000 0.000 Household size 0.062*** 0.013 0.063*** 0.013 Number of children -0.003 0.017 0.004 0.017 Number of older adults -0.047 0.045 -0.034 0.046 Number of migrants 0.051* 0.028 0.067** 0.028 Ethnicity (Reference: Tajik) Uzbek 0.032 0.045 0.067 0.046 Russian -0.422*** 0.161 -0.431*** 0.156 Other -0.488*** 0.124 -0.613*** 0.145 Education (Reference: No Education) Primary -0.208* 0.113 -0.086 0.113 Basic -0.218** 0.110 -0.184* 0.109 General Secondary -0.155 0.105 -0.112 0.105 Special Secondary -0.147 0.112 -0.083 0.113 Technical Secondary -0.154 0.116 -0.089 0.116 Higher Education 0.091 0.113 0.162 0.113 Graduate School 0.602** 0.257 0.468* 0.253 Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed -0.012 0.018 0.007 0.018 Agriculture / Fishing / Forestry 0.057 0.065 0.041 0.066 Manufacture / Mining 0.034 0.088 0.034 0.090 Services 0.333** 0.133 0.404*** 0.124 Construction -0.095 0.076 -0.070 0.078 Public Administration / Education / Health 0.097 0.067 0.108 0.067 Sales and Services 0.201*** 0.062 0.229*** 0.063 Other -0.084 0.103 -0.142 0.114 Agriculture (Reference for holdings: No Land) Small Holding 0.055 0.064 -0.006 0.066 Medium Holding 0.113 0.080 0.071 0.080 Large Holding 0.135* 0.077 0.133* 0.079 Any livestock, poultry, etc -0.004 0.055 -0.005 0.058 Geography (Reference: Dushanbe) Sogd Urban -0.099 0.067 -0.086 0.070 Sogd Rural -0.150* 0.077 -0.200** 0.078 Khatlon Urban -0.497*** 0.085 -0.287*** 0.089 Khatlon Rural -0.525*** 0.072 -0.527*** 0.072 RRP Urban -0.242** 0.108 -0.316*** 0.109 24 (1) (2) (3) (4) PCE + PCE + PCE + household PCE household vignettes controls + controls vignettes Coef. se Coef. se Coef. se Coef. se RRP Rural -0.210*** 0.075 -0.263*** 0.076 Gbao Urban 0.370*** 0.098 0.439*** 0.097 Gbao Rural -0.079 0.076 0.003 0.079 Vignette 1 (positions 3 to 6 omitted) vign1==1 0.139 0.134 0.161 0.145 vign1==2 0.168 0.132 0.152 0.144 Vignette 2 (position 1 omitted) vign2==2 0.497*** 0.179 0.498** 0.198 Vign2==3 0.207 0.182 0.226 0.201 Vign2==4 -0.353* 0.205 -0.311 0.225 Vign2==5 -0.075 0.433 -0.195 0.416 Vignette 3 (positions 1 and 2 omitted) Vign3==3 -1.143*** 0.290 -1.137*** 0.314 Vign3==4 -1.739*** 0.293 -1.722*** 0.318 Vign3==5 -1.778*** 0.297 -1.772*** 0.321 Vign3==6 -2.110*** 0.437 -2.258*** 0.446 Vignette 4 (positions 1 to 3 omitted) Vign4==4 -0.112 0.388 -0.212 0.399 Vign4==5 -0.137 0.390 -0.309 0.402 Vign4==6 -0.172 0.392 -0.351 0.403 Number of observations 4,771 4,771 4,771 4,771 2 Pseudo R 0.017 0.075 0.042 0.101 Note: *** p<0.01, ** p<0.05, * p<0.1 25 Figure 2 Figure 1 Figure 3 Figure 4 26 Figure 5 Figure 6 Figure 7 27 Appendix 1: Summary statistics (weighted) Variable mean sd Log expenditure per capita (Tajik somoni) 5.0 0.54 Household Demographics Female-headed household 0.19 Age of household head 50.6 13.74 Household size 6.3 2.80 Number of children (<15) 2.2 1.69 Number of older adults (65+) 0.3 0.57 Number of migrants 0.3 0.68 Ethnicity Tajik 0.79 Uzbek 0.18 Russian 0.01 Other 0.02 Education of household head No education 0.04 Primary (grades 1-4) 0.07 Basic (grades 1-8) 0.12 Secondary general (grades 9-10) 0.34 Secondary special 0.13 Secondary technical 0.11 Higher education 0.19 Graduate school/aspirantura 0.002 Employment characteristics of Household, Head's occupation Number of employed 1.76 1.33 Not employed 0.37 Agriculture, fishing and forestry 0.18 Manufacture and mining 0.04 Services (electricity, gas, hot water, etc.) 0.02 Construction 0.07 Public administration, education, health 0.13 Sales and services 0.16 Other 0.03 Agriculture No land holdings 0.37 Small holding (1-10 acres) 0.29 Medium holding (11-20 acres) 0.15 Large holding (21+ acres) 0.20 Any livestock, poultry, beehives, fish etc. 0.55 28 Appendix 2: The vignettes in the TLSMS Vignette 1: Family A can only afford to eat meat on very special occasions. During the winter months, they are able to partially heat only one room of their home. They cannot afford for children to complete their secondary education because the children must work to help support the family. When the children are able to attend school, they must go in old clothing and worn shoes. There is not enough warm clothing for the family during cold months. The family does not own any farmland, only their household vegetable plot. Vignette 2: Family B can afford to eat meat only once or twice a week. During winter months, they can heat several rooms, but not the entire house. They cannot afford for all their children to complete secondary education. Their clothing is sufficiently warm, but they own only simple garments. In addition to their household vegetable plot, they own a small plot of poor quality farmland that is distant from their home. Vignette 3: Family C can afford to eat meat everyday. During the winter months, generally they are able to keep their home warm. They can afford for all their children to complete secondary education. They have sufficient clothing to keep warm in the winter. Their everyday clothing is simple, but they also have some fancy items for special occasions. In addition to their household vegetable plot, they have a larger plot of good quality farmland, not too distant from their home. Vignette 4: Family D can afford to eat whichever foods they would like, including sweets and imported food. During the winter months, they have no problems with heating and are able to keep their entire house warm. They can afford for all of their children to complete their education, and then to continue at a local university. They are able to afford a variety of fancy traditional clothes and also imported brand clothing. The family owns property, including a good car. The family also has a large farm and acts as landlord to others in their area. 29 Appendix 3: Probit for perversely-ordered vignettes Coeff. s.e. Log expenditure per capita 0.045 0.092 Household Head Demographics Female 0.190 0.141 Age -0.028 0.025 Age squared 0.000 0.000 Ethnicity (Reference: Tajik) Uzbek -0.227 0.155 Russian 0.184 0.439 Other 0.629** 0.289 Education (Reference: No Education) Primary -0.368 0.244 Basic -0.536** 0.266 General Secondary -0.561** 0.235 Special Secondary -0.771** 0.309 Technical Secondary -0.356 0.265 Higher Education -0.800*** 0.291 Household Characteristics Household size -0.017 0.052 Number of children (<15) -0.028 0.062 Number of elderly (65+) -0.278** 0.133 Number of migrants -0.071 0.086 Head's Employment (Reference for Occupation: Unemployed) Number of employed 0.086 0.060 Agriculture / Fishing / Forestry 0.065 0.187 Manufacture / Mining -0.509 0.363 Services 0.314 0.306 Construction 0.467** 0.219 Public Administration / Education / Health 0.078 0.200 Sales and Services 0.265 0.190 Other 0.055 0.323 Ownership of Land / Livestock Small Holding 0.174 0.191 Medium Holding 0.153 0.243 Large Holding 0.449** 0.214 Any livestock, poultry, etc. 0.214 0.166 Geography Sogd Urban 0.219 0.225 Sogd Rural -0.389 0.262 Khatlon Urban -0.393 0.317 Khatlon Rural -0.348 0.271 RRP Urban 0.507** 0.248 RRP Rural -0.396 0.256 Gbao Urban -0.030 0.450 Gbao Rural 0.348 0.252 Relative Subjective Welfare Neighbor's Step -0.247*** 0.074 Number of observations 4,852 Pseudo R2 0.133 30 Appendix 4: Vignette Regressions (Ordered Probit) Vignette 1 (1) (2) PCE + household PCE characteristics Coef. se Coef. se Log expenditure per capita 0.008 0.038 0.044 0.043 Household Demographics Female headed household 0.047 0.059 Age of household head -0.009 0.011 Age of household head squared 0.000 0.000 Household size 0.030** 0.014 Number of children -0.017 0.020 Number of older adults -0.073 0.054 Number of migrants -0.006 0.032 Ethnicity (Reference: Tajik) Uzbek -0.074 0.052 Russian 0.041 0.166 Other 0.146 0.151 Education (Reference: No Education) Primary -0.072 0.121 Basic -0.207* 0.112 General Secondary -0.023 0.109 Special Secondary 0.018 0.119 Technical Secondary 0.037 0.123 Higher Education -0.020 0.118 Graduate School -0.098 0.458 Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed -0.020 0.021 Agriculture / Fishing / Forestry 0.074 0.074 Manufacture / Mining 0.101 0.103 Services 0.341** 0.173 Construction 0.046 0.095 Public Administration / Education / Health 0.021 0.082 Sales and Services -0.009 0.072 Other 0.077 0.116 Agriculture (Reference for holdings: No Land) Small Holding 0.168** 0.072 Medium Holding 0.086 0.092 Large Holding 0.057 0.089 Any livestock, poultry, etc 0.039 0.062 Geography (Reference: Dushanbe) Sogd Urban 0.095 0.082 Sogd Rural 0.009 0.087 Khatlon Urban -0.670*** 0.133 Khatlon Rural -0.115 0.083 RRP Urban 0.101 0.120 RRP Rural 0.132 0.080 Gbao Urban -0.121 0.123 Gbao Rural -0.281*** 0.088 Number of observations 4,771 4,771 Pseudo R2 0.000 0.022 31 Vignette 2 (1) (2) PCE + household PCE characteristics Coef. se Coef. se Log expenditure per capita 0.057 0.036 0.047 0.059 Household Demographics Female headed household -0.024 0.056 Age of household head 0.009 0.011 Age of household head squared -0.000 0.000 Household size -0.018 0.014 Number of children 0.023 0.019 Number of older adults 0.042 0.053 Number of migrants 0.037 0.031 Ethnicity (Reference: Tajik) Uzbek 0.006 0.050 Russian 0.063 0.165 Other -0.072 0.147 Education (Reference: No Education) Primary 0.093 0.113 Basic 0.154 0.108 General Secondary 0.182 0.116 Special Secondary 0.216* 0.119 Technical Secondary 0.152 0.116 Higher Education -0.330 0.596 Graduate School -0.024 0.056 Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed 0.041** 0.020 Agriculture / Fishing / Forestry -0.032 0.070 Manufacture / Mining -0.051 0.094 Services 0.223 0.139 Construction 0.110 0.087 Public Administration / Education / Health 0.021 -0.017 Sales and Services 0.000 0.067 Other -0.113 0.111 Agriculture (Reference for holdings: No Land) Small Holding -0.164** 0.066 Medium Holding -0.137 0.084 Large Holding -0.124 0.084 Any livestock, poultry, etc 0.013 0.059 Geography (Reference: Dushanbe) Sogd Urban 0.232*** 0.077 Sogd Rural 0.164** 0.083 Khatlon Urban 0.743*** 0.110 Khatlon Rural 0.263*** 0.081 RRP Urban 0.070 0.105 RRP Rural 0.067 0.077 Gbao Urban -0.269* 0.139 Gbao Rural 0.175** 0.085 Number of observations 4,771 4,771 Pseudo R2 0.000 0.018 32 Vignette 3 (1) (2) PCE + household PCE characteristics Coef. se Coef. se Log expenditure per capita 0.092*** 0.035 0.070* 0.039 Household Demographics Female headed household 0.043 0.053 Age of household head 0.012 0.010 Age of household head squared -0.000 0.000 Household size -0.017 0.013 Number of children 0.029 0.018 Number of older adults 0.048 0.052 Number of migrants 0.021 0.029 Ethnicity (Reference: Tajik) Uzbek 0.089* 0.047 Russian 0.035 0.148 Other -0.177 0.143 Education (Reference: No Education) Primary 0.183* 0.110 Basic 0.126 0.107 General Secondary 0.191* 0.114 Special Secondary 0.165 0.118 Technical Secondary 0.128 0.114 Higher Education -0.429 0.682 Graduate School 0.043 0.053 Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed 0.036* 0.019 Agriculture / Fishing / Forestry -0.068 0.066 Manufacture / Mining -0.028 0.089 Services 0.108 0.149 Construction 0.055 0.084 Public Administration / Education / Health 0.021 -0.025 Sales and Services -0.019 0.062 Other -0.021 0.119 Agriculture (Reference for holdings: No Land) Small Holding -0.114* 0.066 Medium Holding -0.091 0.080 Large Holding 0.035 0.081 Any livestock, poultry, etc 0.005 0.056 Geography (Reference: Dushanbe) Sogd Urban -0.050 0.072 Sogd Rural -0.173** 0.082 Khatlon Urban 0.466*** 0.098 Khatlon Rural 0.006 0.078 RRP Urban -0.127 0.111 RRP Rural -0.166** 0.075 Gbao Urban 0.372*** 0.138 Gbao Rural 0.220*** 0.083 Number of observations 4,771 4,771 Pseudo R2 0.001 0.016 33 Vignette 4 (1) (2) PCE + household PCE characteristics Coef. se Coef. se Log expenditure per capita 0.119*** 0.033 0.083** 0.037 Household Demographics Female headed household 0.043 0.053 Age of household head 0.002 0.011 Age of household head squared -0.000 0.000 Household size -0.016 0.013 Number of children 0.027 0.018 Number of older adults 0.064 0.052 Number of migrants 0.012 0.028 Ethnicity (Reference: Tajik) Uzbek 0.078* 0.047 Russian 0.099 0.154 Other -0.162 0.162 Education (Reference: No Education) Primary 0.089 0.122 Basic 0.163 0.119 General Secondary 0.194 0.126 Special Secondary 0.200 0.129 Technical Secondary 0.123 0.125 Higher Education -0.325 0.452 Graduate School 0.043 0.053 Employment Characteristics of Household, Head's Occupation (Reference for Occupation: Unemployed) Number of employed 0.079*** 0.020 Agriculture / Fishing / Forestry -0.118* 0.066 Manufacture / Mining 0.032 0.087 Services -0.191 0.152 Construction -0.006 0.082 Public Administration / Education / Health 0.021 0.002 Sales and Services -0.047 0.062 Other -0.002 0.117 Agriculture (Reference for holdings: No Land) Small Holding -0.156** 0.067 Medium Holding -0.165** 0.082 Large Holding -0.031 0.082 Any livestock, poultry, etc 0.030 0.059 Geography (Reference: Dushanbe) Sogd Urban -0.301*** 0.068 Sogd Rural -0.384*** 0.079 Khatlon Urban 0.203* 0.104 Khatlon Rural -0.219*** 0.077 RRP Urban -0.247** 0.112 RRP Rural -0.340*** 0.072 Gbao Urban 0.140 0.131 Gbao Rural 0.140* 0.084 Number of observations 4,771 4,771 Pseudo R2 0.002 0.019 34 References Angoff, William H. 1993. "Perspectives on Differential Item Functioning Methodology." in Paul Holland and Howard Wainer (eds) Differential Item Functioning, Lawrence Erlbaum Associates. Bago d'Uva, Teresa, Eddy Van Doorslaer, Maarten Lindeboom and Own O'Donnell. 2008. "Does Reporting Heterogeneity Bias the Measurement of Health Disparities?" Health Economics 17(3): 351-75. Cantril, H. 1965. The Pattern of Human Concern. New Brunswick: Rutgers University Press. Clark, Andrew E., and Andrew J. Oswald. 1994. "Unhappiness and Unemployment." Economic Journal 104 (424): 648­59. ______________ and ______________ 1996. "Satisfaction and Comparison Income," Journal of Public Economics 61: 359-381. Conti, Gabriella and Stephen Pudney. 2008. "If You're Happy and You Know it, Clap Your Hands! Survey Design and the Analysis of Satisfaction." Institute for Social and Economic Research Working Paper 2008-39. Das, Jishnu, Jeffrey Hammer and Kenneth Leonard. 2008. "The Quality of Medical Advice in Low-Income Countries." Journal of Economic Perspectives 22(2): 93-114. Diener, Ed, Eunkook Suh, Richard E. Lucas and Heifi L. Smith. 1999. "Subjective Well- Being: Three Decades of Progress." Psychological Bulletin 125: 276-302. Di Tella, Rafael and Robert MacCulloch. 2006. "Some Uses of Happiness Data in Economics." Journal of Economic Perspectives 20(1): 25-46. Easterlin, Richard A. 1995. "Will Raising the Incomes of all Increase the Happiness of all?" Journal of Economic Behavior and Organization 27: 35-47. Ferrer-i-Carbonell A. 2005. "Income and Well-Being: An Empirical Analysis of the Comparison Income Effect," Journal of Public Economics, 89: 997-1019. Fafchamps, Marcel and Forhad Shilpi, F., 2009. "Isolation and Subjective Welfare: Evidence from South Asia," Economic Development and Cultural Change 57(4): 641-83. Frank, Robert H. 1997. "The Frame of Reference as a Public Good," Economic Journal 107: 1832-1847. Frey, Bruno and Alois Stutzer. 2002. "What Can Economists Learn from Happiness Research?" Journal of Economic Literature XL: 402­435. 35 Furnham, Adrian and Michael Argyle. 1998. The Psychology of Money, London: Routledge. Hirshman, Albert O. 1973. "The Changing Tolerance for Income Inequality in the Course of Economic Development, With a Mathematical Appendix by Michael Rothschild", Quarterly Journal of Economics 87: 544-566. Kapteyn, Arie, Peter Kooreman and Rob Willemse. 1998. "Some Methodological Issues in the Implementation of Subjective Poverty Definitions." Journal of Human Resources 23(2): 222- 242. Kapteyn, Arie, James P. Smith and Arthur Van Soest. 2008. "Comparing Life Satisfaction." Working Paper WR-623, Rand Coopoeration. King, Gary and Jonathan Wand. 2007. "Comparing Incomparable Survey Responses: Evaluating and Selecting Anchoring Vignettes." Political Analysis 15(1): 46-66. King, Gary, Christoper J.L. Murray, Joshua A. Salomon and Ajay Tandon. 2004. "Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research." American Political Science Review 98(1): 191-207. Kristensen, Nicolai and Edvard Johansson. 2008. "New Evidence on Cross-Country Differences in Job Satisfaction Using Vignettes." Labor Economics 15(1): 96-117. Lokshin, Michael. 2006. "Difference-Based Semiparametric Estimation of Partial Linear Regression Models." STATA Journal 6(3): 377-384. Luttmer, Erzo. 2005. "Neighbors as Negatives: Relative Earnings and Well-Being." Quarterly Journal of Economics 120(3): 963-1002. McBride, M. 2001. "Relative Income Effects on Subjective Well-Being in the Cross-Section." Journal of Economic Behavior and Organization 45: 251-278. Oswald, Andrew J. 1997. "Happiness and Economic Performance." Economic Journal 107: 1815-1831. Pradhan, Menno and Martin Ravallion. 2000. "Measuring Poverty Using Qualitative Perceptions of Consumption Adequacy." Review of Economics and Statistics 82(3): 462-471 Ravallion, Martin and Michael Lokshin. 2000. "Who Wants to Redistribute? The Tunnel Effect in 1990s Russia," Journal of Public Economics 76(1): 87-104. ______________ and ______________. 2001. "Identifying Welfare Effects from Subjective Questions." Economica 68(271): 335-357 ______________ and ______________. 2002. "Self-rated Economic Welfare in Russia." European Economic Review 46(8): 1453-1473. 36 _____________ and ____________. 2009. "Who Cares About Relative Deprivation?" Journal of Economic Behavior and Organization, in press. Seidl, Christian. 1994. "How Sensible is the Leyden Individual Welfare Function of Income?", European Economic Review, 38: 1633-1659. Senik, Claudia. 2004. "When Information Dominates Comparison: Learning from Russian Subjective Panel Data," Journal of Public Economics 88: 20099-2123. van de Stadt, Huib, Arie Kapteyn, and Sara van de Geer. 1985. "The Relativity of Utility: Evidence from Panel Data," Review of Economics and Statistics 67: 179-187. Van Praag, Bernard M.S. 1968. Individual Welfare Functions and Consumer Behavior, Amsterdam: North-Holland. Van Praag, Bernard M.S. and Arie Kapteyn. 1994. "How Sensible Is the Leyden Individual Welfare Function of Income? A Reply." European Economic Review 38(9): 1817-25. Veenhoven, Ruut, Joop Ehrhardt, Monica Sie Dhian Ho and Astrid de Vries. 1993. Happiness in Nations: Subjective Perceptions of Life in 56 Nations 1946-1992, Erasmus University, Rotterdam. Winkelmann, Liliana and Rainer Winkelmann. 1998. "Why Are the Unemployed So Unhappy: Evidence from Panel Data." Economica 65: 1-15. World Bank. 2008. "Basic Information Document: 2007 Tajikistan Living Standards Measurement Study." Mimeo, World Bank, Washington, DC. 37