WPS3711 An Econometric Method of Correcting for Unit Nonresponse Bias in Surveys Anton Korinek, Johan A. Mistiaen, and Martin Ravallion1 Development Research Group, World Bank 1818 H Street NW, Washington DC, USA Abstract: Past approaches to correcting for unit nonresponse in sample surveys by re-weighting the data assume that the problem is ignorable within arbitrary subgroups of the population. Theory and evidence suggest that this assumption is unlikely to hold, and that household characteristics such as income systematically affect survey compliance. We show that this leaves a bias in the re-weighted data and we propose a method of correcting for this bias. The geographic structure of nonresponse rates allows us to identify a micro compliance function, which is then used to re-weight the unit-record data. An example is given for the US Current Population Surveys, 1998 ­ 2004. We find, and correct for, a strong household income effect on response probabilities. Keywords: Sample surveys, selective unit nonresponse bias. JEL: C42, D31, D63, I3 World Bank Policy Research Working Paper 3711, September 2005 The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org. 1 Helpful comments on the paper were received from Francesco Brindisi, Phoebus Dhrymes and the journal's editor and anonymous referees. Anton Korinek gratefully acknowledges financial support from the Austrian Academy of Sciences, DOC Fellowship. 1. Introduction This paper considers the potential bias that can occur when some portion of the sampled population does not respond to a sample survey. If the decision to respond is statistically dependent on the variables under investigation then the sub-sample of survey respondents will not accurately reflect the true distribution of the variables of interest in the population and this will in turn result in systematically biased sample-based inferences, even in large samples. Survey noncompliance is manifested either as "item" nonresponse -- while participating in the survey, the respondent does not answer some question(s) -- or as "unit" nonresponse, when a sampled respondent does not participate in the survey at all, either because of a failure to establish contact or explicit refusal to participate. The paper develops an ex-post approach to correcting for selective unit nonresponse bias in surveys. Well-designed surveys aim to minimize nonresponse ex-ante (i.e., prior to field implementation) by carefully selecting the most appropriate interview medium (e.g., print/electronic mailings, in-person or telephone calls) in combination with additional preventive approaches (e.g., personalization or organizational endorsement of the survey, reward based incentives, and careful training of interviewers) in addition to monitored call-backs or follow-up requests.2 However, in most surveys a non-negligible fraction of designated respondents still fail to provide all the requested data items or fail to respond altogether.3 Dealing with item nonresponse is facilitated by the fact that some information about the units who did not respond 2 Moser and Kalton (1972) provide an insightful overview. On rewards and monetary incentives, see for instance Philipson (1997). And, as noted early by Deming (1953), depending on the inference variable of interest, accounting for the frequency of call-backs and follow-up requests could be equally relevant to correct for potential biases as the ultimate incidence of nonresponse. 3 Nonresponse rates in income surveys, for example, can range from virtually zero to around 30 percent (Holt and Elliot, 1991; Scott and Steele, 2004). In Internet surveys, nonresponse rates are often close to 100%. 2 to a certain question was collected in the survey.4 However, correcting for unit nonresponse requires that some structure is imposed on the set of nonrespondents without observing a single requested variable in the survey. One approach, sometimes termed an "identification study," aims to assess how the likelihood of response is affected by certain variables, e.g., by investigating how the response rate varies across subgroups of the sample or in relation to certain auxiliary data. However, this requires knowledge about the size of these subgroups or the distribution of the auxiliary data in the total population. Hence, identification studies are best applied when the sample is chosen from a population about which some characteristics are known; examples include employees of a given set of companies (as in Gannon et al., 1971) or students of a given set of schools (Kalsbeek et al., 1974). Implicitly, identification studies assume that within a certain subgroup, or given certain auxiliary data, the decision to respond is independent of the measured variable. Another imputation technique involves substitution of nonresponding units, which is employed when the number of observations in the sample has to be kept constant regardless of survey nonresponse (Hansen and Hurwitz, 1946). Typically, another unit from the same sampling subclass as the initially designated unit substitutes for the nonrespondent. Again, this assumes implicitly that within a subclass, the decision to respond is independent of the measured variable; see the discussion in Chapman (1983). Alternatives to the imputation methods discussed above are found in the literature on adjustment procedures and model-based methods to correct for nonresponse. The common 4 The most common way of correcting for this type of nonresponse is explicit imputation, whereby an imputed value is assigned to the missing item based on the recorded values for other items. This imputed value is usually taken from another surveyed unit that has responded and that resembles the unit with missing data as closely as possible, such as determined by a score estimated on commonly observed variables. For a general discussions of this approach see Kalton and Kasprzyk (1982) and Little and Rubin (1987). 3 approach is to determine a weighting factor for each observed individual that adjusts the sample for nonresponse. Various methods for determining these weighting factors have been suggested in the literature. One proposal has been to infer the weights on the basis of the time or number of solicitation attempts required to respond (Politz and Simmons, 1949). An alternative method infers the weights from the distribution of nonrespondents across certain identifiable subgroups of the sample, called "adjustment cells" (Thomsen, 1973). External data sources, such as a population census, have also been employed to determine the number of units in the various subgroups of the population (Hansen et al., 1953). Again, such methods assume implicitly that the decision to respond and the variables of interest are independent within each subgroup. Our contribution in this paper is to present a new method of correcting for unit nonresponse in surveys samples with multiple strata that does not rely on any additional information. Our method follows the classical view of nonresponse, in that we assume that the variables in the total population are fixed values.5 Our approach falls in the category of adjustment procedures that generate weighting factors for all individuals in order to correct for nonresponse. However, our method is in marked contrast with those methods that assume that the decision to respond is independent of other variables within subgroups of the sample, which we will call the ignorability assumption. As we will show below, this assumption entails almost always an under-correction for nonresponse bias. Furthermore, the assumption is at odds with both the predictions of theoretical models of the decision to respond to a survey (Korinek et al., 2005) and with the (limited) amount of evidence available on unit nonresponse. For example, 5 The alternative to the classic approach is based on the assumption that the variables of interest in a population as well as the decision to respond are the realizations of random variables that follow a given stochastic process. This is sometimes called the stochastic view of nonresponse. The parameters of the assumed stochastic model can be estimated using the data from all observed units, and can be used to make inferences about the statistical properties of the total population. Examples of this approach are Rubin (1977), which employs a Bayesian approach, or Cassel et al. (1983). These approaches generally use an auxiliary data set in order to make inferences about the statistical properties of nonrespondents. 4 Groves and Couper (1998, Chapter 5) report evidence based on compliance with the long schedule of the U.S. Census (administered to a random sample) indicating that compliance tends to fall with individual income. By our proposed method the assessed probability of nonresponse varies with the characteristics of each sampled unit, even within the smallest observable subgroup. Our method has two main advantages. Firstly, it does not assume that within the smallest defined subgroups the decision to respond is independent of the variables of interest (i.e. we allow the probability of nonresponse to differ for every single individual with different characteristics). Secondly, our method relies solely on data from the survey that is to be corrected and does not require any external data sources or repeated survey; in particular, it does not rely on information about the number of solicitation attempts until a given unit responds or on assumptions about how this information can be used to infer the characteristics of nonresponding units. Our method requires that all variables that systematically affect nonresponse are either observable for all respondents or are independent of the partitioning of the population into subgroups. While this is somewhat restrictive, it should be noted that the variables that are generally most thought of as systematically affecting the probability of response in a survey are often observable, such as income, age, gender, race, religion and urban location. It is also possible to include region-specific dummy variables in the specification of the probability of response, as long as the number of regional dummies is lower than the number of geographical areas that are identified in the survey. The following section outlines our estimation method in detail, while section 3 presents results using the Current Population Surveys for the US. Section 4 concludes. 5 2. Estimation method Survey data on non-responding households are by definition unobservable. However, survey response rates across geographical areas are observable. In this section, we develop a statistical model that allows us to estimate the survey response probability of participating households as a function of their observable data. By re-weighting the observed sample accordingly, we can impute these data for non-responding households. The proposed estimation method hinges on the assumption that the survey sample is representative of the population within each geographical subgroup. We define the population as a continuum H of households of mass M that can be partitioned into I non-overlapping groups Hi, where households within a given group are observationally identical and have a vector of characteristics Xi. Assume that the set H can also be geographically partitioned into J non-overlapping subsets Hj of mass Mj. The intersection of these two partitions can be denoted as a collection of mutually exclusive sets Hij = Hi Hj, each of weight Mij. From each of these J areas, a sample of households Sj Hj of mass mj < Mj was selected to collect survey data on the realizations of the vector X. The set of households with characteristics Xi in the sample of households Sj in area j is denoted by Sij Sj with corresponding mass mij. Since we aim to investigate only the effects of survey nonresponse and not of sample design, we assume that each of the J area samples Sj is statistically representative of Hj. A representative sample Sj of the area population is defined as one that comprises households of all I groups in area j and one for which the total weight mij of sampled households of each group i is proportional to Mij and thus, for a given area j, i mij = mj.6 6 Note that our definition here assumes that household characteristics are drawn from a discrete distribution, which also implies that at least one of the observed households has the maximum realization of the total population. This is clearly a counter-factual assumption, but as the sample size increases, the 6 For each sampled household Sij, there is a Bernoulli variable Dij with the realization Dij = 1 if the household responds to the survey and Dij = 0 in the case of unit nonresponse. We assume that these random variables are i.i.d. within each observationally identical group i of households and independent across groups. The probability that the household responds is denoted as: P Dij = 1 Xi, = Pi ( ) (1) where is an unknown parameter vector from a compact parameter space. Note that, consistent with the i.i.d. assumption on the random variables within an identical group of households, subscripts j and are superfluous on the right hand side of equation (1). We assume that the probability of a household to respond has a stable parametric form, for instance, a logistic function:7 P Dij = 1 Xi, = ( ) eXi (2) 1+ eXi Denote the mass of all respondents in group i and area j as the random variable mij [0,mij ]: 1 mij mij = Dij d 1 (3) 0 with an expected value of: E mij = mij Pi [ ] 1 (4) resulting bias tends towards zero. An alternative would be to assume that household characteristics are continuously distributed, and that S is a random sample of this distribution. However, this requires specifying the exact form of the distribution of characteristics, which is problematic given that the true distribution is unobservable. 7 The functional form used must be twice continuously differentiable in with outcomes bounded by the (0,1] interval. Thus, alternatively one could proceed on the basis a probit model, but this would complicate the estimation procedure. 7 The total mass mij of households in group i is unobservable ­ only mij can be observed. 1 In order to establish an estimation method, we divide (4) by the probability Pi so that: E mij 1 = mij (5) Pi The sum of all the fractions mij / Pi for a given j minus their expected value is given by: 1 1 () = mij1 1 = ij (6) j mP - mj i Pi - E mij Pi i i where mj, the total mass of sampled households in geographical area j, is observed. By the law of iterated expectations, the expected value E[j()] = 0. Thus, we can stack the moment conditions () for all geographical areas j into a vector () , which in turn allows us to j estimate the unknown parameter using a minimum distance estimator of the form: ^ = argmin()'W () -1 (7) This estimator is consistent for any positive definite weighting matrix W, providing three technical conditions are fulfilled. First, for the true , plim j() = 0 for all j. By (5) and the assumption that all individual realizations of Dij are independent, this follows from the strong law of large numbers. Second, the parameter space must be compact (by assumption). And finally, () converges in probability uniformly to a continuous function, and the minimum of that limiting function on is reached uniquely at the true parameter value (by assumption). The most efficient weighting matrix W is the covariance matrix of the vector () , or any matrix proportional to it (Hansen, 1982).8 The GMM approach to deriving this weighting 8 To be precise, the described estimator does not fall into the category of GMM estimators, since the variable mij in condition (5) is unobservable. We can thus only use the aggregates j() thereof. 8 matrix would be to calculate the sample covariances of all the individual moment conditions. However, since all mij are unobservable and only their area aggregates are known, we must 1 adopt an alternative procedure. By our assumption of independence of the response decisions of all households between the J areas, the off-diagonal elements of the covariance matrix will be zero, thus we can confine our attention to the diagonal elements. We assume that the variance of () for each state j is proportional to the mass of the sampled household population, with a j factor of proportionality 2, i.e., Var () = mj ( ) 2 (8) j This factor of proportionality, which can also be interpreted as the variance for a sample of weight one, can be estimated consistently as: ()2 ^2 = j m (9) j Since all the elements of our constructed variance-covariance matrix are scaled by 2, we can ignore the factor of proportionality in our optimization procedure and use the weighting matrix: m1 L 0 W = M O M (10) 0 L mJ so that the covariance matrix of () is simply 2W. Since () is twice continuously differentiable, the asymptotic covariance matrix of our proposed estimator ^ is given by: Var(^) = ^ 2 ()'W -1()-1 (11) However, by extension, the approach proposed by Hansen (1982) to determine the most efficient weighting matrix applies analogously here. 9 where, when using the logit model specified in (2), () 1 1 j = - ijPi = - ij Xi (12) mP 2 me Xi i i i We note there is an alternative approach to derive the variance-covariance matrix. Since all the individual Dij are observed Bernoulli variables, the variance of Dij is given by Var(Dij ) = Pi(1 ­ Pi) and thus: mij Var(mij) = Var(Dij )d = mij Pi(1-Pi) 1 (13) 0 The Var(j()) could then be determined as: Var j = Var ( ( )) i mij 1 = 1 Pi (14) Pi (Pi)2 1 i Varmij = mij 1-Pi ( ) 1 i However, because all Pi are initially unknown, this would require applying a two-step estimation procedure. First, we would assume P to be constant across all i's, and--as in the method outlined above--cancel the term (1 ­ Pi)/Pi to obtain a diagonal weighting matrix with the mass of responding households m1j along the diagonal. In a second step, we could then use the estimated to compute the value of the variances of (). However, this analytical expression is derived j solely from estimates of without taking into account the observed second moments of the data. We thus recommend use of equation (8) to estimate the variance rather than the two-step procedure based on (14). In comparing applications of the two approaches, the differences in estimates obtained using this theoretically derived weighting matrix were not significant. To further check for the robustness of equation (11), we also determined the standard deviation of our parameter estimates numerically using bootstrapping. As reported later in the paper, the resulting values were also of similar magnitude as our theoretical estimates from equation (11). 10 Finally, let us formally demonstrate that the ignorability assumption commonly employed in existing re-weighting methods to correct for unit nonresponse systematically underestimates the nonresponse bias under quite general assumptions: Proposition: If the probability to respond of a unit is a strictly monotonic function of a scalar or a vector of independent variables X of that unit, then the ignorability assumption biases estimates of X so as to underestimate the effects of nonresponse. (The proof is found in the Appendix.) This result motivated our concern to implement an econometric method of correcting for selective compliance in sample surveys that does not assume that the problem is ignorable within sub-groups. 3. Unit nonresponse bias in the US Current Population Survey Geographically referenced survey response rates are available for the US Current Population Survey (CPS) conducted annually between 1998 and 2004 (Census Bureau, 2002, Chapter 7).9 These surveys contain a record for each sampled household--i.e. for responding households as well as for "non-interview" households. The latter are distinguished by the reason for the non-interview into categories A, B, and C. Type B and C non-interviews refer to housing units that are vacant or that were demolished, i.e. these records do not represent household units in the sense of the CPS. Type A non-interviews comprise households that explicitly refused to be interviewed or that could not be interviewed because nobody was at home. In this application, we regard only type A households as non-responding and excluded type B and C observations from the data sets we use. The sample size and the number of non-responding 9 The CPS data and survey methodology details are available from the US Census Bureau and can be accessed on-line at: http://www.census.gov/hhes/www/income.html. 11 households in the CPS March Supplements from 1998 to 2004 are summarized in Table 1. The average nonresponse rate is about 8%. The CPS adjusts the initial household sampling weights to correct for various factors, including for nonresponse (described in Census Bureau, 2002, Chapter 10).10 In dealing with unit nonresponse, the CPS divides all sampled households into 254 adjustment cells. Generally, these consist of areas within the same metropolitan statistical area (MSA) or an MSA of similar size and within the same state. MSAs are further split into central and non-central city cells, and non-MSA areas are split into urban and rural cells. For each of these adjustment cells, the sampling weight of nonrespondents is re- distributed to the other households in the cell. In other words, the Census Bureau assumes that nonresponse is ignorable within adjustment cells. The Census Bureau acknowledges that this may not be valid and may lead to a nonresponse bias. As we have demonstrated in the previous section, the described adjustment procedure results in a nonresponse bias. Ideally we would observe the original CPS sampling weights net of corrections for nonresponse. Alas, the CPS data sets made available to the public provide only one weight (called "final weight") for each household, and that weight reflects various adjustments, including for nonresponse, sample design, and post-stratification. Thus, since we cannot disentangle the CPS adjustment for nonresponse from other adjustments, we cannot use the reported individual CPS household weights in our empirical analysis. Instead, we assign equal weights to every household within a state. According to the Census Bureau (2002), "most of the state samples in the CPS come close to being self-weighting," in other words, "...all units in [the] sample have the same probability of selection." This implies that our assumption of equal 10 For a critical assessment of the imputation methods used by the Census Bureau in correcting estimates for income nonresponse see Lillard, Smith and Welch (1986). 12 household weights within a state will not introduce a bias into our estimations. However, the variance of our state-level inferences will be higher by disregarding these Census Bureau final weights (i.e., our estimates will be somewhat less efficient). The March Supplement to the CPS is the source for official national estimates of income poverty rates and levels in the United Sates as well as the distribution of income (Census Bureau, 2004). Thus, in this application, we will focus on income selected unit nonresponse bias. In other words, we will determine whether the probability of response of households sampled in the CPS is a function of their per capita income levels. Using our econometric approach, we then examine the feasibility of correcting the income distribution for this bias. Table 2 reports the results of a naïve regression of the reported CPS weights on income, both with and without State-level fixed effects. We find that there is a significant positive relationship between per-capita income and the CPS weights, which incorporate the Census Bureau's own corrections for nonresponse as well as survey design effects. Thus the Census Bureau's correction methodology implicitly acknowledges that their uncorrected survey is biased towards under-representing higher incomes. The large difference between the OLS and State fixed effects regression coefficient on income indicates that the bulk of the current CPS correction is between States rather than within them, though there is still a significant income effect for the within estimator. Since the CPS was designed to be representative of the US State level, we can use the 51 States as the geographical areas in our estimation methodology indexed by j. It can be seen from Table 3 that in 2004, nonresponse rates varied from 3.4% in Alabama to 15.3% in the District of Columbia. 13 3.1 Specification for the compliance probability as a function of income To illustrate our estimation approach, we specify the following functional form: Pi = e f( yi ) (15) 1+ e f( yi ) where f(y) is a smooth parametric function and yi is the per-capita income of group i. In our data, the total number of groups varies between 30,618 in year 2001 and 43,896 in 2003, where a group comprises all households that report identical per capita income. Table 4 shows the joint Akaike Information Criterion (AIC) of our minimum distance estimates for various parametric specifications based on (15) for the years 1998 to 2004. In our specification tests, we included models with a constant, ln(y), ln(y)2, and y, as well as all possible combinations of up to three out of the four variables.11 For larger models, the estimated coefficients tended to be insignificant, since the number of geographic areas in our dataset is not sufficiently large. Including ln(y)3 or y2 in addition to one of the other variables than the constant caused problems of multi-collinearity, i.e. the variance-covariance matrix was near singular. The specification that best fits the nonresponse behavior exhibited by the data, i.e. that yields the lowest AIC, is specification 3, P = logit[1 + 2 ln(y)], which we thus use in most of our analysis in the rest of this paper. Since several of the specifications yielded an AIC that was very close to the -275.37 observed for specification 3, we plot the functional relationship between compliance and log income for the three best fitting specifications in Figure 1.12 As can 11 In order to apply logarithms to income, we excluded all observations with an income per capita of less than or equal 1 from our sample. In year 2004 for instance, this affected 755 observations. To check whether this changed our results, we also performed an estimation where we used ln(max(y,1)) instead of ln(y) and included all observations. This changed the estimated coefficients in all specifications by less than 2%. 12 The same observation holds if we include all specifications with an AIC below -250 in Table 4. However, the resulting graph becomes very clogged and is thus omitted here. 14 be seen, the resulting curves almost coincide, i.e. using a different specification does not have a significant effect on our corrections for nonresponse. The estimated parameters for the different specification are given in the respective rows of Table 5. To verify the robustness of our calculated standard errors, we also derived them numerically using bootstrapping. We randomly sampled 51 states with replacement from the given set of states and applied our estimator to this sample. After 500 repetitions of this process we calculated the bootstrapped standard errors as the average squared deviation of the bootstrapped estimates from the original estimate. For specification 3, they were 3.294 versus the theoretically derived 1.708 for 1 and 0.304 versus 0.155 for 2. These values are of similar magnitude, though the bootstrapping results are somewhat larger. This might be the case because both methods are limited by the fact that our dataset contains only 51 state observations. Furthermore, (11) is only an asymptotic result. To further investigate the sensitivity of our correction method to the exact choice of specification, we report estimation results for a number of other specifications, for which the AIC in Table 4 suggests that they explain the data well. The results can be found in Table 5 (for the 2004 CPS). Our interest here is whether the different specifications have significantly different implications for the distribution of income. As can be seen from the Gini coefficient, the choice of specification does not affect the correction of the distribution significantly: all corrected Gini coefficients are significantly higher than the uncorrected Gini coefficient of 44.80%, but within one standard deviation of each other, between 49.23% and 49.76%. It is of interest to see how much the estimated parameters vary over time. Table 6 gives illustrative results of estimating specification 3 for data from 1998 to 2004, and in the last line for a dataset that includes all households from 1998 to 2004 (with income chained to 1998 prices 15 using the regional CPI from the Bureau of Labor Statistics). The parameter estimates of the individual years are all close to each other, located within a 95% confidence-interval around the estimate obtained from bundling all years into one data set. From visual inspection of Table 6, there seems to be no systematic time trend in these parameters. However, when we tried to verify this proposition by estimating the specification, P = logit[1 + year*3 + (2 + year*4) ln(y)], allowing for a linear time trend in both the coefficient of income and the constant term, it turned out that both the estimates for 3 and 4 were significant. Also, the value of the AIC improves when adding any of the two additional parameters. The estimation results can be seen at the bottom of Table 9. According to these parameter estimates, survey response seems to be falling over time since 3 < 0, but the negative effect of income on nonresponse seems to be mildly declining, since 4 > 0. We tested the sensitivity of our results to making an allowance for geographic cost-of- living differences, on the presumption that real income should matter more for individuals' behavior. Ideally we would want to deflate each individual's income by an indicator of local consumer prices. Unfortunately, the Bureau of Labor Statistics does not publish data on consumer prices for the 51 states, but only for four regions (north-east, south, mid-west, and west) and for metropolitan statistical areas. Furthermore, the published series for these regions are consumer price indices rather than levels, i.e. they are chained to an average of the prices in the years 1982-84 of the respective area (rather than to a common denominator), and thus they only allow comparing prices within a given area over time, but not across areas. To check sensitivity to this data issue, we used the cost of living indicators of Friar and Leonard (1998), which are based on a publication of the Bureau of Labor Statistics (1981) comparing the cost of living of households across different states in that year. Consequently, they 16 inflated these indicators by the relative increase in the consumer price indices of the metropolitan statistical areas, which are contained in the respective states, and the respective regional CPI for rural areas, both of which are published by the Bureau of Labor Statistics. These indicators in Friar and Leonard (1998) refer to the year 1997. For all following years, we inflate the relative cost of living measure by the appropriate regional CPI and normalize the indicators so that the average across states is 1.00. The correction of incomes by these relative cost of living indicators does not significantly affect our estimation results. For a comparison of estimation results for 2004 see Table 7. We also investigated whether household characteristics other than income have additional explanatory power for survey compliance. This point is important because omitted variables might bias our results. Depending on which results we are interested in, we can differentiate between two kinds of biases. The first refers to the case when we are interested in estimating the exact functional form of the response function P(Dij |X , ). In such a situation, the omission of i any variables that are correlated with both the probability of response and Xi causes a bias in our estimate of . Suppose, for example, that a certain characteristic A is positively correlated with income and positively correlated with compliance. If A is not included in our estimations, then our estimate of the effect of income on compliance will be biased upwards (i.e. in absolute terms, the parameter will be biased downwards). However, the effect of this bias on the income distribution will be offset to the extent that the variation in A is captured by its correlation with income, so that we arrive at an unbiased estimate for the corrected income distribution. The second omitted variable bias is of importance when our object of interest is the income distribution itself. It arises when we omit a variable that is correlated with the probability to respond, but uncorrelated to the other variables we include. In this case, our parameter 17 estimate of for the included variables in the function P(Dij |X , ) is unbiased, but the corrected i income distribution is biased, since it does not reflect the impact of the omitted variable on response. In real world applications, it is likely that many omitted variables can be attributed in part to both of these categories. The additional household characteristics we considered were household size (hhsize), and dummy variables for whether the interviewed household is located in a metropolitan area (IMSA) and whether the household owned the house/apartment in which it lived (Ihomeowner). In addition, we included various characteristics of the household head, such as gender (dummy variable Ifemale), race (Icaucasian), employment status (dummy variables for Iworking and Iunemployed), education (measured by an index that indicates the years of schooling, i.e. edu; and alternatively by dummy variables for attaining different levels of education, of which attaining a graduate degree was most significant, i.e. Iedu master ), and age, which we use both as a level and squared. Our results are given in Table 8. The first observation we can make is that the estimated coefficient on income is highly robust to these changes. We found the included household characteristics, i.e. household size, metropolitan status and home ownership to be insignificant; this is also reflected in the AIC for these specifications. However, there are some characteristics of the household head that should be included according to the AIC: education, age, and age squared. The impact of the education dummy on survey nonresponse is strongly negative, but only significant at the 10% level. Since education and income are positively correlated, it can be expected that the omission of education in our estimations would bias the estimated coefficient on income upwards. The estimated coefficient on income in the regression that includes the education dummy is indeed somewhat higher, though not significantly so. 18 The effect of age is curious: in a linear specification where only the level of age is included, our estimate is insignificant and the AIC increases. However, if we include age as well as age squared in the specification, the coefficient on both becomes significant at the 5% level. According to our estimates, survey response is high for young people, then it decreases until people reach their mid-50s, after which it increases again. This might be in part explained by people's working pattern. We also estimated a specification that included both age variables and the education dummy. This yielded an even lower AIC, indicating a better fit with the data. However, our parameter estimate on income is not significantly changed. Arguably, this could be due to the low number of geographical areas in our sample, which results in higher standard errors and therefore a low power for the test of whether the coefficients change. Another set of variables that we included were regional dummy variables for the Census Bureau's four main regions of the US, the North East (1), the Mid-West (2), the South (3) and the West (4), where we drop the first variable to avoid multi-collinearity. Our estimation results show that location in the Mid-West significantly increases survey compliance, and the AIC increases markedly when we add a dummy variable for this region. Since many of the coefficients in the enhanced specifications of Table 8 yielded the expected sign but were insignificant, we combined all data from 1998 to 2004 ­ with income chained to 1998 prices ­ in one dataset in order to increase the significance of our results. The results of our estimations are presented in Table 9. Among the household variables, household size has a strongly significant negative effect on survey response. However, the inclusion of household size has little effect on our estimated 19 coefficient on income. As in our analysis of 2004 data, the dummy variable for metropolitan areas is insignificant. There are a large number of characteristics of the household head that have a significant impact on survey response. According to our estimates, both female and Caucasian household heads exhibit a lower probability of response than the general population. The same holds for unemployed household heads. Note that as before, the inclusion of these variables does not significantly affect the parameter estimate for income, even though the standard errors are smaller now. With the enlarged dataset, both the estimations using the education index and using the dummy for graduate studies yield significant parameters. Note that the education dummy also has a significant effect on the parameter estimate for income now. 3.2 Implications for the empirical distribution of income The implications for the empirical distribution of income will depend crucially on how the individual compliance probability varies with income. We saw in Figure 1 that compliance falls monotonically with income. In Korinek et al., (2005) we study the theoretical implications of this property for measures of inequality and poverty. Here we summarize the implications for the empirical distribution of income based on the 2004 CPS. The effect of correcting for selective compliance on the distribution of income per capita can be seen from Figures 2 ­ 4 (again using specification 3 for 2004 data). The uppermost (dotted) line in Figure 2 shows the uncorrected income distribution, i.e. the observed distribution if all individuals in a given state are assigned an equal weight, which consists of the population divided by the size of the sample in the given state. It can be seen that both the corrected CPS weights and our estimate for a corrected income distribution first order dominate the measured 20 distribution. For the CPS weights, this dominance seems to be particularly strong for relatively lower-income households. For our estimation methodology, the correction, and thus the first- order dominance, is stronger for higher income levels. Consequently, our correction method assigns comparatively less weight to lower income households and comparatively more weight to higher income households (roughly above an income of $70,000) than the Census Bureau's method. The results indicate that ignoring selective compliance according to income appreciably understates the proportion of the population in the richest income quantiles and slightly overstates the population shares in lower quantiles. What is observed as the highest income percentile in the survey, for example, is estimated to comprise 2.21% (+/­ 0.47%) of the population after correcting for its lower probability of survey compliance, and the highest observed decile actually makes up for 12.95% (+/­ 0.61%) of the population. By contrast, the poorest observed decile and percentile in the unadjusted data actually comprise only 9.34% (+/­ 0.04%) and 0.93% (+/­0.01%) respectively of the corrected population. The correction method of the Census Bureau, by contrast, assigns 1.60% and 15.74% of the population weight respectively to the top observed percentile and decile, and 6.95% and 0.88% to the bottom decile and percentile. Using our correction method, median income per person rises from an uncorrected $16,096 to $17,085, while the mean increases from an uncorrected $22,039 to $25,735 per capita. Using the weights provided by the Census Bureau, median income rises to $19,333, and mean income to $26,958. Figure 3 shows a magnification of the lower 25% of the distribution. It can be seen that using our correction method, the impact on poverty incidence is small for poverty lines 21 commonly used in the U.S., giving poverty rates around 12% (Census Bureau, 2001). However, since there is first-order dominance, poverty measures using the uncorrected, equally-weighted distribution of incomes unambiguously overestimate poverty. Note that the correction methodology of the Census Bureau leads to a significant underestimation in the estimated level of poverty according to our results. Figure 4 depicts the Lorenz curves for the uncorrected income distribution, the distribution according to the Census Bureau's weights, and according to our correction method. The effect of our correction for selective response is a marked downward shift in the Lorenz curve, implying higher inequality. However, there is not strict Lorenz dominance, with an intersection of the Lorenz curves for the corrected and uncorrected distributions occurring at the extreme upper end of the income range. Korinek et al., (2005) show that this intersection is a theoretical implication of a monotonic income effect on compliance. By inverting the CDF to obtain the quantile function for the original distribution we can calculate the income correction at each percentile of income that was observed in the raw survey. We do this for the correction implied by the Census Bureau weights, and the corrected distribution according to our method. The results are given in Figure 5. For the Census Bureau's correction, income at any given percentile shifts up almost uniformly by about 20%. This implies that the Census Bureau's correction method affects the national average, but is almost distribution neutral. For our method, the correction is quite low (around +2 to +3%) for the bottom 9 deciles and then rises sharply, to reach almost +100% for the uppermost percentile. Figure 6 depicts the weight correction of each observed income percentile. This figure reveals why the Census Bureau's correction method has almost no effect on inequality: their methodology heavily reduces the weights of low-income individuals (by almost 40% for some of 22 the bottom percentiles) and attributes this weight to the uppermost third of the income distribution. Our method, in contrast, reduces the weight of bottom four-fifth only by roughly 3%, and redistributes this weight to the top percentiles. The above results have been based on one specification of the compliance probability model, specification 3. In Korinek at el., (2005) we also report results for measures of inequality for the various alternative specifications discussed in section 3.1 and we show that the measures obtained are quite robust to the changes in model specification. 4. Conclusions Past empirical work has either ignored the problem of selective compliance in surveys or made essentially ad hoc corrections. We have shown how the latent income effect on compliance can be estimated consistently with the available data on average response rates and the measured distribution of income across geographic areas. Thus we are able to re-weight the raw data to correct for the problem. In an example using US data, we find that we can reject the assumptions made in past ad hoc correction methods. A highly significant negative income effect on survey compliance is indicated by our results. Our method also indicates higher inequality than implied by the survey's internal weights. An upward revision to the overall mean is also called for to correct for selective compliance. Ideally, the adjustment methods employed by the Census Bureau to correct for various sampling errors as well as the post-stratification methods could be combined with our correction method for nonresponse to obtain the most efficient estimate of population statistics possible and to balance off the biases that are introduced by the various methods. Technically, this would be no problem. However, the CPS dataset did not provide us with the sample weights before correction for nonresponse or the detailed data used for post-stratification, which would both be 23 required for such a calculation. We thus recommend to the Census Bureau to include sample weights that are unadjusted for non-response in future data releases. There can be no presumption that our quantitative results will hold elsewhere. Possibly in poorer settings one will find greater under-representation of the poor than in the US. Or one might find a less (more) steep income gradient of compliance in countries with lower (higher) inequality than the US. These are conjectures. However, the data and computational demands of the method we have proposed are quite modest, so other applications can easily be implemented. 24 Appendix The proof of the Proposition in Section 2 is as follows. Let {Sj}j = 1...Jbe J samples of households that can be partitioned into I subsets with characteristics Xi each. Suppose w.l.o.g. that i mij = mj = 1, and that the probability of response P(Dij = 1|Xi, ) is strictly increasing in Xi. Then let us show that for any geographical subgroup j, the observed average X 1j is in expectation lower than the actual (but unobserved) average X = I j i=1Ximij . To prove this we establish that the contrary yields a contradiction. For suppose that: E[X 1j ] =i=1XiPimij I I i=1Pimij I i=1Ximij This can be re-written as: i I Pimij Xi -mij 0 =1 i I =1Pimij Note that we can rewrite I i=1Pimij = P , since it represents an average probability of response. Now observe that, since Pi(|Xi) is strictly increasing in Xi, there must be some X~ such that Pi Xi P, Xi X~ and Pi ( ) ( X ) P , X X~ . We can use this to re-express i i the equation above as: I (X Pimij I -mij 0 or (X )m i - X~ i- X~ ij i=1 ) P i=1 Pi P -1 0 It is straightforward to see that all addends on the LHS are zero or negative; for Xi X~ the first brackets are positive or zero and the second brackets are negative or zero. For Xi < X~ , the first brackets are negative and the second brackets are positive. Thus we have a contradiction and it must be the case that E[X 1j ] < X . Having shown that the observed average X 1j is in j expectation below the actual average X for every single area j, the claim in the proposition j follows readily by averaging over all J areas. 25 References Bureau of Labor Statistics, 1981, Urban family budgets and comparative indexes for selected urban areas. U.S. Department of Labor, Washington DC. Cassel, C-M., Sarndal, C-E., Wretman, J.H., 1983, Some uses of statistical models in connection with the nonresponse problem, in: Madow, W.G. Olkin, I. (Eds.), Incomplete Data in Sample Surveys, Vol. 3. Academic, New York. Census Bureau, 2001, Poverty in the United States: 2001. Current Population Report P60-219. U.S. Department of Commerce, Washington, DC. Census Bureau, 2002, Current Population Survey: Design and Methodology. Technical Paper 63RV. U.S. Department of Commerce, Washington DC. Census Bureau, 2004, Income, poverty, and health insurance coverage in the United States. Current Population Report P60-226. U.S. Department of Commerce, Washington DC. Chapman, D.W., 1983, The impact of substitution on survey estimates. In: Madow, W.G. Olkin, I. (Eds.), Incomplete Data in Sample Surveys, Vol. 2. Academic Press, New York. Deming, W.E., 1953, On a probability mechanism to attain an economic balance between the resultant error of response and the bias of nonresponse. Journal of the American Statistical Association 48, 743-772. Friar, M.E., Leonard, H.B. 1998. Variations in cost of living across States. Taubman Center for State and Local Government, John F. Kennedy School, Harvard University. Gannon, M.J., Nothern, J.C., Carroll, S.J., 1971, Characteristics of nonrespondents among workers. Journal of Applied Psychology 55, 586-588. Groves, R.E., Couper, M.P., 1998, Nonresponse in Household Interview Surveys. Wiley, New York. Hansen, L.P., 1982, Large sample properties of Generalized Method of Moments estimators. Econometrica 50, 1029-1054. Hansen, M.H., Hurwitz, W.N., Madow, W.G., 1953, Sample survey methods and theory, Vol. 1. Wiley, New York. Hansen, M.H., Hurwitz, W.N. 1946, The problem of nonresponse in sample surveys. Journal of the American Statistical Association 41, 516-529. 26 Hendricks, W.A., 1949, Adjustment for bias caused by nonresponse in mailed surveys. Agricultural Economic Research 1, 52-56. Holt, D., Elliot, D., 1991, Methods of Weighting for Unit Nonresponse. The Statistician 40, 333- 342. Kalsbeek, W.D., Folsom Jr., R.E., Clemmer, A.F., 1974, The national assessment no-show study : an examination of nonresponse bias. American Statistical Association Proceedings of the Social Statistics Section, 180-189. Kalton, G., Kasprzyk, D., 1982, Imputing for missing survey response. American Statistical Association Proceedings of the Survey Research Methods Section, 21-33. Korinek, A., Mistiaen, J.A., Ravallion, M., 2005, Survey Nonresponse and the Distribution of Income. Policy Research Working Paper No. 3543. World Bank, Washington DC. (http://econ.worldbank.org/external/default/main?pagePK=64165259&theSitePK=46938 2&piPK=64165421&menuPK=64166093&entityID=000012009_20050322112823) Lessler, J.T., Kalsbeek W. D., 1992, Nonsampling Error in Surveys. Wiley, New York. Lillard, L., Smith, J.P., Welch F., 1986. What Do We Really Know about Wages? The Importance of Nonreporting and Census Imputation. Journal of Political Economy 94, 489-506. Little, R.J.A., Rubin D.B., 1987, Statistical Analysis with Missing Data. Wiley, New York. Moser, C.A., Kalton, G., 1972, Survey Methods in Social Investigation. Basic Books, New York. Philipson, T., 1997. Data markets and the production of surveys. Review of Economic Studies 64, 47-72. Politz, A.N., Simmons, W.R., 1949, An attempt to get `not-at-homes' into the sample without call-backs. Journal of the American Statistical Association 44, 9-31. Rubin, D.B., 1977, Formalizing subjective notions about the effects of nonrespondents in sample surveys. Journal of the American Statistical Association 72, 538-543. Scott, K., Steele, D., 2004, Measuring Welfare in Developing Countries: Living Standards Measurement Study Surveys. In: Surveys in Developing and Transition Countries: Design, Implementation and Analysis. United Nations, New York. Thomsen, I., 1973, A note on the efficiency of weighting subclass means to reduce effects of non-response when analyzing survey data. Statistisk Tidskrift 4, 278-283. 27 Table 1. Sample Sizes and Nonresponse Rates for the CPS (1998 ­ 2004) Year Total number of Type A households Rate of nonresponse (%) households 1998 54,574 4221 7.73 1999 55,103 4318 7.84 2000 54,763 3747 6.84 2001 53,932 4299 7.97 2002 84,831 6566 7.74 2003 85,092 6782 7.97 2004 84,116 6967 8.28 All years 472,411 36,900 7.81 Table 2. CPS 2004 Final Weight Regressions Intercept Income per-capita OLS 11.034 0.5969 (0.031) (0.0031) Sate fixed effects 10.920 0.0712 (0.014) (0.0014) Table 3. Summary Statistics by State (2004 CPS, sorted by response rate) State Response Sample Size Income per State Response Sample Size Income per Rate (%) (Households) capita ($) Rate (%) (Households) capita ($) Alabama 96.47 1,189 15,183 Missouri 92.04 1,269 16,251 North Dakota 96.03 1,082 15,415 Virginia 92.04 1,470 19,322 Indiana 95.73 1,500 16,667 Tennessee 91.62 1,014 14,167 South Dakota 95.53 1,164 14,763 Texas 91.51 3,864 12,547 Utah 95.35 1,010 14,205 Colorado 91.50 1,788 17,816 Wisconsin 95.29 1,528 17,294 Massachusetts 91.49 1,540 19,856 Arkansas 95.29 976 12,704 Michigan 91.46 2,319 16,700 Montana 94.60 871 13,013 Rhode Island 91.44 1,518 17,018 Georgia 94.55 1,175 16,049 Maine 91.44 1,366 15,098 Iowa 93.69 1,379 16,904 Connecticut 91.36 1,574 20,779 Louisiana 93.67 979 12,550 Ohio 91.34 2,517 17,102 Florida 93.51 3,680 15,400 North Carolina 90.78 1,811 14,251 Kansas 93.41 1,441 16,085 South Carolina 90.53 1,162 14,904 Wyoming 93.35 1,128 15,561 Hawaii 90.53 1,193 17,377 Illinois 93.28 2,945 16,898 New Mexico 90.46 1,090 12,000 Arizona 93.23 1,167 13,750 Washington 90.19 1,509 17,751 Nevada 93.23 1,594 15,999 California 90.06 5,984 14,908 Delaware 93.16 1,082 18,039 Oregon 89.99 1,289 15,442 Oklahoma 93.12 1,047 13,667 Vermont 89.04 1,277 17,710 West Virginia 92.91 1,170 13,150 Alaska 88.64 1,206 16,523 Mississippi 92.81 904 13,440 New Hampshire 88.50 1,400 20,367 Idaho 92.81 973 12,494 New Jersey 88.50 2,200 20,208 Minnesota 92.51 1,535 19,194 Maryland 88.00 1,408 20,255 Nebraska 92.47 1,302 16,086 New York 87.56 4,245 16,141 Kentucky 92.18 1,138 14,700 District of 84.66 1,180 17,210 Pennsylvania 92.14 2,964 17,385 Columbia 28 Table 4. AIC for various specifications, 1998 ­ 2004 data Specification AIC 1: zi = 1 -69.27 2: zi = 1 ln(yi) -42.20 3: zi = 1 + 2 ln(yi) -276.14 4: zi = 1 ln(yi)2 -16.04 5: zi = 1 + 2 ln(yi)2 -275.37 6: zi = 1 ln(yi) + 2 ln(yi)2 -273.45 7: zi = 1 + 2 ln(yi) + 3 ln(yi)2 -270.04 8: zi = 1 yi 88.78 9: zi = 1 + 2 yi -193.05 10: zi = 1 ln(yi) + 2 yi -159.03 11: zi = 1 + 2 ln(yi) + 3 yi -273.04 12: zi = 1 ln(yi)2+ 2 yi -119.01 13: zi = 1 + 2 ln(yi)2 + 3 yi -273.31 14: zi = 1 ln(yi) + 2 ln(yi)2 + 3 yi -273.77 Note: The probability of response is modeled as P = logit(z) for all given models. In order to determine the Akaike Information Coefficient (AIC) for the various specifications, we estimated each specification with data from all 7 years and used the resulting residuals j to calculate AIC = J log ( (^) ) 2 J + 2m, where J is the number of residuals, i.e. 7*51 here, and m is j the number of estimated parameters, i.e. 7, 14 or 21 in our application. The lowest value for the AIC (i.e. here the highest absolute value) indicates that specification 3 (underlined in the table above) best fits the nonresponse behavior exhibited by our data. For our estimations we are using Matlab 6.5. The source code of our program can be downloaded at http://econ.worldbank.org/programs/poverty/topic/2678/ 29 Table 5. Various Specifications for 2004 CPS Gini Specification 1 2 3 index (%) 3: z = 1 + 2 ln(y) 19.112 -1.613 49.23 (1.708) (0.155) (0.92) 5: z = 1 + 2 ln(y)2 10.108 -0.07165 49.41 (0.747) (0.00611) (0.90) 6: z = 1 ln(y) + 2 ln(y)2 1.8091 -0.1519 49.60 (0.1165) (0.0105) (0.87) 7: z = 1 + 2 ln(y) + 3 ln(y)2 -1.1568 2.017 -0.1611 49.63 (9.7906) (1.766) (0.0791) (0.93%) 9: z = 1 + 2 y 2.900 -1.232*10-5 49.56 (0.055) (4.368*10-7) (0.62%) 11: z = 1 + 2 ln(y) + 3 y 7.968 -0.5113 -8.704*10-6 49.62 (3.878) (0.3865) (2.755*10-6) (0.69) 13: z = 1 + 2 ln(y)2 + 3 y 5.396 -0.02541 -8.221*10-6 49.66 (1.896) (0.01885) (3.072*10-6) (0.69) 14: z = 1 ln(y) + 2 ln(y)2 + 3 y 1.0752 -0.07891 -7.199*10-6 49.76 (0.3615) (0.03610) (3.328*10-6) (0.70) Note: Standard errors are in brackets. The uncorrected Gini coefficient for 2004 data (with households equally weighted within states) is 44.80%, and using the official CPS weights it is 45.20%. 30 Table 6. Specification P = logit[1 + 2 ln(y)] ­ Estimates for 1998 ­ 2004 1998 1999 2000 2001 2002 2003 2004 All 1 19.904 18.100 22.207 20.111 17.807 17.388 19.113 18.838 (2.071) (2.420) (2.545) (1.728) (1.920) (2.100) (1.708) (0.793) 2 -1.696 -1.528 -1.890 -1.702 -1.490 -1.454 -1.613 -1.599 (0.188) (0.223) (0.230) (0.156) (0.176) (0.193) (0.155) (0.073) Note: standard errors in brackets Table 7. Specification P = logit[1 + 2 ln(y)] using cost-of-living adjustment for 2004 data Income y in nominal terms Income y in real terms 1 19.113 18.337 (1.708) (2.501) 2 -1.613 -1.542 (0.155) (0.229) Note: standard errors in brackets 31 Table 8. Augmented specifications for 2004 data Specification 1 2 3 4 5 AIC zi = 1 + 2 ln(yi) 19.113 -1.613 [baseline] (1.708) (0.155) -23.881 zi = 1 + 2 ln(yi) + 3 hhsize 18.092 -1.545 0.1315 (2.545) (0.197) (0.2623) -22.205 zi = 1 + 2 ln(yi) + 3 IMSA 20.010 -1.705 0.1462 (1.896) (0.178) (0.1790) -22.568 zi = 1 + 2 ln(yi) + 3 Ihomeowner 18.436 -1.648 1.107 (1.571) (0.151) (0.678) -23.271 zi = 1 + 2 ln(yi) + 3 Ifemale 18.804 -1.569 -0.3703 (1.808) (0.18) (0.7412) -22.204 zi = 1 + 2 ln(yi) + 3 Icaucasian 17.669 -1.499 0.2607 (2.290) (0.199) (0.2799) -22.689 zi = 1 + 2 ln(yi) + 3 Iworking 19.143 -1.612 -0.0455 (1.715) (0.172) (1.2631) -21.883 zi = 1 + 2 ln(yi) + 3 Iunemployed 18.709 -1.57 -1.4699 (1.766) (0.163) (1.3966) -22.241 zi = 1 + 2 ln(yi) + 3 edu 17.304 -1.183 -0.2567 (2.38) (0.437) (0.2456) -23.231 zi = 1 + 2 ln(yi) + 3 Iedu 11.347 -0.821 -1.9618 master (4.51) (0.479) (1.1183) -26.625 zi = 1 + 2 ln(yi) + 3 age 19.866 -1.629 -0.0114 (2.327) (0.162) (0.0221) -22.345 zi = 1 + 2 ln(yi) + 3 age + 127.215 -1.784 -3.934 0.03596 + 4 age2 -36.922 (50.518) (0.138) (1.850) (0.01671) zi = 1 + 2 ln(yi) + 3 age + 97.365 -1.572 -2.909 0.02653 -0.6948 + 4 age2 + 5 Iedu -38.500 master (44.926) (0.215) (1.661) (0.01500) (0.3835) zi = 1 + 2 ln(yi) + 3 Iregion + 2 16.991 -1.428 0.2762 0.1020 0.0744 + 4 Iregion + 5 Iregion -26.122 3 4 (1.794) (0.164) (0.1012) (0.0809) (0.0816) zi = 1 + 2 ln(yi) + 3 Iregion 17.319 -1.453 0.2126 2 (1.813) (0.166) (0.0935) -28.219 Note: standard errors in brackets 32 Table 9. Augmented specifications for pooled data from 1998 to 2004 Specification 1 2 3 4 AIC zi = 1 + 2 ln(yi) [baseline] 18.838 -1.599 (0.793) (0.073) -262.51 zi = 1 + 2 ln(yi)+ 3 hhsize 21.383 -1.759 -0.342 (1.022) (0.085) (0.068) -275.65 zi = 1 + 2 ln(yi)+ 3 IMSA 18.892 -1.605 0.010 (0.925) (0.089) (0.092) -260.52 zi = 1 + 2 ln(yi)+ 3 Ifemale 18.383 -1.521 -0.812 (0.813) (0.080) (0.308) -270.74 zi = 1 + 2 ln(yi)+ 3 Icaucasian 19.004 -1.611 -0.116 (0.776) (0.071) (0.038) -273.01 zi = 1 + 2 ln(yi)+ 3 Iworking 18.808 -1.617 0.281 (0.802) (0.082) (0.411) -261.14 zi = 1 + 2 ln(yi)+ 3 Iunemployed 18.472 -1.561 -1.336 (0.807) (0.075) (0.438) -264.78 zi = 1 + 2 ln(yi)+ 3 edu 16.587 -1.162 -0.223 (1.041) (0.167) (0.085) -271.22 zi = 1 + 2 ln(yi)+ 3 Iedu 15.482 -1.231 -1.020 master (1.434) (0.158) (0.445) -269.95 zi = 1 + 2 ln(yi)+ 3 age 19.274 -1.614 -0.006 (1.065) (0.076) (0.009) -261.04 zi = 1 + 2 ln(yi)+ 3 age + 4 age2 20.180 -1.607 -0.045 0.0004 (2.302) (0.077) (0.081) (0.0007) -259.47 zi = 1 + 2 ln(yi)+ 3 year 18.858 -1.595 -0.020 (0.785) (0.072) (0.008) -269.98 zi = 1 + year*3 + [2 + year*4] ln(yi) 22.179 -1.898 -0.948 0.0849 (1.293) (0.117) (0.310) (0.0283) -273.42 Note: standard errors in brackets 33 Figure 1: Probability of response function, top three specifications, 1998 ­ 2004 data 1 0.95 0.9 ecn 0.85 lia 0.8 mpocfo 0.75 y ilitbaborp 0.7 0.65 bottom percentile 0.6 top percentile specification 3 specification 5 0.55 specification 14 0.5 $1000 $2000 $4000 $10,000 $20,000 $40,000 $100,000 Y ... income/capita Note: The graphs of the three specifications of nonresponse that match the data most closely almost coincide, indicating that the exact choice of specification is not of major importance. 95% confidence intervals were computed, but are visibly almost indistinguishable from the graphed functions themselves and were thus omitted. The two dotted vertical lines indicate the interval in which the median 98% of income observations are located. 34 Figure 2: Empirical and compliance corrected cumulative income distribution Corrected income distribution for 2004 data 1 0.9 0.8 0.7 oni at 0.6 populfo 0.5 onit 0.4 acrf 0.3 0.2 uncorrected CPS weights 0.1 corrected 0 $0 $50,000 $100,000 $150,000 income per capita Note: The upper (dotted) line represents the income distribution from raw survey data without the Census Bureau's weight adjustments. The dash-dot line depicts the income distribution using the adjustment weights of the Census Bureau. Finally, the solid line shows the distribution according to our correction method. A 95% confidence interval for our corrected distribution line was computed but is omitted here, since it was visibly almost indistinguishable from the line itself. 35 Figure 3: Lower segment of cumulative income distribution from Figure 2 Corrected income distribution for 2004 data 0.25 uncorrected CPS weights corrected 0.2 notialupopfo 0.15 noticafr 0.1 0.05 0 $0 $2,000 $4,000 $6,000 $8,000 $10,000 income per capita Note: The figure gives a magnification of the lower part of the cumulative income distribution reveals that our correction method assigns comparatively less weight to lower income households than the Census Bureau's correction method. 36 Figure 4: Observed and corrected Lorenz curves Lorenz curves for 2004 data 1 uncorrected 0.9 CPS weights corrected 0.8 0.7 e mocnil 0.6 ta tofo 0.5 noticafr 0.4 0.3 0.2 0.1 0 0 0.2 0.4 0.6 0.8 1 fraction of population Note: The top line (dotted) shows the Lorenz curve using un-weighted data, the second (dash-dot) line is the Lorenz curve using the weights provided by the Census Bureau. This line hardly differs from the un-weighted data. The bottom (solid) line is the Lorenz curve according to our correction for survey nonresponse: it shows a marked increase in inequality as compared to the previous two cases. We also calculated a 99% confidence interval for our correction method. However, since this visibly almost coincides with the depicted line, it is omitted in the graph here. 37 Figure 5: Percentage correction of income by percentile of income distribution Implied income correction for 2004 +50% Our correction CPS correction +40% Baseline niotcerroc +30% e mocin +20% d lie +10% imp 0% -10% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% percentile of income Note: The figure shows by how much the income of a given income percentile in the corrected distribution is revised with respect to the income of the same percentile in the equally weighted income distribution. The Census Bureau's method implies a relatively uniform shift of incomes in each percentile by roughly 20% upwards. Our correction method shifts the income of lower income percentiles only modestly upwards, whereas the mean income of the top percentile is corrected by almost 40%. 38 Figure 6: Weight correction for each observed percentile Weight correction for 2004 +100% eli Our correction +80% CPS correction entc Baseline per edvre +60% obs +40% orf onit ecr +20% orc ghteiw 0% edipl -20% mi -40% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% percentile of income Note: The figure presents the data from the previous graph in a different format: Instead of comparing percentiles in the un-weighted distribution with percentiles in the corrected distribution, we depict the correction in the sum of weights of all households contained in a given percentile of the un-weighted income distribution. As can be seen, the Census Bureau's method strongly reduces the weights of the lower income percentiles in the un-weighted distribution and increases the weights of the upper third of the observed income distribution. Our correction method slightly decreases the weights of all households below the 83rd percentile and strongly increases the weights of the households in the top observed percentiles. 39