Does the same question asked of the same population yield the same answer in face-to-face interviews when other parts of the questionnaire are altered? If not, what would be the implications for proxy-based poverty measurement? Relying on a randomized household survey experiment implemented in Malawi, this study finds that observationally equivalent as well as same households answer the same questions differently when interviewed with a short questionnaire versus the longer counterpart that, in a prior survey round, would have informed the prediction model for a proxy-based poverty measurement exercise. The analysis yields statistically significant differences in reporting between the short and long questionnaires across all topics and types of questions. The reporting differences result in significantly different predicted poverty rates and Gini coefficients. While the difference in predictions ranges from approximately 3 to 7 percentage points depending on the model specification, restricting the proxies to those collected prior the variation in questionnaire design, namely demographic variables from the household roster and location fixed effects, leads to same predictions in both samples. The findings emphasize the need for further methodological research, and suggest that short questionnaires designed for proxy-based poverty measurement should be piloted, prior to implementation, in parallel with the longer questionnaire from which they have evolved. The fact that at the median it took 25 minutes to complete the food and non-food consumption sections in the long questionnaire also implies that the implementation of these sections might not be as overly costly as usually assumed.
Details
-
Author
-
Document Date
2015/01/01
-
Document Type
Policy Research Working Paper
-
Report Number
WPS7182
-
Volume No
1
-
Total Volume(s)
1
-
Country
-
Region
-
Disclosure Date
2014/12/31
-
Disclosure Status
Disclosed
-
Doc Name
Same question but different answer : experimental evidence on questionnaire design's impact on poverty measured by proxies
-
Keywords
Surveys and Methods;impact on poverty;national household survey data;sociological methods;female head of household;ordinary least squares regression;Poverty Measurement;household questionnaire;Poverty & Inequality;household consumption expenditure;food consumption module;headcount poverty rate;treatment effect;Proxy Means Tests;household and individual;consumer price index;household survey instrument;Social Safety Nets;labor force survey;propensity to consume;small area estimation;marginal effect;fixed effect;durable good;household level;standard error;Durable goods;statistical significance;consumption datum;poverty estimate;regression results;labor module;model specification;behavioral processes;consumption regression;measurement error;durable asset;subjective assessment;coping strategy;total consumption;demographic variables;private communication;data quality;economics research;poverty focus;sample mean;research assistance;opinion survey;general population;interaction effect;open access;consumption distribution;cooking oil;confidence interval;differences in results;welfare level;statistical method;farm implement;survey implementation;survey design;comparative assessment;question answering;cognitive process;official poverty;poverty trend;national statistical;high frequency;steep decline;empirical investigation;substantial variation;international community;welfare aggregate;methodological research;poverty predictor;enumeration area;screening question;field staff;regional location;urban residence;development policy;multivariate regression;error component;data processing;household size;targeting performance;survey questions;empirical evidence;research community;
- See More
Downloads
COMPLETE REPORT
Official version of document (may contain signatures, etc)
- Official PDF
- TXT*
- Total Downloads** :
- Download Stats
-
*The text version is uncorrected OCR text and is included solely to benefit users with slow connectivity.
Citation
Kilic,Talip Pave Sohnesen,Thomas
Same question but different answer : experimental evidence on questionnaire design's impact on poverty measured by proxies (English). Policy Research working paper,no. WPS 7182,LSMS Washington, D.C. : World Bank Group. http://documents.worldbank.org/curated/en/818271468278349475/Same-question-but-different-answer-experimental-evidence-on-questionnaire-designs-impact-on-poverty-measured-by-proxies