Policy Research WORKIN0 PAPERS Poverty Analysis and Policy Africa TechnicalDepartment The World Bank January 1993 WPS 1079 How Useful Are Integrated Household Survey Data for Policy-Oriented Analyses of Poverty? Lessons from the Cote d'lvoire Living Standards Survey Christiaan Grootaert Simpler and bigger are better, for household living standards surveys. And better collection of price data, in an independent survey, should be a priority. PolicyReseah Wotkinghp adimnate the rmdingsofwork inprogres and enoouagetheexchangeof ideas among Bank staff and aliothesintersted indevelopmat issues. hese paprs carry the names of te authom reflect adly their views. and shuld be used and cited accordingly. The findings, intequstiou and cmclusios are the authors own. Thy should not beattnibuted to the World Bank. its Board of Ditechsa its m anagemnt. or any of its mmber counties. Policy Research Poverty Analysis and Policy WPS 1079 This paper- a product of the Poverty and Social Policy Division, Africa Technical Department - is part of the output of the research project "Poverty and the Social Dimensions of Structural Adjustment in Cote d'Ivoire, 1985-88: A Policy-Oriented Analysis" (RPO 675-26). Copies ofthis paper are available free from the World Bank, 1818 H Street NW, Washington DC 20433. Please contact Elena Vitanov, room J2-24 1, extension 38400 (January 1993, 23 pages). Grootaert reflects on the pros and cons of using It is important to involve analysts and policy- integrated household survey data in empirical makers in survey design. They need to identify up analysis aimed at providing a quantitative basis front, using current knowledge, the important for policy decisions affecting welfare, poverty, socioeconomic and target groups on which the and the fulfillment of basic needs. The experi- survey must be able to report. The sample designer ence examined is that of using four years of data can then compose the sample in such a way that from the CMe d'Ivoire Living Standards Survey certain groups will be undersampled and others (1985-88) to link changes in poverty and welfare oversampled, to make the analysis of the rsulting to macroeconomic trends. sample as useful as possible. Grootaert groups the lessons learned from Frequency of data collection. Grootaert this work around four themes. recommends that an integrated survey of the CILSS type be undertaken every four or five Survey content. When survey data are rich, years, to provide benchmark data and to pernit transparency of methodology is iniprtant. It is in-depth analysis of household behavior and essential that analysts provide explicit informa- response to policy, if the country has the capabil- tion about how their income and spending ity to fully analyze the data. In the intervening aggregates were constructed. These aggregates years, a much simpler collection of household must be deflated with a regional price index, but spending and basic needs data can be used to prices should be collected separately from monitor changes in welfare and poverty. household survey data. Data on household spending and basic needs fulfillment are the key The role of panel data. To be really useful, information for poverty analysis. panel data collection should be extended over longer periods than two years, although this Sample size and design. Bigger and simpler increases the costs and difficulties of finding the is better. Grootaert recommends increasing (at same households. If a country undertakes an least doubling) sample size in future living integrated survey every four to five years and a standards surveys; this could be done without lighter monitoring survey in between, a small, increasing the cost of the survey by reducing or parallel, panel survey could be conducted. The eliminating the income modules of the question- monitoring sample and the panel sample should naire. be drawn from the same master sample. ThePolicy ResearchWorking PaperSeriesdisseminates thefindings of work under way in theBank. Anobjectiveof the series is to get these findings out quickly, even if presentations are less than fully polished. The findings, interpretations, and conclusions in these papers do not necessarily represent official Bank pohcy. Produced by the Policy Research Disse.nination Center Poverty and Social Policy Division Technical Department Africa Region The World Bank HOW USEFUL ARE INTEGRATED HOUSEHOLD SURVEY DATA FOR THE POLICY-ORIENTED ANALYSIS OF POVERTY? LESSONS FROM THE C6TE D'IVOIRE LIVING STANDARDS SURVEY Christiaan Grootaert This paper is an output of the research project "Poverty and the Social Dimensions of Structural Adjustment in Cote d'Ivoire, 1985-88 - A Policy-Oriented Analysis" (RPO 675-26). The author would like to thank Lionel Demery and Chris Scott for many helpful discussions on the subject of this paper and for useful comments on an earlier draft. 1. Introduction This paper offers some reflections on the pros and cons of using integrated household survey data in empirical analysis aimed at providing a quantitative basis for policy decisions concerning welfare, poverty and the fulfillment of basic needs. The experience reflected here is that of using four years of data from the C6te d'Ivoire Living Standards Survey (1985-88) in a research effort attempting to link changes in poverty and welfare to macro-economic trends. The design of this research is written-up in Grootaert and Kanbur (1990) and the main findings are discussed in Grootaert (1992). The lessons we learned from this work have been grouped according to four themes: survey content, sample size and design, frequency of data collection, and the role of panel data. One section of this paper is devoted to each theme. A conciuding comment offers some recommendLtions for future surveys. 1 2. The C6te d'Ivoire Living Standards Survey (CILSS) The CMte d'Ivoire Living Standards Survey (CILSS) was conducted from 1985 to 1988 by the Direction de la Statistique in Cdte d'Ivoire, with financial and technical support from the World Bank during the first two years. The sample size each year was 1600 households and the sample design was a rotating panel, i.e. 50% of the households were revisited the following year and the other half was replaced by new households. The survey thus produced a sequence of four cross-sectional data sets, each of which is representative of the country as a whole, as well as three overlapping panels of approximately 800 households each (1985-86, 1986-87, 1987-88). The survey collected detailed information on expenditures, income, employment, assets, basic needs and other socio-economic characteristics of households and individuals. Over the four years, coverage and methodology of the data collection were held constant so results are comparable over time. A more detailed discussion of the methodology and content of the CILSS can be found in Ainsworth and Munoz (1986), Grootaert (1986), and Daho (1992). T1he CILSS data have been the subject of much analysis of welfare and poverty, but most studies have only used the 1985-86 data (for a review, see Glewwe, 1990, and Grootaert, 1992). The reflections in this paper are based on research which used both the four years of cross-sectional data and the three overlapping panels.l/ 1/A general review of living standards surveys can be found in Glewwe (1990); Grosh (1991) reviews the Jamaica Living Standards Survey. 2 3. Survey Content: When Data Are Too Rich 3.1 Income and Expenditure Aggregates The first question one must ask about the CMte d'Ivoire Living Standards Survey is whether it achieved its prime objective of measuring households' levels of living. The answer is a clear "yes." The survey contains all necessary information to construct a complete set of current accounts for the household. Both income and expenditure are collected in great detail, and savings can be derived as the residual. With respect to the capital account, the survey collects a fair amount of information on household assets and liabilities, although no attempt was made to be comprehensive in coverage (mainly because of expected respondent resistance to detailed inquiries about financial assets and liabilities). However, in order to achieve its objective, the CILSS can.e equipped with some extraordinarily lengthy and complex questionnaire modules on income and expenditure. In total, there are about a thousand questions in the survey and the questionnaire takes an average of six hours of interview time in the field, spread over two visits. The questionnaire is a highly non-linear design, with complex skip patterns, yielding data with a multi-level hierarchical structure (households, individuals, household enterprises, crops, expenditure categories, loans, recipients of transfers, etc.) All this not only represents a significant challenge for the data collection in the field, but it adds tremendously to the analytical complexity and cost. One overwhelming lesson learned in the course of analyzing the C LSS data is the difficulty and time-consuming nature of constructing the household accounts from this survey. It speaks for itself that it took more than three years, after the end of the field work, before the first analysis which used all four years of CILSS data on income and expenditure was completed (see Grootaert, 1992). In view of the complexity and detail of the different income and expenditure modules, it is not surprising that there is more than one way to build the relevant income 3 and expenditure aggregates at the household level. This is not a trivial point because the varying assumptions and methodologies used in doing this can make a tremendous difference to the final outcomes. As an illustration, we show in Tables 1 and 2 total household income and expenditure and their regional distributions as calculated by various autho1s, but based on exactly the same raw data. The differences are striking, not only in the country-wide means, but also in the regional distributions. For example, Kanbur (1990) estimates the average expenditure level in Abidjan as 86% above the country average, while Grootaert (1992) estimates this difference as 58%, even though the Abidjan mean is about the same in both studies. The estimates of Kanbur (1990) and Glewwe (1987) of welfare in Savannah are almost 50% lower than Grootaert's estimates (Table 1). On the income side, Kozel's (1990) estimate of country-wide mean household income is one third higher than that of Grootaert, and her estimate for Abidjan is two third& higher (Table 2). Under such conditions, transparency of methodology becomes very important, and it is essential that analysts provide explicit information on how their incomne and expenditure aggregates were constructed. In the poverty research project underlying t;,:s paper, the methodology was documented in detail in Johnson, McKay and Round (1990) and Oh and Venkataraman (1992). 3.2 Data Quality There is a fairly general consensus among the various users of the data that the quality of the CILSS results is good. Some attempts have been made to validate the CILSS results with other data sources (population census and other household surveys). While in general those comparisons were favorable (see Grootaert, 1992), they need to be treated with caution since it is not always obvious that the reference data are of better quality than the CILSS. Nevertheless, the quality of the CILSS data is not uniform across the different modules of the survey. We found two weaknesses. First, the information on non-farm 4 Table 1: Alternative Estinates of Household Expenditure Per Capita (CFAF/year) based on the 1985 CILSS Grootaert Glewwe Kanbur l___________ (1992) (1987) (1990) Abidjan 376,108 402,300 377,600 Other Cities 271,864 253,300 237,000 East Forest 164,472 146,500 136,500 West Forest 239,134 182,200 170,100 Savannah 152,573 107,700 100,800 Cote d'Ivoire 237,853 216,500 202,800 Table 2: Alternative Estimates of Household Income (CFAF/year) based on the 1985 CILSS Grootaert Kozel (1992) (1990) Abidjan 1,704,295 2,823,011 Other Cities 1,402,945 1,934,893 East Forest 925,730 } West Forest 1,228,056 }1,022,597 Savannah 767,173 } Cote d'Ivoire 1,206,414 1,600,743 S household enterprises is dubious. Our calculations showed that in each year, the majority of household enterprises reported negative net income, calculated as the difference between revenues and cost (each was recorded in detail in the survey). Apparently, this was due to both over-estimation of expenditures of the enterprise and under-reporting of revenues. Respondents also had difficulties in separating transactions made for the enterprise from those made bor the household. This problem had been noted already in the first year of the survey and had led the survey managers to add, as of the second year, two direct but aggregate questions about net revenues from enterprises. It turned out that this line of questioning (which most survey practitioners would view with great suspicion) provided an apparently better estimate of household enterprise income - which of course does not mean that it was accurate. Experimental survey work is needed to improve data collection on the costs and revenues of household enterprises. A special study of informal sector enterprises in Nairobi and Dakar is currently underway to understand better how the budget of a household enterprise relates to the budget of the household who owns it. In Madagascar, household enterprises are being asked to keep a very simple type of daily account. These efforts will hopefully lead to better ways to collect data on the enterprise budget. Second, the survey results show that on average, household savings are negative. (This result was obtained after replacing the negative incomes of household enterprises with the results from the direct questions). This is a customary finding in surveys which collect both income and expenditure data and it is generally believed that this is due to under-reporting of income, especially by high-income earners. This is confirmed in the case of the CILSS since tne negative savings rate increases with the level of income. While this result is therefore not surprising, it is distinctly disappointing that even with an instrument of the detail and complexity of the CILSS, it has not proved possible to improve on the experience of other surveys. While the under-reporting of household income in the CILSS is clearly bad news, it is by no means disastrous for the analysis of household welfare or poverty. Indeed, most such analyses undertaken with CILSS data have used expenditure data, including the work in this research project. Household expenditure is a conceptually preferred 6 measure because it reflects better permanent income and is less subject to short-term fluctuations than current income. In addition, the general experience is that expenditure is typically collected with greater accuracy than income. This is an important point because the income modules assume well over half of ali questions in the CILSS. In other words, dropping the income modules would yield a much simpler and leaner instrument which would reduce only marginally the value of the data from the point of view of analyzing welfare and poverty. We would argue that in the context of a typical low-income developing country it is not a top priority for welfare and poverty analysis to iAentify household savings. In the context of linking welfare changes to the macro- economic environment, the key information is not so much the level of housel' )ld income, but the composition of income and how it is changing when the external economic environment is changing. For that purpose, it is feasible to introduce in a survey a line of qualitative questioning to obtain information on the pattern of income, and how it is changing over time, without going into the quantitative detail needed to build up total income. For example, questions could be asked of the sort "How much does wage income contribute to total household income?", with the answer codes indicating ranges such as "less than 25%", "between 25% and 50 %", etc. To capture change, questions could be added such as "Is your wage income now higher, lower or the same relative to twelve months ago?" or "Has the importance of wage income increased, decreased, or remained about the same in your household's total income?" An approach along these lines is currently being tested in a dozen African countries, as part of the administration of Priority Surveys (see Grootaert and Marchant, 1991). Accepting then, that most welfare and poverty analysis can usefully be based on household expenditure and that this should be the focus of data collection, there still remain unresolved issues relating to the quality of expenditure data. A main query pertains to the optimal reference and recall periods for each expenditure category.2/ 2/The reference period is the period over which the respondent is asked to report. The recall period is the time interval over which the respondent is asked to search his/her memory, at any given interview. In a one-visit survey, the two periods 7 It is usually believed that for regularly purchased items, accuracy of recall increases when the recall period is shortened (Scott and Armenuvegbe, 1990). For items which are infrequently purchased, it is more meaningful to use a longer reference as well as recall period. The length of the reference period is important for the precision of annual estimates - the customary time period for welfare analysis. The shorter the reference period, the more difficult it becomes to extrapolate the results to an annual basis. In the extreme case, if for all expenditure items a one-day reference period were used, the estimate of mean annual expenditure would have to be obtained by multiplying survey responses by 365. And while, strictly speaking, 'his would still yield an unbiased estimate of mean annual expenditure, the resulting error variance would be so large as to make the results practically meaningless. The extrapolation to an annual basis of expenditures recorded over shorter reference periods will always yield an estimate of the variance of expenditures which overestimates the true variance (Scott, 1993). The reason is that the correlation of household expenditures over successive time-periods is less than unity. The lower this correlation is in reality, the more the variance of extrapolated expenditures will be biased upward. The problem is compounded when annual expenditure is built-up from components which each were recorded over different reference periods (as is the case in the CILSS). One solution to all this is to visit the household repeatedly and to obtain multiple observations over short recall periods which in total add up to the year, but this is obviously a very costly solution which would greatly increase the organizational complexity of a survey. An alternative is to obtain estimates of the over-time correlation of expenditures by repeatedly visiting a small sample of households. The results of such exercise could then be usAd to correct the data from the main sample. While some limited research has been undertaken to address these issues (see e.g. Chesher, 1991), coincide. In a multi-visit scheme, a reference period can be covered by multiple recall periods, e.g. if expenditures are to be reported over a one-month reference period, this can be achieved in four visits, each with a recall period of one week. 8 much further work remains to be done to improve the quality of expenditure data collection. 3.3 Prices In order to use household expenditure as a measure of welfare, it is necessary to take into account the fact that prices are not the same in different parts of a country. It is thus necessary to deflate household expenditure with a regional cost of living index. This had been recognized by the designers of the CILSS and they added to the household survey a price questionnaire which called for the enumerators to obtain prices in markets for a limited number of items. For each item, three observations were to be made. The experience with this has been disappointing. First, as a practical matter, the collection of prices was insufficiently integrated with the rest of the survey so that delays in collection and data processing occurred and many price questionnaires were filled out incompletely. In addition, the three observations proved insufficient in light of the typically wide price variation in African markets, especially for perishable food items. Lastly, the commodity coverage was inadequate (seventeen food items and four non-food items). According to Glewwe (1987), the non-food price information could not be used at all and it was necessary for the construction of the regional price index to rely on one non-perishable food item, a can of tomato paste, as a proxy for all non-food items. The limitations of such a price index are obvious and it was an important element of this research to construct a more comprehensive price index (see Grootaert and Kanbur, 1992a). The lesson from this effort is a positive one. The index was constructed on the basis of price data collected in CMte d'Ivoire in 1985 for the International Comparisons Project. In other words, the price data were not related to the CILSS. Nevertheless, it proved relatively easy (although time-consuming) to match the prices from the ICP with the expenditure categories in the CILSS. The resulting regional price index was based on prices for 260 product categories in the ICP which were matched with 27 food categories and 25 non-food categories from the CILSS. 9 From this experience, we are inclined to recommend that price collection best be conducted as an exercise separate from the household survey data collection. This could usefully occur in the context of an upgrading of existing price collection for the Consumer Price Index, which in most African countries currently covers only the capital city, or at most, selected urban areas. Our empirical results for CMte d'Ivoire indicate that regional price variation is significant and makes a major difference in the proper estimation of welfare and poverty levels, particularly in rural areas. It is thus essential that price coverage includes rural areas. Likewise, the regional price pattern for food items was found to be very different from that of non-food items, so that a good coverage of both types of expenditures is necessary. 3.4 Basic Needs Basic needs information collected in the CILSS included health, education and housing. The general experience with the CILSS modules covering these topics is very positive. We did not discover any major measurement problems. The modules contain rnore information than what is needed to construct indicators which would capture the basic needs aspects of welfare (they were in fact designed to permit full analysis of the demand for these services). Nevertheless, the education section missed some useful information for assessing the impact of economic conditions on school attendance. In particular, it would have been useful to be able to calculate drop-out and repetition rates, especially in primary school, since those are warning signals of deteriorating household conditions which force parents to pull their children out of school. The CILSS results have made it clear that it is essential to combine in poverty analysis information on household expenditure and on basic needs since the two do not necessarily display the same pattern. The incidence of poverty in CMte d'Ivoire, as measured by expenditure, increased by 50% between 1985 and 1988, while most basic needs indicators showed only relatively moderate changes. Combining the two items of information yields a poverty profile which is very useful for the design of poverty- 10 oriented policy. A key lesson though is that such a profile can be constructed from' significantly less information than that available in the CILSS. In that sense, we conclude that the CILSS is truly too rich a data base if the objective is measuring levels of living and poverty. Since it is not costless to collect and mine information, a thorough review is needed of the Living Standards Survey questionnaire. Given how much analysis has been undertaken with the data, it should not be too difficult to identify and eliminate these lines of quesnoning which are not or are only rarely used. In the context of basic needs, we need to refer to the community module which was appended to the CILSS in order to provide information on the supply of health, education and other facilities. Unfortunately, it displayed some of the same problems as the price data, namely a large amount of missing items in the filled-out questionnaires due to the fact that the data collection and processing was insufficiently integrated with the rest of the survey. This has limited the use of this information by researchers (Glewwe, 1991, is a notable exception). Nevertheless, community-level data are potentially very valuable for the analysis of the basic needs dimensions of welfare (see Demery, Ferroni & Grootaert, forthcoming). Future surveys should treat the collection of this type of data with the same care and priority as the household level data. 11 4. Sample Size and Design: Bigger and Simpler is Better The CILSS had a sample size of only 1600 households each year. This restraint in sample size was necessary in view of the complexity of the questionnaire and the six hours needed for each interview. Resource constraints are such that there is a clear trade-off between sample size and the length of the questionnaire. In our experience a sample size of 1600 households is too small for genuinely policy-oriented analysis. It was not possible to distinguish more than five regions or six or seven socio-economic groups to disaggregate the analysis. Within urban areas, it proved very difficult to push the analysis beyond the differentiation between Abidjan and all other cities combined. Rural areas could only be split in three: East Forest, West Forest and Savannah. Socio- economic classifications usually distinguished farmers from non-farmers. Farmers could be split further in export and food crop farmers, while non-farming households could be broken down according to sector (e.g. formal/informal). While other disaggregations are possible, there is a general upper limit of six to seven groups because the number of observations per group becomes prohibitively small. The problem is that most policy interventions are targeted at groups which are narrower than this. In particular, it is extremely useful for policy to be able to consider jointly the regional and socio-economic classification of households. But this would yield 30-35 cells and that is clearly too much for a sample size of 1600 households. One recommendation would thus be to increase the sample size in future living standards surveys.3/ Within a given resource constraint, this could easily be accommodated if our earlier suggestion of reducing or eliminating the income modules of the questionnaire is adopted. Before starting our analysis, we undertook a detailed scrutiny of several key variables in the survey, and we discovered several errors in the data related to sampling 3/The Ghana Living Standards Survey was conducted over a sample of 3200 households. 12 deficiencies (for details see Grootaert, 1992). One was the observation that average household size, as recorded in the survey, dropped from about eight persons to about six persons over the four-year period. It was clear that this did not correspand to a demographic reality and the culprit was found to be a flawed listing procedure, before the start of the 1985 data collection. Indeed in 1984, a listing operation was undertaken within each of the 100 retained primary sampling units (PSU). Contrary to customary survey practice, these PSU's were not listed exhaustively (because the units selected were too large). The field staff responsible for the listing was instructed to start at a random point in the PSU and to list every fourth househola until 64 households were listed. In practice, true randomness was not achieved and larger dwellings with larger households turned out to have been over-listed and hence were over-represented in the final sample. This flaw in the listing could easily have been corrected at the time of field operations, had design and supervision been better. However, it required a significant effort at the analytic stage to compensate for this error through the construction of ex-post corrective sampling weights (see Demery and Grootaert, 1992). In 1987, the listing techniques were revised. Each PSU was now completely listed: this eliminated the earlier problem and the 1987-88 household size corresponds closely to that recorded in the 1988 population census. It is clear that, in principle, one should avoid making ex-post corrections to data and this experience underlines the importance of close supervision of the field work underlying the selection of the sample, and of keeping the field work simple. An exhaustive listing of small PSU's is simpler and less error-prone than the requirement that enumerators undertake random selections during the field work. Any error made at the sampling stage will inevitably require often time-consuming ex-post corrections at the analytic stage. (In addition to the correction for household size, corrections were also necessary for over-sampling of high income PSU's in Abidjan in 1985-86, and under-sampling of rural PSU's in 1987-88 - see Grootaert, 1992.) The CILSS was designed with a self-weighting sample. One reason offered for this design was that self-weighting data are easier to analyze. In practice, it did not work 13 out that way because of the need to apply ex-post weights to correct for various sampling deficiencies. However, a self-weighting design is also non-optimal from an analytic point of view, if the total sample size is small. The design implies that each socio-economic group in the population or each region is represented in the sample in exactly the same proportion as it is represented in the total population. With a sample size of 1600 households, this means that every socio-economic group or region which is less than 10-15% of the population cannot be analyzed with any significant degree of confidence. In such circumstances, it is preferable to over-sample small socio-economic groups or sparsely populated regions when these are relevant analytic entities, e.g. because they are policy target groups. In this fashion sampling weights (inverses of the selection probability) can be constructed a priori. The argument that unweighted data are easier to analyze no longer holds in an era of computerized data analysis. The typically available statistical software, such as SAS, SPSS, etc. can easily accommodate the use of weighted data with little or no added complexity for the user. A lesson that emerges from all this is the importance of involving analysts and policy makers in survey design. They need to identify a priori, using existing knowledge, what the important socio-economic and target groups are on which the survey must be able to report. The sample designer can then compose the sample in such a way that certain groups will be under-sampled and others will be over-sampled, in order to maximize the analytic usefulness of the resulting sample. 14 S. Frequency of Data Collection: The Essence of Monitoring The CILSS results on the incidence of poverty very clearly demonstrate the importance of monitoring changes in poverty and welfare annually. While the direction of change in poverty in CMte d'Ivoire was not unexpected in light of the macro-economic evolution in the country, the magnitude of the year-by-year changes and the speed with which the macro-economic decline in 1987 and 1988 was transmitted to the household level, were surprising. These changes would not have been picked-up if the survey had not been repeated each year. This being said, it is clear that monitoring should not take place with a survey of the complexity of the CILSS. Like many aspects of management, the complexity of data management increases more than linearly with the amount of data that needs to be managed and the potential for error likewise increases exponentially. An integrated household survey with full income and expenditure accounting can provide a useful benchmark or starting point for successive monitoring exercises, but it need not be repeated every year. We suggest therefore that an integrated survey of the CILSS-type be undertaken every four or five years, if the country has the analytic capacity to take full advantage of the data. In the intervening years, a much simpler survey could be used to monitor the evolution of welfare and poverty. The key variable to be monitored is household expenditure, for conceptual reasons and because our research indicates that it reacts rapidly to changes in the macro-economic environment. Beyond that it is also necessary to include basic needs indicators because they do not necessarily evolve in the same direction or show the same regional or socio-economic patterns. Depending upon the policy priorities in the country, we suggest that a set of well-chosen indicators, which rarely would need to number more than 15 or 20, would be adequate to do the job. The "priority analysis" proposed in Demery, Ferroni and Grootaert (forthcoming) has some suggestions for the selection of such indicators. 15 6. The Role of Panel Data: An Unavoidable Challenge? The CILSS contained a rotating panel. This means that the longitudinal data derived from the survey did not span the four-year period, but consisted of three overlapping panel data sets, each covering two years. We have analyzed the panel data and compared the findings on poverty with the cross-sectional results (see Grootaert and Kanbur, 1992b). In general they confirm one another, but not entirely. The panel component was undertaken with only 10-15% attrition in the second year (some households could not be found or refused to participate in the survey a second time). While such attrition rate is not excessive, the households who dropped out of the panel were not a random subset. In particular, the main attrition occurred among better-off households. This was the opposite of the expectation a priori, that poor and more mobile households would be those most likely not to be found again. The main usefulness of the panel data in the context of welfare and poverty analysis was to identify the extent of "permanent poverty" and the characteristics of the "hard-core poor". A surprising result was that mobility in and out of poverty was much greater than had been thought. But the characteristics of the hard-core poor were not significantly different from those groups where the poverty incidence had been found to be highest on a cross-sectional basis. It is clear that the identification of people who remain in poverty over several years, as distinct from the transitorily poor who can escape poverty on their own means, is of key importance for targeting poverty alleviation policies. This information can only be learned fronm longitudinal data, but our experience suggests that two years may not be a long enough period for this. Also, if policy considerations provide a strong argument for panel surveys, the practical question is whether such panel data should be colle-ted as 2at of an otherwise cross-sectional survey. We feel that the limited gain in understanding poverty from a two-year rotating panel was not worth the added complexity in survey design, field implementation and data use which this feature 16 introduced. Our recommendation is to continue the attempts to collect panel data in developing countries, but to do so in a separate survey. To be really useful, panel data collection would have to be extended over longer periods than two years, although this will of course raise difficulties and costs involved in finding the same households over subsequent years. If a country undertakes an integrated survey every four to five years and a lighter monitoring survey in the intervening years, a small survey panel could be conducted in parallel. Analytically it would be helpful if the monitoring sample and the panel sample are drawn from the same master sample, because this would allow panel results to be corrected for any loss of representativeness resulting from non-random attrition. 17 7, VWhat To Do In Future Surveys? Our experience in analyzing the four years of the CMte d'Ivoire Living Standards Survey data in an attempt to trace the evolution of household welfare and poverty, has clearly shown that this type of survey has all the necessary data for policy oriented research on poverty. It contains in fact much more information than is needed. Our work, however, has also brought out some difficulties and shortcomings of the surveys. On the basis of this experience we would like to suggest what to keep and what to change in future surveys. The list of what to keep includes the combination of expenditure and basic needs data in one survey, and the annual monitoring of welfare and poverty. The latter should however occur with a "lighter" survey. The full-fledged living standards survey could be undertaken once every 4-5 years, to provide benchmark data and to permit in-depth analysis of household behavior and response to policy. The collection of panel data should be continued, but in a separate survey. The list of what to change includes a simplification of the questionnaire according to actual use of information, a doubling (at least) of the sample size and an amended sample design to reflect better analytic priorities. Improved price data collection, but independent from the survey, should receive high priority. The table on the next page summarizes the recommendations suggested in the paper. 18 Recommendations for the Monitoging of Poverty and Welfare through a Household Survey L - I - CILSS Recommendation Content Houcehold Expenditure full detail full detail Household Income full detail income composition based on qualitative questions Basic Needs full detail selected indicators Data Quality Income from non-farm cost-minus-revenue yields to be revised following household enterprises negative incomes experimental survey work Household expenditure mixed reference periods to be revised based on ongoing research Prices insufficient coverage remove from survey - enhanced price collection as part of CPI Community Survey too much rmissing infornmation integrate better into household survey Samole size and design Sample size 1600 households 3000-4000 households Sample design self-weighting unequal representation of regions and groups based on analytic priorities Sample selection flawed listing procedure simplify procedures and improve field supervision Freauencv Monitoring 4 consecutive years undertake once every 4-5 years; 'lighter' monitoring survey in between Panel Data Panel design rotating panel separate panel survey 19 What then would be the "ideal" data collection scheme in a developing country, aimed at policy-relevant analysis of poverty and welfare? Base-line data would be provided by an integrated survey collecting full information on household expenditure and income and on basic needs fulfillment. The survey would incorporate methodological improvements particularly with regards to the reference and recall periods for expenditures, and the collection of income of non-farm household enterprises. This survey would be repeated every five years and cover a sample of three- to four-thousand households, using a sample design which ensures adequate representation of all important socio-economic and target groups. These groups would be identified before the design of the survey collaboratively by data producers and data users. In the intervening years, poverty would be monitored with a lighter survey focusing on household expenditure and selected basic needs indicators. The sample could now be much larger if resources permit this and the survey would be repeated at least once a year, possibly more frequently if the country is undergoing rapid macro- economic or structural change. Parallel to this, a small panel of households would be surveyed, drawn from the same master sample and runniig over the same period as the monitoring survey. Both the integrated and the monitoring survey would have a well-integrated community module to obtain information on health and education facilities and other relevant infrastructure. At the same time, price collection in the country would be improved to cover more items, especially non-food, and be extended to the rural areas. In this way, the consumer price index would be useable as a deflator for household expenditure in the context of welfare analysis. We do not know that any one country in Africa is currently embarked on this road, but several changes have been introduced recently in various countries which go in the right direction. We are particularly looldng forward to the analysis of the Priority Surveys which in the past twelve months have been started in more than a dozen Sub- Saharan African countries and which come close to the monitoring survey we have in 20 _ mind. Neither data collection nor analysis will ever be "ideal", but the important feature is the feedback between analysts and data collectors which leads to gradual improvements in data collection and analysis. Our hope in writing this note is that some of the lessons we learned while analyzing the CILSS will be useful in this process. 21 References Ainsworth, M. and J. Munoz (1986). "The C6te d'Ivoire Living Standards Survey - Design and Implementation." Living Standards Measurement Study Working Paper, No. 26, The World Bank, Washington, D.C. Chesher, A. (1991). "Two Econometric Issues in the Analysis of Household Surveys." Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed.) Daho, B. (1992). "La Qualite des Donnees de l'Enquete Permanente Aupres des Menages de Cote d'Ivoire." Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed.) Demery, L., M. Ferroni and C. Grootaert (eds.) (forthcoming). "Understanding The Social Dimensions of Adjustment ." The World Bank, Washington, D.C. Demery, L. and C. Grootaert (1992). "Correcting for Sampling Errors in the Measurement of Welfare and Poverty: The Case of the Cote d'Ivoire Living Standards Survey." Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed.) Glewwe, P. (1987) "The Distribution of Welfare in the Republic of C6te d'Ivoire in 1985. "Living Standards Measurement Study Working Paper, No. 29, The World Bank, Washington, D.C. Glewwe, P. (1990) "Improving Data on Poverty in the Third World - The World Bank's Living Standards Measurement Study." PRE Working Paper, No 416, The World Bank, Washington, D.C. Glewwe, P. (1991) "Investigating the Determinants of Household Welfare in the Cote d'Ivoire." Journal of Development Economics, Vol. 35. Grootaert, C. (1986). "Measuring and Analyzing Levels of Living in Developing Countries: An Annotated Questionnaire." Living Standards Measurement Study Working Paper, No. 24, The World Bank, Washington, D.C. Grootaert, C. (1992). "The Evolution of Welfare and Poverty during Structural Change and Economic Recession - The Case of C6te d'Ivoire, 1985-1988. " Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed). Grootaert, C. and R. Kanbur (1990). "Policy-Oriented Analysis of Poverty and the Social Dimensions of Structural Adjustment - A Methodology and Proposed Application to C6te d'Ivoire, 1958-88." Social Dimensions of Adjustment Working Paper, No. 1, The World Bank, Washington, D.C. 22 Grootaert, C. and R. Kanbur (1992a). "A New Regional Price Index for C6te d'Ivoire Using Data from the International Comparisons Project." Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed). Grootaert, C. and R. Kanbur (1992b). "The Lucky Few Amidst Economic Decline: Distributional Change in C6te d'Ivoire as seen through Panel Da a Sets, 1985- 88."Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed). Grootaert, C. and T. Marchant (1991). "The Social Dimensions of Adjustmen; Priority Survey - An Instrument for the Rapid Identification and Monitoring of Policy Target Groups."Social Dimensions of Adjustment Working Paper, No. 12, The World Bank, Washington, D.C. Grosh, M.E. (1991). "The Household Survey as a Tool for Policy Change - Lessons From the Jamaican Survey of Living Conditions." Living Standards Measurement Study Working Paper, No. 80, The World Bank, Washington, D.C. Johnson, M., A. McKay and J.I. Round (1990). "Income and Expenditure in a System of Household Accounts - Concepts and Estimation." Social Dimensions of Adjustment Working Paper, No. 10, The World Bank, Washington, D.C. Kanbur, R. (1990). "Poverty and the Social Dimensions of Structural Adjustment in C8te d'Ivoire." Social Dimensions of Adjustment Working Paper, No. 2, The World Bank, Washington, D.C. Kozel, V. (1990). "The Composition and Distribution of Income in C6te d'Ivoire." Living Standards Measurement Study Working Paper, No. 68, The World Bank, Washington, D.C. Oh, G.T. and M. Venkataraman (1992). "Construction of Analytic Variables and Data Sets from the C6te d'Ivoire Living Standards Survey, 1958-88: Concepts, Methodology and Documentation." Poverty and Social Policy Division, The World Bank, Washington, D.C. (Processed.) Scott, C. (1993). "Estimation of Annual Expenditure From One-Month Cross-sectional Data in a Household Survey." Interstat, no. 8. Scott, C. and B. Amenuvegbe (1990). "Effect of Recall Duration on Reporting of Household Expenditure - An Experimental Study in Ghana." Social Dimensions of Adjustment Working Paper, No. 6, The World Bank, Washington, D.C. 23 Policy Research Working Paper Serles Contact Title Author Date for paper WPS1060 Economies of the Former Soviet Dmitri Steinberg December 1992 E. Zamora Union: An Input-Output Approach 33706 to the 1987 National Accounts WPS1061 Dynamic Response to Foreign Klaus Schmidt-Hebbel December 1992 A. Maranon Transfers and Terms-of-Trade Luis Serven 31450 Shocks in Open Economies WPS1062 Economic Development, Robert E. B. Lucas December 1992 WDR Environmental Regulation, and the David Wheeler 31393 International Migration of Toxic Hemamala Hettige Industrial Pollution, 1960-88 WPS1063 Mongolia: Privatization and System Cevdet Denizer December 1992 A. Oropesa Transformation in An Isolated Alan Gelb 39075 Economy WPS1064 More Evidence on Income George R. G. Clarke December 1992 R. Martin Distribution and Growth 39065 WPS1065 Strengthening Uganda's Policy Thomas Owen Eisemon January 1993 C. Cristobal Environment for Investing in John Sheehan 33640 University Development George Eyoku Franklin Van Buer Delane Welsch Louisa Masutti Nat Colletta Lee Roberts WPS1 066 Pollution Control in a Decentralized Antonio Estache January 1993 A. Estache Economy: Which Level of Kangbin Zheng 81442 Government Should Subsidize What in Brazil WPS1067 Returns to Investment in Education: George Psacharopoulos January 1993 G. Psacharopoulos A Global Update 39243 WPS1068 Enterprise Reform in Eastem Europe Sweder van Wijnbergen January 1993 B. Pacheco 37033 WPS1069 Why Is There Proportionately More Estelle James January 1993 D. Evans Enrollment in Private Schools in Some 37496 Countries? WPS1070 Economic Fundamentals of Road Timothy Hau December 1992 Pricing: A Diagrammatic Analysis WPS1071 Congestion Charging Mechanisms: Timothy Hau December 1992 An Evaluation of Current Practice Pollcy Research Working Paper Serles Contact Title Author Date for paper WPS1072 Costs of Alternative Treatments for Brooke R. Johnson January 1993 0. Nadora Incomplete Abortion Janie Benson 31091 Janet Bradley Aurora RFbag3 Ordohiez Catia Zambrario Leonard Okoko Leticia Vazquoz Chavez Paulina Quiroz Khama Rogo WPS1 073 Fiscal Aspects of Developing Peter J. Monticl January 1993 R. Vo Country Debt Problems and Debt-Service 31047 Reduction Operations: A Conceptual Framework WPS1074 How Moving to World Prices Affects David G. Tarr January 1993 N. Artis the Terms of Trade in 15 Countries 38010 of the Former Soviet Union WPS1075 Many Paths to Skilled Employment: Adrian Ziderman January 1993 PHREE A Reverse Tracer Study of Eight Robin Horn 33680 Occupations in Colombia WPS1 076 A Presumptive Pigouvian Tax on Gunnar S. Eskeland January 1993 C. Jones Gasoline: Analysis of An Air Pollution 37699 Control Program for Mexico City WPS1077 Risk Management Prospects for Panos Varangis January 1993 D. Gustafson Egyptian Cotton Elton Thigpen 33714 Takamasa Akiyama WPS1078 The Evolution of Welfare and Poverty Christiaan Grootaert January 1993 E. Vitanov Under Structural Change and Economic 38400 Recession in C6te d'lvoire, 1985-88 WPS1079 How Useful Are Integrated Household Christiaan Grootaert January 1993 E. Vitanov Survey Data for Policy-Oriented 38400 Analyses of Poverty? Lessons from the C6te d'lvoire Living Standards Survey WPS1080 A New Regional Price Index for C6te Christiaan Grootaert January 1993 E. Vitanov d'lvoire Using Data from the Ravi Kanbur 38400 Intematicnal Comparisons Project WPS1081 Correcting for Sampling Bias in the Lionel Demery January 1993 E. Vitanov Measurement of Welfare and Christiaan Groota,art 38400 Poverty: The Case of the C6te d'lvoire Living Standards Survey