LSA I2qq I il 1,1' 'It III LI dt- 1' liE t V , I iRil t,,.,l Poverty Comparisons and Household Suirvev Design ,rtevn Howeib iaildT~~ OIac£3n:anin~iw Poverty Comparisons and Household Survey Design The Living Standards Measurement Study The Living Standards Measurement Study (LsMs) was established by the World Bank in 1980 to explore ways of improving the type and quality of household data collected by statistical offices in developing countries. Its goal is to foster increased use of household data as a basis for policy decisionmaking. Specifically, the LSMS is working to develop new methods to monitor progress in raising levels of living, to identify the consequences for households of past and proposed government policies, and to improve communications between survey statisticians, analysts, and policymakers. The LSMS Working Paper series was started to disseminate intermediate prod- ucts from the LSMS. Publications in the series include critical surveys covering dif- ferent aspects of the LSMS data collection program and reports on improved methodologies for using Living Standards Survey (Lss) data. More recent publica- tions recommend specific survey, questionnaire, and data processing designs and demonstrate the breadth of policy analysis that can be carried out using Lss data. LSMS Working Paper Number 129 Poverty Comparisons and Household Survey Design Steven Howes and Jean Olson Lanjouw The World Bank Washington, D.C. Copyright X) 1997 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First printing April 1997 To present the results of the Living Standards Measurement Study with the least possible delay, the typescript of this paper has not been prepared in accordance with the procedures appropriate to formal printed texts, and the World Bank accepts no responsibility for errors. Some sources cited in this paper may be informal documents that are not readily available. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors or the countries they represent. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imnply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. Requests for permission to reproduce portions of it should be sent to the Office of the Publisher at the address shown in the copyright notice above. The World Bank encourages dissemination of its work and will normally give permission promptly and, when the repro- duction is for noncommercial purposes, without asking a fee. Permission to copy portions for classroom use is granted through the Copyright Clearance Center, Inc., Suite 910, 222 Rosewood Drive, Danvers, Massachusetts 01923, U.S.A. ISBN: 0-8213-3862-5 ISSN: 0253-4517 Steven Howes is an economist at the World Bank; this paper was written while he was in the Poverty and Huma;n Resources Division of the Bank's Policy Research Department. Jean Olson Lanjouw is Assistant Professor in the Department of Economics at Yale University. Library of Congress Cataloging-in-Publication Data Howes, Stephen R., 1964- Poverty comparisons and household survey design / Stephen R. Howes and Jean Olson Lanjouw. p. cm. - (LSMS working paper, ISSN 0253-4517; no. 129) Includes bibliographical references. ISBN 0-8213-3862-5 1. Poverty-Statistical methods. 2. Household surveys- Statistical methods. I. Lanjouw, Jean Olson. II. Title. III. Series. HC79.P6H68 1997 339.2'2'-dc2l 96-53397 CIP Table of Contents Foreword ................................................ vii Abstract .................. ix Acknowledgments .................. xi 1. Introduction .................. 1 2. Household survey designs .................. 3 3. Estimators of totals and means and their variances appropriate for complex survey designs.. ................ 9 4. Poverty and other welfare measures .................. 19 5. Some examples .................. 20 6. Concluding comments .................. 25 Appendix I: Proofs .................. 27 References .................. 34 Tables Table 1: Features of sample design from some recent national household consumption surveys, and a comparison with simple random sampling. 8 Table 2: Sample design for Pakistan and Ghana LSMS surveys .20 Table 3: Sample design effects for mean expenditure, household size and various poverty measures for two household surveys .23 v Foreword The ability to monitor poverty is crucial to assessing the success of policies designed to improve standards of living. With the LSMS household surveys and others now available, many developing countries now have the data-base required to undertake this policy monitoring. Earlier LSMS papers have shown how to approach the measurement of poverty statistically, so as to be able to distinguish real changes from sampling variation. This paper extends the earlier work to show how to take into account typical sample designs in calculating statistical measures of poverty change. It uses LSMS data sets both to show how household surveys differ greatly from the "simple random sample" paradigm and to illustrate the importance of basing statistical formulae on the actual sample design used. I S t Lyn Squire, Director Policy Research Department vii Abstract Poverty comparisons - an increasingly important starting-point for welfare analysis - are almost always based on household surveys. They therefore require that one be able to distinguish underlying differences in the populations being compared from sampling variation: standard errors must be calculated. So far, this has largely been done on the assumption that the household surveys are simple random samples. But household surveys are more complex than this. We show that taking into account sampling design has a major effect on standard errors for well-known poverty measures: they can increase by around one-half. We also show that making only a partial correction for sample design (taking into account clustering, but not stratification, whether explicit or implicit) can be as misleading as not taking any account at all of sampling design. ix Acknowledgments We would like to thank, for their provision of data, information and/or comments: Benu Bidani, Gaurav Datt, Mark Foley, Paul Glewwe, Margaret Grosh, Dean Jolliffe, Peter Lanjouw, Martin Ravallion, Chris Scott, Kinnon Scott, Salman Zaidi, and Qing-hua Zhao. Mr Ranzam of Pakistan's Federal Bureau of Statistics also provided us with useful information. xi I. Introduction Has poverty increased or fallen? Is urban or rural poverty higher? Will some policy under consideration reduce or increase poverty? These are typical of the questions asked in poverty analyses. To provide answers, recourse is required to household surveys. But a survey is not a census. It is a sample, with a size typically numbering in the thousands of households, from which conclusions concerning populations typically numbering in the millions must be drawn. This leads to the fundamental problem that any comparative analysis must distinguish population differences from sampling variation. A series of recent papers have stressed the importance of this and have provided the tools by which standard errors can be calculated (Howes, 1993, Kakwani, 1993, Pudney and Sutherland, 1994, Ravallion, 1994). The problem with the current state of play is that, in presenting statistical methods and results for use in poverty comparisons, the assumption has been made that the household surveys being analyzed are simple random samples of the populations from which they are drawn. In fact, however, they are not. Household surveys are far more complex in their design. This can best be seen by analogy. Consider each household in the population to be represented by a number written on a piece of paper. All pieces of paper are of equal size and are placed in a hat. Then a household survey would be a simple random sample if it were selected by blindly drawing numbers from the hat. Household surveys differ in a number of ways from this simple model: * One may have many hats from which sub-samples are drawn: often populations are first divided into strata, each of which may be considered a separate hat or sub-population. * There will probably be "hats within hats". A random selection of clusters, such as villages, is invariably first made from the population (or from each stratum). Households are then randomly drawn from these smaller clusters. * Some numbers (households) have a higher probability of selection than others. * The selection of numbers may not be "blind", that is, random. Instead, it may be systematic: the numbers may be lined up and every nth one chosen. As we will see, this is only the start of a fairly long list of complexities which household surveys incorporate. What are the implications for statistical poverty (and welfare or inequality) analysis of these various features? This is the central question which this paper addresses. We present estimators of the variance of poverty measures appropriate for typical survey designs. And we assess the influence departures from a simple random sample are likely to have on the precision with which poverty estimates can be made. Which departures have a substantial impact and which can be safely ignored for the sake of convenience? We show that, under sample designs commonly in use, conventional formulae may lead to estimates of standard errors for poverty measures which are only two-thirds the size they should be. That is, ignoring sample design can make us think estimates are considerably more precise than they actually are. It should be noted at the outset that the key results we make use of - relating to the variances of sample means - have been known since the fifties, and are presented in several textbooks on the subject of sample design.' However, we have not found any work which links the general results available for complex survey designs to the typical features of household surveys, let alone to poverty analysis. Moreover, the fact that these general results have been almost completely overlooked in the empirical and theoretical literature on poverty measurement suggests that there is a need to set out clearly the formulae required and to provide a strong motivation for their use.2 In the next section, we provide a more formal and detailed treatment of the various ways in which household surveys can differ from the simple random sample model. In Section 3, we provide the basic formulae. Section 4 applies these formulae to poverty measures. Section 5 gives some examples of the importance of taking into account sample design. Section 6 concludes. The appendix provides proofs of the paper's key results. 1. Kish (1965) is the classic on this subject. It gives what is still probably the most comprehensive treatment, though not the simplest. Som (1973) provides a very clear presentation of results. Hansen, Hurwitz and Madow (1953) provide proofs. Levy and Lemeshow (1991) provide an introduction, as does Scheaffer, Mendenhall and Ott (1990). 2. Deaton (1994) provides an excellent introduction to and analysis of household surveys and to some extent fulfills these two aims. However, his treatment does not cover many of the, common problems raised by the use of household surveys (such as when one has stratification and clustering, or raising factors and clustering, or all three, or when one is estimating per capita (rather than household) means). Scott and Amenuvegbe (1989, pp.55-57) cover - in relation to a survey of Mauritania - the joint use of clustering and (implicit) stratification (not raising factors). However, their intention is to provide only formulae, and they add neither motivation nor explanation. Rodgers and Rodgers (1992) use a alternative method to that provided in this paper (balanced repeated replications - see footnote 16), but provide only results. In general it would seem to be that poverty analysis of developing countries pays less attention to sample design and its implications than analysis of developed countries (see Rodgers and Rodgers, 1992, and Duncan and Rodgers, 1991, as examples for the United States). Incorporation of sample design also seems to be much more widespread among demographers than economists (see Cleland and Scott, 1987). 2 2. Household survey designs In this section, we discuss eight sampling features one needs to be aware of when analyzing household surveys. We then (in 2.9) present examples showing how different surveys incorporate various of them. This section, like the rest of the paper, draws most of its examples from surveys conducted as part of the World Bank Living Standards Measurement Study (LSMS). This is simply because we are more knowledgeable about these surveys' designs than others. However, LSMS surveys do present a range of household survey designs. Since each survey in the series is carried out in conjunction with the statistical bureau of the country in which it is being conducted, different LSMS surveys incorporate different designs, depending on prevailing practice in the countries concerned. 2.1. Clustering One feature which most household surveys share is that they are clustered. That is, the first selection (from the population or sample frame) is not of households, but of some higher level units such as villages or street blocks, known as clusters. This is the case for all nationwide household surveys, though some very small surveys (of one or several villages, say) are not clustered. As we will see, clustering leads to higher variances. Its justification is purely practical. By concentrating sampled households in a small number of geographical areas, clustering drastically reduces survey costs per household. Under some sample selection procedures, a cluster can be selected more than once: that is, more than one group of households, say, can be selected from a single cluster. To avoid confusion, we refer to each selection of a group of households (or "ultimate sampling units" - see 2.3 below) from the cluster as a 'cluster take'. 2.2. Stratification Many surveys are stratified, typically into geographical regions, such as urban/rural and provincial, but also by other characteristics. The difference between strata and clusters is simply explained. Both strata and clusters divide the sampling frame exhaustively and exclusively. If both are present, the clusters sub-divide the strata. All strata are included in the sample (each with its designated sample size), but only a selection of clusters are included in the sample. Stratification is very common, but not universal. Stratification with equal sample rates in the strata ensures a more representative sample overall, and so reduces variance. It also can also be used to ensure that one obtains sufficient observations from small sub-populations of interest. 3 Note that what we call 'stratification' here is sometimes referred to as 'explicit stratification' to distinguish it from implicit stratification, a form of systematic sampling discussed in 2.7. We analyze both forms of stratification, but when we use the term 'stratification' without qualification we are referring to the explicit variety.3 2.3. Ultimate sampling unit The ultimate sampling unit is the smallest level of population unit sampled by the survey. For most household surveys, the ultimate sampling unit is the household. That is, after first selecting clusters, and then after possibly some intermediate selection stages (see 2.4), the fmal selection of elements is of households. But this is not universal. In the Nicaragua LSMS, the ultimate sampling unit was groups of five households: each cluster was diviided into groups of five households (on the basis of geographical proximity) and a selection of these groups, rather than of individual households, was made. Of course, by selecting groups of- households one is selecting individual households. However, the ultimate sampling unit is the lowest level at which sampling occurs (and hence is the unit in which sample size is measured). In the Nicaraguan case, there was no sampling below the group level: all households within any selected group were chosen.4 2.4. Number of random selection stages The selection process of many samples is two-stage. That is, once clusters have been selected, the ultimate sampling units, typically households (see above), are selected directly from the clusters. However, more than two stages may also be used, especially in large countries. For example, in the Russian Longitudinal Monitoring Survey, raions (regions) provide the first-level clusters. Then voting districts are selected from the chosen raions. These serve as second-stage clusters. In a third stage, households are chosen from the selected voting districts. Note that the reference here is to the number of sampling stages, so stratification is never regarded as the first stage. 3. About half of the World Bank LSMS surveys are explicitly stratified. As far as we know, all the rest are implicitly stratified. 4. Compare the typical case in which the final sampling is of households. Then households are the ultimate sampling unit, and not individuals, even though by selecting households one is selecting individuals. By contrast, if one samples from a list of individuals or if one samples individuals within households (as in the case of some fertility surveys), then individuals do indeed become the ultimate sampling units. 4 2.5. Unequal probabilities of selection Many household surveys are not self-weighting. This means that some households have a higher chance of being selected than others. Variable weights (known as raising or expansion factors) have to be used to prevent estimators being biased as a result. Formally, raising factors can be defined as a set of weights such that the weighted sum of the sample observations of a given variable is an unbiased estimator of the population total of the variable. There are three main reasons why a survey may not be self-weighting. First, when strata are used, the sample may not be distributed over the strata in accordance with the distribution of the population. Instead disproportionate stratification may be used, and some areas deliberately over- represented. These may be areas, typically urban, in which sampling is cheaper or they may be small sub-national political units, such as small provinces, for which one wants to ensure a minimum sample size. Second, even if the survey is intended to be self-weighting, it can end up not being so, owing, for example, to non-response. Finally, a common method of selecting clusters - discussed immediately below - often leads to the sample being non-self-weighting. 2.6. Selection of clusters according to size or estimated size Some clusters are bigger than others in terms of the number of ultimate sampling units (typically households) they contain. It is a widespread practice to give larger clusters a higher probability of selection. If exact cluster sizes were known, then selection could be by probability proportional to size (pps), which has the advantage of allowing an equal number of households to be drawn from each cluster at the second-stage with the sample remaining self-weighting (at least within each stratum). With household surveys, however, the exact size of a cluster at the time of sampling is generally not known prior to sampling. The most recent count may be the last census, which may be a decade or even more out of date. If the discrepancies are expected to be small, they can be ignored and pps can be used (this was done in the Russian Longitudinal Monitoring Survey, for example). Alternatively, the discrepancies can be formally dealt with. In this case, one explicitly recognizes that one is using a proxy for size - typically the out-of-date census but possibly something as weakly correlated with cluster size as area - and is choosing clusters on this basis. This is known as selection by probability proportional to estimated size (ppes). Under ppes, selected clusters are re-enumerated (re-listed). The new list (typically of households in the selected clusters) not only provides the basis for the selection of the ultimate sampling units. It also gives the desired exact measure of size of the selected clusters. The ratio of actual to proxied size can be used in two ways. If the size of the cluster take is held fixed, the ratio can be used to re-weight observations (to give, for example, greater weight to those observations from clusters whose actual exceeds their proxied size, and which are thus under-represented in the survey). Alternatively, the size of the cluster take can be adjusted up or downwards (from some average size) by this ratio. This has the disadvantage of resulting in differently-sized cluster takes, but the advantage of removing the need for cluster weights (see footnote 11 for a formal statement of this). Both methods allow 5 unbiased estimates to be derived from a survey even if only very roujgh or out-of-date information on cluster sizes is available. Both methods are commonly used and both require the same statistical treatment. Note that pps can be regarded as a special case of ppes in which estimated size equals, or is assumed to equal, actual size. 2.7. Systematic or random sampling Systematic sampling is selection at a fixed interval, say s, from a list of units, starting from a random point not more than s units from the start of the list. Its usage in household surveys, at all stages, is widespread. At the final sampling stage, its usage would seem to be universal. What happens here is that ultimate sampling units (say households) from the chosen clusters are listed. Then a sampling interval, s, equal to the total number of households in the cluster list divided by the desired cluster sample size, is defined. Then a random number is chosen less than or equal to s. That household is chosen, as is every sth household from then on until one reaches the end of the list.5 With regards to the choice of clusters (the first stage), the situation is more complex. In some textbooks (e.g., Scheaffer, Mendenhall and Ott, 1990, p.79), pps and ppes are defined to be random. However, in practice, in many surveys, systematic sampling is used at the first stage. These two methods work as follows. In both cases, all clusters in a stratum are placed in a list and their (estimated or actual) sizes, Mc, are cumulated, giving a series with typical element R1=E1=,M,. Say one wants to choose n clusters from a total of N. Using random sampling, one would simply pick n random numbers between 1 and RN. Using systematic sampling, one would define the sampling interval, s=RN/n. A random number, r, less than s would be chosen and the list of numbers {r, r+s, r+2s, ..., r+(n-1)s} compiled. Both methods - random and systematic - result in a list of n numbers, each of which picks out a cluster using the rule that a number in the range (Rl1,Rj] picks out the ith cluster. Both methods make it possible to have clusters chosen more than once. In interpreting systematic sampling, the ordering of the elements in the sample frame (list of clusters or households) is very important. There are two cases. In the first, the ordering of units is random with respect to the variables of interest: an example might be an alphabetical listing (though one can think of exceptions). In the second and more common case, the ordering is non-random: often neighboring clusters or households are placed next to one another. This is sometimes referred to as implicit stratification (see 1.2). The implications of these two methods of ordering are very different and are discussed in the next section (see 3.5). 5. For simplicity, we assume that s is an integer. See Kish (1965, pp.115-116) for details of the procedure to be followed when it is not. 6 Systematic sampling is popular because it is easier to implement than random sampling. Under systematic sampling, only the starting point of the sample selection need be generated randomly. After that, no more attention need be paid to random number generation. In addition, the implicit stratification method generally leads to a more representative sample, and so reduces variance (though it can also make variance estimation difficult - again see 3.5). 2.8. Sampling with or without replacement Say one has random (as against systematic) sampling, and a household or cluster is removed from the sample frame once it has been selected in the sample, so that no unit can be selected twice. This is known as (random) sampling without replacement.6 If the household or cluster is replaced, one has (random) sampling with replacement. (The random ppes sampling described in 2.7 is with replacement.) Since the with/without replacement issue only applies to random sampling, and since most household surveys use systematic sampling, the relevance of the distinction may seem unimportant. It is, in fact, very germane, as, in presenting formulae, we will be forced to approximate systematic by random sampling. The issue is discussed further in 3.5. 2.9. Examples Table 1 gives some examples of some recent household surveys from various developing countries. All of them are clustered and use systematic sampling, but otherwise a diversity of features is present. Table 1 also presents the features which an imaginary household survey conducted as a simple random sample (srs) would possess to highlight the many differences between the srs and real-world designs. It should be noted that the complexity of surveys is not necessarily exhausted by the detail of Table 1. As notes 3 and 4 to the table illustrate, many surveys have methods of selection specific only to that survey.7 6. In practice, random sampling without replacement implies that once a random number has picked out a household or cluster any subsequent random numbers which identify the same element are ignored. 7. The Jamaican Survey of Living Conditions (JSLC, described in PRDPH, 1994) provides another illustration of the individual features which surveys can have, for which there is no solution without approximation. The survey is drawn from the Jamaican Labor Force Survey (LFS), a stratified, clustered, two-stage sample. The JSLC is based on a selection of the LFS's strata, turning these strata into new first-stage clusters. Yet the proportion of LFS strata chosen is high: 100% for one year for some parts of the country, and one- or two-thirds for other years and other parts. How is one to treat these LFS strata? Where less than 100% of them are chosen, they are, strictly speaking, clusters. Yet with the selection rate so high, the assumption of sampling with replacement would give a poor approximation. The without-replacement formulae, however, make informational demands which can not be met with the survey data publicly available (see 3.5). One alternative is to regard the survey as one not of all Jamaica, but of those LFS strata surveyed, and treat the LFS strata as JSLC strata. Applications of survey findings 7 Table 1 Features of sample design from some recent national household consumption surveys, and a comparison with simple random sampling Sample Clus- Strat- Ultimate Number Self- Method Use of If random tered ified sampling of stages weighting' for systematic sampling, unit selection saampling at with or of first without clusters stage/secod replacement ________________________ __________ ~~ ~~stage Simple random sampling No No Household One Yes N/A No (only Either2 one stage) Pakistan Integrated Yes Yes Household Two No ppes Yes / Yes N/A Household Survey 1991 _ Ghana Living Standards Yes No Household Two Yes ppes3 Yes / Yes N/A Survey 1987-88 __ _ Nicaragua LSMS 1993 Yes Yes Groups of 5 Two No ppes Yes / Yes N/A households Russian Longitudinal Yes Yes Household Three4 Yes pps Yes / Yes N/A Monitoring Survey 1993 _ I.__Ye Notes: 1. Most surveys are not self-weighting on account of differential non-response. In the table above, however, we are concerned with whether the surveys are intended to be self-weighting or not. 2. Some authors define srs to be sampling without replacement. Others allow it to be either with or without replacement. We follow the latter convention. 3. The Ghana survey used ppes, adjusting the size of the cluster take on the basis of the ratio of actual to estimated cluster size. However, an integer restriction was used so that cluster takes were either nought, once, twice (etc.) times the target cluster take size (in this case, 16 households). See Scott and Amenuvegbe (1989, 1991) and Section 5 of this paper for further discussion. 4. Most strata have three stages, a couple have only two. to Jamaica as a whole would then depend on the (probably perfectly acceptable) judgement that the survey's sample frame differed little from the country as a whole. 8 3. Estimators of totals and means and their variances appropriate for complex survey designs Before turning to poverty measures, we consider, in this section, the estimation of means of variables. We focus on means not because they are attractive welfare measures, but on account of their simplicity. As we show in Section 4, the estimation of poverty and welfare indices follows on as a direct analogue from the estimation of simple means. Since a mean is the ratio of two totals, we begin with the estimation of (consumption or income) totals. The complexity of survey design is very easily incorporated into the calculation of unbiased estimators of population totals. Suppose that we have a sample of N households, subscripted by j, each with income of yj. To each of the households is attached an expansion factor or weight, wj. Given the correct choice of weights, an unbiased estimator of T, the total income of the population, can be written N t= Wjyj (1) If the sample is self-weighting, then wj is constant for all households. Using weights in this way is both simple and is commonly done, at least implicitly, to obtain means (dividing (1) by the sum of wj).8 Matters are more complex when estimating variances. Now one must consider not just the use of expansion factors but rather all the departures from simple random sampling implied by the sample design. To make any progress, one must therefore be explicit about the type of sample design being dealt with. We assume (for the moment) that the survey being analyzed: (i) is clustered, (ii) uses ppes for the selection of clusters (with pps as a special case) (iii) has any number of strata (with one stratum equivalent to no stratification), (iv) has two stages, (v) has households as the ultimate sampling unit, (vi) was selected by random sampling with replacement at the first stage, and by systematic or random sampling with or without replacement but with equal probability of selection at the second stage, and, (vii) is self-weighting or has cluster-level expansion factors due to the use of ppes and/or disproportionate stratification. 8. Typically, if the sample is self-weighting, one does not have the required expansion factors. This means totals cannot be estimated. Means, however, can be, and still using the method given in this paper. Since, if the weights are constant, they will cancel out in a ratio of totals (which a mean is), one can set the weights to an arbitrary constant, say one. See 3.4 for further detail on this. 9 The two key features of sample design - stratification and clustering - can be separated and dealt with independently. That is, whatever use is made of clustering, stratification is always dealt with in the same way, and vice versa. We first consider these separately (in 3.1 and 3.2), and then bring them together (in 3.3). In 3.4, we move from totals to means. Some of the above assumptions may be questioned, in particular that of random sampling at the first stage, since, as claimed in 2.7, systematic sampling is far more common if not universal. However, the reasons for making these assumptions will soon beco,me evident. In 3.5, we discuss extensions and, more importantly, the extent to which results based on these assumptions can serve as, and are needed as, approximations for results valid for alternative sampling designs. 3.1 Stratification Suppose the population is divided into H strata. (If there is no stratification, H1=1.) In the hth stratum, let th be an unbiased estimator of Th, the total income in that stratum. Then one can re-write equation (1) as H t v ti, (2) h=1 The variance of t is given by H H H Var{F t^ =E Var(t^) +2 E Cov(th,tj,) h=1 h=1 h=1 h4^h (3) H =E Var(t.) h=1 so that if Var(th) is an unbiased estimator of Var(th,), Var(t) can be estimated unbiasedly by summing over the Var(th). Ignoring stratification will overestimate standard errors,9 since the between-stratum variances, which should be ignored, will be included in the estimate. Intuitively, this is because, with stratification, one ensures that no part of the sampling frame goes unrepresented (Kish, 1965, pp. 139-142). Note that stratification must be taken account of whether it is proportionate or disproportionate. 9. Of course, this true only in expectation. We take this qualifier to be implicit here and throughout the paper. By "ignoring stratification" we mean ignoring the fact that different observations come from different strata. We do not mean ignoring the fact that different observations are associated with different expansion factors, even if these expansion factors in fact come from disproportionate stratification (it can be, for example, wrongly assumed that they come from the use of a ppes selection of clusters). 10 3.2 Clustering Given that equations (2) and (3) hold whatever use of clustering is made within each stratum, we can focus on the derivation of th and Vir(tQ), that is, on estimators for a single stratum. Let there be Nh clusters in this stratum from which nh cluster takes are chosen with replacement. Let MhC be the true size of (number of households in) the cluster from which the cth cluster take is selected in the hth stratum and let ZhC be the estimated size of the cth cluster.'° The total estimated stratum population size is Zh = EZh,. Randomly select one cluster with probability proportional to estimated size, i.e. with probability Zc/Zh, and sample mhC of the MhC households. Define the cluster-level expansion factor: W l = Zh Mc (4) Zhc Mhc Note that whC takes into account any or both of disproportionate stratification and the use of ppes. If pps is used, ZhC=Mhc and, assuming mhc to be constant within a stratum, the weights reflect only the former (and so will vary only between strata)."' Since the selection of clusters is random, any one cluster take can be used to form an unbiased estimator of total income, Th, namely thC: mh, thc = Whc Yhd (5) i=1 Note that this is estimated mean income in the cth cluster times Zh*(MhJZh), the estimate of the size of the hth stratum based on cluster c. Hence th, is an unbiased estimator of Th. It is not, however, efficient. A more efficient estimator can be obtained by utilizing information from all cluster takes. From nh cluster takes we can construct a combined unbiased estimator of Th as the unweighted mean of the thc: =1 (6) th = th ( nhc=l Note that if we define (for some h and all c) wj = whC/nh then we can rewrite (6) in the form we 10. c is used to refer both to the cluster and to the cluster take. In reality, if more than one take can be selected from a cluster, the cth cluster take may come from cluster b. However, to avoid a proliferation of notation, this possibility is ignored in the exposition though it is covered by the results provided. 11. If ppes is used and nm, is held constant then wb will vary across clusters. The alternative ppes method set out in 2.6 adjusts mi: according to the ratio MhJ/Zm to hold ww constant within the hth stratum. 11 began with, namely, analogously to (1), as th =E wjYj (7) jEh However, (6) is more useful for our purposes, since it leads directly to the variance of th. Let Shc be the number of possible takes which could be drawn from the cth cluster if it were selected at the first stage. Define Sh to be the sum of Shc over all Nh clusters. Then, two-stage sampling with replacement is no more than sampling with replacement nh of these Sh elements. Hence Var(th) = E(th-.rh)2 = 1 2 si_ 2 (8) 8hC"l s=l1 where each cluster is only counted once (from 1 to Nh). It follows then that an unbiased estimator of Var(th) is given by V&r(th) = 1 (tk - th (9) nh(nj - 1)(c=j (See Appendix 1.1 for a proof, which parallels the one used for simple random samples.) This is a remarkable and useful result, basic to the sampling literature, but rarely applied to poverty analysis. The result is remarkable for its simplicity in the face of all the com]plexities of sample design which it is capturing. It is useful for its very weak assumptions about sampling at the sub- cluster level. In particular, the formula takes no explicit account of sampling variability within each cluster. All that is assumed is that the thc are unbiased estimators of Th. thc may be obtained using random sampling with or without replacement or systematic sampling. 12 Unlike the case of stratification, ignoring clustering will often lead to an underestimate of Var(t). If one (wrongly) assumed simple random sampling, one would assume Var(th)= Var(tj (10) Mh where th,i=yhc,whjnt, is an estimator of Th based on the ith household in cluster c in stratum h and mh = , . In fact the correct variance is given by (8). As shown in Appendix I.3, if there is two-stage sampling, and we make the simplifying assumption that sampling is with replacement at both stages, then (8) may also be written 12. (9) does assume that one has at least two cluster takes per stratum (so that n--l > 0). If this condition is not satisfied, say because data is missing, one can construct "pseudo-strata" by joining neighboring strata until all are made up of at least two cluster takes. 12 Var(t.) 1 [1 + (mh - Dpj( Mh where n^, ph = hCo @,y m*= M (12) Ph Var(thci) h mh Ph iS the intra-cluster correlation coefficient for the hth stratum. If Ph > 0 (that is, if there is homogeneity within the cluster with respect to the variable of interest), which will be the case in practice, then assuming simple random sampling will yield an estimator of Var(th) which is biased downward. The intuition is the same as that for stratification, though here it works in the opposite direction: clustering leads to a less uniform coverage of the population, and so reduces precision. Both (9) and (11) - the latter with caps added - provide estimators of Var(1Q). The former is simpler, since it involves no computation of covariances. Moreover, if sampling at the second- stage is systematic, then any estimation of Ph based on a single sample will reflect the influence of the ordering on which the systematic selection was based as well as the degree of intra-cluster correlation, and so may be biased. Equation (9), by contrast, is valid for both systematic and random sampling at the second stage. 3.3 Stratification and clustering For a sample which is both stratified and clustered, use the formulae for th and Var(th) in 3.2 in the formulae for t and Var(t) in 3.1. Bringing together our earlier remarks on the effects of clustering and stratification, for a given sample size and population variance, the variance of estimators is smaller the greater the homogeneity between clusters within strata and the greater the heterogeneity within clusters. The introduction of clustering increases within- stratum homogeneity since clusters differ by less than households do. This means that stratification becomes a more potent tool for increasing estimator precision once one has clustering (Kish, 1965, p.164). We will see this at work in Section 5. 3.4 Means An estimated mean is a ratio of two totals, with the estimated total of the variable of interest as the numerator and estimated or actual population size as the denominator. The population size will either be the number of households (if one wants a household mean) or the number of individuals (if one is after a per capita mean). We can handle both cases using ph, 13 an estimator of the population (however defined) in the hth stratum based on data from the cth cluster take in that stratum. Now, Ph. is exactly analogous to thc defined in (5): for a household mean, replace the Yhci in that equation by ones; for a per capita mean replace Yhci by household size (hhj. Likewise, Ph will be defined analogously to th in (6) as the simple mean of the ph, and an estimator of its variance will be derived identically (see (9)). Finally, p will be defined analogously to t in (2) as the sum of the Ph. Average income can then be estimated by y, defined as Y=t (13) p Since this estimator is a ratio of two random variables, it will be biased but consistent. Using a Taylor's expansion, a consistent estimator of its variance is Va&r = 1 [ Var(t) + y2 Var(p) - 2y Cov(t, p)]. (14) p The covariance is the only new term. If there are strata, then the covariance for the entire sample will be the sum of the covariances within each stratum. The covariance within a single stratum is estimated by nk E (Qhe th)07P&-Ph (15) CoV(th, Ph) = ) These formulae may also be used to calculate means and their standard e rrors for sub- populations of interest. If whole clusters are contained within a sub-population, for example, urban or rural, then only those clusters are used. If clusters have elements frcm various sub- populations, for example, literate or illiterate households, then, for a given category, the relevant households are included and other households are given a weight of zero. In either case, but especially the second, the number of cluster takes per stratum may fall below two in which case neighboring strata may be merged (see footnote 12). What if the sample is self-weighting, and one does not have the weights given by (4)? Then totals cannot be calculated, but means still can be. Since the (unknown) weights - the wj of equation (1) - will not vary across clusters or strata, and since they will appear in both the denominator and numerator of (13), they will cancel out, and so can be ignored from the start. Hence, for a self-weighting sample, one can arbitrarily set whC to nh for all c ancd h (and thus wj to 1 for all observations - see p.11). More generally, as long as the weights give unbiased estimates of some multiple or fraction of the relevant population totals, one can use the formulae of this section to calculate means and their variances. This is important since often weights provided with survey data sets are normalized. As long as one is only interesteid in calculating means and not totals, the normalization is not important. 14 Note finally that having a self-weighting sample does not in general imply that Var(p) is zero. If one is analyzing a per capita mean, variation in household size will lead to a non-zero variance for p.l3 However, if one has a self-weighting sample selected using pps and one is estimating a household mean then there will be no variation in cluster sample size and the variance of p will be zero."4 In this case, (14) simplifies to VfrG)= 1 V&r(t). (16) p2 3.5 Alternative assumptions We now considering relaxing some of the initial assumptions. First, say there are more than two stages. This poses no problem as long as we assume the first-level cluster takes are selected with replacement. One takes the number of cluster takes as given by the number of these first-level cluster takes. Expansion factors may not then be constant within the clusters thus defined, since lower-level cluster takes may be chosen using ppes, but unbiased estimates of Th from each first-level cluster can still be obtained. Likewise if individuals or groups of households are the ultimate sampling units. We have already incorporated the case of no stratification (simply set H= 1), but say there is no clustering. Then Var(th) will depend on the precise sample design used within each stratum. Take the simplest case of stratified random sampling (with replacement). Then Var(th) becomes the element variance of in the hth stratum divided by the number of households sampled from that stratum. Hence assumptions (i), (iv) and (v) are easily relaxed, while (ii) and (iii) are not restrictive. What if the absence of self-weighting is due to non-response rather than the use of ppes (our assumption (vii))? If, as sometimes happens, non-response is corrected at the cluster level using weights, this is exactly analogous to having cluster weights due to ppes.15 But in other cases, non-response is assumed to be a function of other-than-cluster-level variables, such as age groups. One is then left with weights which can vary within clusters. For discussion of this more 13. This must of course be taken account of even if one has a simple random sample. Hence the formula presented by Kakwani (1993) for individual-based poverty measures are invalid even if applied to simple random samples (they are valid only for household-based measures). 14. Another way of looking at this is that, if pps is used, the population size must be known (or at least assumed to be known) by the sample designer, though not necessarily the sample analyst. 15. This is assuming that the respondents have the same characteristics as those questioned. In many contexts this will not be the case and non-response will introduce sample selection biases in estimates. There is no general solution to this problem. Note that to correct for non-response in this way, one needs to know whether existing expansion factors assume target or realized cluster sample sizes. This may not always be clear. 15 difficult case see Hansen et al. (1953), Kish (1965), and Pudney and Sutherland (1994). The arguments of the latter authors would suggest there is little loss in accuracy from treating these as random weights and thus in the same way as expansion factors which vary within first-level clusters due to the use of multi-stage clustering (see the previous paragraph). We now come to assumption (vi), the restrictive part of which is that the sampling of clusters is random and with replacement. In fact, we know from 2.7 that many if not all surveys use systematic sampling at the first stage. Why not assume this then? If the ordering on which the systematic sampling is based is approximately random, then systematic sampling will approximate random sampling. This means that the assumption of random sampling is valid for the first type of systematic sampling discussed in 2.7. But say that, as is much more common, sampling is systematic with non-random ordering - the case of "implicit stratiiication". Since the randomness enters only once (at the start) of a systematic draw, unless one has several systematic draws, one does not have enough variation in the sample to estimate variances. So, unless there is more than one sub-sample (known as "replicated sub-sampling"), variances simply cannot be estimated from systematic samples.'6 Since implicit stratification adds precision, we do know, however, that variances calculated under the assumption that the sample is random will tend to be upper bounds on the true variances and so will be conservative. 17 Moreover, we can do better than this. Implicit stratification leads to precision for the same reason as explicit stratification, and we car, in fact treat samples derived using the former method as if they had been derived using the latter, using a method given by Kish (1965). Say that all the clusters in the hth explicit stratum have been listed from 1 to Nh and nh cluster takes have been selected systematically. Group pairs of cluster takes closest to each other into sub-strata (starting with the first two). Calculate Vir(th) as the sum of estimated variances for each sub-stratum, where the latter are calculated using the pairs of clusters takes and (9). 16. There are some samples which are based on replicated sub-samples. For example, the Nicaraguan LSMS is based on four systematic samples of clusters within each stratum, each with a different random start). However, most surveys either are not designed in this way or, if they are, do not record which cluster is part of which replicate. Moreover, the small number of sub-samples involved will lead to unreliable variance estimates (Kish and Frankel, 1970, p.1088). Replicates can also be created by randomly sub-sampling from the sample once it has been collected. However, this muist be done based on the design of the original sample. Taking random sub-samples which ignore any clustering or stratification present and basing variances on the variation in the mean estimates from these different sub- samples will lead to biased variance estimates. The method of "balanced repeated replications" (Kish and Frankel, 1970) is one type of sub-sampling which does take into account sample design. But in this case one might just as well, wherever possible, use the same assumptions used to replicate (post-survey) to calculate the variances analytically. It is only when this is not possible (when one has statistics more complex than unconditional means) that post-survey sub-sampling is appropriate. 17. The qualifier 'tend' is required since one can think of cases in which implicit stratification will reduce precision. This will be the case if the elements display periodicity: so that every sthl element will be simnilar of if the ordering displays very strong trends (Kish, 1965, pp.120-121). Neither case will be relevant in the household survey context. 16 Var(th), thus calculated, would be an unbiased estimator of the hth stratum variance if that hth stratum had been divided into nh/2 sub-strata - each containing, among others, the relevant pair of sampled clusters - and two cluster takes had been selected at random and with replacement from each of these sub-strata. Moreover, Var(th) will also be only slightly biased upwards as an estimator of Var(tb) given actual sample design. There would be no overestimation at all if the ordering of clusters within each sub-stratum was random. Since clusters which are close to each other in a list used for implicit stratification are unlikely to differ much, this assumption is unlikely to be seriously violated and so the overestimation will be small (see Section 5 for an example). Certainly, when there is implicit stratification, the assumption that the ordering is random only within each (artificial or created) sub-stratum is better than the assumption that it is random within each (explicit) stratum. In any case, working with nh/2 sub-strata makes the number of clusters within sub-strata as small as possible subject to the requirement of at least two clusters be chosen per sub-stratum and so makes the overestimation as small as possible."8 Note that this method requires that one knows the original ordering of clusters, from which the sample was systematically drawn. Unfortunately, this information is often not available to analysts. In this case, one has no choice but to ignore implicit stratification altogether. So, the assumption of random sampling will lead to little loss of accuracy if applied in the manner suggested, and in any case we have no better alternative available. However, in assumption (vi), we assume not only random sampling at the first stage, but random sampling with replacement. Why not assume random sampling without replacement? Formulae are certainly available for this case. For simplicity, assume all clusters are of equal size and that equal-probability random sampling (without replacement) occurs at both stages. Then, as shown in Appendix 1.2, an unbiased estimator of the variance of th is given by: ZA, nk mih NV n E (thl-th) 2 E E (th¢g.-thC)2 (17) _aZ __=_h h_ c=l +__ hmh c=1 i=1 Vair(th) = (Nh n) Z= Mh Nn n(nl Nh nh(nh-1) Mh N- nh ) = (i 1) where Nh (nh) is the total (sample) number of clusters in stratum h and Mh (lnlh) the common population (sample) cluster size. (9) will generally exceed (17) (Kish, 1965, p.160) so why not use the latter? There are four reasons. First, (17) is more complex. Once cluster sizes are allowed to vary and pps or ppes used, the unequal probabilities of selection at the first stage must be taken account of (even if the sample if self-weighting) by a weighting-scheme involving single and joint inclusion probabilities for each cluster (see Shah et al., 1993). Second, it won't make much difference. As Nh increases, the second term goes to zero. Third, (17) makes 18. Though Kish (1965, pp. 202 and pp. 285-286) provides this method, his exposition (in terms of collapsing pairs of strata) is slightly different. See Kish also for the case in which nh is not even. Basically, the idea is to end up with as fine an explicit stratification as possible. Kish (p. 286) also provides an alternative method based on a double pairing (the first cluster with the second, the second with the third, and so on). This will give a more precise estimate, though whether it is worth the extra trouble is debatable. 17 greater informational demands. To make the finite population corrections one needs to know the total number of clusters, Nh, and the population of the clusters, Mh, information that the statistical office which collected the data may know, but which the data analyst often will not. By contrast, the information required for the formulae we have presented will typically be available to the analyst. Finally, whatever happens at the first stage, sampling at the second will invariably be systematic, which means the second term of equation (17) will no longer be valid. The advantage of (9), by contrast, is that, to re-iterate, it is valid whatever the method of sampling used at the second stage (as long as it gives rise to unbiased estimators). For all these reasons, the use of formulae valid for sampling with replacement is widely recommended in the sampling literature (see Kish, 1965, p.160; Som, 1973) even if it does lead to a slight overestimation of variances. 18 4. Poverty and other welfare measures We deal with the class of additive poverty measures for which the analogy with a simple mean is direct. Let 7r(khc;) be a measure of household poverty for the ith household in the cth cluster in the hth stratum. There are two aspects to 7r(khc,). First, we define khi = f(Yhd,Xh,.). Yhci has already been defined as total household income/consumption and Xhd is a vector of variables such as prices and household size which allow one to normalize yhCi such that households with equal k's are assumed to be equally well off. In the simplest case, khi will be per capita nominal income (YhcI/hhci) Second, a functional form needs to be chosen for w. The indicator function, I(khCi < k), where k is the poverty line, plays a crucial role here. If the expression inside the brackets is false then I(.) is zero. If it is true then I(.) equals one if one is interested in poverty among households; it equals hhci, the number of people in the household, if one interested in the number of individuals poor. We can now define some specific measures. For the headcount ratio, 7r(ko,) =I(khCi C k). For the poverty gap, w(kh,) =I(khCi S k)*(l-khc/k). More generally, the well-known and much-used FGT family is given by 7r,(khd)=I(khcd < k)*(l-khC,/k)u for a 2 0 (see Ravallion, 1994, for further details). In order to calculate aggregate poverty measures and standard errors, simply replace Yhi in equation (5) with ir(kh,). y, in equation (13), then becomes an estimator of aggregate poverty, with a variance estimator given by (14). Use of the simple formulae given here also enables statistical poverty stochastic dominance analysis to be undertaken. First-class stochastic dominance is based on the headcount ratio and second-class is based on the poverty gap, both over some range of poverty lines. Combining the estimators with the testing method presented in Howes (1993) enables one to test for statistically significant stochastic dominance rather than just work with sample outcomes. The attention paid by this paper to poverty measures is simply to provide a focus. One can, of course, think of a more general class of welfare measures, of which the negatives of poverty measures are one sub-class. The mean itself is one type of welfare measure, and there are many others within the class of separable functions with more desirable (egalitarian) properties. With some extensions to cater for mean-normalization, one can also derive variances for estimators of inequality indices. 19 5. Some examples We illustrate the quantitative importance of these considerations with analyses of household surveys from two developing countries, Pakistan and Ghana. What we measure for both countries are sample design effects. These give the ratio of the standard error one obtains given a particular set of assumptions about sample design to the standard exror one obtains assuming that the sample is a simple random one. This ratio has two interpretations. First, it gives the magnitude of error made when ignoring sample design. Second, it represents the efficiency gain or loss from moving away from simple random sampling. Since our interest is in analyzing rather than designing surveys, our interest is primarily in the former interpretation, though we also briefly address the second. For the second interpretation, one nieeds to be able to estimate what the standard error of the estimator of interest would be for the same population under some alternative sample design. The fact that one does not actually have that alternative sample design does in some cases introduce bias: for example, the standard estimator for the element variance is biased downward in a clustered sample. However, one can show that such biases are negligible for the cases considered here (Kish, 1965, Deaton, 1994). We deliberately focus on two surveys which, although both LSMS surveys, are quite different in nature. Both are of course clustered and use systematic sampling and both are also two-stage with the household being the second-stage sampling unit. But in other important ways the two surveys differ. The Pakistani sample is designed in a traditional, highly-stratified way (described in Howes and Zaidi, 1994). Its 4745 observations are collected from 300 clusters in 104 strata (see Table 2).19 The Pakistani sample is also non-self-weighting. Its cluster-level weights reflect the usage of ppes as well as disproportionate stratification, with, in particular, urban households over-represented. Table 2 Sample design for Pakistan and Ghana LSMS surveys Strata Number of Cluster Sample Average cluster clusters weights size samLple size Pakistan Integrated 104 300 Yes 4745 15.82 Household Survey, 1991 Ghana Living Standards None 170 No 3181 18.71 Survey, 1987-88 _ I I I The Ghanaian sample is made up of 3181 households from 170 clusters (see Scott and Amenuvegbe, 1989, for a detailed description). As mentioned in the notes to T'able 1, ppes is 19. For simplicity, we delete one stratum since it only has one cluster. Alternatively, we could have followed the procedure recommended in 3.3. 20 used and cluster-take sizes are adjusted by the ratio of actual to estimated cluster size. However, the survey contains an innovation: this ratio was rounded off to the nearest integer to restrict all cluster-take sizes to a multiple of 16 (for reasons of administrative convenience). Scott and Amenuvegbe (1989 and 1991) show that this restriction introduces only a negligible bias so the standard ppes formula can be used. 141 cluster takes have a size of approximately 16 (a very small number of households is lost to data cleaning, 19 in all20), 28 of 32, and 1 of 48 (an additional 30 clusters originally selected were dropped since their ratio of actual to estimated size was less than half). The Ghanian sample is not explicitly stratified at all. However, implicit stratification was used: clusters were selected on the basis of a geographical ordering. To deal with this, we follow the method recommended in 3.5, and approximate the effects of this implicit stratification by a fine explicit stratification. It is assumed, on the basis of informed advice, that the current ordering of clusters reflects their original ordering in the sampling frame. On this basis, the 170 clusters were divided into 85 strata, starting with the first two clusters in the first stratum. As noted in 3.5, this procedure may leave a slight overestimation of the 'actual' standard errors, since the assumed stratification may not capture all of the precision induced by systematic sampling. Figure 1, however, shows that the procedure is likely to be very effective. It gives standard errors assuming that the implicit stratification can be captured by explicit stratification of differing degrees of fineness, beginning with no stratification (assuming a random ordering), then with 2, 3, 5, 10, 21, 42 and 85 strata, the latter being the finest possible degree of stratification consistent with the data.21 It is clear from the figure that most of the gains from finer stratification are exhausted by the introduction of twenty strata. In particular, in this case from 42 to 85 strata actually slightly increases the estimated standard error (recall that the fall is in expectation). One can deduce from this that the approximation made of random ordering of clusters within each of the constructed 85 strata causes very little overestimation indeed. In Table 3 we present standard errors for both means (of household size and aggregate expenditure) and the three most used poverty measures: the headcount, poverty gap and FGT2 (a =2) index. Both the poverty indices and the expenditure means are per capita. Since the exercise is purely illustrative we do not worry about equivalence scales or regional deflators, and arbitrary poverty lines are set at 3780 rupees per capita per month for Pakistan and of 37,900 cedis per capita per annum for Ghana. Both lines are chosen to put approximately 1/3 of the sample in poverty. 20. Since attrition due to cleaning is so low, no adjustment is made to correct for it. Likewise, no adjustment is made to the Pakistani data for non-response since (i) we are unsure whether the weights assume actual or target sample sizes (see footnote 15) and (ii) we tried both assumptions and it made no difference due to the very high response rates. 21. These degrees of stratification were obtained as described in the text. Where clusters could not be divided evenly among the strata, any residual clusters were added into the last stratum. 21 Figure 1 The impact on standard errors of approximating implicit by dilrering degrees of explicit stratification: an example from Ghana 2400 Stidad err for per capita mean consunpfon 2300 S 2200 - 2100 2000 - 1900 1800 - 1700 - 1600 . 1500 l l l l l l 0 10 20 30 40 50 60 70 80 90 Nunter of stata For each measure, four standard errors are given. Each is an estimate of the standard error which would be correct for the measure of interest for some sample design. (The means are correct under all four sets of assumptions, since in all four cases, the same weights are assumed - though their rationale would have to change.) The first ("stratification and clustering") is correct for the set of assumptions outlined at the start of Section 3 and the degree of stratification and clustering found in each survey (with the necessary approximations made where necessay to fit the surveys to this model). The second ("clustering without stratification") would be correct for the same set of assumptions if the surveys were conducted without stratification. The third ("stratification without clustering") would be correct for the same set of assumptions except that instead of clustering one had random sampling of households (with replacement) within each stratum. The fourth ("simple randomi sample") would be correct if the surveys were simple random samples (with replacement) in all respects except that they were not self-weighting. 22 Table 3 Sample design effects for mean expenditure, household size and various poverty measures for two household surveys Mean household Mean Headcount Poverty-gap FGT2 (a=2) size expenditure index Pak. Ghana Pak. Ghana Pak. Ghana Pak. Ghana Pak. Ghana Estimate 7.17 4.52 5934 71276 .336 .333 .092 .098 .0364 .0403 Standard A. Clustering .072 .081 125 1935 .012 .016 .0046 .0072 .0024 .0039 errors and stratification B. Clustering .087 .098 158 2323 .017 .020 .0063 .0083 .0032 .0043 w/o strat'n C. Strat'n w/o .059 .053 86 918 .009 .010 .0033 .0042 .0018 .0024 clustering D. Simple .061 .055 91 1002 .009 .011 .0035 .0044 .0019 .0025 random sample Ratio of A/D 1.18 1.47 1.37 1.93 1.33 1.45 1.31 1.64 1.26 1.56 standard errors B/D 1.43 1.78 1.74 2.32 1.89 1.82 1.80 1.89 1.68 1.72 C/D 0.97 0.96 0.95 0.92 1 .00 0.91 0.94 0.95 0.95 0.96 AIB 0.83 0.83 0.79 0.83 0.71 0.80 0.73 0.87 0.75 0.91 Notes: See the text for explanation of the different standard errors. We begin with Pakistan. For mean household size, the design effect is not that large. Standard errors increase by only 18% over a simple random sample (A/D). This is what one would expect as there will typically be a great deal of variation within any one cluster with respect to household size. But for mean expenditure and the poverty measures, the design effect is much larger. For the mean, it is 37 %; for the headcount 33 %; for the poverty gap 31 %; and for the FGT2 index 26%. On average, standard errors increase by around a third. Ignoring the sample design of the Pakistan survey leads to calculated standard errors which are substantially smaller than the correctly estimated standard errors. It is clear from the table that stratification is very important. With clustered but unstratified data, standard errors would be between 68% and 89% (depending on the welfare/poverty measure) greater than under srs (B/D). But note that stratification is only important when the sample is clustered. Take mean expenditure as an example. Without clustering, stratification reduces the standard error on mean expenditure by 5% from 91 to 86 rupees (C/D). With clustering, stratification reduces standard errors by four times as much, that is, by 21 % from 158 to 125 rupees (A/B). Taking into account clustering but not stratification can be as misleading as taking neither into account. For the mean, the former strategy's estimate of 158 overestimates the correctly calculated standard error of 125 by as much as the latter 23 strategy's estimate of 91 underestimates it.' The sample design effects from Ghana are qualitatively similar and quantitatively larger. Sample design increases the standard error of household size by 47% (A/D). For the other estimators, increases due to sample design range from a low of 45% for the headcount to 55- 65 % for the poverty gap and FGT2 index to 93 % for mean expenditure. Agrain the importance of taking stratification (here implicit) into account is evident. With cluste-ring, stratification reduces standard errors by 9-20% (A/B); without it, stratification reduces standard errors by only 4-9% (C/D). Of course, whatever the standard error estimated, the t-ratio associated with its poverty measure will be very large. This should not surprise, since all it means is the rejection of the null hypothesis of zero poverty. As indicated by the paper's title, however, of greater interest are poverty comparisons. Here the inaccuracies in standard error computation are much more important. Take the case of Pakistan. Say we had another survey of identical size and design a few years later. Assuming that the new survey would give approximately the same standard errors, how much of a change in poverty would the new survey have to register for that change to be judged significant (at a level of .05), that is, not due to sampling variation? For the headcount, for a simple random sample, the, say, decrease would have to be from 33.6% to 31.1 %, a decline of 6.6%. With clusters and strata, a 10.1 % decline would be required from 33.6% to 30.2%. One can imagine many cases in which two survey points would register a change in poverty of 7-10%: at least in Pakistan, these are the cases in which taking account of sampling design is imperative.23 22. Of course, one might prefer the former on the grounds that it is better to err on the side of caution. 23. Rodgers and Rodgers (1992) calculate sample design effects for permanent (multi-year) poverty measures using U.S. data and the balanced repeated replication method mentioned in footnote 16 above. They find larger sample design effects of 50 to 200% (see their footnote 22). Comparing their results with those of this paper suggests that data source used by Rodgers and Rodgers is not highly stratified (so that the clustering effect is hardly offset at all), though the actual degree of stratification is not given in the paper. 24 6. Concluding comments Household surveys are far from being simple random samples. Typically, they are clustered and highly stratified, to name only the two most important departures from the srs model. Although these departures have been ignored by recent papers on statistical poverty analysis, they are of considerable importance. The examples of the previous section show that not taking into account sample design can increase standard errors by around one-half. The main recommendation of this paper is therefore that standard errors for poverty and welfare measures should be calculated taking proper account, where possible, of sample design. Both stratification (explicit or implicit) and clustering should be factored in. Taking account only of one, and not the other, can lead to inaccuracies as great as taking account of neither. Often the complexity of sample design will necessitate approximations, but these are much less egregious than just assuming simple random sampling.24 Examining sample design from a statistical perspective also leads to several ancillary conclusions about data availability, analysis and sample design itself. On data availability, the obvious point is that the information required to take account of sample design should be made available with the sample information itself. Thus each household should have a cluster identifier and, if appropriate, a stratum identifier. Not only should raising factors be made available, but their derivation should be documented, as should the sample design itself. All surveys should be describable in terms of the list of features presented in Section 2. In particular, if implicit stratification is used to pick clusters, the relationship of the cluster numbering to the original ordering of clusters should be given. For many surveys, precisely because attention has not been paid to sample design by end-users, these basic requirements are not fulfilled (see also Harpman and Scott, 1987, p.375). On data analysis, we have been concerned in this paper with the calculation of standard errors for estimators of unconditional means (whether of original or transformed variables). What about standard errors for conditional means, such as are obtained by regression analysis? It has recently become much more widespread to take into account clustering when calculating standard errors for regression coefficients (using methods given in Deaton, 1994). Whether or not stratification should be similarly accounted for is more contentious. But assume that one either has a highly explicitly stratified or an implicitly stratified sample. In either case, one could regard this as a survey with n cluster takes and, approximately, n/2 strata. In such a case, it is unlikely that one would want to handle stratification by introducing dummy variables, as one might do with a smaller number of strata (a common strategy with urban/rural or provincial strata). However, if there is heterogeneity across strata, one would like to take account of the greater precision induced by this stratification. If one does not, and only factors in the clustering, one runs the risk of exaggerating the design effect, at least if the examples using 24. Standard data-processing software cannot always be relied upon to provide the correct estimators, but specialist packages are available (for example, Sudaan, PC Carp and Clusters). 25 unconditional means presented in this paper are any guide. There are methods available for incorporating both clustering and stratification into regression analysis (see .Deaton, 1994; Shah et al., 1993; and Kish and Frankel, 1970). The imnportance of implementing these methods is clearly a topic worthy of further research. Finally, we turn to sample design, in particular to the question of sitratification. Given the large precision gains induced by stratification when clustering is also being used, the current practice of using some degree of stratification at the first stage would certainly seem warranted, though one might argue over the optimal degree. The question remains, however, of whether explicit or implicit stratification should be availed of.2 Many survey designers favor the latter on the grounds of simplicity (see Scott and Amenuvegbe, 1989, p. 17, for example). But from an analytical point of view, one must favor the explicit variety, since, for the calculation of variances, one is forced to assume that this is what is used. Opposing this, implicit stratification will tend to produce slightly more precise estimates than even the finest possible degree of explicit stratification (two clusters per stratum), though here we have stressed that the qualifier 'slightly' must be emphasized. We are unclear which type of sampling is easier to implement, but suspect that there is little difference. We also suspect that the information required to take account of explicit stratification information is easier to record than that required for implicit stratification. The former requires a single, stratum-representing variable. The latter requires that significance be given to the ordering of the actual cluster numbers chosen, information which could easily be lost, especially in a complex list.26 Taking all of these factors into account, we would argue that the precision gains of explicit stratification should first be exhausted before one turns to implicit stratification. That is, one should not use systematic sampling at the first stage until one has divided the sampling frame into N/2 strata, where N is the desired number of clusters. Only once one has done this should one consider use of systematic; selection (to select the two clusters from each stratum) in order to achieve even greater precision (though by an unquantifiable amount) through implicit stratification.27 If one uses fewer than N/2 strata, it is hard to see the rationale for any use of systematic sampling, since in this case one could increase precision in an analytically tractable way with more extensive use of explicit stratification. 25. This is of course in relation to the first stage. Implicit stratification at the second stage creates no problems at all. 26. Certainly we have never seen such information documented, though this may be because there has never been a demand for it. 27. Alternatively, one can, as Kish (1965, pp.284-285) recommends, divide each of the strata into half- strata and randomly pick one cluster from each. This avoids the use of systematic sampling altogether. 26 Appendix I: Proofs We begin with a summary of the relevant notation required for these proofs. th =yhci*Zh*MhcIZhc is an unbiased estimator of the population parameter Th based on observation i from cluster c in stratum h. th, defined in (5), is the simple mean of t,-, (summed over i and divided by mhc, the sample size drawn from cluster c). And th, defined in (6), is the simple mean of the thc (summed over c and divided by nh, the number of clusters drawn from the hth stratum). 1. An unbiased variance estimator for two-stage cluster sampling with replacement We prove that (9) is an unbiased estimator of Var(tb). In fact, the proof is trivial. We give it only to make clear the importance of the with-replacement assumption. Note that in the proof, as in the text, an important distinction is made between the summation c = 1 to n. and that c = 1 to Nh. The latter is simply a list of all clusters in the stratum. In the former one is selecting n1h cluster samples, with the possibility that one is sampling from the same cluster more than once. Each of the Nh clusters contains Sh,, possible sampling combinations (depending on the second-stage sampling size and rules). The sum of the Shc over all c is Sh. Two-stage sampling with replacement is no more than the random selection with replacement of nh of these Sh possible sampling combinations, each with population parameter thc. Proof: nhE(Var(th)) = 1 (A.1) n cI=1 n- nk Skc nA nh EE thcs (A.2) E(E =E) E(hc) =nh c=1 c=1 Sh E(th) = Var(t) +?h (A.3) i 1 nk nh h (A.4) Var(th) Var( - th) = Cov( t t) nh c=l nh Cfl C/=l Since sampling is with replacement Cov(thC,th,)=O for coc'. So (A.4) simplifies to 27 Var(th) = -E Var(tj) = a(t,c) (A.5) nh t=1 nh Now substitute (A.5) into (A.3). Substitute the result of this and (A.2) into (A. 1) to give nh I th2 _ Var(tk_ nh- Shc=l s=1 nh n. NA Si. Var(t) = h ( 1 s ) (A.6) = (Var(thc) - ar(th) Var(thc) Combining (A.6) with (A.5) completes the proof. 2. An unbiased variance estimator for two-stage cluster sampling without replacement With single-stage sampling, without-replacement estimators are obtained simply by multiplying the with-replacement variance estimator by a degrees-of-freedom correction. But, with multi-stage sampling, matters are more complex. Once any one cluster is picked, one is removing from the draw not one, but Shc elements, all from the one cluster, and so all correlated. We assume that sampling is without replacement at both stages, and that clusters are all of equal and known size, Mh, and that mi is sampled from each cluster. In this case, for all c, whC=NhMh/mfh, so thc is simply the stratum population times mean income in cluster c. Relaxation of these assumptions is discussed at the end of the sub-section. To prove that (17) is a unbiased estimator of Var(th) in this case, it is necessary first to define the latter. So we prove first that Var(th) = 1 Nhhn [ A (rhC Ch) M& (A.7) Nhnh Nh-i c=1 C=1 where V(thc) is the within-cluster variance of th, (see (A.16) for the case of sampling without replacement), and is to be distinguished from Var(th). 28 Proof: Var(th) = VI [E2(th)] + E1 [V2(th)] (A.8) where the subscript 2 refers to the indicated operation with clusters fixed and the subscript 1 refers to the operation over all samples of clusters. nb E the V[E2)A = VI E21 1 ]I nh E E2(thc) = V1 I1 n, nh (A.9) (N-nh) V (h Nh-i nh Ni Nh-nh 1 1 (Thc_h2 Nh-1 nh Nb Note here the without-replacement correction (Nh-nh)/(Nh-l). ns i t E1 [V2(th)] = E1 [V2I [C1 ]] nh nh E V2(t^¢) (A.10) = El [¢ 1 2 ] nn, NA 1 V(tk) nh Nh Substituting (A.9) and (A. 10) into (A.8) gives (A.7), which completes the proof. 29 We now prove that (17) is an unbiased estimator of Var(th) as defined by (A.7). Proof: (17) is made up of two terms, the first of which is nh N -nh E (thc th) (A.11) Nhnh c=1 Nh nh(nh 1) The expected value of (A. 11) can be derived as follows n I1 {E2 [ 1 S (th 2 1 =Es E2 [ thc nt£ t2 nh 1 nh c=l =El[ [ [ (1 - I ) thc2 _2E tt] nt1 nh C=1 = E1 [- E [ ( t1 - -) (Z2thj + sc hlc] h 1 4h nh n =E E 1 E[ (1 --) ( -2 +* h( 2 1 E th --Y thCthAdJJ n.-I C=1~ nkC1 hC=i C Ch deoLacuez 41 order. COter cErimton el e Ddtusion Aricaines Sales & Crculaion Depaorlent Umnversdy Pressn limed 9a, Loupachrily Pereulok CH1807 Blnay (CEDA) Seabird House, uni 110142 JAMAICA Three Crowns Building Jercho Moseow 101831 Tel (41 21) 943 2673 Oka del LbAm lntemacooa 04B P 541 22-28 Wyndhan Street, Central Ian Rondo Pubishers Ltd Private Mail Bag 5095 Tel (7095)9178749 Fan (4121)9433505 Av CordolLbaolnt1877 ioa AbidjanC04 HongKong 206OldHopeRoad Ibadan Fax p7095)9179259 Au CoBusba 1A77 Tel (225)246510246511 Tel (852) 2530-1409 Kngston6 Tel (23422)41-1356 TANZANIA TlB (541)815-8L Fax (225)250567 Fax (852)2526-1107 Tel 809-927-2085 Fax (23422)41-2056 SINGAPORE, TAIWAN, Oxtord Unoversny Pross Fax (541)8t-8356 E-mail salesOasa200M orD hk Fax 809-9774243 MYANMAR, BRUNEI Makaba Street Fan (541)981548150 CYPRUS URL h#pJiMvw ansk2060 com hk E-mail rptcolso crom NORWAY Akahgate Publishig Aua Paedic Pte Ltd PO Box 5299 AUSTRAUIA, FIJI, PAPUA NEW GUINEA Center or Applied Reswrch NIC JInf B ea 41 Kallang Pudring Road 904-03 Dar es Sataam SOLOMON ISLANDS, VANUATU, AND CyPrus Co"lg INDIA JAPAN Book Department Golden Wheel Buildug Tot (255 51) 29209 WESTERN SAMOA 6, Drogenes Street, Enrgomi Allied Pubishers Ltd Eastem Book Servce PO Box 6125 Eterstad Singapore 349316 Fax (255 51) 46822 DA IfS TioNSA PO BOa 2006 751 Mount Road 3-13 Hong 3-chorne, Bunkyo-ku N40602 Osb 6 Tl (65)741-51b6 D R ma Ncoon Mudras-600002 Tokyo113 Tot (47 22) 57-3300 Fax (65)742-9356 THAILAND 648 WhftoleRoadNcsaMars-6D00ooy 1 Mittham 3132 Tel (357 2) 44-1730 Tel (91 44) 852-3938 Tel (81 3) 38184861 Fax (47 22) 68-1901 E-miu ashguteslg asnanconnect com Central Books Distrbution VMlonam312 Fax (3572)46-2051 Fax (9144)8524349 Fax (813)38180864 306 Silom Road E-mall svt-ebsrppp bekkoame or up PAKISTAN SLOVENIA Bangkok 10500 Tel (61)392107t77 CZECH REPUBUC INDONESIA URL hftpJlwww- bekkoameortl-svut-ebs Mima Book Agency Gospodarski Vestnik Publishing Group Tel (66 2)235-540D Fax (61)392107788 NatIonal Intomation Center Pt Incita Lanmed 65, Shahrah-e-4uaid-e-Azam Dtnaka costa S Fax (662) 237-8321 E-mail nervlce0ladradct coom en URL hflpJt ww dadect com au prodnpa, Konvstska s Jatun Borobudur 20 KENYA Lahore 5400D 1000 .jublioana CS -llsS 7PraguelI P0 Ron 191 Alnca Book Service(E A) Ltd Tel (92 42) 735 3601 Tot (386 61) 133 93 47, 122 12 30 TRINIDAD &TOBAGO, Tel (422) 2422-9433 Jukauta 10320 Quaran House, Mtangano Street Fax (9242)7585283 Fax (386 61)133B030 AND THE CARRIBBEAN AUSTIRtA andCFax (42 2)2422-1484 Tel (62 21) 390-4290 PO Box 45245 E-mai belcd@gneunksn SystoratcStudiesUri Woirberggaand 2e URL htlpiowwm nisczt Fan (6221)4214289 Nairobi Oxtord UninersiyPress 9WattsSreet WOihburggasse 26 Tel (254 2) 223 641 5 Bangabre Town SOUTH AFRICA, BOTSWANA Curepe A-1011 Wien DEN17ARK IRAN Fax (2542)330272 Sharae Fasal fornsigle ties Tnnoldd, West llades Tel (431) 51247-31-2 SarmrndsLnteratur Ketab Sara Co Publishers PO BRx 13033 Oxiord Unversny Press Southem Afwca Tel (909) 662-5654 URL hFlpiax m gerold co/at onl5in Rosernoms AIle 11 Khaled Eslombuli Ave, KOREA, REPUBLIC OF Karachi-75350 PO BRx 1141 Fax (909)662-5654 UDK-1970 Fredenksberg C 6th Street Daelon Trading Co Ltd lbt (92 21)446307 Cope Townu8000 E-mail tobhelltodld net BANGLADESH Tel (4531)351942 Kusheh Delatrooz No 8 PO Boxn34, Ynuda Fax (92 21)4547640 el (2721)45-7266 Micro laduolrleo Development Fax (45 31) 357822 PO bx 15745-733 706 Seoun Bldg -E-mail oupoup dii erum corn pk Fax (27 21)45-7265 UGANDA MAmldsatnces Savely MDAS)n Tehran 4446 YoudD-Dong, Yeongchengpo-Ku Gustro Ltd AsHsstancd Soiet(MIDAS) EGYPT, ARAB REPUBLIC OF Tel (98 21) 8717819,8716104 Seoul Pak Book Corporation Forsubscnpipon orders PO Box 9997, Madhvani Building HousebS,oRoad A6 Al Ahram Disbution Agoocy Fax (98 21)8712479 Tel (822)785-1631/4 Azrz Chamber 21 lnlematonal Subscnrption Service Plot 1614 Jinja Rd DnMrK 129 ea Al Galaa Street E-mail ketab-sara@neda net ir Fax (822)7844315 Oueens Road PO BRx 41095 Kampal Dhaka 1249 CurrJo Lahure Craighall Tel (25641) 254 763 Tel (880 2) 329427 Tel (2D2) 5784083 Kowkab Publishers MALAYSIA Tel (92 42)636 3222,963 0885 Johannesburg 2024 Fax (256 41) 251 468 Fax (880 2) 811188 Fax (262) 5784833 PO Box 19575-511 Univensty ol Malaya Cooperative Fau (92 42) 636 2328 Tel (2711) 880-1448 Tehran Bookshop, Limded E-mail pbc@bram nel pk Fax (2711) 8804248 UNITED KINGDOM BELGIUM The Middle East Obseiver Tel (98 21) 258-3723 PO Box 1127 E-mail ss@scoza Micrornto Ltd an De Lanroy 41, Shee Street Fax (9821)258-3723 Julan Pantai Baru PERU PO Box3 Av du Rol 202 Caio 59700 Kuala Lumpur Edional Desarmolo SA SPAIN Anon, Hampshire GU34 2PG 1060 B3usse5s Tel (42 2) 393-9732 IRELAND Tel (60 3) 756-5000 Apadado 3824 Mundi-Prensa Libros, S A Engand Tel (322) 53-5169 Fax (202)393-9732 Govenmnent Supplies Agency Fax (603)7554424 Len I Castello 37 Tel (44 1423) 98648 Fax (32 2) 5384tt41 OVig an ISoftAair Tel (5114) 295380 28001 Madnid Fax (441420) 89889 BRAZIL FINLAND 4-5 Harcoun Road MEXICO Fax (51 14)286628 Tel (341)431-3399 E-nail wbank@ukmntocdemoncouk Publcacoes Tecnrcan tarnmcranars Ltda Akateemmen Kiijakauppa Dublin 2 INFOTEC Fan (341) 575-3999 URL htlpiMw micrornto couk RaPxooGmcs209 P0 Ron 129 Tel1(3501)661-3111 An San Fernando No 37 PHILIPPINES E-muil libreoia@mundiprenso en Run PeSao PGo,Srdep' FIN040101 Helsinki Fan (3531)475-2670 Col Tonello Guerra Intemaonal Booksource Cenler Inc URL httpJ/wvw mundiprensa est VENEZUELA T01(55o11)u25946 Tel (3580)12141 14050 Mexico, D F 1127-AAntpoloSt Tecni-Ciencra Libos, S A Fan (5511)2594694950 Fan (358 0)1214441 ISRAEL Tel(525)624-2800 Barangay, Venezuela Mundi-Prensa Barcelona Centro CuKidd Comercoal Tmanuco E-mil postnaoter@pti uou hr URL. httpAwAw akateemien comt Yoznort Lerature Ltd Fan (525)624-2922 Makati OlC Conseil de Cent, 391 Nlvel02 PO BoxO 56055 E-mail inotecirtn net m Tel (632)8966501,6505,6507 08009 Barcelona Caracas URL htlpJAvww el br FRANCE 3 Yohanan Hasandlar Street URL httpl/r/ net ma Fan (632)8961741 Tel (343) 488-3492 Tel (582)9595547.5035,0016 Word Bank PublcMions Tel Aoo 61560 Fux (343)487-7659 Fan (592)959 5636 CANADA 9,aeedeeTe(92)2-37NPLPOLAND E-mail barcelonu@mundiprensa en R536n9 aPubotge C Lld 75116 Pans Fax (9723)5285-397 Everest Mearlntemational Services (P)Ud Intematonal Publishng Service ZAMBIA 5369CaantekaRoad Tel (331)4049-30-5'vS7 GPO Box 5443 Ul Piekna 31/37 Unwuersy Bookshop, Unrvernsy ou Zambua Olta(613)r745269J5 Fax (331) 4049-3048 ROY lntemabeonal Kathmandu 00477 Warzawa SRI LANKA, THE MALDIVES Great East Road Campus Fax (613) 745-2660 PO Box 13056 Tel (9771) 472152 Tel (48 2) 6284089 Lake House Bookshop PO Box 32379 E- 1il renouf0foxvishica GERMANY Tel Avn 61130 Fax (977 1)224431 Fan (482)621-7255 100, Sir Chotampalam Gardiner Mawath Lusaka URL, httpJ/ www fox nsm cat-renouf UNO-Vedag Tel(97235461423 NETHERU SE-mail books%psiipatmrocm p Colombo 2 Tel)(2601)252576 URL htPJ/ mon to nutr cOI-eeout Poppelsodorer Allee 55 Fan (972 3) 5461442 NETHERLANDS URL htlpilwww pscg mum pbipsfexporVl Tel (94 1) 32105 Fax (260 1) 253 952 CHINA 53115 Bonn E-mail ropillinetvrison net il De LideboondnOr-Publkalties Fax (941)432104 Chma Firmanckt & Economic Tel1(49 228) 212940 P0 BOo 232 PORTUJGAL E-mail LHL@un lanka net ZIMBABWE Publshiang House Fan (49 228) 217492 Paleutinian Authonty/Middle East 7480 AE Haaksbergen Livrana Podugal Logman Zimbabwe (Pte)Ltd hng FoSIDusg Jie Indenx Inomiation Seorces Tel (31 53) 574-004 Apartado 2681 SWEDEN Toude Road, Ardbenne 8, Da Fo Si Dong Jie GREECE POB 19502 Jerusalem Fax (31 53)572-9296 Rua Do Carmo 70-74 Wennergren-Willim AB PO BoxoST125 B 0ei7ng Papasotirou S A Tel (9722) 6271219 E-mail lmdeboomworldonlinenI 1200 Lisbon P0 Box 1305 Soulhedon FaxI t86 10) 640173657 35, Stoumara SIr Fau (9722)6271634 URL h8pJAiww woitbnlme nrl-fndeboo Tel (1)3474982 S-171 25 Solna Harare F(K (861O)64 7365 106 82 Athens Fax (1)347-0264 Tel (408)705-97-50 Tel (2634)6216617 T1l (30 1)364-1826 Fax (468) 2740-71 Fax (2634)621670 Faxi (301) 3644254 E-mail maildww0 so WM Ti ii WVVi. I i) 13 \\K IS I S I tt I t . \N kV. r~~~~. X;A04:: W:: X popoin-4e (24 47391 Tol~x: M~ 641415 w*) MO 2W3 W0ftL~~~~~~~~~~~~~~~~~~~~~~~~*ANK~~138 CaNe Addrq.4: W40M0A7; .Es0ia: b00 I:f 4 b:arE0 IcH:a''' ISBNJ ( ' ; 3e ).