WPS7915 Policy Research Working Paper 7915 A Poor Means Test? Econometric Targeting in Africa Caitlin Brown Martin Ravallion Dominique van de Walle Development Research Group Human Development and Public Services Team December 2016 Policy Research Working Paper 7915 Abstract Proxy-means testing is a popular method of poverty tar- poverty. Some methodological changes perform better, with geting with imperfect information. In a now widely-used a poverty-quantile method dominating in most cases. Even version, a regression for log consumption calibrates a proxy- so, either a basic-income scheme or transfers using a simple means test score based on chosen covariates, which is then demographic scorecard are found to do as well, or almost implemented for targeting out-of-sample. In this paper, as well, in reducing poverty. However, even with a budget the performance of various proxy-means testing methods sufficient to eliminate poverty with full information, none is assessed using data for nine African countries. Standard of these targeting methods brings the poverty rate below proxy-means testing helps filter out the nonpoor, but about three-quarters of its initial value. The prevailing excludes many poor people, thus diminishing the impact on methods are particularly deficient in reaching the poorest. This paper is a product of the Human Development and Public Services Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at dvandewalle@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team A Poor Means Test? Econometric Targeting in Africa Caitlin Brown, Martin Ravallion and Dominique van de Walle1 Keywords: Poverty, cash transfers, means test, targeting, Africa JEL: I32, I38 1 Brown and Ravallion are with the Department of Economics, Georgetown University, and van de Walle is with the Development Research Group, World Bank. For their comments the authors thank Arthur Alik-Lagrange, Kathleen Beegle, Mary Ann Bronson, Raphael Calel, Phillippe Leite, Mead Over, Lucian Pop, Adam Wagstaff, anonymous reviewers from the World Bank’s Africa region and seminar participants at Georgetown University. The authors are grateful to the World Bank’s Strategic Research Program for funding assistance. 1. Introduction While universal social programs—whereby everyone is covered—are excellent at reaching the poorest, the beneficiaries can include many people who do not need this form of public help. Governments have tried many ways of assuring better “targeting,” with the explicit aim of concentrating the benefits of a social policy on poor people. The means used vary in their data requirements, methodological sophistication and costs (both administrative and broader social costs). Readily measurable proxies for consumption or income are often used in efforts to reduce poverty in settings in which the means-testing of benefits is not an administratively feasible option, as in most low-income countries (and many middle-income countries). Efficiency considerations point to the need for indicators that are not easily manipulated by actual or potential beneficiaries. Proxy variables, such as gender and education, family size and housing conditions, have been common.2 A score based on these variables is used in validating other targeting methods, such as those based on community-level subjective assessments of who is “poor.” The scores are also entering many social-protection registries—national data bases that are used in various ways including to flag ineligible households in future schemes. The main challenge has been in setting the score’s weights. Various “poverty scorecards” or “basic needs indicators” have been used. Some versions use ad hoc weights, such as taking a simple average of the scores across components.3 Practitioners have turned to more sophisticated statistical methods in an effort to further improve targeting accuracy. These methods have come to be known as proxy means testing (PMT).4 This paper assesses an increasingly popular solution in which the weights in the PMT are identified from regression coefficients for household consumption or income as a function of readily observed covariates. The regression is calibrated to survey data and then used to make the out-of-sample predictions for the relevant population. This has the intuitive attraction that the dependent variable is a 2 This idea appears to have emerged in social policy making in Chile in the 1980s (Grosh, 1994, Ch.5). Grosh et al. (2008) provides a useful overview of PMT and other targeting methods found in practice in developing countries, with details on many examples. 3 A popular example of the poverty scorecard was proposed by Schreiner (2010); the Progress out of Poverty Index uses Schreiner’s (2015) method. The scorecard includes 10 easily measured correlates of poverty which are used to form a composite index. Diamond et al. (2016) argue that the predictive ability of such scorecards can be improved by calibrating the variables and their weights to local (sub-national) conditions, for which purpose they advocate econometric methods. 4 This term appears to be due to Grosh and Baker (1995, p. ix), who define PMT as “a situation where information on household or individual characteristics correlated with welfare levels is used in a formal algorithm to proxy household income, welfare or need.” 2 well-established measure of household economic welfare and, indeed, the same variable is typically used in measuring poverty.5 To distinguish it from other methods of means testing, we will use the term “econometric targeting” to refer to any PMT based on a regression model. An influential early contribution by Grosh (1994) compared numerous social programs in Latin America and concluded that this class of methods produced the best targeting outcomes, measured in terms of reducing inclusion errors, whereby a nonpoor person is counted as poor. Various versions of econometric targeting have since been proposed, and the method has been widely implemented in developing countries.6 Econometric targeting has also been criticized for its seemingly poor predictions about who is poor and who is not. For example, Kidd and Wylde (2011, p.ii) refer to the method’s “considerable inaccuracy at low levels of coverage.” Transparency has also been a concern. Sometimes the score variables and weights are deliberately kept secret for incentive reasons. In other cases, the method and formula are too complicated, or too poorly explained, for public consumption. Either way, observers on the ground do not always understand why some people are selected and some are not based on these targeting methods. With reference to a conditional cash transfer scheme in Nicaragua using PMT, field work by Adato and Roopnaraine (2004, p.15) led them to write that: “…the targeting process as a whole is poorly understood at the community level in both geographical- and household-targeted communities. When asked why some households were beneficiaries and others not, informants offered a range of explanations, from divine intervention to a random lottery. For example, one informant from a geographically-targeted community noted: ‘Well, some people wonder why they weren’t targeted even though they live in this same area. So we tell them that the Bible says that many are called but few are chosen.’” In the context of a PMT in Indonesia, Cameron and Shah (2014) argue that considerable local social unrest was generated by this lack of transparency in why some people were deemed beneficiaries and some not. This came with an erosion of local social capital and greater distrust of local administrators. Another critique relates to the goals of social protection policies, which can be thought of as involving both protection from uninsured risks as well as promotion from poverty over the longer-term.7 Some observers have questioned the effectiveness of PMT in responding to shocks or targeting insurance. Instead, it is argued that, because it is largely based on long-term assets, PMT is suitable “…for identifying the chronic poor and determining eligibility for programs that provide long-term support” (Del Ninno and Mills, p.22). Nonetheless, PMT is widely used in implementing policies that 5 For a critical review of the methods used see Ravallion (2016, Part 2). 6 Useful overviews can be found in Mills et al. (2015) and USAID’s website on Poverty Assessment Tools. 7 On this distinction and the implications for assessing social protection policies see Ravallion et al. (1995). 3 offer short-lived benefits and are seen to be justified by their provision of insurance or emergency relief such as public works and cash transfer schemes. The criticisms of econometric targeting could reflect either methodological inadequacies or informational/data limitations. On the former, standard regression-based calibration of the PMT score will tend to work less well toward the extremes of the distribution of household consumption. By its design, a standard regression line passes through the means of the data. The residuals will be positively correlated with the dependent variable (more so the higher the variance of the residuals given exogenous regressors).8 One can expect the method to have a tendency to overestimate living standards for the poorest and underestimate them for the richest, though the degree to which this is problematic for targeting accuracy is unclear. Indeed, it is theoretically possible that the PMT method predicts that nobody lives below a poverty line for which even a sizable share of the population is deemed to be poor based on observed consumptions. Another possibility is that the variables used are not sufficiently good proxies for household consumption. In other words, that there is an information problem. The paper aims to provide a systematic assessment of the reliability of econometric targeting as a tool for social policies aiming to reduce poverty. We assess what appears to be the most common form of what we call “Basic PMT,” as well as some alternative methods using extra covariates and methods that are arguably more appropriate when it is recognized explicitly that the PMT is for antipoverty policy making. A natural counterfactual for assessing any form of PMT is a uniform allocation—the same for everyone. Various other counterfactuals of interest to policy makers are examined. The study also considers less finely-targeted options to econometric targeting, which are uniform only within stipulated categories. While Latin America has attracted the bulk of the past research on PMT, we study the method using survey data for the world’s poorest region, Sub-Saharan Africa (SSA). This is also the region where existing social spending has been least effective in reaching the poorest.9 The specific countries studied are Burkina Faso, Ethiopia, Ghana, Malawi, Mali, Niger, Nigeria, Tanzania and Uganda, being all those countries in SSA with recent and reasonably comparable surveys in the World Bank’s Living Standards Measurement Study (LSMS).10 For a subset of these countries we also have panel data. 8 If the regression model is y   x   with Cov ( x ,  )  0 then Cov ( y , ˆ )  Var (ˆ )  0 (in obvious notation). 9 For evidence on the point see Ravallion (2016, Chapter 10). 10 Existing government safety net programs in Ghana, Malawi, Nigeria, and Tanzania, and World Bank projects in Burkina Faso, Ethiopia, Ghana, Malawi and Niger use PMT, often in combination with geographic and community-based targeting. At the time of writing, PMT is also being considered for Mali. 4 In advocating and assessing PMT, social policy making in developing countries has often emphasized the need to avoid the “leakage” of benefits to the non-poor, and to assure broad coverage of the poor. Following the literature, one can term failures with regard to these two aspects of targeting as the aforementioned “errors of inclusion” (i.e., counting someone as poor who is not) and “errors of exclusion” (i.e., counting someone as non-poor who is in fact poor).11 The difference is important when deciding how much to spend on a program. Inclusion errors are generally costly to the public budget while exclusion errors save public money. Governments and international financial institutions concerned about the fiscal cost of social policies have thus put greater emphasis on avoiding inclusion errors as a means of cutting the cost to the government without hurting poor people.12 Some observers have questioned this prioritization, arguing that exclusion errors should get higher weight when the policy objective is to minimize poverty.13 In this paper we consider various measures of both targeting performance and impacts on poverty. Some assessments of econometric targeting are already available in the literature.14 The methods appear to vary considerably across the studies to date, such as in how many variables are used in the PMT, how targeting performance is assessed, and what poverty cutoff point is used. However, documentation is rarely ideal, often leaving the reader to guess what has been done. This makes it difficult to compare results. We provide similar tests on a consistent basis. We go further than past work in a number of other respects. We consider alternative econometric methods for calibrating the PMT scores. These include methods that recognize explicitly that the goal of PMT is poverty reduction rather than obtaining unbiased estimates of conditional means. We also simulate stylized policies to see how well econometric targeting works. Here we consider simpler alternatives to PMT that have a long history, going back to the state-contingent transfers that were introduced under England’s Poor Laws, and the various proposals that have been made over the last 200 years for a “basic-income scheme.”15 Additionally, we compare PMT to optimally differentiated 11 The distinction between these two targeting errors goes back to Weisbrod (1970) who called them “vertical” and “horizontal targeting efficiency.” Smolensky et al. (1995) called them “errors of inclusion” and “errors of exclusion.” Some authors refer to exclusion as “under-coverage” and refer to inclusion as “leakage.” In development contexts, influential early contributions were made by Cornia and Stewart (1995) and Grosh and Baker (1995). 12 This emphasis on reducing inclusion errors appears to have emerged during macroeconomic adjustment efforts, notably in Latin America in the 1980s (Smolensky et al., 1995). 13 See Cornia and Stewart (1995), Smolensky et al. (1995) and Ravallion (2009). 14 See Grosh and Baker (1995) (Jamaica, Bolivia, Peru), Ahmed and Bouis (2002) (Arab Republic of Egypt), Narayan and Yoshida (2005) (Sri Lanka), Sharif (2009) (Bangladesh), Stoeffler et al. (2015) (Cameroon), Pop (2015) (Ghana) and Cnobloch et al. (2015) (Malawi). 15 On the history of these policy options see Ravallion (2016, Part 1). 5 transfers based on the same information set. In considering these options, we focus directly on the impacts on poverty rather than looking solely at measures of targeting performance. Here we take the view that “better targeting” should not be seen as an end in itself but rather as a possible means of assuring a greater impact on poverty. The panel data that are available for a subset of countries help us address concerns about measurement errors in consumption. To some degree, what are called “targeting errors” are likely to be measurement errors (Ravallion 2008). By using the panel data to calculate time-mean consumption we can at least partly reduce the effect of measurement error, as a robustness test of our main findings. Another departure from past work is that we allow for likely lags in implementation; past assessments have ignored the fact that PMT invariably entails such lags, given that the score must be set in advance of implementation. There are lags between the survey year and the release of the PMT formula, and further lags to implementation.16 We can expect a degree of churning, with households moving in and out of poverty.17 So implementation lags are likely to constrain the performance of econometric targeting in identifying the currently poor. We exploit the panel nature of our data for a subset of countries to explicitly introduce lags. There are a number of issues that we do not take up. One of these is whether household consumption obtained from a survey is an adequate welfare indicator. The methods of econometric targeting studied here make that assumption, and we accept it for the purpose of evaluating the performance of these methods. Another issue not taken up here is how well a low level of household consumption identifies deprived individuals; Brown et al. (2016) take up this issue in the context of attempts to reach undernourished women and children. While we do address the performance of econometric targeting for stylized cash transfer programs we do not consider alternatives such as self- targeting using work requirements (“workfare”) or community-based targeting in which local communities are engaged directly in deciding who is poor and who is not.18 Nor do we consider the (economic, social and political) costs of targeting, which have received some attention in the literature.19 16 For example, even for a relatively simple PMT such as the Progress out of Poverty Index, we find that across the 59 countries for which the index is currently available, the number of years between the survey year and the release date of the index ranges from 1 to 9, with a mean of 3.9 years and a median of 3.5. 17 The implications of such churning for assessing the performance of social protection policies are examined further in Ravallion et al. (1995). 18 On workfare see, for example, Murgai et al. (2016) and on community-based targeting see Alatas et al. (2012), Karlan and Thuysbaert (2013) and Stoeffler et al. (2016). Barrientos (2013) provides a useful overview of the whole class of social assistance policies in developing countries. 19 See the discussions in van de Walle (1998), Gelbach and Pritchett (2000) and Ravallion (2016, Ch. 10). 6 For example, we do not discuss behavioral responses, social stigmas, or implications for social cohesion and political support for poverty programs.20 A few caveats are also in order. The paper uses existing household consumption surveys to simulate the targeting performance of various methods and the likely impacts on poverty of using the methods to allocate a fixed budget. In a sense, we are creating a “laboratory.” While this is inevitably somewhat artificial, we believe it to be essential for properly testing the performance of the various targeting methods found in practice. It must be acknowledged, however, that things happen in the real world that cannot be predicted in our laboratory setting. None of the methods considered here could be implemented flawlessly. While we do allow for lags in implementation, idiosyncratic mistakes will no doubt also be made. Most likely the methods will perform less well than our calculations suggest. We present and compare the best case results for the various methods reviewed, unaffected by potentially differential costs, ease of implementation, and susceptibility to manipulation and corruption. In addition, there are potential benefits of each method in practice that we cannot identify in our laboratory setting. Our purpose here is to judge the methods solely on how accurately they reach the poor and by how much transfers targeted in these ways would reduce poverty. These are undeniably important as goals for targeted programs and are usually explicit in the justifications provided by donors and governments. Nonetheless, there may be other objectives in practice such as capacity building or building systems. Donors, governments and NGOs implementing these methods do sometimes make considerable effort to include communities in the selection of beneficiaries, build robust grievance and redress procedures, and continuously monitor and evaluate the performance of the targeting systems. Such efforts could well be important in building administrative capacity for social protection over the longer term and also improve targeting. It is not obvious how an accounting for such potential benefits would alter the relative assessments of alternative methods based on our simulations. The paper finds that when the counterfactual is a uniform allocation of the same budget, even our Basic PMT allows a substantial reduction in the rate of inclusion errors; in this setting it should be possible to roughly halve the rate of inclusion errors using econometric targeting. When judged against a fixed poverty line, this success at avoiding leakage to the nonpoor comes with seemingly weak coverage of poor people—a high rate of exclusion errors. In other words, the method helps exclude the poor as well as the non-poor. The paper finds that econometric targeting typically provides at most 20 Smolensky et al. (1995) conclude that none of these issues is likely to be decisive for or against targeting. Atkinson (1995) argues that broader objectives of social policy (including social solidarity) warn against targeting. 7 modest gains in the poverty impacts over other policy-relevant alternatives. Indeed, in a number of cases and depending on the country and the nature of its poverty profile, simpler state-contingent targeting methods or even a “basic-income scheme” (in which everyone is covered) dominate in certain policy- relevant cases, such as when one allows for plausible lags in PMT implementation. However, none of these methods can be considered to perform especially well. Prevailing methods do not reliably reach the poorest. The costs of each method in practice may then be decisive in the choice. The following section describes the PMT method that we assess, while Section 3 describes the measures we use in assessing econometric targeting. Section 4 studies the basic version of PMT, while Section 5 turns to various extensions and revisions to that version. For stylized transfer programs, Section 6 compares the poverty impacts of econometric targeting to those of less methodologically sophisticated methods, including un-targeted (universal) transfers and simple demographic “scorecard” methods. Section 7 presents our results for (informationally-feasible) differentiated transfers, including optimal transfer schemes for poverty reduction with a given budget but limited information. Section 8 uses the panel surveys to introduce lags in implementation. Section 9 offers some cross-country comparisons of the performance of econometric targeting; here we ask how much the impacts of PMT on poverty for a given budget are explicable in terms of the alternative targeting measures and the predictive ability of the PMT regressions. Section 10 concludes. 2. Econometric targeting Quite generally, we can think of any PMT as some weighted function of a vector of covariates xijt . The specific form of this function that has become popular and that we focus on uses household- consumption regression coefficients as the weights. We can write the following empirical regression function for the consumption of household i in country j at date t on a vector of covariates xijt using a survey sample of size N jt : y ijt   jt   jt x ijt   ijt (i=1,…, N jt ) (1) The PMT is then based on: ˆ ijt   y ˆ jt  ˆ x jt ijt (2) The most common method in practice for estimating  jt and  jt in (1) appears to be Ordinary Least Squares (OLS) using log consumption per capita as the dependent variable. As usual, OLS 8 chooses the parameter estimates to minimize the sum of squared errors with no difference in the weights attached to poor versus non-poor households (i.e., choosing  ˆ jt and   ˆ ijt ˆ to minimize  jt 2 for each j, t). We also considered the option of using a binary indicator for whether a household’s actual consumption falls below the poverty line as the dependent variable (equal to one if a household is poor, and zero otherwise). We tried this for both OLS (giving a linear probability model) and a Probit. However, we found that targeting errors were substantially higher with a binary dependent variable in all cases. So we confine attention to the continuous dependent variable in the rest of this paper. Another option to OLS in estimating equation (1) is to try to better tailor the estimator to the specific policy problem, in this case poverty reduction. Two ways of doing this can be suggested. The first is the quantile regression method of Koenker and Bassett (1978). This is more robust to outliers than OLS, and (importantly) the method can be tailored to the problem at hand in that the quantile can be set at the overall poverty rate.21 In other words, we calibrate the PMT score to how that specific quantile in the distribution of log consumption, given the covariates, changes with those covariates. The second method entails placing higher weight on the squared errors of poorer people, giving “poverty- weighted least-squares” (PLS). Among the various weighting schemes that might be used, we choose the method proposed by Mapa and Albis (2013), which weights equally all observations below the poverty line but gives zero weight to those above the line. In other words, we run the regression on poor households only. We extend this method by including households somewhat above the line. Once we have the PLS parameter estimates we calculate the revised PMT scores using the actual values of xijt . Any PMT method is likely to be quite constrained in practice in the choice of covariates. Practitioners are restricted to using xijt variables that are considered easy to observe or verify in the field. There are feasibility constraints associated with the number and nature of the variables used in practice; administrative costs almost certainly rise with the number of variables. There are also incentive constraints, stemming from the scope for manipulation by local agents when there are many variables in the PMT (Niehaus et al. 2013). The variables used in practice typically cover readily observed living conditions of the household, such as basic consumer durables or assets, demographic variables (size and composition) and attributes of the head.22 Two important exclusions are notable. First, prices are rarely used and assets are 21 For example, this is one of the methods used by USAID (2011) in calibrating a PMT for Peru. This method is also discussed in Mills et al. (2015). 22 See, for example, the various studies in the compilation by Del Ninno and Mills (2015). 9 identified in broad categories; clearly, two households can each own a “fridge” but in one case it is 30 years old and works poorly while in the other case it is a fancy new model. Second, an important exclusion is that one cannot use fine geographic effects, such as at the level of the village, since one is constrained to estimating on a sample survey that will typically only cover a sample of villages (typically determined by the first stage of a two-stage sampling design). One does not know the geographic effect for the population, as required for implementing the PMT.23 However, in one version we include community-level variables that go some way toward addressing this concern. There is a degree of judgement required in selecting covariates. Here we consider various options, starting with a “Basic PMT” that seems to capture well the set of variables found in practice. We also consider “Extended PMT” methods that include variables that have extra explanatory power; while this provides a useful indication of the gains from more data, it is acknowledged that this version may not be easily implemented in the field. The Statistical Addendum provides descriptive statistics. 3. Measures of targeting and poverty An early strand of the literature formulated the targeting problem as that of choosing a schedule of transfer payments across types of households to minimize a measure of poverty subject to a budget constraint.24 The subsequent literature has instead emphasized “targeting efficiency,” defined in terms of reducing targeting errors as defined below. Here we shall study both types of measures. We start with targeting measures. Measures of targeting performance: The relevant counts corresponding to the joint distribution ˆ ijt are shown in Table 1, which helps clarify our notation and some of the properties of our of yijt and y measures. We focus on three main measures of targeting performance. The first is the Inclusion Error Rate (IER), defined by the proportion of those identified as poor who are not. This can be written as:25  N jt ˆ ijt  z jt ) wijt 1( y ijt  z jt y IER jt  i 1 (3)  N jt ˆ ijt  z jt ) w 1( y i 1 ijt 23 The same limitation is shared by small-area estimation methods (“poverty mapping”) as in Elbers et al. (2003). 24 The idea was developed in theoretical terms by Kanbur (1987) and the problem was formulated and solved numerically in Ravallion and Chao (1989) for the squared poverty gap index of Foster et al. (1984). Glewwe (1992) generalized this approach to allow for continuous variables. 25 The indicator function 1(.) takes the value unity when the condition in parentheses is true and zero otherwise. 10 Here the poverty line (in consumption space) is z jt and the sample size is N jt with households indexed i=1,…, N jt and wijt denotes the appropriate sample weights (to deal with differences in household size  N jt and sample design); i 1 w ijt  1 . Inclusion errors have received much attention in efforts to reduce the budgetary cost of social policies aiming to use transfer payments (in cash or kind) to reduce poverty. Inclusion errors imply a fiscal cost without any direct impact on poverty. For a uniform transfer paid to all those who are deemed to be poor, the IER gives the share of the transfers going to the non-poor.26 If everyone is deemed “poor,” so the transfer payment is universal, then IER is simply one minus the poverty rate. The IER is often normalized by the poverty rate when the latter varies, which we will also do in some cases. The resulting measure has been used extensively—clearly more than any other targeting measure—in comparing the targeting performance of social programs across developing countries.27 Critics of the focus on reducing inclusion errors have pointed to a number of issues, including measurement errors and the need for more inclusive policies in the interest of social coherence/stability.28 The second measure is the Exclusion Error Rate (EER), given by the proportion of the poor who are not identified as poor. (Sometimes the term “coverage rate” is used instead, which is simply one minus the EER.) For a social program providing a uniform transfer payment to all—variously called a “basic income guarantee” or “citizenship income”—the EER is of course zero, since everyone is covered. One might expect measures based on the EER to be better predictors of a social program’s impact on poverty.29 While that is intuitive—the more the poor are covered, the greater their expected gain—it does not necessarily hold as it will depend on the measure of poverty used, the distribution of coverage and the budget.30 The Exclusion Error Rate can be written as:  N jt i 1 ˆ ijt  z jt y ijt  z jt ) w ijt 1( y EER jt  (4) i 1 wijt 1( y ijt  z jt ) N jt 26 This is what Weisbrod (1970) dubbed “vertical efficiency.” 27 This normalized share of transfers going to the poor was used by Coady et al. (2004a, b) to compare 85 programs across many countries. 28 Weisbrod (1970) raised concerns about focusing solely on reducing inclusion errors (vertical efficiency in his terms). On measurement errors in targeting see the discussion in Ravallion (2008). 29 See Ravallion (2009) who finds supportive evidence using data for a large cash transfer program in China. 30 For example, for the headcount index of poverty one focuses on whether there is exclusion at the poverty line. 11 To better understand the properties of these measures it helps to also think of IER and EER in probabilistic terms as: IER jt ˆ ijt  z jt )  Pr( y ijt  z jt , y  Pr( y ijt  z jt y ˆ ijt  z jt ) ˆ ijt  z jt ) Pr( y (5.1) EER jt ˆ ijt  z jt y ijt  z jt )  Pr( y  Pr( y ˆ ijt  z jt , y ijt  z jt ) Pr( y ijt  z jt ) (5.2) Plainly, when the predictions are perfect ( y ijt  y ˆ ijt for all i,j,t) IER jt  EER jt  0 for all j,t. Note that: ˆ ijt  z jt )  Pr( y ijt  z jt , y Pr( y ijt  z jt , y ˆ ijt  z jt ) ˆ ijt  z jt )  Pr( y ˆ ijt  z jt )  Pr( y ijt  z jt , y Pr( y ijt  z jt , y ˆ ijt  z jt )  Pr( y ijt  z jt ) ˆ ijt  z jt )  Pr( y ijt  z jt , y ˆ ijt  z jt )  Pr( y ijt  z jt ) implies Pr( y ijt  z jt , y Also note that Pr( y ˆ ijt  z jt ). Then, from (5.1) and (5.2), we see that IER jt  EER jt . Thus the two error rates are equalized (though ˆ jt  H jt ), i.e., not at zero unless all levels are predicted correctly) when the poverty rates are equal ( H ˆ ijt  z jt )  Pr( y ijt  z jt ) implies that IER jt  EER jt . Intuitively, this is because each time a person Pr( y who is in fact poor (based on the survey-based consumption) is incorrectly identified as non-poor, that person has to be replaced by someone who is in fact non-poor, so as to keep the total count of the poor constant. In other words, every exclusion error must generate an inclusion error once the poverty rate is identical when comparing actual and predicted values. Of course, we do not expect the actual and predicted poverty rates to be equal in general. However, in the methodology of PMT there is the option of fixing the poverty rate for predicted values according to the survey-based measure using actuals. For example, if the survey indicates that 20% of the population is poor then one targets the poorest 20% based on the PMT scores. When the poverty rate is fixed this way we will simply refer to the “Targeting Error Rate” (TER). The third measure is the Normalized Targeting Differential (NTD). In the context of a transfer program, the (ordinary) Targeting Differential (TD) is defined as the mean transfer made to the poor less that made to the non-poor.31 For a uniform transfer paid to all those who are deemed eligible, the TD becomes the difference between the proportion of the poor who are predicted to be poor and the proportion of the non-poor who are predicted to be poor. (In the case of a specific antipoverty program it is the difference between the program’s coverage rate for the poor and that for the non-poor.) The NTD divides this measure by the mean transfer receipt, to make the resulting measure more comparable 31 This measure was proposed by Ravallion (2000). Also see Galasso and Ravallion (2005) and Ravallion (2009) on the properties of this measure and the discussions in Stifel and Alderman (2005) and Stoeffler et al. (2016). 12 across countries and programs. For a basic income guarantee, NTD=0. When only the poor get help from the program and all of them are covered, the NTD reaches its upper bound of 1; when only the non-poor get the program and all of them do, the NTD is at its lower bound of -1. For a uniform transfer to all recipients in the amount  jt we have:  N jt TD jt i 1 ˆ ijt  z jt y ijt  z jt ) wijt 1( y ˆ jt  EER jt 1 H NTD jt   1  EER jt   (6)  jt  1  H jt N jt w 1( y ijt  z jt ) i 1 ijt where H jt is the headcount index of poverty (or poverty rate), defined as the proportion of the relevant ˆ jt is the population living in households with consumption per person below the poverty line, and H headcount index obtained based on predicted consumptions. Another concept of targeting errors occasionally found in the literature makes the distinction between “Type 1” (T1) and “Type 2” (T2) errors of targeting (borrowing the terms from statistics).32 The former is defined as the proportion of the (ineligible) non-poor who are assigned a program targeted to the poor; thus, in this context:33  N jt i 1 ˆ ijt  z jt y ijt  z jt ) wijt 1( y ˆ jt  (1  EER jt ) H jt H T 1 jt   (7)  (1  H jt ) N jt w 1( y ijt  z jt ) i 1 ijt ˆ jt  H jt  H for all (j, t)), T 1 jt is directly proportional to EER jt ; When the poverty rate is fixed ( H specifically, T 1 jt  EER jt H /(1  H ) . On the other hand, the Type 2 error rate is T 2 jt  EER jt . This yields another interpretation of the NTD as (one minus) the aggregate of Type 1 and 2 errors: NTD jt  1  (T 1 jt  T 2 jt ) (8) When the poverty rate is fixed the NTD is also a simple linear transform of the exclusion rate; i.e. NTD jt  1  EER jt /( 1  H ) ). We will not use T1 and T2 given that they are so closely related to EER and NTD. 32 The designation of which is Type 1 and which Type 2 is arbitrary, and usage has varied. For example, Wodon (1997) and Ravallion (2009) define them our way but Grosh and Baker (1995) and Barrientos (2013) swap the two labels while Van Domelen (2007) has both usages. Appeals to statistics (whereby a Type 1 error is the incorrect rejection of a true null hypothesis while Type 2 is the failure to reject a false null) cannot resolve the matter since one can define the relevant null hypotheses consistently with either interpretation. (For our interpretation the hypothesis being tested is that a specific person is poor; the null is that she is not poor.) Readers are free to swap the labels and nothing substantive changes in our argument.  ˆ ijt  z jt y ijt  z jt )  H jt N jt   i 1jt w ijt 1( y ˆ ijt  z jt y ijt  z jt ) H jt N jt (1  EER jt ) . 33 N jt N Note that i 1 w ijt 1( y 13 Poverty measures: Given that poverty reduction is typically the primary (or even sole) objective of this class of policies it is appropriate that we also study impacts on poverty measures. The first measure we use is the popular headcount index, defined already. We denote the empirical cumulative distribution function (CDF) of consumption as p  F jt ( y )  [ 0 ,1] , which gives the proportion of the population of country (or group) j at date t consuming less than the amount y  [ y min , y max ] . Then the headcount index can be written as: H jt  F jt ( z jt )  i 1 N jt wijt 1( y ijt  z jt ) (9) ˆ jt . ˆ ijt (i=1,…, N jt ) is H The calculated poverty rate when based on the empirical distribution of y While H is (by far) the most popular measure in practice, its limitations are widely appreciated, notably that the measure does not reflect changes in living standards below the poverty line. We also consider two “higher-order” measures. The first is the poverty gap index, as given by the mean distance below the poverty line as a proportion of the line where the mean is taken over the whole population, counting those above the line as having zero gap.34 The poverty gap index can be written as: PGjt  w ijt (1  yijt / z jt ) (10) yijt  z jt We also make use of a distribution-sensitive measure, namely the Watts index proposed by Watts (1968) given by the mean proportionate poverty gap (counting the non-poor as having zero gap). This measure penalizes inequality among the poor, by putting higher weight on poorer people.35 The Watts index can be written as: W jt  w ijt ln( z jt /( yijt )] (11) yijt  z jt Optimal transfers for a given budget: It appears to be a near-universal practice to provide a uniform transfer payment to all those who are identified as poor by the PMT. Transfer size may vary according to the number or age of children in the household (as in some conditional cash transfer schemes) but not with respect to predicted poverty levels based on the PMT. The popularity of such uniform transfers to those predicted to be poor can be thought of as a feasibility constraint on PMT; in the field it is likely to be difficult to make finely differentiated transfers. However, it is still of interest to see how much this constraint is limiting the impact on poverty. 34 The Statistical Addendum gives selected results for the squared poverty gap index of Foster et al. (1984). 35 The Watts index is known to have a number of other desirable theoretical properties, as detailed in Zheng (1993). 14 We explore the effect of this constraint in two ways. The first is to vary the size of transfers based on the PMT scores. The second is to reformulate the problem as one of optimizing the transfers as a function of the variables going into the PMT. Quite generally one can think of the informationally- feasible transfers as a function of m observed x’s. The policy maker only observes the covariates x for each person; it is not known who is poor and who is not. However, the policy maker has a survey with much more information available for a sample. The problem is to choose the parameters of a score for assigning the real-world transfers based on the x’s, as given by: m  ijt   ( k k  jt xijt )  0 (12) k 0 0 (Here xijt  1 so that  0 jt is the intercept—the transfer received by someone with xijt  0 for k=1,…,m.) k In one version transfers are linear in the x’s, i.e.,   1 . We call this the linear optimization. We also estimate a nonlinear version with   2 , which introduces squared terms and interaction effects among the x’s.36 The choice of the score parameters  k jt is made to minimize the Watts index in the sample survey data: W jt  w ijt ln[ z jt /( yijt   ijt )] (13) yijt  z jt The choice is constrained by the budget: N jt B jt   wijt ijt (14) i 1 We solve this problem numerically.37 One start value we use for the optimization is the uniform jt  B jt and  ijt  0 for k=1,…,m. Other start values are tested. When there are case obtained by setting  0 k multiple local optima the solution for the parameters  k jt that gives the lowest value of the poverty measure is chosen. Data: The data for implementing these measures come from the World Bank’s well-known LSMS.38 Table 2 lists the countries, years of survey, and numbers of households surveyed. In keeping 36 Glewwe (1992) recommended this in his formulation of the optimal targeting problem. 37 We use the “fmincon” program in Matlab. 38 The LSMS has designed and implemented household surveys across many countries since the 1980s. These are nationally representative multi-purpose surveys spanning a quite wide range of topics. Further information can be found at the LSMS website. All surveys except for Ghana are LSMS-ISA surveys. 15 with the bulk of the literature, our dependent variable is log total consumption per capita.39 Consumption is measured in local currency units. Spatially deflated consumption values are available for all countries except Burkina Faso.40 We use two poverty lines, corresponding to H jt  0.2 and 0.4 for all (j, t). The 40% figure coincides fairly closely with the overall poverty rate found for the Africa region using the World Bank’s international line.41 The 20% rate allows us to focus on how well the method does at identifying those who can be considered extremely poor. When comparing the actual values and the PMT scores one can chose to either fix the poverty rate (at 0.2 or 0.4) or fix the poverty line in the 1 1 consumption space (i.e., fixing z jt  F jt ( 0 .2 ) or F jt ( 0 .4 ) ). This choice makes a difference and practice varies so we present results for both options. 4. Results for Basic PMT The “Basic PMT” follows our reading of prevailing practice. Variables used include the type of toilet a household has; floor, wall and roofing material; type of fuel used for cooking; certain characteristics of the head, including gender, education and occupation; the household’s religion and demographic size and composition. All regressions have dummy variables for categories of household size, age of head, month of survey and region of residence; the latter is measured at an aggregate level (typically a state or province) for which the surveys can be considered representative. The Statistical Addendum gives the OLS regression results for the Basic PMT. The simple average R2 is 0.53, with a range from 0.32 (for Ethiopia) to 0.64 (Burkina Faso); Table 3 provides summary statistics for the Basic PMT (as well as for the extended version discussed below). This explanatory power is typical of past studies.42 We did not try to prune this model, by either ad hoc or more systematic methods (such as stepwise regression). This would reduce the number of predictors but (of course) also reduce R2 and probably increase targeting errors. However, we do consider stepwise regression as an option in Section 5 when using a much larger set of explanatory variables. 39 We also considered the option of using log consumption per equivalent single adult using the scales provided by the LSMS. We focus on the “per capita” case in this paper although the Addendum also gives regressions and key results using scales. 40 We use nominal consumption for Burkina Faso. 41 Using the World Bank’s international line of $1.90 a day at 2011 purchasing power parity, 43% of the population of Sub- Saharan Africa are found to be poor in 2013 (based on PovcalNet). 42 A seemingly representative set of studies is Grosh and Baker (1995) (R2 from 0.3 to 0.4), Ahmed and Bouis (2002) (R2=0.43), Narayan and Yoshida (2005) (R2=0.59), Sharif (2009) (R2=0.57), Stoeffler et al. (2015) (R2=0.62), Pop (2015) (R2=0.54) and Cnobloch et al. (2015) (R2=0.5 to 0.7). The simple average is 0.52. 16 ˆ ijt , Using the lower poverty line, fixed across the comparisons between distributions of yijt and y the Basic PMT substantially under-predicts the poverty rate in all countries. This is not unexpected given the properties of OLS, as noted in the Introduction. The extent of the problem can be seen in Table 4. While the poverty rate based on the data is 20%, that based on the predicted values ranges from 0 (for Mali) to 12% (Nigeria); the simple (population weighted) average is 8%. This improves considerably 1 ˆ jt is 37% with a range when one switches to the higher poverty line ( F jt ( 0.4 ) ), for which the average H from 29% to 44%. OLS results for Basic PMT: We turn now to the targeting measures. Table 5 gives the results. Let us focus first on the fixed poverty line case with H=0.2. On average, the rate of inclusion errors implies that 48% of those identified as poor by the Basic PMT method are in fact non-poor, i.e., just over half of those identified as poor using Basic PMT are in the poorest 20% when measured using the survey-based consumption. For a poverty rate of 20% and a fixed line, the PMT method has nearly halved the rate of inclusion errors that would be obtained with a uniform transfer payment. However, this has come at the expense of exclusion. The average exclusion error is sizeable, with 81% of those who are in the poorest 20% in terms of survey-based consumption being incorrectly identified as non- poor by the PMT method. There is considerable variation across countries, with IER ranging from 33% to 100%, and EER from 55% to 100%. In the country with the lowest coverage rate of the poor implied by PMT, Mali, all poor families are incorrectly identified as non-poor. Unsurprisingly, we also find a tendency for the PMT to do better at correctly identifying poor households when the R2 in the PMT regression is higher (comparing Tables 5 and 3). However, the proportion of households correctly included is less than half of those who are poor under a poverty line corresponding to H=0.2. Both inclusion and exclusion errors are lower for H=0.4. Taking a (population weighted) average of our estimates of IER and EER for H=0.4, we find that 36% of those who are poor are excluded on average, while 31% of those who are deemed poor are actually not poor. So we again find that econometric targeting halves the inclusion error rate of 0.6 that would be implied by uniform transfers. There is also less spread in the values across countries with IER ranging from 25% to 40% and EER from 24% to 56%. The finding that the errors tend to be higher using the lower poverty line again suggests that econometric targeting may have difficulty in identifying those who are very poor. This is supported by Figure 1, which plots actual consumption against predicted consumption. The lines represent the poverty 17 lines at H=0.2. The bottom left quadrant represents households that are correctly identified as poor by the Basic PMT. The top left quadrant is the inclusion error, and the bottom right quadrant is exclusion. In one case, Mali, there are no data points in the bottom left quadrant; only one household is (incorrectly) predicted to be poor (thus giving the result that IER=EER=1.0 from Table 5). While Mali is exceptional in this respect, the point remains that PMT is missing many of the poorest households in all countries. Figure 2 gives the implied residuals. As expected, these tend to be lower (more negative) for poor people, but it is notable just how much the PMT regression is over-estimating the living standards of the poorest. For the poorest 20% in terms of actual consumption, the mean residual ranges from -0.73 to -0.37, implying that the PMT regressions yield predicted consumptions for the poor between 50% and 100% above their actual consumption.43 (The fact that consumptions of the poor are overestimated by the PMT regressions at the poverty line, as is evident in Figure 2, echoes our finding above that Basic PMT underestimates the poverty rate.) Looking at Figures 1 and 2, one can understand why many of those accepted or rejected might be tempted to believe that econometric targeting is something like a random lottery, or maybe even divine intervention (with reference to the quote from Adato and Roopnaraine, 2004, in the Introduction). At a given level of consumption, the predicted values generated from the PMT can vary considerably – see, for example, Ethiopia. A more encouraging finding is that households who are incorrectly included do not seem to be among the wealthiest households, that is, many of these households have actual consumption values that are relatively close to the poverty line. So far we have focused on PMT using a fixed poverty line in consumption space. As we have seen, this tends to predict far fewer households as poor than the actual poverty rate, particularly when the poverty line corresponds to H=0.2 (Table 4). Table 5 also provides the results for the case where we instead fix the poverty rate. For example, we calculate the mean targeting error for the poorest 20% in the distribution of predicted consumption to be 51%, falling to 32% using H=0.4. Note that fixing the poverty rate instead of the poverty line will typically increase the number of predicted poor households thus resulting in higher IER and lower EER. As noted in the introduction, “targeting errors” may reflect to some extent time-varying measurement errors in the cross-sectional data. For those countries with panel data we can address this problem by assessing targeting performance using the time-mean consumption instead of current 43 The mean residuals for the poorest 20% by country are -0.371 (Burkina Faso), -0.711 (Ethiopia), -0.497 (Ghana), -0.564 (Malawi), -0.725 (Mali), -0.402 (Niger), -0.401 (Nigeria), -0.555 (Tanzania), and -0.543 (Uganda). 18 consumption. This will reduce, though probably not eliminate, any bias due to time-varying measurement errors. The lower panel of Table 5 gives the results. In the majority of cases, the measures of targeting performance improve, although this is less evident for exclusion errors than inclusion errors when using a fixed line. (Ethiopia accounts for about half of the exceptions.) Overall, the results are broadly consistent with the view that measurement errors are playing some role, but the panel data do not overturn our main conclusions about PMT. (Section 8 returns to the panel data, as a means of allowing for implementation lags in PMT.) Results using “poverty-focused” estimation methods: As noted, the OLS method used for the Basic PMT chooses the parameter estimates to minimize the unweighted sum of squared errors. Recall that we consider two “poverty-focused” options to OLS. The first is a quantile regression using the poverty rate as the quantile. For this estimator, Table 6 gives the analogous results to Table 5. This method allows a substantial reduction in the exclusion error rate using a fixed poverty line. This comes at the cost of higher inclusion errors, especially when using the lower poverty line (anchored to a quantile of 0.2). Targeting errors are similar to those for Basic PMT when using a fixed poverty rate instead. Table 7 reports the targeting errors using our PLS method when a fixed poverty line is used to classify predicted poor households, as well as the results when a fixed poverty rate is used instead. (The Statistical Addendum gives the coefficients for our PLS regression with the Basic PMT variables.) In both cases, the weighted regressions correctly include almost all poor households. However, as with the poverty-quantile regression, inclusion errors are also high. The PMT using PLS regression is better at covering the poor but predicts that too many households are poor. An alternative is to include some households who are above the poverty line in the PLS regression. We did this by also including in the sample all households at or below the poverty line, plus the next 20% of households, as ranked by their consumption. For example, at the poverty line for H=0.2, the bottom 40% of households is used in the regression. For H=0.4, the bottom 60% is used. Table 8 provides the inclusion and exclusion errors for this version. There is a decrease in the IER relative to Table 7, but with higher EER (though still lower than for the OLS). Targeting subgroups within a national population: So far we have used a basic PMT calibrated to national populations. However, when using PMT to target programs meant for a specific group it will typically be better to calibrate the PMT to that group. We tested this by estimating the PMT model on the samples restricted to two groups of households, namely those containing elderly and/or disabled 19 members, and those households with children under 5. Next we compared the targeting measures based on the predicted values for each group with those predicted for the same household subgroups using a nationally calibrated PMT. We found that there is a modest improvement in targeting performance when using the sub-group-specific PMT. For example, to focus on the elderly and disabled subgroup case: for a fixed poverty rate of 0.2, average targeting errors go down from 0.50 to 0.47. Both inclusion and exclusion errors also fall using a fixed line, from 0.40 to 0.37 and 0.74 to 0.70, respectively. But again the gains are small (see Statistical Addendum for full results). 5. Extended PMT We now test an extended specification with far more data, including the household’s water source; more detailed information about housing materials; the number of household members per room; whether the household has a separate room for cooking; whether the household has electricity; household assets; and more details on the characteristics of the household. Regression results for the extended PMT are shown in the Addendum. The values of R2 are higher but in most cases the gains are relatively small; although the number of explanatory variables has almost doubled there are clearly some strong correlations between the extra variables and those in the core set used for the Basic PMT. As expected, the Extended PMT does better than Basic PMT with respect to targeting errors (Table 9). However, the improvement would have to be judged as modest (comparing Tables 9 and 5). For example, many more than half of the poorest 20% are still misidentified as non-poor. Table 4 also gives the average proportion of households that are predicted as poor using the poverty line methodology under the extended PMT method. At H=0.2, we see a slight improvement over the Basic PMT, with 11% of the sample predicted to be poor using the poverty line corresponding to H=0.2 (as compared to 8% using Basic PMT). The results are more similar between Basic PMT and Extended PMT for a line corresponding to H=0.4. We also reran the poverty-weighted PMT regressions as in Section 3 for the extended PMT model. The Addendum gives the regression results when the extended PMT model is estimated on the bottom 20th, 40th and 60th percentiles. The Addendum also gives the targeting errors for the poverty line and poverty rate method when the PMT is fitted using poor households only as well as the results when the poor plus the next 20 percent of the distribution are used. The key findings for Basic PMT using the poverty-weighted regression (Table 6) were confirmed using the Extended PMT. 20 The field implementation of a PMT formula with many variables is expensive and difficult, so some practitioners have opted for stepwise regression to obtain a more parsimonious PMT. We tested a backwards stepwise regression on the extended model to identify the key variables in the PMT. We used a cut-off of p = 0.01. The targeting errors for the more parsimonious regressions are given in the Addendum. We see a modest increase in the targeting errors, which are now back to approximately the same values we found for Basic PMT. A further methodological change we considered is to include variables that are not as readily available as those in our Extended PMT regressions, but are likely to have extra explanatory power. In one case we used extra data on households’ food security as well as on any shocks the household may have experienced. (Note that these variables are only available for four countries.) We augment the extended PMT model with these food security and shock variables. (The Addendum lists the variables, their means and the regression results for this model.) The R2 increases slightly for all countries (Table 3). However, this version produced negligible improvement in targeting (Addendum). In another variation on the Extended PMT we included a range of community-level variables; again this was not possible for all countries. And (again) there was only a modest reduction in targeting errors, as can be seen in the Addendum. We also tried other versions of PMT that might be of interest. In one case we used quantile regression at the median (in both the Basic and Extended PMTs). In another we used log consumption per equivalent single adult as the dependent variable. The Addendum gives the results. There was little improvement in the targeting performance of the PMT. So far we have focused solely on the inclusion and exclusion rates as the measures of targeting performance. These appear to be the most popular measures in the literature, though others have been proposed and used in some studies. Probably the most promising example of the latter when the policy objective is poverty reduction is the targeting differential (Ravallion, 2000, 2009). Recall that the normalized TD is in the range [-1, 1], with zero corresponding to a uniform (un-targeted) transfer. Table 10 gives summary statistics on the normalized targeting differential using both the Basic and Extended PMTs. The mean NTD for Basic PMT is 0.21, meaning that if program participation was based on the PMT scores the participation rate for the poor would be 21% points higher than that for the non-poor. Using Basic PMT, in four of the five countries with panel data, the NTD is higher using time- mean consumption; this rises to five out of five using the Extended PMT. Returning to the cross-section surveys, the poverty-quantile method yields the highest NTD, at around 0.49 on average for Basic PMT, 21 rising to 0.53 for the Extended PMT. For all nine countries, the poverty-quantile regression method comes out best. The poverty-weighted method does almost as well provided that the 20% of households above the line are included. 6. Poverty impacts of stylized transfer schemes using various targeting methods PMT is typically used to identify eligible recipients of a specific transfer scheme with the aim of reducing poverty. So we now study the poverty impacts of stylized transfers that are allocated according to various PMT specifications and selected counterfactuals. Our comparisons are all budget neutral with the budget for each stylized scheme set at the aggregate poverty gap ( PG jt z jt N jt ) for that country.44 We assume a poverty line corresponding to H=0.2. (The Addendum gives the average transfer amounts by country.) If the PMT worked perfectly— so that predicted consumption equaled actual consumption—then the transfers differentiated to exactly fill the poverty gaps would eliminate poverty. In this section we confine attention to uniform transfers among those deemed eligible, as is common in practice; in the next section we consider more finely differentiated transfers. A natural benchmark is a universal (“basic income”) scheme in which every person (whatever their characteristics) receives the same transfer payment. We then calculate the impacts on poverty of transfers using the various versions of PMT discussed above. We measure the impact of a uniform transfer per capita given to all households who are predicted to be below the line according to the PMT. The total transfer amount for a given country (as given by the country’s aggregate poverty gap) is divided by the total number of individuals who reside in designated poor households, and distributed to households according to their size. (For example, if a poor household has two members, the transfer will be two times the per capita amount.) We also consider counterfactual policies that use categorical targeting rather than PMT. These policies make uniform transfers within a specified category of people, as defined by a “poverty scorecard.” Here we consider an especially simple form of demographic scorecard.45 The first category is the set of persons 65 years or older. The second is any person who is a (female) widow, disabled (where disabled is defined as an illness or condition that significantly impairs a person over the age of 44 We make no claim about this being a realistic budget but we do consider it to be an obvious reference point. 45 Indeed, our method is even simpler than the “Simple Poverty Scorecard” developed by Schreiner (2010, 2015) and used for the Progress out of Poverty Index. 22 14 and their ability to work or study), or orphaned (defined as any child 14 or younger whose parents have both died or whose whereabouts are unknown). The third is a combination of the first two: a transfer to the elderly, widowed, disabled or orphaned. Note that if a person fits two categories, the score and (hence) transfer is doubled. The fourth transfer is a payment to households with children – whereby up to three children are each alloted a transfer. Finally, the last scheme combines all previous schemes, where children, the elderly, widowed, disabled or orphaned are eligible. (Recall that all stylized schemes considered have the same aggregate budget.) Table 11 shows the implied headcount index for each case. (Recall that the baseline headcount index across all countries is 20%.) Most methods bring the poverty rate down to around 16%, well short of eliminating poverty; indeed, more than three-quarters of the poor remain poor. On average, Basic PMT does only slightly better than the universal basic income with the same budget, and Basic PMT does not do as well as the universal transfer in one third of the countries. Using the time-mean consumptions for the countries with panel data makes little difference on average. The quantile regression method does noticeably better on average, bringing the poverty rate down to one percentage point below the level attainable with the Basic PMT. Extended PMT does slightly better. However, it is notable how well categorical targeting does in many cases. On average, categorical targeting to households with elderly, widows, disabled and children does as well as Basic PMT. Nevertheless, categorical targeting never does as well as the poverty quantile regression method, which typically has the greatest impact on poverty. While categorical targeting does not have quite as much impact on poverty as the Basic PMT, it clearly comes close and is simpler and more transparent. Tables 12 and 13 give the corresponding results for the poverty gap index and Watts index respectively; the pre-transfer poverty measures are shown in the first row. Aggregating across countries, the Basic PMT methods reduce the poverty gap by around 27% and the Watts index by 28%. As for the headcount index, the Extended PMT gives a larger reduction, namely 35% and 39% respectively. Simply giving a uniform transfer based on household size does as well as Basic PMT on average for both PG and the Watts index. 7. Allowing differentiated transfers So far we have focused on the standard practice of giving the same transfer payment to all those predicted to be poor using PMT. While this is the most relevant case in practice, differentiating the transfers could be expected to work better if the predicted poverty gaps are quite accurate. However, we 23 have already seen that this is not the case—that PMT works poorly in predicting the levels of living of the poorest. So it is unclear on a priori grounds whether differentiated transfers will have larger impacts on poverty. How much better can PMT do using the same information if the transfers are differentiated, with more going to those who appear to be poorer? To put the question another way: how much does the constraint of relying on uniform transfers to the “predicted poor” limit the effectiveness of PMT? We address these questions in two ways. First, we simply fill the predicted poverty gaps, scaling up (or down) to attain the same budget. That is, each household predicted as poor receives the difference between the poverty line and its predicted consumption value, scaled such that the sum of all transfers equals the aggregate poverty gap. “PMT Gap” refers to this first method. The allocation of transfers obtained this way need not be optimal in the sense of minimizing an agreed poverty index for a given budget. Following Ravallion and Chao (1989) and Glewwe (1992), we also devised a program for calculating the optimal allocation based on the set of PMT covariates. We chose the Watts index as the objective function given its desirable properties as a poverty measure (Section 2), which also provides suitable curvature to the objective function. Multiple solutions were common but we also found that the objective function tended to be quite flat in the sub-set of the parameter space corresponding to the various solutions found. Indeed, for all nine countries the minimum value of the Watts index was the same up to two decimal places whatever start value we used (though the parameter estimates themselves often differed for a given country). Table 14 gives the results for the Watts index for H=0.2 using differentiated transfers that are determined by both the PMT gaps and optimization. (The Addendum gives those for the headcount index, though note that the solutions are only optimal for the Watts index.) Overall, filling the predicted gaps does little to reduce the poverty measure. As expected, the non-linear specification in the optimization routine (   2 in equation 12) does better than the linear one in reducing poverty, and the nonlinear version does as well on average as the PMT. 8. Allowing for lags in PMT implementation Lags in the implementation of a PMT are almost certainly universal. It takes some time to set up the data and the administrative apparatus for implementation. Yet there is undoubtedly some “churning” 24 in living standards over time, even when using consumption as the welfare indicator. So the lags in implementation have bearing on the performance of PMT in reducing current poverty. We have panel data for a subset of our study countries, namely Ethiopia, Malawi, Nigeria, Tanzania and Uganda. By exploiting the panel data, we can introduce a 1 to 2 year lag in the implementation of PMT. The precise lags are one year for Uganda, and two years for the other countries.46 In other words, we develop the PMT on the Round 1 survey data and then apply it in Round 2. If anything, our lags appear to be less than found in practice.47 We consider two types of lags. In the first (Method 1), we take the regression parameters from Round 1, but use the covariates from the Round 2 data. Here there is no lag in the observations of the covariates; the lag is only due to the need to estimate the PMT scores. In the second (Method 2), we simply use the PMT score from Round 1, which we then compare to the survey data on consumptions in Round 2. The lag then applies to all aspects of the PMT method (both parameter estimates and covariate values). The targeting errors obtained using Methods 1 and 2 are found in Tables 15 and 16, respectively. Comparing the results in Table 15 (top panel) with Table 5 we see that allowing for lags increases the targeting errors on average.48 For the lower line, we now find that, on average, about half of those predicted to be poor are not in fact poor based on the survey data (a mean IER of 0.553, as compared to 0.481 from Table 5). Exclusion errors are also affected, though these errors rose less markedly. Using the Extended PMT we also find a substantial increase in the targeting errors, especially for inclusion, when we allow for lags using Method 1. A similar pattern is found for Method 2. In Table 17 we give the targeting differentials. Allowing for lags, we find mixed results. Ethiopia and Nigeria have better pro-poor targeting with lags, while Malawi, Tanzania and Uganda see increases in targeting of the non-poor. Malawi in particular has negative TD’s, indicating that the correction for lags now means that a uniform (un-targeted) policy would do better. The post-transfer poverty rates allowing for lags are provided in Table 18. PMT still brings the poverty measures down, but by about two percentage points less when allowing for lags. For example, allowing for lags achieves an average post-transfer headcount index of 19% instead of 17%. The stylized categorical targeting schemes now attain similar or somewhat lower post-transfer poverty rates. On 46 The survey years are as follows: Ethiopia 2011/12 and 2013/14; Malawi 2010/11 and 2013; Nigeria 2010/11 and 2012/13; Tanzania 2010/11 and 2012/13; Uganda 2010/11 and 2011/12. 47 For example, recall that the mean lag between the survey year and the release date of the Progress out of Poverty Index is 3.9 years (Introduction). 48 Switching to the panel samples changes the targeting measures somewhat but the following observations still hold. 25 average, the simple demographic scorecards bring the poverty rate down by an extra one and in some cases two percentage points once one allows for plausible lags in PMT implementation. The Addendum gives results for other poverty measures, which follow a similar pattern. 9. Predictors of poverty impacts We can bring a number of the results from previous sections together to quantify the relative importance of targeting errors and estimator fit to the impacts of PMT on poverty. With only nine country observations, cross-country comparisons should be treated with caution. Nonetheless, some strong patterns emerge even with so few degrees of freedom. As noted in the Introduction, inclusion errors have tended to receive more attention in the policy community although it has been argued by some that exclusion errors may well be more important to the impacts on poverty. There is a clear pattern in our results whereby the exclusion error rate is generally a better predictor of the poverty impact of PMT than the inclusion error rate. This can be seen in Table 19 which gives regressions of the final “post-PMT” poverty measure on the two error rates (with controls for the initial poverty measure, which is of course a constant in the case of the headcount index). In all instances, the EER is the stronger predictor of impacts on poverty, and it is a strong (statistically significant) predictor in all but one case (the exception being for the method of filling the “PMT gap” described in section 7). Another plausible predictor of both the targeting performance of PMT and its impact on poverty is the R2 of the original regression used to calibrate the PMT. Indeed, it appears that this is often the main parameter that practitioners focus on in PMT applications. Table 20 gives the regressions across countries for both Basic and Extended PMT. The value of R2 has only weak predictive power for the main measure used in practice, the headcount index. The R2 does emerge as a strong predictor of the poverty impacts of standard PMT for the higher-order poverty measures (PG and the Watts index). This is not the case when we allow for differentiated transfers using either of the methods from the last section, although (as noted) the practical relevance of differentiated transfers is a moot point. For the more relevant case of uniform transfers across those predicted to be poor, R2 can only be considered a useful predictor of impacts on poverty for the higher-order measures. 10. Conclusions 26 Highly imperfect information and limited administrative capabilities create challenges for implementing effective antipoverty programs in most developing-countries. Practitioners have often turned to some form of proxy means test. While these methods have an a priori appeal, users should have realistic expectations of what the methods can deliver. Our results point to both strengths and weaknesses of standard econometric targeting methods. While these methods can substantially reduce inclusion errors in an antipoverty program—in most cases studied here the inclusion error rate can be at least halved—this comes at the cost of substantial exclusion errors when judged against the data on household consumption used to calibrate the test scores. Standard methods found in practice may look fine when the sole aim is to reduce inclusion errors—to prevent non-poor people receiving benefits when judged against a fixed poverty line. However, if poverty-reduction relative to a fixed line is the objective then policy makers with a given budget should be more worried about exclusion errors than inclusion errors. When attention switches to the problem of assuring broad coverage of the poor (reducing exclusion errors), better methods can be proposed, which give higher weight to performance in predicting the living standards of poor people. The method we find to generally perform best from the point of view of reducing exclusion errors is a “poverty-quantile regression.” This method generates more inclusion errors than prevailing PMT methods, though still less than un-targeted transfers. When judged in terms of the impact on poverty for a given budget (set equal to the aggregate poverty gap), we find that what appears to be the most widely-used form of PMT in practice does only slightly better on average than a universal basic income, in which everyone gets the same transfer, whatever their characteristics. One can achieve somewhat larger impacts on poverty using other PMT methods considered here, with either a richer data set or using the poverty-quantile regression method. However, even under seemingly ideal conditions, the “high-tech” solutions to the targeting problem with imperfect information do not do much better than age-old methods using state-contingent transfers or even simpler basic income schemes. We find that an especially simple demographic “scorecard” method can do almost as well as econometric targeting in terms of the impacts on poverty. Indeed, on allowing for likely lags in implementing PMT, the simpler categorical targeting methods perform better on average in bringing down the current poverty rate. This conclusion would undoubtedly be strengthened once the full costs of fine targeting are taken into account. We were surprised that econometric targeting only allowed such small (or even negative) gains in reaching poor people compared to simpler methods. For practitioners deciding on targeting methods 27 going forward, we suspect that other criteria besides targeting accuracy should take precedence in the choice, such as the specifics of the poverty profile, administrative capabilities and cost, the need for transparency, and the scope for fine targeting to undermine political support for social policies. Not everything relevant to performance on the ground can be quantified using the kind of “laboratory experiment” done here. Field implementation may introduce idiosyncratic mistakes or allow ad hoc efforts to improve beneficiary selection on the ground. Nor have we allowed for the pursuit of other objectives besides poverty reduction such as social cohesion or administrative capacity building. That said, looking at our findings as a whole, it would be fair to say that none of the examined methods performs particularly well when one is striving to reduce poverty. When the budget required for a set of transfer payments that would eliminate poverty (ignoring behavioral responses) is allocated by any of these methods, about three-quarters of the original (pre-intervention) count of poor people remain poor. 28 References Adato, Michelle, and Terry Roopnaraine, 2004, “A Social Analysis of the Red de Proteccion Social (RPS) in Nicaragua.” IFPRI Report, International Food Policy Research Institute, Washington DC. Ahmed, Akhter and Howarth Bouis, 2002, “Weighing what’s Practical: Proxy Means Tests for Targeting Food Subsidies in Egypt.” Food Policy 27: 519-540. Alatas, Vivi, Abhijit Banerjee, Rema Hanna, Benjamin Olken and Julia Tobias, 2012, “Targeting the Poor: Evidence from a Field Experiment in Indonesia.” American Economic Review 102(4): 1206-1240. Atkinson, Anthony B., 1995, “On Targeting Social Security: Theory and Western Experience with Family Benefits,” in Dominique van de Walle and Kimberly Nead (eds) Public Spending and the Poor, Johns Hopkins University Press. Barrientos, Armando, 2013, Social Assistance in Developing Countries, Cambridge: Cambridge University Press. Beckerman, Wilfred, 1979, “The Impact of Income Maintenance Payments on Poverty in Britain, 1975,” Economic Journal 89: 261-279. Brown, Caitlin, Martin Ravallion and Dominique van de Walle, 2016, “How well do Household Poverty Data Identify Africa’s Nutritionally Vulnerable Women and Children,” mimeo. Cameron, Lisa, and Manisha Shah, 2014, “Mistargeting of Cash Transfers, Social Capital Destruction, and Crime in Indonesia,” Economic Development and Cultural Change 62(2): 381-415. Cnobloch, Rodica, and Kalidni Subbarao, 2015, “Targeting Methods to Identify the Poorest in Malawi.” In C. Del Ninno and B. Mills (Eds.), Safety Nets in Africa: Effective Mechanisms to Reach the Poor and Most Vulnerable. The World Bank, Washington D.C. Coady, David, Margaret Grosh and John Hoddinott, 2004a, “Targeting Outcomes Redux,” World Bank Research Observer 19(1): 61-86. __________, 2004b, Targeting Transfers in Developing Countries: Review of Lessons and Experience, Washington D.C.: World Bank. Cornia, Giovanni, and Frances Stewart, 1995, “Two Errors of Targeting,” in Dominique van de Walle and Kimberly Nead (eds) Public Spending and the Poor, Johns Hopkins University Press. Diamond, Alexis, Michael Gill, Miguel Rebolledo Dellepiane, Emmanuel Skoufias, Katja Vinha, and Yiqing Xu, 2016, “Estimating Poverty Rates in Target Populations: An Assessment of the 29 Simple Poverty Scorecard and Alternative Approaches,” Policy Research Working Paper 7793, World Bank. Del Ninno, Carlo and Bradford Mills, 2015 (Eds.), Safety Nets in Africa: Effective Mechanisms to Reach the Poor and Most Vulnerable. The World Bank, Washington D.C. Elbers, Chris, Jean Lanjouw and Peter Lanjouw, 2003, “Micro–Level Estimation of Poverty and Inequality,” Econometrica 71(1): 355-364. Flores, Mario González, 2014, “Evaluation of the Progress Out of Poverty Index (PPI): The Case Of Peru,” in Three Essays on Evaluation and Measurement in Developing Economies, Doctoral Dissertation, American University. Foster, James, Joel Greer, and Erik Thorbecke, 1984, “A Class of Decomposable Poverty Measures,”Econometrica 52: 761-765. Galasso, Emanuela and Martin Ravallion, 2005, “Decentralized Targeting of an Antipoverty Program,” Journal of Public Economics 85: 705-727. Gelbach, Jonah and Lant Pritchett, 2000, “Indicator Targeting in a Political Economy: Leakier Can be Better,” Journal of Policy Reform 4: 113–45. Glewwe, Paul, 1992, “Targeting Assistance to the Poor: Efficient Allocation of Transfers when Household Income is not Observed,” Journal of Development Economics 38(2): 297-321. Grosh, Margaret, 1994, Administering Targeted Social Programs in Latin America: From Platitudes to Practice. Washington D.C.: World Bank. Grosh, Margaret and Judy Baker, 1995, “Proxy Means Tests for Targeting Social Programs: Simulations and Speculation,” Living Standards and Measurement Study Working Paper 118, World Bank, Washington DC. USAID, 2011, “Poverty Assessment Tool Submission for Peru,” USAID. Kanbur, Ravi, 1987, “Measurement and Alleviation of Poverty,” IMF Staff Papers 36: 60-85. Karlan, Dean, and Bram Thuysbaert, 2013, “Targeting Ultra-Poor Households in Honduras and Peru,” NBER Working Paper 19646. Kidd, Stephen and Emily Wylde, 2011, “Targeting the Poorest: An Assessment of the Proxy Means Test Methodology,” AusAID Working Paper, Government of Australia. Koenker, R., and G. Bassett, 1978, “Regression Quantiles,” Econometrica 46: 33-50. 30 Mapa, Dennis S. and Manuel Leonard F. Albis, 2013, “New Proxy Means Test (PMT) Models: Improving Targeting of the Poor For Social Protection,” 12th National Convention on Statistics (NCS) EDSA Shangri-La Hotel. Mills, Bradford, Carlo Del Ninno and Phillippe Leite, 2015, “Effective Targeting Mechanisms in Africa: Existing and New Methods,” in Del Ninno and Mills (2015). Murgai, Rinku, Martin Ravallion and Dominique van de Walle, 2016, “Is Workfare Cost Effective against Poverty in a Poor Labor-Surplus Economy?” World Bank Economic Review, forthcoming. Narayan, A. and Yoshida, N., 2005, “Proxy Means Test for Targeting Welfare Benefits in Sri Lanka.” South Asia Region Working Paper. Report No. SASPR-7. Niehaus, Paul, Antonia Atanassova, Marianne Bertrand and Sendhil Mullainathan, 2013, “Targeting with Agents,” American Economic Journal: Economic Policy 5(1): 206-238. Pop, Lucian, 2015, “Options for Improving the Targeting of Safety Nets in Ghana,” In C. Del Ninno and B. Mills (Eds.), Safety Nets in Africa: Effective Mechanisms to Reach the Poor and Most Vulnerable. The World Bank, Washington D.C. Ravallion, Martin, 2000, “Monitoring Targeting Performance when Decentralized Allocations to the Poor are Unobserved,” World Bank Economic Review 14(2): 331-45. ______________, 2008, “Miss-Targeted, or Miss-Measured?” Economics Letters, 100: 9-12. ______________, 2009, “How Relevant is Targeting to the Success of the Antipoverty Program?” World Bank Research Observer 24(3): 205-231. ______________, 2016, The Economics of Poverty. History, Measurement, and Policy. New York: Oxford University Press. Ravallion, Martin, and Kalvin Chao, 1989, “Targeted Policies for Poverty Alleviation Under Imperfect Information: Algorithms and Applications,” Journal of Policy Modeling 11(2): 213-224. Ravallion, Martin, Dominique van de Walle and Madhur Gautam, 1995, “Testing a Social Safety Net,” Journal of Public Economics 57(2): 175-199. Schreiner, Mark, 2010, “A Simple Poverty Scorecard for Jordan,” mimeo. _____________, 2015, “A Comparison of Two Simple, Low-Cost Ways for Local, Pro-Poor Organizations to Measure the Poverty of Their Participants,” Social Indicators Research 124: 537–569. 31 Smolensky, Eugene, Siobhand Reilly and Erik Evenhouse, 1995, “Should Public Assistance be Targeted?” Journal of Post Keynesian Economics 18: 3-28. Stifel, David and Harold Alderman, 2005, “Targeting at the Margin: The ‘Glass of Milk’ Subsidy Programme in Peru,” Journal of Development Studies 41(5): 839-864. Stoeffler, Quentin, Bradford Mills, and Carlo Del Ninno, 2016, “Reaching the Poor: Cash Transfer Program Targeting in Cameroon,” World Development 83: 244-263. Stoeffler, Quentin, P. Nguetse-Tegoum, and Bradford Mills, 2015, “Generating a System for Targeting Unconditional Cash Transfers in Cameroon.” In C. Del Ninno and B. Mills (Eds.), Safety Nets in Africa: Effective Mechanisms to Reach the Poor and Most Vulnerable. The World Bank, Washington D.C. van de Walle, Dominique, 1998, “Targeting Revisited,” World Bank Research Observer, 13(2): 231-48. Van Domelen, Julie, 2007, “Reaching the Poor and Vulnerable: Targeting Strategies for Social Funds and other Community-Driven Programs,” Social Protection Discussion Paper No. 711. Watts, Harold W., 1968, “An Economic Definition of Poverty,” in Daniel P. Moynihan (ed.), On Understanding Poverty, New York, Basic Books. Weisbrod, Burton, 1970, “Collective Action and the Distribution of Income: A Conceptual Approach,” in R. H. Haverman and J. Margolis (eds.) Public Expenditure and Policy Analysis, Chicago, Ill.: Markham. (Reprinted in Nicholas Barr (ed.) Economic Theory and the Welfare State, Volume 2, Edward Elgar, Cheltenham, UK, 2001.) Wodon, Quentin, 1997, “Targeting the Poor using ROC Curves,” World Development 25(12): 2083- 2092. Zheng, Buhong, 1993, “Axiomatic Characterization of the Watts Index,” Economics Letters 42: 81-86. 32 Figure 1: Actual and predicted consumption for Basic PMT 33 Note: The figure shows actual and predicted consumption in logged values using Basic PMT. The red lines represent the poverty line at the 20th percentile in logged values. Points in the top left corner are incorrectly predicted as poor (inclusion errors). Points in the bottom right corner are incorrectly predicted as non-poor (exclusion errors). Points in the bottom left and top right corners are correctly predicted as poor and non-poor respectively. 34 Figure 2: Residuals for Basic PMT plotted against log real consumption per capita Note: The figure shows log real household consumption per capita and the residuals for the predicted consumption values. Basic PMT is used to predict consumption. The red lines represent the poverty line at the 20th percentile. 35 Table 1: Counts of inclusion and exclusion Based on proxy-means test ˆ ijt  z jt y ˆ ijt  z jt y Poor either way Exclusion errors H jt N jt  w 1( y  w 1( y N jt N jt ˆ z y z ) i 1 ijt ijt jt ijt jt i 1 ˆ z y z ) ijt ijt jt ijt jt y ijt  z jt   w 1( y  z y   w 1( y  z y N jt N jt ˆ z i 1 ijt ijt jt ijt jt ) i 1 ˆ z ijt ijt jt ijt jt ) Based on survey data Inclusion errors Non-poor either way (1  H jt ) N jt  w 1( y  w 1( y N jt N jt y ijt  z jt ˆ z y z ) i 1 ijt ijt jt ijt jt i 1 ˆ z y z ) ijt ijt jt ijt jt   w 1( y  z y   w 1( y  z y N jt N jt ˆ z i 1 ijt ijt jt ijt jt ) i 1 ˆ z ijt ijt jt ijt jt ) ˆ jt N jt H (1  H ˆ jt ) N jt N jt Note: Notation explained in the main text. Table 2: Countries and survey rounds Country Year N Burkina Faso 2014 10,265 Ethiopia 2013/14 5,017 Ghana 2009 4,224 Malawi 2013/14 3,931 Mali 2014 3,212 Niger 2011 3,833 Nigeria 2012/13 3,720 Tanzania 2012/13 4,753 Uganda 2011/12 2,650 Note: All surveys except for Ghana are LSMS-ISA surveys. 36 Table 3: Summary table for the regressions Burkina Faso Ethiopia Ghana Malawi Mali Niger Nigeria Tanzania Uganda Basic PMT Basic PMT R2 0.644 0.319 0.561 0.573 0.418 0.634 0.581 0.585 0.498 N 10265 5017 4224 3931 3212 3833 3720 4753 2650 Poor 20 R2 0.175 0.136 0.290 0.151 0.126 0.156 0.274 0.176 0.231 N 10265 5017 4224 3931 3212 3833 3720 4753 2650 Poor 20 Probit R2 0.227 0.183 0.361 0.222 0.192 0.239 0.332 0.28 0.25 N 9151 5017 4224 3498 2776 3193 3491 4123 2558 Poor 40 R2 0.285 0.192 0.393 0.242 0.228 0.261 0.392 0.306 0.299 N 10265 5017 4224 3931 3212 3833 3720 4753 2650 Poor 40 Probit R2 0.292 0.186 0.368 0.244 0.223 0.271 0.383 0.319 0.275 N 10265 5017 4224 3734 2922 3638 3720 4753 2647 Weighted Bottom 20 R2 0.112 0.192 0.214 0.091 0.150 0.110 0.203 0.182 0.193 N 1395 762 755 558 422 456 871 618 628 Weighted Bottom 40 R2 0.112 0.152 0.277 0.105 0.101 0.113 0.264 0.162 0.206 N 3024 1473 1508 1186 927 961 1685 1385 1105 Weighted Bottom 60 R2 0.170 0.143 0.329 0.155 0.162 0.155 0.353 0.202 0.257 N 4895 2307 2325 1929 1599 1629 2427 2293 1595 Adult Equivalent Consumption R2 0.287 0.542 0.595 0.544 0.502 N 5017 3931 3833 4753 2650 Rural Only R2 0.465 0.201 0.475 0.462 0.356 0.355 0.538 0.449 0.426 N 6298 3148 2557 2900 2068 2326 2627 3089 2120 Urban Only R2 0.710 0.310 0.446 0.685 0.452 0.614 0.522 0.561 0.539 N 3967 1869 1667 1031 1144 1507 1093 1664 530 Extended PMT Extended PMT R2 0.724 0.381 0.587 0.674 0.520 0.718 0.666 0.647 0.596 N 10265 5017 4224 3931 3212 3833 3720 4753 2650 Weighted Bottom 20 R2 0.191 0.242 0.259 0.148 0.226 0.186 0.292 0.228 0.288 N 1395 762 755 558 422 456 871 618 628 Weighted Bottom 40 R2 0.193 0.205 0.321 0.181 0.173 0.166 0.355 0.226 0.292 N 3024 1473 1508 1186 927 961 1685 1385 1105 Weighted Bottom 60 R2 0.267 0.192 0.358 0.256 0.235 0.225 0.435 0.274 0.357 N 4895 2307 2325 1929 1599 1629 2427 2293 1595 37 Stepwise (p=0.01) R2 0.687 0.344 0.579 0.676 0.534 0.701 0.606 0.632 0.553 N 10265 5017 4224 3931 3212 3833 3720 4753 2650 Stepwise (p=0.05) R2 0.688 0.349 0.582 0.679 0.539 0.703 0.607 0.634 0.559 N 10265 5017 4224 3931 3212 3833 3720 4753 2650 Household Shocks and Food Security R2 0.726 0.395 0.690 0.722 0.670 0.654 0.604 N 10265 5017 3931 3833 3720 4753 2650 Shocks, Food Security and Community Variables R2 0.400 0.698 0.725 0.672 0.655 0.607 N 5017 3931 3833 3720 4753 2650 Note: Values are taken from the regression tables presented in the Addendum. (Results from the quantile regressions are not included in the above table.) Table 4: Proportion of sample predicted to be poor using PMT regressions       Fixed poverty line -1 z = F (0.2) z = F-1(0.4) Basic PMT Extended PMT Basic PMT Extended PMT Burkina Faso 0.083 0.112 0.359 0.358 Ethiopia 0.023 0.049 0.291 0.348 Ghana 0.115 0.126 0.350 0.360 Malawi 0.042 0.108 0.329 0.356 Mali 0.000 0.017 0.316 0.339 Niger 0.054 0.092 0.429 0.404 Nigeria 0.117 0.151 0.393 0.406 Tanzania 0.059 0.111 0.419 0.403 Uganda 0.104 0.144 0.439 0.424 Mean 0.079 0.113 0.372 0.387 Note: Predicted values are calculated from the Basic PMT and Extended PMT regressions (Addendum). Statistic are population weighted. 38 Table 5: Targeting errors using Basic PMT Inclusion error Exclusion error Inclusion error Exclusion error Targeting Targeting rate rate rate rate Error Error (IER) (EER) (IER) (EER) (TER) (TER) Fixed poverty line Fixed poverty rate z = F -1 (0.2) z = F-1(0.4) H = 0.2 H = 0.4 Burkina Faso 0.401 0.751 0.304 0.375 0.522 0.329 Ethiopia 0.515 0.945 0.396 0.562 0.621 0.413 Ghana 0.354 0.628 0.257 0.350 0.428 0.288 Malawi 0.431 0.880 0.333 0.451 0.553 0.373 Mali 1.000 1.000 0.348 0.485 0.553 0.375 Niger 0.539 0.875 0.384 0.340 0.584 0.362 Nigeria 0.332 0.548 0.247 0.243 0.392 0.244 Tanzania 0.396 0.822 0.323 0.291 0.513 0.314 Uganda 0.357 0.663 0.350 0.294 0.455 0.335 Mean 0.481 0.807 0.309 0.359 0.505 0.319 Using time-mean consumption from panel data Ethiopia 0.310 0.947 0.366 0.746 0.638 0.427 Malawi 0.311 0.837 0.321 0.429 0.517 0.341 Nigeria 0.309 0.544 0.245 0.261 0.412 0.249 Tanzania 0.340 0.765 0.291 0.339 0.461 0.303 Uganda 0.370 0.687 0.328 0.293 0.483 0.318 Mean 0.317 0.691 0.276 0.397 0.482 0.307 Note: Errors are calculated using the predicted values from the regression given in the Addendum. Statistics are population weighted. 39 Table 6: Targeting errors for Basic PMT using quantile regression centered at the poverty line Inclusion Exclusion Inclusion Exclusion Targeting Targeting error rate error rate error rate error rate Error Error (IER) (EER) (IER) (EER) (TER) (TER) Fixed poverty line Fixed poverty rate -1 -1 z = F (0.2) z = F (0.4) H = 0.2 H = 0.4 Burkina Faso 0.627 0.218 0.365 0.228 0.525 0.326 Ethiopia 0.684 0.260 0.441 0.292 0.621 0.420 Ghana 0.540 0.163 0.301 0.228 0.426 0.290 Malawi 0.636 0.267 0.383 0.304 0.548 0.364 Mali 0.660 0.231 0.401 0.253 0.630 0.375 Niger 0.663 0.199 0.408 0.212 0.603 0.378 Nigeria 0.519 0.136 0.299 0.164 0.372 0.241 Tanzania 0.632 0.173 0.364 0.153 0.528 0.327 Uganda 0.661 0.147 0.407 0.172 0.488 0.336 Mean 0.615 0.191 0.363 0.204 0.505 0.324 Note: Errors are calculated using the predicted values from the regressions given in the Addendum. A quantile regression centered at the poverty line at the 20th and 40th percentile is used to estimate the model. Statistics are population weighted. Table 7: Targeting errors for Basic PMT using a poverty-weighted regression Inclusion Exclusion Inclusion Exclusion Targeting Targeting error rate error rate error rate error rate Error Error (IER) (EER) (IER) (EER) (TER) (TER) Fixed poverty line Fixed poverty rate -1 -1 z = F (0.2) z = F (0.4) H = 0.2 H = 0.4 Burkina Faso 0.798 0.000 0.582 0.000 0.695 0.380 Ethiopia 0.799 0.002 0.598 0.000 0.707 0.512 Ghana 0.788 0.004 0.567 0.003 0.589 0.335 Malawi 0.795 0.000 0.597 0.000 0.624 0.385 Mali 0.790 0.004 0.596 0.000 0.646 0.418 Niger 0.798 0.000 0.593 0.000 0.729 0.434 Nigeria 0.756 0.007 0.560 0.004 0.619 0.325 Tanzania 0.791 0.001 0.588 0.001 0.721 0.398 Uganda 0.782 0.002 0.581 0.001 0.573 0.408 Mean 0.781 0.004 0.579 0.002 0.655 0.391 Note: Errors are calculated using the predicted values from the regression given in the Addendum with full weight on the bottom 20 and 40 percentiles respectively. Statistics are population weighted. 40 Table 8: Targeting errors using a Basic PMT weighted regression with “poor plus 20 percent” Inclusion Exclusion Inclusion Exclusion Targeting Targeting error rate error rate error rate error rate Error Error (IER) (EER) (IER) (EER) (TER) (TER) Fixed poverty line Fixed poverty rate z = F -1 (0.2) z = F-1(0.4) H = 0.2 H = 0.4 Burkina Faso 0.666 0.181 0.501 0.038 0.566 0.339 Ethiopia 0.756 0.172 0.577 0.024 0.663 0.475 Ghana 0.539 0.197 0.425 0.062 0.439 0.297 Malawi 0.681 0.160 0.521 0.040 0.564 0.363 Mali 0.702 0.173 0.520 0.040 0.653 0.380 Niger 0.708 0.099 0.525 0.027 0.624 0.390 Nigeria 0.573 0.166 0.428 0.051 0.485 0.274 Tanzania 0.653 0.172 0.466 0.049 0.584 0.334 Uganda 0.695 0.127 0.500 0.037 0.508 0.369 Mean 0.665 0.164 0.491 0.042 0.551 0.347 Note: Errors are calculated using the predicted values from the regression in the Addendum with full weight on the bottom 40 and 60 percentiles respectively. Statistics are population weighted. Table 9: Targeting errors using the Extended PMT Inclusion Exclusion Inclusion Exclusion Targeting Targeting error rate error rate error rate error rate Error Error (IER) (EER) (IER) (EER) (TER) (TER) Fixed poverty line Fixed poverty rate -1 -1 z = F (0.2) z = F (0.4) H = 0.2 H = 0.4 Burkina Faso 0.334 0.626 0.257 0.336 0.449 0.282 Ethiopia 0.419 0.857 0.373 0.455 0.541 0.405 Ghana 0.349 0.591 0.256 0.331 0.421 0.267 Malawi 0.439 0.698 0.295 0.374 0.470 0.315 Mali 0.444 0.951 0.344 0.444 0.572 0.341 Niger 0.458 0.751 0.328 0.323 0.539 0.327 Nigeria 0.330 0.496 0.228 0.217 0.384 0.223 Tanzania 0.403 0.670 0.283 0.279 0.481 0.281 Uganda 0.379 0.552 0.313 0.279 0.467 0.307 Mean 0.362 0.639 0.283 0.308 0.456 0292 Note: Errors are calculated using the predicted values from the extended PMT regressions shown in the Addendum. Statistics are population weighted. 41 Table 10: Targeting differential for various PMT specifications Burkina Faso Ethiopia Ghana Malawi Mali Basic PMT covariates Basic PMT 0.207 0.040 0.321 0.098 0.000 Using means from panel data n.a. 0.380 n.a. 0.378 n.a. Poverty quantile regression 0.453 0.338 0.591 0.412 0.395 Poverty weighted: Poor only 0.012 0.004 0.068 0.031 0.060 Poverty weighted: Poor + 20 0.411 0.185 0.568 0.392 0.339 PMT with Urban/Rural 0.210 0.073 0.321 0.123 0.001 Extended PMT covariates Extended PMT 0.327 0.117 0.354 0.243 0.039 Using means from panel data n.a. 0.182 n.a. 0.354 n.a. Poverty quantile regression 0.523 0.372 0.605 0.507 0.427 Poverty weighted: Poor only 0.117 0.042 0.124 0.138 0.147 Poverty weighted: Poor + 20 0.494 0.283 0.557 0.492 0.443 Stepwise (p=0.01) 0.292 0.064 0.350 0.289 0.162 HH Shocks + Food Security 0.327 0.121 0.298 Shocks, Food Security + Community Variables 0.120 0.324 Niger Nigeria Tanzania Uganda Mean Basic PMT covariates Basic PMT 0.088 0.339 0.149 0.289 0.214 Using means from panel data n.a. 0.382 0.319 0.259 0.366 Poverty quantile regression 0.408 0.594 0.471 0.437 0.485 Poverty weighted: Poor only 0.010 0.222 0.051 0.100 0.107 Poverty weighted: Poor + 20 0.354 0.554 0.437 0.375 0.421 PMT with Urban/Rural 0.095 0.383 0.163 0.313 0242 Extended PMT covariates Extended PMT 0.196 0.442 0.275 0.380 0.309 Using means from panel data n.a. 0.317 0.455 0.411 0.334 Poverty quantile regression 0.455 0.635 0.524 0.509 0.531 Poverty weighted: Poor only 0.122 0.353 0.084 0.201 0.196 Poverty weighted: Poor + 20 0.359 0.606 0.491 0.450 0.484 Stepwise (p=0.01) 0.169 0.307 0.274 0.373 0.249 HH Shocks + Food Security 0.225 0.464 0.300 0.386 0.333 Shocks, Food Security + Community Variables 0.227 0.451 0.308 0.405 0.331 Note: The targeting differential is computed using the poverty line at the 20th percentile. Statistics are population weighted. 42 Table 11: Headcount index of poverty post transfer Burkina Faso Ethiopia Ghana Malawi Mali Universal (basic income) 0.176 0.171 0.166 0.171 0.166 Basic PMT covariates Basic PMT 0.152 0.190 0.149 0.176 0.200 Using means from panel data n.a. 0.189 n.a. 0.167 n.a. Poverty quantile regression 0.154 0.160 0.154 0.157 0.155 Poverty weighted: Poor only 0.175 0.171 0.164 0.170 0.167 Poverty weighted: Poor + 20 0.162 0.174 0.156 0.163 0.159 PMT with Urban/Rural 0.152 0.182 0.153 0.170 0.200 Extended PMT covariates Extended PMT 0.147 0.172 0.147 0.154 0.190 Using means from panel data n.a. 0.182 n.a. 0.151 n.a. Poverty quantile regression 0.153 0.158 0.153 0.149 0.147 Poverty weighted: Poor only 0.173 0.170 0.163 0.167 0.162 Poverty weighted: Poor + 20 0.155 0.169 0.156 0.155 0.152 Stepwise (p=0.01) 0.153 0.184 0.151 0.150 0.163 HH Shocks + Food Security 0.147 0.171 0.154 Shocks, Food Security + Community Variables 0.170 0.148 Categorical targeting Elderly 65+ 0.176 0.184 0.178 0.185 0.178 Widowed or disabled 0.179 0.186 0.171 0.177 0.180 Elderly, widows & disabled 0.176 0.186 0.169 0.180 0.175 Children under 14 (max 3) 0.178 0.168 0.161 0.166 0.174 Elderly, widows, disabled & children 0.175 0.170 0.166 0.168 0.170 Female heads with children 0.189 0.189 0.181 0.176 0.191 Shock: drought, flood or livestock death 0.198 0.196 0.195 Shock: drought, flood, livestock death, job loss 0.198 0.197 0.195 Niger Nigeria Tanzania Uganda Mean Universal (basic income) 0.177 0.169 0.183 0.168 0.171 Basic PMT covariates Basic PMT 0.175 0.149 0.165 0.153 0.163 Using means from panel data n.a. 0.147 0.153 0.150 0.159 Poverty quantile regression 0.166 0.150 0.162 0.154 0.155 Poverty weighted: Poor only 0.177 0.166 0.180 0.166 0.170 Poverty weighted: Poor + 20 0.164 0.156 0.165 0.157 0.162 PMT with Urban/Rural 0.173 0.145 0.161 0.147 0.159 43 Extended PMT covariates Extended PMT 0.159 0.144 0.150 0.149 0.154 Using means from panel data n.a. 0.151 0.137 0.128 0.155 Poverty quantile regression 0.168 0.151 0.159 0.146 0.154 Poverty weighted: Poor only 0.175 0.163 0.181 0.162 0.168 Poverty weighted: Poor + 20 0.168 0.151 0.164 0.151 0.157 Stepwise (p=0.01) 0.161 0.153 0.154 0.149 0.160 HH Shocks + Food Security 0.155 0.140 0.148 0.149 0.155 Shocks, Food Security + Community Variables 0.156 0.142 0.147 0.146 0.157 Categorical Targeting Elderly 65+ 0.185 0.182 0.185 0.171 0.181 Widowed or disabled 0.192 0.181 0.187 0.174 0.182 Elderly, widows & disabled 0.182 0.180 0.188 0.169 0.180 Children under 14 (max 3) 0.179 0.169 0.178 0.165 0.170 Elderly, widows, disabled & children 0.179 0.170 0.183 0.163 0.171 Female heads with children 0.191 0.190 0.179 0.166 0.185 Shock: drought, flood or livestock death 0.192 0.196 0.196 0.197 0.196 Shock: drought, flood, livestock death, job loss 0.192 0.196 0.195 0.198 0.197 Note: Eligible households receive uniform per capita transfers. The total transfer amount for each country is equal to the country’s aggregate poverty gap. In the top two panels uniform transfers are based on their predicted consumption under the various PMT models. The poverty line is used to determine whether a household is eligible. The statistics in the table give the change in the country’s headcount index following the transfer. The starting value of the headcount index is 0.2. Statistics are population weighted. Categorical targeting gives transfers to each household member who meets the specified category. If a member meets the category twice he receives two transfers (e.g. elderly and disabled). The number of children who can receive transfers under the Children category is capped at 3. If a household satisfies either of the Shock categories, every household member receives a transfer. 44 Table 12: Poverty gap index post transfer Burkina Faso Ethiopia Ghana Malawi Mali Pre-transfer poverty gap 0.037 0.055 0.054 0.047 0.049 Universal (basic income) 0.030 0.045 0.040 0.039 0.039 Basic PMT covariates Basic PMT 0.027 0.051 0.034 0.039 0.049 Using means from panel data n.a. 0.042 n.a. 0.037 n.a. Poverty quantile regression 0.025 0.037 0.032 0.033 0.033 Poverty weighted: Poor only 0.030 0.045 0.039 0.038 0.039 Poverty weighted: Poor + 20 0.026 0.042 0.031 0.034 0.035 PMT with Urban/Rural 0.026 0.048 0.033 0.038 0.049 Extended PMT covariates Extended PMT 0.021 0.046 0.032 0.032 0.046 Using means from panel data n.a. 0.040 n.a. 0.030 n.a. Poverty quantile regression 0.025 0.037 0.032 0.033 0.033 Poverty weighted: Poor only 0.030 0.044 0.038 0.037 0.038 Poverty weighted: Poor + 20 0.024 0.041 0.031 0.031 0.033 Stepwise (p=0.01) 0.024 0.049 0.032 0.030 0.036 HH Shocks + Food Security 0.022 0.043 0.029 Shocks, Food Security + Community Variables 0.043 0.028 Categorical targeting Elderly 65+ 0.031 0.049 0.047 0.043 0.043 Widowed or disabled 0.032 0.050 0.042 0.042 0.044 Elderly, widows & disabled 0.031 0.049 0.043 0.041 0.043 Children under 14 (max 3) 0.031 0.044 0.038 0.038 0.042 Elderly, widows, disabled & children 0.031 0.044 0.040 0.038 0.043 Female heads with children 0.035 0.051 0.046 0.040 0.047 Shock: drought, flood or livestock death 0.036 0.053 0.045 Shock: drought, flood, livestock death, job loss 0.036 0.053 0.045 Niger Nigeria Tanzania Uganda Mean Pre-transfer poverty gap 0.039 0.050 0.053 0.059 0.051 Universal (basic income) 0.034 0.039 0.044 0.045 0.041 Basic PMT covariates Basic PMT 0.034 0.029 0.041 0.038 0.037 Using means from panel data n.a. 0.026 0.031 0.033 0.031 Poverty quantile regression 0.026 0.030 0.036 0.038 0.033 Poverty weighted: Poor only 0.034 0.038 0.044 0.044 0.040 45 Poverty weighted: Poor + 20 0.029 0.031 0.037 0.040 0.035 PMT with Urban/Rural 0.033 0.028 0.040 0.036 0.036 Extended PMT covariates Extended PMT 0.028 0.027 0.033 0.033 0.033 Using means from panel data n.a. 0.025 0.022 0.026 0.029 Poverty quantile regression 0.026 0.030 0.036 0.038 0.033 Poverty weighted: Poor only 0.033 0.036 0.044 0.043 0.039 Poverty weighted: Poor + 20 0.029 0.029 0.035 0.038 0.033 Stepwise (p=0.01) 0.031 0.032 0.034 0.032 0.035 HH Shocks + Food Security 0.026 0.026 0.031 0.032 0.033 Shocks, Food Security + Community Variables 0.026 0.027 0.031 0.032 0.034 Categorical targeting Elderly 65+ 0.036 0.044 0.048 0.044 0.045 Widowed or disabled 0.036 0.044 0.047 0.048 0.045 Elderly, widows & disabled 0.036 0.043 0.047 0.044 0.044 Children under 14 (max 3) 0.034 0.039 0.043 0.045 0.040 Elderly, widows, disabled & children 0.034 0.040 0.043 0.044 0.041 Female heads with children 0.038 0.047 0.045 0.046 0.047 Shock: drought, flood or livestock death 0.038 0.049 0.051 0.057 0.050 Shock: drought, flood, livestock death, job loss 0.038 0.049 0.051 0.057 0.050 Note: The initial value of the poverty gap index is shown in the first row. See notes to Table 11 for other details. 46 Table 13: Watts index post transfer Burkina Faso Ethiopia Ghana Malawi Mali Actual 0.044 0.074 0.069 0.060 0.064 Universal (basic income) 0.036 0.059 0.049 0.049 0.050 Basic PMT covariates Basic PMT 0.032 0.069 0.042 0.050 0.064 Using means from panel data n.a. 0.053 n.a. 0.046 n.a. Poverty quantile regression 0.029 0.048 0.038 0.040 0.041 Poverty weighted: Poor only 0.035 0.059 0.048 0.049 0.049 Poverty weighted: Poor + 20 0.030 0.055 0.037 0.042 0.044 PMT with Urban/Rural 0.031 0.064 0.040 0.048 0.064 Extended PMT covariates Extended PMT 0.025 0.061 0.040 0.041 0.059 Using means from panel data n.a. 0.050 n.a. 0.036 n.a. Poverty quantile regression 0.027 0.047 0.038 0.037 0.041 Poverty weighted: Poor only 0.035 0.058 0.047 0.047 0.048 Poverty weighted: Poor + 20 0.027 0.053 0.037 0.038 0.041 Stepwise (p=0.01) 0.027 0.066 0.040 0.037 0.046 HH Shocks + Food Security 0.025 0.057 0.036 Shocks, Food Security + Community Variables 0.057 0.034 Categorical targeting Elderly 65+ 0.037 0.066 0.060 0.055 0.056 Widowed or disabled 0.038 0.067 0.052 0.053 0.057 Elderly, widows & disabled 0.037 0.065 0.053 0.052 0.056 Children under 14 (max 3) 0.037 0.057 0.047 0.048 0.055 Elderly, widows, disabled & children 0.037 0.058 0.049 0.048 0.055 Female heads with children 0.042 0.069 0.059 0.051 0.061 Shock: drought, flood or livestock death 0.043 0.071 0.058 Shock: drought, flood, livestock death, job loss 0.043 0.071 0.058 Niger Nigeria Tanzania Uganda Mean Actual 0.048 0.064 0.069 0.085 0.067 Universal (basic income) 0.040 0.049 0.057 0.062 0.052 Basic PMT covariates Basic PMT 0.041 0.036 0.053 0.049 0.048 Using means from panel data n.a. 0.031 0.037 0.040 0.038 Poverty quantile regression 0.030 0.037 0.046 0.050 0.041 Poverty weighted: Poor only 0.040 0.047 0.056 0.061 0.051 47 Poverty weighted: Poor + 20 0.034 0.038 0.047 0.054 0.044 PMT with Urban/Rural 0.040 0.035 0.051 0.047 0.046 Extended PMT covariates Extended PMT 0.033 0.032 0.042 0.040 0.041 Using means from panel data n.a. 0.030 0.026 0.032 0.035 Poverty quantile regression 0.030 0.036 0.043 0.047 0.040 Poverty weighted: Poor only 0.039 0.045 0.056 0.059 0.050 Poverty weighted: Poor + 20 0.035 0.035 0.044 0.050 0.041 Stepwise (p=0.01) 0.038 0.039 0.043 0.042 0.045 HH Shocks + Food Security 0.031 0.032 0.040 0.040 0.041 Shocks, Food Security + Community Variables 0.031 0.032 0.039 0.039 0.042 Categorical targeting Elderly 65+ 0.043 0.055 0.063 0.058 0.058 Widowed or disabled 0.043 0.056 0.062 0.066 0.058 Elderly, widows & disabled 0.043 0.054 0.062 0.058 0.056 Children under 14 (max 3) 0.041 0.049 0.055 0.064 0.052 Elderly, widows, disabled & children 0.041 0.050 0.056 0.061 0.052 Female heads with children 0.045 0.061 0.059 0.067 0.061 Shock: drought, flood or livestock death 0.046 0.062 0.067 0.083 0.065 Shock: drought, flood, livestock death, job loss 0.046 0.062 0.067 0.083 0.065 Note: The initial value of the Watts index is shown in the first row. See notes to Table 11 for other details. 48 Table 14: Watts index post transfer using differentiated transfers Optimal transfers Actual PMT PMT Gap Linear Non-linear Burkina Faso 0.044 0.032 0.044 0.038 0.036 Ethiopia 0.074 0.069 0.074 0.056 0.053 Ghana 0.069 0.042 0.066 0.048 0.044 Malawi 0.060 0.050 0.060 0.051 0.044 Mali 0.064 0.064 0.064 0.057 0.053 Niger 0.048 0.041 0.048 0.042 0.039 Nigeria 0.064 0.036 0.062 0.049 0.043 Tanzania 0.069 0.053 0.069 0.060 0.051 Uganda 0.085 0.049 0.083 0.061 0.053 Mean 0.067 0.048 0.066 0.054 0.049 Note: This table shows the Watts index for each country following the transfers made using the Basic PMT and the differentiated transfers implied by the optimization procedure based on both linear and quadratic transfers as a function of the same variables used in the PMT with weights chosen to minimize the Watts index (see text). 49 Table 15: Targeting errors with lags using Method 1 Inclusion Exclusion Inclusion Exclusion Targeting Targeting error rate error rate error rate error rate error rate error rate (IER) (EER) (IER) (EER) (TER) (TER) Basic PMT Fixed poverty line Fixed poverty rate -1 -1 z = F (0.2) z = F (0.4) H = 0.2 H = 0.4 Ethiopia 0.000 0.993 0.302 0.882 0.677 0.445 Malawi 0.674 0.244 0.491 0.085 0.593 0.382 Nigeria 0.333 0.959 0.183 0.704 0.481 0.303 Tanzania 0.481 0.848 0.319 0.321 0.556 0.321 Uganda 0.489 0.699 0.376 0.429 0.541 0.393 Mean 0.553 0.903 0.304 0.650 0.551 0.351 Extended PMT Fixed poverty line Fixed poverty rate z = F -1 (0.2) z = F-1(0.4) H = 0.2 H = 0.4 Ethiopia 0.540 0.989 0.284 0.831 0.598 0.422 Malawi 0.644 0.178 0.470 0.053 0.514 0.332 Nigeria 0.163 0.948 0.140 0.723 0.455 0.287 Tanzania 0.424 0.773 0.277 0.348 0.501 0.295 Uganda 0.472 0.474 0.348 0.290 0.475 0.342 Mean 0.496 0.869 0.276 0.640 0.502 0.328 Note: The parameters of the PMT score are estimated using Round 1 data, then predicted values are generated using Round 2 covariate values. Underlying regressions are found in the Addendum. Statistics are population weighted. 50 Table 16: Targeting errors with lags using method 2 Inclusion Exclusion Inclusion Exclusion Targeting Targeting error rate error rate error rate error rate error rate error rate (IER) (EER) (IER) (EER) (TER) (TER) Basic PMT Fixed poverty line Fixed poverty rate -1 -1 z = F (0.2) z = F (0.4) H = 0.2 H = 0.4 Ethiopia 0.207 0.985 0.322 0.873 0.677 0.446 Malawi 0.570 0.720 0.409 0.434 0.604 0.418 Nigeria 0.184 0.964 0.182 0.745 0.475 0.304 Tanzania 0.444 0.851 0.345 0.382 0.569 0.352 Uganda 0.547 0.851 0.385 0.390 0.576 0.387 Mean 0.436 0.935 0.295 0.690 0.548 0.355 Extended PMT Fixed poverty line Fixed poverty rate z = F -1 (0.2) z = F-1(0.4) H = 0.2 H = 0.4 Ethiopia 0.359 0.967 0.343 0.793 0.613 0.437 Malawi 0.568 0.648 0.396 0.404 0.585 0.401 Nigeria 0.259 0.928 0.142 0.696 0.482 0.301 Tanzania 0.395 0.731 0.334 0.340 0.507 0.333 Uganda 0.465 0.658 0.354 0.382 0.504 0.362 Mean 0.407 0.880 0.273 0.637 0.523 0.346 Note: The PMT is calibrated using Round 1 panel data, then predicted values are compared to actual consumption values in Round 2 panel data. Regressions are found in the Addendum. Statistics are population weighted. Table 17: Targeting differentials for panel PMTs Basic PMT Extended PMT No lags Method 1 Method 2 No lags Method 1 Method 2 Ethiopia 0.100 1.000 0.585 0.126 -0.079 0.282 Malawi 0.102 -0.349 -0.141 0.178 -0.288 -0.136 Nigeria 0.317 0.334 0.633 0.403 0.675 0.482 Tanzania 0.207 0.039 0.111 0.212 0.153 0.209 Uganda 0.188 0.022 -0.094 0.221 0.056 0.070 Mean 0.275 -0.106 0.128 0.321 0.007 0.187 Note: The targeting differential is computed using the poverty line at the 20th percentile. Method 1 uses Round 1 PMT calibration and Round 2 data to generate predicted values. Method 2 uses Round 1 predicted values and Round 2 actual consumption. “No lags” refers to results when the PMT is calibrated and compared to Round 2 data (i.e. the panel structure is not used but only panel households are included). Statistics are population weighted. 51 Table 18: Headcount index post transfer, round 2 Ethiopia Malawi Nigeria Tanzania Uganda Mean PMT Targeting Basic PMT 0.187 0.176 0.146 0.165 0.153 0.161 Extended PMT 0.172 0.155 0.137 0.153 0.146 0.150 Method 1 Basic 0.199 0.160 0.193 0.170 0.143 0.186 Method 1 Extended 0.198 0.154 0.191 0.160 0.143 0.183 Method 2 Basic 0.197 0.163 0.194 0.176 0.170 0.189 Method 2 Extended 0.194 0.161 0.187 0.157 0.152 0.181 Categorical Targeting Household size 0.168 0.164 0.171 0.173 0.164 0.170 Elderly 65+ 0.182 0.180 0.176 0.179 0.168 0.178 Widowed or disabled 0.177 0.173 0.181 0.178 0.173 0.179 Elderly, widows& disabled 0.180 0.169 0.177 0.178 0.163 0.176 Children under 14 (max 3) 0.166 0.163 0.171 0.168 0.157 0.168 Elderly, widows, disabled & children 0.164 0.160 0.169 0.171 0.158 0.167 Female heads with children 0.181 0.171 0.190 0.167 0.158 0.182 Note: Method 1 uses Round 1 PMT calibration and Round 2 data to generate predicted values. Method 2 uses Round 1 predicted values and compares to Round 2 actual consumption. The Basic and Extended PMT methods (rows 1 and 2) are using Round 2 data only (i.e. no lags). Only panel households are included. Statistics are population weighted. 52 Table 19: Targeting errors as predictors of the post-transfer poverty measures obtained by PMT Headcount index Poverty gap index Watts index Basic PMT Extended Basic PMT Extended Basic PMT Extended PMT gaps Optimal PMT PMT PMT (nonlinear) Constant 0.083*** 0.116*** -0.035*** -0.031*** -0.047*** -0.035** -0.041* -0.001 (0.012) (0.014) (0.004) (0.006) (0.010) (0.014) (0.020) (0.004) Initial poverty n.a. n.a. 0.750*** 0.731*** 0.657*** 0.584*** 1.119*** 0.483*** measure (0.062) (0.074) (0.095) (0.098) (0.182) (0.041) Inclusion error 0.028 0.070 0.005 -0.008 0.005 -0.015 -0.025 0.003 rate (0.015) (0.045) (0.003) (0.014) (0.009) (0.032) (0.016) (0.004) Exclusion 0.090*** 0.098*** 0.043*** 0.046*** 0.065*** 0.065*** 0.034 0.018** error rate (0.020) (0.014) (0.004) (0.004) (0.012) (0.010) (0.022) (0.005) R2 0.927 0.913 0.986 0.980 0.954 0.946 0.886 0.969 F (prob) 38.240 31.337 114.373 81.257 34.458 29.259 12.917 48.725 (0.000) (0.001) (0.000) (0.000) (0.001) (0.001) (0.009) (0.000) Note: Standard errors in parentheses. ***: significant at 1% level; **: 5%; *: 10%. N=9. Table 20: R-squared in PMT regression as a predictor of the post-transfer poverty measures Headcount index Poverty gap index Watts index Basic PMT Extended Basic PMT Extended Basic PMT Extended PMT gaps Optimal PMT PMT PMT (nonlinear) Constant 0.228*** 0.209*** 0.080*** 0.079*** 0.110*** 0.106*** 0.001 0.041** (0.029) (0.024) (0.021) (0.022) (0.025) (0.024) (0.028) (0.012) Initial poverty n.a. n.a. -0.079 -0.048 -0.055 -0.038 0.927*** 0.298** measure (0.273) (0.263) (0.219) (0.195) (0.239) (0.106) R2 from PMT -0.113* -0.085* -0.071*** -0.071*** -0.107*** -0.102*** -0.027 -0.027* regression (0.052) (0.038) (0.019) (0.018) (0.026) (0.023) (0.028) (0.013) R2 0.401 0.415 0.767 0.803 0.796 0.827 0.847 0.842 F (prob) 4.682 4.964 9.875 12.240 11.735 14.301 16.559 15.969 (0.067) (0.061) (0.013) (0.008) (0.008) (0.005) (0.004) (0.004) Note: Standard errors in parentheses. ***: significant at 1% level; **: 5%; *: 10%. N=9.